How to implement mono camera for spatial in SDK

jsiic

Hello,

I am trying to do image & spatial recognition in low light settings by using mono camera in SDK.

from depthai_sdk import OakCamera
import depthai as dai

with OakCamera() as oak:
    left = oak.create_camera('left')
    nn = oak.create_nn('yolov8n_coco_640x352', left, spatial = True)

    nn.config_spatial(
        bb_scale_factor=0.5, # Scaling bounding box before averaging the depth in that ROI
        lower_threshold=300, # Discard depth points below 30cm
        upper_threshold=10000, # Discard depth pints above 10m
        #Average depth points before calculating X and Y spatial coordinates:
        calc_algo=dai.SpatialLocationCalculatorAlgorithm.AVERAGE
    )
    oak.visualize([nn.out.main], fps=True)
    #oak.show_graph()
    oak.start(blocking=True)

However, I get the following error:

[2023-06-13 09:00:10] INFO [root.__init__:137] Setting IR laser dot projector brightness to 800mA
[2023-06-13 09:00:11] INFO [root.__exit__:328] Closing OAK camera
Traceback (most recent call last):
  File "C:\Users\kimjs\PycharmProjects\OakD\yolo8.py", line 17, in <module>
    oak.start(blocking=True)
  File "C:\Users\kimjs\PycharmProjects\OakD\venv\Lib\site-packages\depthai_sdk\oak_camera.py", line 362, in start
    self._oak.device.startPipeline(self.pipeline)
RuntimeError: MonoCamera(5) - Specified camera board socket already used
Sentry is attempting to send 2 pending error messages
Waiting up to 2 seconds
Press Ctrl-Break to quit

It seems like the issue here is that MonoCamera is being used twice and the SDK doesn't allow that. However, it seems to work just fine in API examples. Could anyone help me figure out how to work through this?

jakaskerl

Hi jsiic
When passing a boolean to as argument to create_nn(spatial =bool), a stereo stream is internally created which reinstantiates both mono cameras. Since you created a mono camera instance beforehand, this will result in runtime error since a socket is already used. You need to pass a StereoComponent as the argument. This will ensure that the sockets (cameras) get reused rather then reinstantiated.

from depthai_sdk import OakCamera
import depthai as dai

with OakCamera() as oak:
    left = oak.create_camera("left")
    right = oak.create_camera("right")
    stereo = oak.create_stereo(left=left, right=right, fps=30)

    nn = oak.create_nn('yolov8n_coco_640x352', left, spatial=stereo)

    nn.config_spatial(
        bb_scale_factor=0.5, # Scaling bounding box before averaging the depth in that ROI
        lower_threshold=300, # Discard depth points below 30cm
        upper_threshold=10000, # Discard depth pints above 10m
        #Average depth points before calculating X and Y spatial coordinates:
        calc_algo=dai.SpatialLocationCalculatorAlgorithm.AVERAGE
    )
    oak.visualize([nn.out.main], fps=True)
    #oak.show_graph()
    oak.start(blocking=True)

Hope this helps,
Jaka

jsiic

jakaskerl Thank you so much!