Hello,

I have an Oak-D Pro W and I'm doing recording of only video flux (RGB, left, right). Here are my post-process steps on computer :

  • YOLO on RGB video to have object bonding boxes
  • Prepare left and right video for stereo computation: cv2.stereoRectify and cv2.initUndistortRectifyMap
  • Stereo computation (disparity then depth map)
  • Find depth of the center of bonding boxes.

However I'm facing a problem, my bounding boxes computed on the RGB video are not aligned with the depth map due to the rectification and undistortion. What is the easier way to do the alignment between both depth and RGB?

Thank you in advance.

François

    Hi fraymackers
    It would be easier if you added a mre of your code. I believe you would get the correct coordinates if you set the depthalign to the rgb camera (example here). Also, what about just using our YoloSpatialDetectionNetwork node? It simplifies the process.

    Thanks,
    Jaka

      5 days later

      For normal aruco detect ,if we don't align RGB-depth,why the right camera is the reference frame?

        Hi ManiRadhakrishnan
        The camera calculates depth from comparing images from left and right rectified cameras. This means that one of those two mono cameras will be the reference frame. Without specifically setting the camera which the depth is aligned to, the depthai will select the right mono camera.

        Thanks,
        Jaka