Hello, I have an Oak-D Pro W and I'm doing recording of only video flux (RGB, left, right). Here are my post-process steps on computer : * YOLO on RGB video to have object bonding boxes * Prepare left and right video for stereo computation: cv2.stereoRectify and cv2.initUndistortRectifyMap * Stereo computation (disparity then depth map) * Find depth of the center of bonding boxes. However I'm facing a problem, my bounding boxes computed on the RGB video are not aligned with the depth map due to the rectification and undistortion. What is the easier way to do the alignment between both depth and RGB? Thank you in advance. François

Hi @"fraymackers"#p11726 It would be easier if you added a mre of your code. I believe you would get the correct coordinates if you set the depthalign to the rgb camera ([example here](https://docs.luxonis.com/projects/api/en/latest/samples/StereoDepth/rgb_depth_aligned/)). Also, what about just using our YoloSpatialDetectionNetwork node? It simplifies the process. Thanks, Jaka

Offline RGB-depth alignement

fraymackers

Hello,

I have an Oak-D Pro W and I'm doing recording of only video flux (RGB, left, right). Here are my post-process steps on computer :

YOLO on RGB video to have object bonding boxes
Prepare left and right video for stereo computation: cv2.stereoRectify and cv2.initUndistortRectifyMap
Stereo computation (disparity then depth map)
Find depth of the center of bonding boxes.

However I'm facing a problem, my bounding boxes computed on the RGB video are not aligned with the depth map due to the rectification and undistortion. What is the easier way to do the alignment between both depth and RGB?

Thank you in advance.

François

jakaskerl

Hi fraymackers
It would be easier if you added a mre of your code. I believe you would get the correct coordinates if you set the depthalign to the rgb camera (example here). Also, what about just using our YoloSpatialDetectionNetwork node? It simplifies the process.

Thanks,
Jaka

ManiRadhakrishnan

For normal aruco detect ,if we don't align RGB-depth,why the right camera is the reference frame?

jakaskerl

Hi ManiRadhakrishnan
The camera calculates depth from comparing images from left and right rectified cameras. This means that one of those two mono cameras will be the reference frame. Without specifically setting the camera which the depth is aligned to, the depthai will select the right mono camera.

Thanks,
Jaka