OAK-D Pro wide OV9782 RGB-Depth Alignment

RyanLee

Hi,

We are using the OAK-D Pro Wide(with OV972). And we would like to check the RGB Depth alignment like below link.

RGB Depth alignment — DepthAI documentation | Luxonis

But based on our test it seems not available its RGB-Depth Alignment function anymore with the OAK-D Pro Wide(with OV9782). Is our test result is right? and are there any way to make this RGB-Depth Alighment with the OAK-D Pro Wide(with OV972)?

<picture explanation>
In the picture below, you can see that the angle of view of the depth image is narrower than that of the RGB image.

There is a person at the edge of the RGB image, but in the Depth image the person is out of the field of view and cannot be seen.

Best regards,
Ryan.

erik

Hi RyanLee ,
Could you please provide full MRE, together with minimal script you are using?
Thanks ,Erik

RyanLee

Based on our test you can check the RGB-Depth angle of view difference by just running the demo code provided by Luxonis like below. How do you think?

Best regards,
Ryan.

erik

Hi RyanLee ,
The OV9782 color image has much wider FOV than the depth (as mono cameras need to be rectified for the depth) - so that's expected.
Thanks, Erik

RyanLee

Hi erik

That means that the RGB Depth alignment is not available for OAK-D Pro wide OV9782 anymore?
<RGB Depth alignment>
https://docs.luxonis.com/projects/api/en/latest/samples/StereoDepth/rgb_depth_aligned/

Best regards,
Ryan.

erik

Hi RyanLee ,
It should be possible to rgb-detph align the model, but you would need to undistort the rgb, as calibration uses undistorted rgb when calculating intrinsics.
Thanks, Erik

RyanLee

HI erik

We tried the code undistort the rgb image. But we got some error result like below.

Please check it out and let us know how to handle this issue.

Best regards,
Ryan.

erik

Hi RyanLee ,
I have developed rgb-depth alignemnt demo that also does rgb undistortion;
I tried it with the OAK-D W and it works as expected.
Thanks, Erik

RyanLee

HI erik

I checked the test code you shared us above.
It has been confirmed that the alignment between RGB/Depth has been significantly improved compared to before.

However, the alignment between the two images still seems insufficient at the lens periphery for use in Spatial Detection.

Is it possible to further improve the alignment between the two images more precisely?

Best regards,
Ryan.

erik

Hi RyanLee ,
Have you modified the script so it uses 800P color camera (which is max of OV9782), and potentially disabled isp scaling?
So the first few lines of the script would be:

import cv2
import depthai as dai
import numpy as np

camRes = dai.ColorCameraProperties.SensorResolution.THE_800_P
camSocket = dai.CameraBoardSocket.RGB
ispScale = (1,1)

RyanLee

Hi erik

This issue is cleared. So it is working now. And there was one of our mistake.

Actually, this issue is cleared with above solution you shared. But we did some mistake (Bug) by ourselves. Thank you for your patience always.

Best regards,
Ryan.