Stereo Depth Alignment with camera Socket CAM_B and CAM_C

HiepNguyen

Hi Luxonis experts,

I'm using the OAK-FFC-4P with 3 AR0234 sensors, and configure the the OAK-FFC-4P for multiple baseline : 5cm, 10cm and 15cm.
- With baseline 15cm, the camera left is CAM_B, the camera right is CAM_C and the RGB camera is CAM_A. In this case, the stereo depth alignment to CAM_A works normally.

However, with baseline 10cm, the camera left is CAM_A, the camera right is CAM_C and the RGB camera is CAM_B, then the stereo depth alignment to CAM_B error with logs below. It's same for baseline 5cm, where left camera is CAM_B, right camera is CAM_A and the RGB camera is CAM_C.

[14442C10D11AE7D600] [1.1] [3.203] [StereoDepth(3)] [error] DepthAlign is only implemented for RGB camera and LEFT and RIGHT stereo inputs!
terminate called after throwing an instance of 'std::runtime_error'
  what():  Communication exception - possible device error/misconfiguration. Original message 'Couldn't read data from stream: 'rgb' (X_LINK_ERROR)'
Aborted (core dumped)

Look like the firmware have not supported the DepthAlign to other socket then RGB camera (CAM_A) yet. Am I correct ?
Could you please guide me how to fix this issue ? Many thank in advance !

Best regards,
Hiep Nguyen

jakaskerl

HiepNguyen Look like the firmware have not supported the DepthAlign to other socket then RGB camera (CAM_A) yet. Am I correct ?

Looks like the color stereo doesn't support aligning to LEFT or RIGHT yet. Mono stereo does. No solution currently (other than to use CAM_A as the socket to which you align to).

I'll notify the team.

Thanks,
Jaka

jakaskerl

HiepNguyen
What's the reason why you would want to align color to the middle camera if all three are RGB?

isk I am trying the same the OAK-D LD and got errors.

What errors are you getting?

Thanks,
Jaka

isk

Sorry for this follow up question: Are you using for stereo depth, a color input or a grayscale input?
I am trying the same the OAK-D LD and got errors.

isk

jakaskerl

Sorry, it is an OAK-D LR and was trying to execute one of the examples (depth_preview.py),
without any alignment of the middle camera.

HiepNguyen

jakaskerl

Hi @jakaskerl ,

To make it clear, I want to align the depth output of the stereo pair with the other camera. It's similar to OAK-D LR camera use case, I believe so.

For example : with 3 RGB AR0234 sensors, I expect to have 3 baselines : 5cm, 10cm and 15cm from the camera, and the depth output from stereo pair is aligned with the other RGB camera. Current status :

The setup as normal OAK-D camera with stereo pair in 2 side, RGB camera in the middle, then the depth output from stereo pair can be aligned with RBG camera.
With RGB camera in the side, 2 sensors left is stereo pair, then the depth output from stereo pair can't be aligned with RGB camera, with error logs:

[14442C10D11AE7D600] [1.1] [3.203] [StereoDepth(3)] [error] DepthAlign is only implemented for RGB camera and LEFT and RIGHT stereo inputs!
terminate called after throwing an instance of 'std::runtime_error'
  what():  Communication exception - possible device error/misconfiguration. Original message 'Couldn't read data from stream: 'rgb' (X_LINK_ERROR)'
Aborted (core dumped)

Could you please help to confirm that the OAK-D LR works as my expectation, and do the OAK-FFC-4P work same as OAK-D LR multiple baseline functionality ?

Many thanks in advance !

Best regards,
Hiep Nguyen

jakaskerl

isk
Can you open a separate thread and elaborate more on the error you are receiving?

Thanks,
Jaka

isk

Thank you for the reply. I found a working example for the OAK-D LR. The main idea is to use isp instead of preview.

jakaskerl

Hi HiepNguyen
Sorry for late answer.
The 4P should work the same I think. Keep in mind that for color stereo, the alignment is hard-coded to CAM_A. This is likely the case in your setup. That's is why alignment doesn't work for the second case, since you have one of the stereo cameras connected to CAM_A.

Is this not the case for you?

Thanks,
Jaka