Hi Luxonis experts,

I'm using the OAK-FFC-4P with 3 AR0234 sensors, and configure the the OAK-FFC-4P for multiple baseline : 5cm, 10cm and 15cm.
- With baseline 15cm, the camera left is CAM_B, the camera right is CAM_C and the RGB camera is CAM_A. In this case, the stereo depth alignment to CAM_A works normally.

However, with baseline 10cm, the camera left is CAM_A, the camera right is CAM_C and the RGB camera is CAM_B, then the stereo depth alignment to CAM_B error with logs below. It's same for baseline 5cm, where left camera is CAM_B, right camera is CAM_A and the RGB camera is CAM_C.

[14442C10D11AE7D600] [1.1] [3.203] [StereoDepth(3)] [error] DepthAlign is only implemented for RGB camera and LEFT and RIGHT stereo inputs!
terminate called after throwing an instance of 'std::runtime_error'
  what():  Communication exception - possible device error/misconfiguration. Original message 'Couldn't read data from stream: 'rgb' (X_LINK_ERROR)'
Aborted (core dumped)

Look like the firmware have not supported the DepthAlign to other socket then RGB camera (CAM_A) yet. Am I correct ?
Could you please guide me how to fix this issue ? Many thank in advance !

Best regards,
Hiep Nguyen

    HiepNguyen Look like the firmware have not supported the DepthAlign to other socket then RGB camera (CAM_A) yet. Am I correct ?

    Looks like the color stereo doesn't support aligning to LEFT or RIGHT yet. Mono stereo does. No solution currently (other than to use CAM_A as the socket to which you align to).

    I'll notify the team.

    Thanks,
    Jaka

      Sorry for this follow up question: Are you using for stereo depth, a color input or a grayscale input?
      I am trying the same the OAK-D LD and got errors.

        HiepNguyen
        What's the reason why you would want to align color to the middle camera if all three are RGB?

        isk I am trying the same the OAK-D LD and got errors.

        What errors are you getting?

        Thanks,
        Jaka

              jakaskerl

              Sorry, it is an OAK-D LR and was trying to execute one of the examples (depth_preview.py),
              without any alignment of the middle camera.

                  isk
                  Can you open a separate thread and elaborate more on the error you are receiving?

                  Thanks,
                  Jaka

                    Thank you for the reply. I found a working example for the OAK-D LR. The main idea is to use isp instead of preview.

                    jakaskerl

                    Hi @jakaskerl ,

                    To make it clear, I want to align the depth output of the stereo pair with the other camera. It's similar to OAK-D LR camera use case, I believe so.

                    For example : with 3 RGB AR0234 sensors, I expect to have 3 baselines : 5cm, 10cm and 15cm from the camera, and the depth output from stereo pair is aligned with the other RGB camera. Current status :

                    • The setup as normal OAK-D camera with stereo pair in 2 side, RGB camera in the middle, then the depth output from stereo pair can be aligned with RBG camera.
                    • With RGB camera in the side, 2 sensors left is stereo pair, then the depth output from stereo pair can't be aligned with RGB camera, with error logs:
                    [14442C10D11AE7D600] [1.1] [3.203] [StereoDepth(3)] [error] DepthAlign is only implemented for RGB camera and LEFT and RIGHT stereo inputs!
terminate called after throwing an instance of 'std::runtime_error'
  what():  Communication exception - possible device error/misconfiguration. Original message 'Couldn't read data from stream: 'rgb' (X_LINK_ERROR)'
Aborted (core dumped)

                    Could you please help to confirm that the OAK-D LR works as my expectation, and do the OAK-FFC-4P work same as OAK-D LR multiple baseline functionality ?

                    Many thanks in advance !

                    Best regards,
                    Hiep Nguyen

                        4 days later

                        Hi HiepNguyen
                        Sorry for late answer.
                        The 4P should work the same I think. Keep in mind that for color stereo, the alignment is hard-coded to CAM_A. This is likely the case in your setup. That's is why alignment doesn't work for the second case, since you have one of the stereo cameras connected to CAM_A.

                        Is this not the case for you?

                        Thanks,
                        Jaka