How to configure depth alignment at runtime and adjust the size of depth map?

rohan

Hello,

I'm currently using StereoDepth.setDepthAlign to set the alignment while building the pipeline. I'm trying to make the alignment configurable at runtime, so I tried using StereoDepthConfig.AlgorithmControl.depthAlign only.

I realized that StereoDepth.setDepthAlign ensures that the depth map and the aligning frame are of the same size. StereoDepthConfig.AlgorithmControl.depthAlign doesn't seem to do this. I can see a "shift" in the depth map view, confirming a change in alignment, but the size stays the size as the mono cam (which is 1280x800). And when I have Decimation filter enabled with factor 2, the depth map's size is 640x400.

I'm working with OAK-FFC-4P, and I want to be able to switch alignment between color cam and mono cam pair. Is it possible to change alignment at runtime and also get the depth map with same size as that of the aligning frame?

I appreciate any help.

Regards,
Rohan

OskarSonc

Hey rohan - yes, this is expected. StereoDepthConfig.AlgorithmControl.depthAlign only changes the depth viewpoint (RECTIFIED_LEFT / RECTIFIED_RIGHT / CENTER). It does not switch to camera-specific alignment or resize the depth output to match that camera, so with decimation enabled you can still get 640x400.

If you want depth aligned to a specific camera and with the same output size as that camera, you need to use the camera-alignment path (StereoDepth.setDepthAlign(CameraBoardSocket...) / inputAlignTo). That part is configured when building the pipeline, so switching between color-cam alignment and mono-cam alignment at runtime is not supported in the same way.

If you need the depth output to always match the target stream size, the best option is to use ImageAlign, or rebuild the pipeline when changing the alignment target.

Docs:
https://docs.luxonis.com/software-v3/depthai/depthai-components/nodes/stereo_depth
https://docs.luxonis.com/software-v3/depthai/depthai-components/nodes/image_align/

Best,
Oskar

rohan

OskarSonc
Hey Oskar, sorry for late reply. I realized that I don't need the depth map to match the size of the output from the aligned camera. I'm switching the alignment at runtime and updating the intrinsic and extrinsic properties for re-projection. I'm getting correct depth/distance measurements.

Thank you.

OskarSonc

rohan - no worries, happy you figured it out!

Best,
Oskar