V3 RGB Depth Align

Shai

The setDepthAlign function is not resizing the stereo output to match the RGB output in depthAI v3 like it used to in v2.
The following is my code:

    auto camRgb = pipeline_->create<dai::node::Camera>()->build(dai::CameraBoardSocket::CAM_A);
    auto monoLeft = pipeline_->create<dai::node::Camera>()->build(dai::CameraBoardSocket::CAM_B);
    auto monoRight = pipeline_->create<dai::node::Camera>()->build(dai::CameraBoardSocket::CAM_C);
    auto stereo = pipeline_->create<dai::node::StereoDepth>();

    monoLeft->requestFullResolutionOutput()->link(stereo->left);
    monoRight->requestFullResolutionOutput()->link(stereo->right);

    stereo->setDefaultProfilePreset(dai::node::StereoDepth::PresetMode::DEFAULT);
    stereo->initialConfig->setMedianFilter(dai::StereoDepthConfig::MedianFilter::KERNEL_7x7);
    stereo->setLeftRightCheck(true);
    stereo->setExtendedDisparity(true);
    stereo->setSubpixel(true);
    stereo->setDepthAlign(dai::CameraBoardSocket::CAM_A);

OskarSonc

Hi @Shai,

setDepthAlign() in DepthAI v3 only does the geometric align; it no longer resizes the depth map.
Add one line to get the old (v2) behaviour:

stereo->setDepthAlign(dai::CameraBoardSocket::CAM_A);  // align to RGB
stereo->setOutputSize(1280, 720);                      // resize to match RGB (Width must be a multiple of 16.)

Hope this helps.

Oskar

Shai

I am getting the error that "StereoDetph | setOutputSize is not supported on RVC4 platform"

OskarSonc

I see. Maybe you can try setting decimationFactor = 2 filter in StereoDepth.

// ↓ add this right after stereo->setDepthAlign(...)
auto cfg = stereo->initialConfig.get();
cfg.postProcessing.decimationFilter.decimationFactor = 2;
stereo->initialConfig.set(cfg);

Shai

How is that going to make the stereo output resolution match the RGB resolution? That will decrease the stereo depth resolution which is already smaller than the RGB resolution.

Shai

For our purposes, we are looking for a point in the RGB image (a chessboard), and we want to find the depth in the stereo depth image. Since depth-ai no longer automatically resized the depth image when calling "setDepthAlign()", I resized the stereo depth manually. However, I noticed that the field of view in the stereo depth was wider than in the RGB image. If the chessboard is near the edge of the RGB image, then the corresponding point near the edge of the resized stereo depth image is not on the chess board, and I get a very inaccurate depth reading for the location of the chess board.

How would you recommend finding the depth of a specific (x, y) pixel in the RGB image?

Shai

The AI chatbot on the Luxonis docs pages said that "setDepthAlign()" is not recommended for DepthAi V3 and that I should use the ImageAlign node instead. Is this true?

OskarSonc

Hi @Shai

setDepthAlign() is still the preferred method when possible — it's more efficient since the alignment happens directly in the stereo engine. However, on RVC-4, it doesn't support resizing the depth map to match the RGB resolution yet.

For now, the best option is to use the ImageAlign. More information here:
https://docs.luxonis.com/software-v3/depthai/depthai-components/nodes/image_align#ImageAlign

Thanks,
Oskar