I'm working on an autonomous wheelchair system that uses OAK-D Pro W cameras for person tracking and obstacle detection. The camera stands at a height of 2 meters, looking down at a 27° angle. The IR laser dot projector is set to 765mA, and the flood IR LED is off. The following options are set in the dai::node::StereoDepth object:

stereo->setDefaultProfilePreset(dai::node::StereoDepth::PresetMode::HIGH_ACCURACY);
stereo->setLeftRightCheck(true);
stereo->setRectification(true);
stereo->setExtendedDisparity(false);
stereo->setSubpixel(true);
stereo->initialConfig.setMedianFilter(dai::MedianFilter::KERNEL_7x7);

I have found that for some very specific tiled surfaces, the camera will underestimate the depth of the ridges between tiles, placing them a lot closer to the camera then they're actually are. However this is only a problem when the IR laser dot is on — artifacts disappear when it is turned off. See the pictures below for examples:

Depth image with IR laser dot turned on — notice the light blue (low depth) artifacts inside the white circle.

The same surface images with the IR laser dot off — no artifacts are seen.

The thing about this issue is that it's extremely hard to reproduce. So far I have been unable to find another environment where it's so consistently found as the one above. For example, none of these tiled surfaces show the same pattern:

On the other hand, this flight of stairs shows depth artifacts even with the laser dot turned off:

Has anyone seen similar issues? Any suggestions on how to address it?

  • I was able to get rid of the depth artifacts, but it took more than lowering the confidence threshold. Specifically, I'm using the following configuration options:

        import depthai as dai

    from depthai import MedianFilter
    DecimationMode = dai.RawStereoDepthConfig.PostProcessing.DecimationFilter.DecimationMode

    # ...

    message = dai.StereoDepthConfig()
    options = message.get()

    options.algorithmControl.leftRightCheckThreshold = 1
    options.costMatching.confidenceThreshold = 200
    options.costMatching.linearEquationParameters.alpha = 1
    options.costMatching.linearEquationParameters.beta = 1
    options.postProcessing.brightnessFilter.maxBrightness = 250
    options.postProcessing.decimationFilter.decimationFactor = 2
    options.postProcessing.decimationFilter.decimationMode = DecimationMode.NON_ZERO_MEAN
    options.postProcessing.median = MedianFilter.KERNEL_7x7
    options.postProcessing.spatialFilter.enable = True
    options.postProcessing.speckleFilter.enable = True

    message.set(options)

    # ...

    For illustration, in the image below you can see the view from the RGB camera, the depth map with (mostly) default settings, and the depth map with the above settings.

Hi @xperroni
I'll have to take a look at that. Will report back when I find a solution.

Thanks for the patience,
Jaka

Hi @xperroni
I find that lowering the confidence threshold improves this (it does make the depth more scarce, but at least the confidence is higher). Try values like 200 and 100.

Thanks,
Jaka

    4 months later

    I was able to get rid of the depth artifacts, but it took more than lowering the confidence threshold. Specifically, I'm using the following configuration options:

        import depthai as dai

    from depthai import MedianFilter
    DecimationMode = dai.RawStereoDepthConfig.PostProcessing.DecimationFilter.DecimationMode

    # ...

    message = dai.StereoDepthConfig()
    options = message.get()

    options.algorithmControl.leftRightCheckThreshold = 1
    options.costMatching.confidenceThreshold = 200
    options.costMatching.linearEquationParameters.alpha = 1
    options.costMatching.linearEquationParameters.beta = 1
    options.postProcessing.brightnessFilter.maxBrightness = 250
    options.postProcessing.decimationFilter.decimationFactor = 2
    options.postProcessing.decimationFilter.decimationMode = DecimationMode.NON_ZERO_MEAN
    options.postProcessing.median = MedianFilter.KERNEL_7x7
    options.postProcessing.spatialFilter.enable = True
    options.postProcessing.speckleFilter.enable = True

    message.set(options)

    # ...

    For illustration, in the image below you can see the view from the RGB camera, the depth map with (mostly) default settings, and the depth map with the above settings.