Poor Depth Quality on OAK FFC 4P with OV9782 Stereo Pair

OskarSonc

Hi @ShreyasK thanks for the detailed report!

Main issue I see is that you’re undistorting in the Camera node and letting StereoDepth rectify, which double-warps the images and degrades matching. I’d suggest starting from our v3 StereoDepth example and adjusting from there:

https://docs.luxonis.com/software-v3/depthai/examples/stereo_depth/stereo_depth/

Also, did you calibrate the cameras? Guide here:
https://docs.luxonis.com/hardware/platform/depth/calibration/

Hope it helps,
Oskar

ShreyasK

Thank you for reply @OskarSonc

I will attach few images for your kind reference:

"Bottle" is placed at a distance of 44cm approx.

This the default stereo.py
This default stereo.py with resolution of 640,400.
This actual camera feed:

Calibration is done this is the output:

After Calibration:

640,400 with default code:
default code(FUllresolution output)
Camera Feed(640,400):

Based on these, I felt calibration improved the stuff majorly. And undistorted feed cannot be used for Stereo.

So few question:

do u have any suggestions on setting etc (I have read configuring stereo depth page but need little info from personal experience with Wide-angle lens and oak FFC module).
So as I am working on a project where it needs undistorted feed for VIO and stereo depth for obstacle avoidance, how can I get two feed at a time. Because undistortion setting can be turned on only in requestOutput?
Or any trick to use undistorted feed with stereo depth?
Left camera feels more blurry at corners (I do not mean extreme corner). Tried cleaning etc but didn't able to fix the same.

jakaskerl

ShreyasK
Looks like like a bad calibration.. Can you send the configuration.json you used during calibration and the command you used to run it?

Thanks
Jaka

ShreyasK

Sure @jakaskerl.

configuration.json file:

I used 32 inch charuco pdf on my 27 inch.

Distance between cameras is 5cm.

And this is the command I used:
python3 calibrate.py -s 3.284 -nx 17 -ny 9 -brd OAK-FFC-4P.json

jakaskerl

ShreyasK
Try specifying this for both stereo cameras:

monoRight = pipeline.create(dai.node.Camera).build(dai.CameraBoardSocket.CAM_C)
monoRight.setSensorType(dai.CameraSensorType.COLOR)

This should force the board to treat OV9782 as a COLOR sensor so it doesn't 2x2 bin it to 400p and break the image.

Thanks,
Jaka

ShreyasK

@jakaskerl Will try the same and check performance.

Is the calibration and stuff correct?

jakaskerl

ShreyasK
Yeah looks fine. Only issue I can see would be the wrong placement of sensors (mix-up of L and R).

Thanks,
Jaka

ShreyasK

Do u mean that right should be left and left should be right?
The left and right configuration is correct, I have rechecked it.
This is the output of using this: monoRight.setSensorType(dai.CameraSensorType.COLOR)

Compared to default mono:
With few filters like and also updated to newer stable release 3.0.0:
stereo.setSubpixel(True)

stereo.setSubpixelFractionalBits(4)

stereo.PresetMode(dai.node.StereoDepth.PresetMode.ROBOTICS)

stereo.setPostProcessingHardwareResources(3, 3)

stereo.initialConfig.postProcessing.temporalFilter.enable = True

stereo.initialConfig.postProcessing.temporalFilter.alpha = 0.5

stereo.initialConfig.postProcessing.temporalFilter.delta = 3

stereo.initialConfig.postProcessing.speckleFilter.enable = True

stereo.initialConfig.postProcessing.speckleFilter.speckleRange = 5

stereo.initialConfig.postProcessing.spatialFilter.enable = True

stereo.initialConfig.postProcessing.spatialFilter.alpha = 0.5

stereo.initialConfig.postProcessing.spatialFilter.delta = 3

stereo.initialConfig.postProcessing.spatialFilter.holeFillingRadius = 2

stereo.initialConfig.postProcessing.spatialFilter.numIterations = 1

stereo.initialConfig.postProcessing.decimationFilter.decimationFactor = 2

Output
This is the output of real sense from same spot the oak camera was kept:
Is there any way we can get output from decimation filter in desired resolution than that short output?

ShreyasK

Hi @jakaskerl
Any inputs from your end?

jakaskerl

ShreyasK
Was on vacation, apologies for late reply. The image looks good, I was mainly concerned with the 400p image since binning is done when downsizing from 800p.

If you want better depth, you need to improve the texture of the surfaces you are looking at. The bottle looks completely black so the algorithm can not perform depthai.

Thanks,
Jaka

ShreyasK

jakaskerl Thank you for your reply.

With regard to binning, as I need grayscale than color, will there be any way on this regard?
Because if I set sensorType to color it give color output which will affect the other systems which need the greyscale image feed.

jakaskerl

Hi ShreyasK
The OV9728 sensor is a color sensor. If it's shows a mono stream then it's already been converted. You can manually convert it to mono using imageManip if you really need to - otherwise, there shouldn't be any issues just using the color stream.

Thanks,
Jaka