Bad depth and point cloud precision for my new OAK-D Pro W PoE

AnimeshSureshGunaki

Hi,

We recently purchased Luxonis OAK-D Pro W PoE. But, the point cloud generated by the sensor looks worse with large parts of the depth information is missing in the point cloud. I analyzed the point cloud generated by both depthai_viewer and the ros2 wrapper. But, both of them provide same results.

I have attached the images of the point cloud where there is a box places infront of the camera. From the image, you can clearly see that, at one side of the box large parts of the depth information is missing. We compared the camera with Intel RealSense D455 which provided way better point cloud than our OAK-D Pro W.

Point Cloud for OAK-D Pro W in depthai_viewer

Cropped Point Cloud of the box kept front of the camera generated using depthai_ros driver and OAK-D Pro W

Point Cloud generated of the box using Intel RealSense D455 (Just for Comparison as I think OAK D Pro W should perform better than Realsense)

Is this the error regarding the hardware or wrong calibration for OAK-D Pro W? Could you please help us to fix this issue?

The Details of my Camera Module are:

Model Name: Luxonis OAK-D Pro W PoE

MXID: 14442C10E1E7FCD600

P/N: A00574 D/C: 1724A3

Awaiting your reply! Thank You!

Regards,

Animesh

jakaskerl

AnimeshSureshGunaki
Are you using this script: luxonis/depthai-rosblob/humble/depthai_ros_driver/launch/pointcloud.launch.py ?

I'm not too familiar with ROS but I assume it is using the pcl config: luxonis/depthai-rosblob/humble/depthai_ros_driver/config/pcl.yaml

Which would mean other parameters are set as default:
https://docs.luxonis.com/software/ros/depthai-ros/driver/#DepthAI%20ROS%20Driver-List%20of%20parameters

So add/modify stereo parameters to improve depth. Especially the i_enable_distortion_correction since you are using WFOV camera which has high distortion.

AnimeshSureshGunaki I would like to add to my previous post. Can you also tell if there is any way to check if the camera was factory calibrated with new procedure?

I can check internally. It was calibrated on 23.6.2024 with new calibration procedure with a good score. How is the overall depth quality (when checking with other scripts like https://docs.luxonis.com/software/depthai/examples/#Depthai%20Examples-StereoDepth)?

Thanks,
Jaka

AnimeshSureshGunaki

I would like to add to my previous post. Can you also tell if there is any way to check if the camera was factory calibrated with new procedure?

AnimeshSureshGunaki

Hi jakaskerl Thank You for your reply!

Yes, you are right the pointcloud.launch.py uses pcl.yaml. I will try to modify the stereo parameters.

However, as I had issues with the depth, I tried to recalibrate the camera again. I have saved the default calibration parameters in a default_calibration.json file. But, currently the camera is running with the new calibration parameters(above results were similar with default parameters).

Is there a way to reset the calibration again to factory calibrated parameters now? So that I can try the above things with the default parameters. I found out an example in depthaipython 'calibration_factory_reset.py'. Should I try this to reset to the default parameters?

jakaskerl

AnimeshSureshGunaki
I'd use luxonis/depthai-pythonblob/main/examples/calibration/calibration_flash.py and flash the default_calibration.json back to the device.

Thanks,
Jaka

AnimeshSureshGunaki

Hi,

I have set the default calibration parameters for the camera. But, the point cloud that I get was still not accurate.

I set the following parameters in pcl.yaml:

/oak:

ros__parameters:

camera:

i_nn_type: none

i_enable_ir: true

i_laser_dot_brightness: 800

stereo:

i_align_depth: true

i_board_socket_id: 2

i_subpixel: true

i_publish_right_rect: true

i_enable_disparity_shift: true

i_enable_distortion_correction: false

i_enable_spatial_filter: true

i_enable_brightness_filter: true

i_enable_speckle_filter: true

i_enable_temporal_filter: true

i_stereo_conf_threshold: 240

But, still I was not getting the expected results as you can see in the images below:

As you can see in the images, the planes of the box look significantly warped. The surfaces should appear straight but, they are not.

I also tried the example:

epipolar_test_online.py from depthai-calibration and I got the following results:

With the default calibration, the epipolar error was between 20 and 21. I believe this error should ideally be below 1 for accurate stereo matching. Therefore, I suspect there may be an issue with the calibration.

It would be nice if you could let me know the next steps as soon as possible.

Regards,

Animesh

jakaskerl

AnimeshSureshGunaki i_enable_distortion_correction: false

Set to true. Should help with distorted depth.

Thanks,
Jaka

AnimeshSureshGunaki

@jakaskerl

I set i_enable_distortion_correction: true. I got the same results.

Following are the results that I got:

I think, I might also have the same issue as posted here: https://github.com/luxonis/depthai-core/issues/397

In the above issue, it was concluded that the bug is caused by hardware itself and it stated to convert the raw calibration values and then to load those meshes with with dai::node::StereoDepth::loadMeshData(). Is this the case for OAK-D Pro W PoE camera also. If yes, then could you please provide detailed steps for the solution provided here?

Also, the epipolar error that I was getting running the epipolar_test_online.py from the depthai_calibration gave me high errors as you can see in my last post.

jakaskerl

AnimeshSureshGunaki
Can you be a bit more specific when you say "still not accurate". The depth looks quite good; what do you wish to improve?

Also make sure you are running depthaiV2.25 or newer, the depth has been improved there.

Thanks,
Jaka

AnimeshSureshGunaki

jakaskerl

Hi,

Thank You! I updated the depthai version to the latest V2.28 and the point cloud looks significantly improved now.

Could you also suggest some ways to fill the holes in the point cloud where the depth data is missing? Because, for our application we need a very dense point cloud of the object of interest.

In the below example:

The plane doesn't have all the points covered. There are areas where there are no points due to missing depth information. I already tried to set the i_stereo_conf_threshold: 240 and i_enable_spatial_filter: true.

Regards,

Animesh

jakaskerl

AnimeshSureshGunaki
Check https://docs.luxonis.com/hardware/platform/depth/configuring-stereo-depth/ for improving depth.

set i_spatial_filter_hole_filling_radius to a higher value (i think 2 is default). cc @Luxonis-Adam on documenting this.

Thanks,
Jaka