• ROS

  • Pointcloud messages getting dropped and improving Depth Accuracy

Hello,

I am working on an application that requires precise depth measurements at distances of 70 cm and beyond while simultaneously capturing high-resolution RGB images to read text as small as 5mm from that distance. To achieve this, I am using the OAK-D Pro with PoE and running the DepthAI ROS Humble wrapper with the following parameters:

Here is the set of parameters I would like to use for my application:

/oak:
  ros__parameters:
    camera:        
      i_pipeline_type: rgbd
      i_nn_type: none
      i_floodlight_brightness: 1000  # 1500 is max
      i_laser_dot_brightness: 1200  # 1200 is max
    rgb:
      i_fps: 10.0                                                                                                                                                                  
      i_resolution: '4K' #'720P' #'4K'       
      i_set_isp_scale: false

    stereo:
      i_width: 1280 #512 1280
      i_height: 720 #288 720
      # i_resolution: '720P'
      i_extended_disp: true
      i_subpixel: false
      i_depth_preset: HIGH_DENSITY
      i_enable_decimation_filter: true
      i_enable_temporal_filter: false
      i_enable_speckle_filter: true
      i_enable_spatial_filter: true
      i_lr_check: true

    left:
      i_publish_topic: true

When I launch the camera.launch.py file with these parameters + pointcloud.enable:=true

the /oak/points topic gets published but all messages are lost. How can I get a more reliable pointcloud publisher with this configuration?

Challenges:

  1. Depth Accuracy at 70cm+ - Despite using HIGH_DENSITY depth preset and enabling i_lr_check, the depth values seem to have inconsistencies at the required range. This is an environment with lots of light so guidance on which parameters would help improve depth accuracy would be great.

  2. PointCloud Message Loss Due to High RGB Resolution (maybe) - I am experiencing issues where pointcloud messages are being lost, which appears to be correlated with running high-resolution RGB alongside depth processing. Even when using best_effort Quality of Service (QoS), the pointcloud messages are dropped. Are there known bandwidth limitations or synchronization issues that could be causing this? Would reducing the RGB resolution or adjusting processing priorities help stabilize pointcloud publishing?

  3. Optimizing RGB and Depth Pipeline - Since my application requires both depth accuracy and the ability to read text at high resolution, I am looking for the best balance between depth quality and RGB clarity. Are there additional settings or alternative configurations I should consider?

Any insights from the community or recommendations from those who have tackled similar issues would be greatly appreciated.

    Hi

    1. You could try out some of the parameters from the default presets that are listed here
      Using one of those presets themselves or just enabling subpixel disparity will also help, although it will increase the bandwidth of the camera.
    2. You could try scaling the RGB message a little bit and use low_bandwidth mode which basically uses VideoEncoder node to compress the messages incoming from the host. Syncing frames can also help with pointcloud calculation https://docs.luxonis.com/software/ros/depthai-ros/driver/#DepthAI%20ROS%20Driver-Syncing
    3. If you are using PoE Camera bandwidth is very limited compared to USB variant, you can read more on that here https://docs.luxonis.com/software/depthai/optimizing.