How to set `sippBufferSize` in the ROS driver?

FredLabrosse

I am trying to filter the depth images produced by my OAK-D-PRO mounted on a small mobile robot to reduce the amount of spurious obstacles being detected. I found the i_depth_preset: ROBOTICS option. However, when I try it, I get:

[4.1] [2.813] [StereoDepth(3)] [critical] 'Median' out of system resources: '126'! You can increase the available memory pool by setting 'sippBufferSize' and/or 'sippDmaBufferSize' using 'setSippBufferSize' and/or 'setSippDmaBufferSize' API.

I can't find a way to set these from a launch file and/or a yaml file. Any ideas?

Cheers
Fred

Oskar-Luxonis

Hey FredLabrosse - The ROS v3 driver does not currently expose sippBufferSize or sippDmaBufferSize as launch or YAML parameters. To set them, you would need to build the driver from source and call pipeline->setSippBufferSize(...) or pipeline->setSippDmaBufferSize(...) before the pipeline starts.

For a YAML-only test, try i_depth_preset: DEFAULT, then enable the filters you need one at a time, such as i_enable_spatial_filter or i_enable_speckle_filter. ROBOTICS enables several filters, including median, so this will give different depth output.

Could you share your ROS distro, driver version, and parameter YAML? That will help us suggest the right change for your setup.

Thanks,
Oskar

FredLabrosse

I have tried all the presets and the only ones that works are FASTACCURACY and HIGHDETAIL. All the others fail in the same way (including DEFAULT), all complaining about the median filter. Interestingly, I can't see an option to enable or not a median filter.

The filter that seems to be doing the better job is the spatial one. Maybe I need to increase further the radius, and maybe try more than one pass. Speckle does not seem to be doing much, even without the spatial filter.

This is the latest depthai-v3 on ROS jazzy.

Here is my configuration:

/camera:
  ros__parameters:
    driver:
      i_publish_tf_from_calibration: false
      i_enable_ir: false
      i_tf_imu_from_descr: "true"
    pipeline_gen:
      i_pipeline_type: RGBD
      i_nn_type: none
      i_enable_imu: true
    rgb:
      i_width: 320
      i_height: 200
      i_resize_mode: STRETCH
      i_fps: 20.0
      i_synced: true # group with stereo's synced depth output via the on-device Sync node
    right:
      i_width: 320
      i_height: 200
      i_resize_mode: STRETCH
      i_fps: 20.0
    left:
      i_width: 320
      i_height: 200
      i_resize_mode: STRETCH
      i_fps: 20.0
    stereo:
      i_width: 320
      i_height: 200
      i_fps: 20.0
      i_synced: true # group with rgb's synced output via the on-device Sync node
      #i_depth_preset: ROBOTICS
      i_depth_preset: FAST_ACCURACY
      #i_depth_preset: HIGH_DETAIL
      #i_enable_decimation_filter: false
      i_enable_spatial_filter: true
      i_spatial_filter_hole_filling_radius: 15
      i_spatial_filter_alpha: 0.3
      i_spatial_filter_delta: 0
      i_spatial_filter_num_iterations: 1
      i_enable_speckle_filter: true
      i_speckle_filter_speckle_range: 50
      #i_enable_temporal_filter: false
      #i_enable_threshold_filter: false
      i_subpixel: true
      i_lr_check: true
      i_stereo_conf_threshold: 0
    imu:
      i_enable_rotation: false
    nn:
      i_nn_config_path: depthai_ros_driver/mobilenet
      i_disable_resize: true

FredLabrosse

This is a typical depth image I get. Unfortunately I did not capture the corresponding RGB, but here is a pair where I have. The noise in the one above is in the gravel part this the pair below. There just happened to be no noise in the gravel part in the pair below when I saved the image, but there is often of the same type as for the first image. This is with hole filling radius of 30 and 2 iterations.

Oskar-Luxonis

FredLabrosse - thanks for the images and configuration. I found an error in our parameter docs: the ROS v3 driver reads stereo.i_spatial_filter_iterations, not stereo.i_spatial_filter_num_iterations. Please use the driver’s name when you test two passes.

You are right about DEFAULT. It also enables the median filter, so please keep FAST_ACCURACY or HIGH_DETAIL for now.

Thanks,
Oskar

FredLabrosse

I changed the stereo.i_spatial_filter_num_iterations option to stereo.i_spatial_filter_iterations: 3 and that helped but still a lot of noise. Obviously, the higher it is the slower the images are published. Here is an example of depth and rgb showing the kind of noise (in the centre part). This creates obstacles in front of the robot all the time. Can this be reduced?

Oskar-Luxonis

FredLabrosse - Thanks for the new images. I can see short horizontal marks near the centre of the depth image. Since three spatial filter passes slow the image rate, please first compare with a hole filling radius of 2, one pass, and the default confidence threshold of 15. Then change one setting at a time.

Could you also stop the ROS driver and run the DepthAI v3 Stereo Depth example on the same camera and scene? Please match your ROS resolution, preset, and filters when you compare the results, as the example has different defaults. Let us know whether the marks also appear outside ROS.

Thanks,
Oskar

FredLabrosse

I have played a lot with the parameters. I will try what you suggest but I have more or less settled on HIGH_DETAIL rather than FAST_ACCURACY and then add filters with more or less defaults. I enable spatial filter and set the to 2 iterations and that gets better results, albeit still noisy but manageable.

Unfortunately I cannot perform the test with non-ROS programs. I am too far from mains and have no display on the robot.

Oskar-Luxonis

FredLabrosse - ah I see. Let me know if you need any more help.