RGB RGBD RGBStereo performane issue

lubkwo

im experiencing a performance issue when configuring streams on the OAK D PRO W POE

When i set rgb to compressed image i get a stable 24 Mhz iage which matches my configuration however when i switch to RGBD or RGBStereo the fps drops to around ~18, and im unable to achieve higher values regardless of the settings
i would like to get rgb with left and right images at the highest possible FPS

could you please advice what might causing the fps drops in RGBD / RGBstereo modes?
what configuration of pipeline shoud i use to achieve wanted performance

Thanks in advance for your help

ps. using ros2 depthai v3

OskarSonc

Hey lubkwo - do you mean 24fps? If that is the case then what you see seems to be normal on PoE.

A compressed RGB stream is the lightest option, so it will usually run faster. RGBD is heavier because it adds stereo depth and RGB/depth sync-alignment, and in depthai_ros_driver that can add extra host-side work as well. RGBStereo is also usually slower than RGB-only if left/right are published raw.

If you want the highest FPS for rgb + left + right, I would avoid RGBD and use RGBStereo or just publish rgb, left, and right. Also enable low-bandwidth/compressed transport for all image topics, not only RGB. If depth is needed, try disabling subpixel and lr_check first, since both can reduce throughput.

Thanks,
Oskar

lubkwo

OskarSonc

in my case rgb left and right are already published as compressed streams im not sending raw images. thats why im suprised by the drops

could this still be caused by stereo processing on device even if i only publish compressed left/right/rgb without depth output enabled?

OskarSonc

lubkwo - Yes, if a stereo/depth node is still part of the pipeline.

If you are really using RGBStereo with only compressed rgb + left + right, then StereoDepth should not be the main reason for the drop. In that case I would look more at sync, hidden extra outputs, host-side processing, or PoE bandwidth.

But if RGBD, i_enable_rgbd, depth, rectified, aligned, or pointcloud is still enabled anywhere, then stereo processing can still lower FPS even if you are not publishing the depth topic.

Can you share code?

Thanks,
Oskar

lubkwo

Yes of course
/oakd_front:

ros__parameters:

driver:

i_device_id:

i_restart_on_diagnostics_error: true

i_usb_speed: SUPER_PLUS

diagnostic_updater:

period: 1.0

pipeline_gen:

i_pipeline_type: RGBStereo

i_nn_type: none

i_enable_imu: false

rgb:

i_fps: 30.0

i_resolution: 800P

i_height: 800

i_width: 1280

i_low_bandwidth: true

i_publish_topic: true

i_publish_compressed: true

i_max_q_size: 1

i_jpeg_quality: 70

left:

i_fps: 30.0

i_resolution: 800P

i_height: 800

i_width: 1280

i_low_bandwidth: true

i_jpeg_quality: 70

i_publish_topic: true

i_publish_compressed: true

right:

i_fps: 30.0

i_resolution: 800P

i_height: 800

i_width: 1280

i_low_bandwidth: true

i_jpeg_quality: 70

i_publish_topic: true

i_publish_compressed: true

OskarSonc

lubkwo - with that config, RGBStereo should not be creating a StereoDepth node, so I would not expect the drop to come from stereo processing itself.

In this case the more likely cause is the transport / host side.
You are still sending three 1280x800 @ 30 FPS compressed streams over PoE, which is a fairly heavy setup even without depth.

I would test the same config with one stream disabled, then with lower resolution or lower JPEG quality, just to confirm where the limit is.

Thanks,
Oskar

lubkwo

OskarSonc mjpeg quality tested from 50 to 70 range did nothing

i dont think that three streams are to much for this camera because i manage to do it in python ;////

edit: mjpeg 35 still nothing

OskarSonc

lubkwo -with your YAML, RGBStereo in depthai_ros_driver should only create rgb + left + right, not StereoDepth. If the same 3-stream setup works in Python, the difference is probably the ROS path itself.
3 encoded streams, 3 ROS image publishers, 3 camera_info publishers, and host-side conversion/message handling.
Also, i_usb_speed does nothing on a PoE device.

lubkwo

OskarSonc Is there anything that i can do?

OskarSonc

lubkwo - I’d try removing i_usb_speedsince it does nothing on PoE, increase i_max_q_sizefrom 1 to at least 4 or 8, and check whether you are also loading camera_info, RViz, republishers, or any extra subscribers. I would also try H264/H265 instead of MJPEG, because three MJPEGstreams can still be expensive in the ROS publish/convert path even when the device itself is fine.

lubkwo

custom node resolve all problems 😉 dont know why driver dont work properly

OskarSonc

lubkwo - intersting, can you share custom node setup?