Syncing the RGB camera with the stereo depth camera in OAKD pro

Robotik123

Hi there,

I am new to the community. I have been trying to enable the sync mode using the pipeline sync enable option, but that seems to fail, and throw an error:

[ERROR] [component_container-11]: process has died [pid 13578, exit code -11, cmd '/opt/ros/jazzy/lib/rclcpp_components/component_container --ros-args -r __node:=oakd_container -r __ns:=/'].

[component_container-11] [INFO] [1786535550.815637936] [oakd]: Pipeline type: RGBD

[component_container-11] [INFO] [1786535550.808528058] [oakd]: Device type: OAK-D-PRO

I try to do this first in my oakd pro yaml file:
/oakd:

ros__parameters:

camera:

i_enable_ir: true

i_floodlight_brightness: 0

i_laser_dot_brightness: 1000 #700

i_nn_type: none

i_pipeline_type: RGBD

i_usb_speed: SUPER_PLUS

i_enable_sync: true

pipeline_gen:

i_enable_sync: true

left:

i_fps: 6.0

rgb:

i_synced: true

i_get_base_device_timestamp: true

right:

i_fps: 6.0

stereo:

i_synced: true

i_get_base_device_timestamp: true

imu:

i_get_base_device_timestamp: true

use_sim_time: false

Please let me know, if i am doing something wrong here.

Oskar-Luxonis

Hey Robotik123 - the failure is happening when the driver enables its timestamp-based Sync wrapper. It should not crash the ROS container. Please first make sure your rgb FPS also matches stereo (rgb.i_fps: 6.0), since it otherwise defaults to 30 FPS.

Also, please update to the newest Jazzy ROS driver if you are not already on it: latest releases. Then retry with the same minimal RGBD + sync configuration.

This sync groups RGB and depth by timestamp; it does not hardware-trigger the sensors. The standalone DepthAI sync example may also help: RGB/depth Sync example.

If it still crashes after updating, please share the exact installed depthai_ros_driver and DepthAI versions, plus the full terminal log.

Thanks,
Oskar

Robotik123

Hi,
@Oskar-Luxonis . Thank you for your reply. I am using Depthai v2 ros driver v2.12.2-jazzy-2-g3170e87, along with the depth ai core 2.30.0-1noble.20250424.110307

I tried doing a few things and found an important factor is

rgb:
  i_publish_topic: false

When setting this to true, solves the error. But i only want cropped out rgb preview images raw images and not the raw images directly from rgb topic, to save bandwidth. Is there a work around this issue.

Adding to the issue:
I want a 640x480 image raw, the aspect ratio is 4:3, but since the original image aspect ratio 1080P is 16:9, I switched to preview rgb raw image, to get the aspect ratio. I know the following options for raw RGB image are 1080P, 4K and 12MP. I would like to have your suggestions if possible to get this 4:3 aspect ratio 640x480 rgb images from above depthai ros v2 and the depthai core version.

Please let me know if i am missing out on something. Thank you once again for supporting me.

Oskar-Luxonis

Hey Robotik123 -thanks for testing this. You are already using the latest legacy v2 Jazzy driver. The crash is a v2 driver issue: with rgb.i_synced: true, disabling rgb.i_publish_topic leaves the Sync wrapper with no RGB publisher to attach, which causes the segfault.

Unfortunately, v2 does not support synchronizing the preview output alone, so there is no parameter-only workaround for this. Keeping i_publish_topic: true avoids the crash, but the full RGB frame is still included in the synced group.

I recommend moving to the current v3 Jazzy driver, v3.2.1-jazzy. There you can configure the RGB stream itself as 640x480 with i_resize_mode: CROP, keep it synchronized with stereo, and avoid sending a separate full-resolution RGB stream. Please note that this is a v2 → v3 migration, and the ROS packages/configuration have changed; the v3 ROS docs are here.

Thanks,
Oskar

Robotik123

Hey Oskar-Luxonis, Thanks a lot for the info. unfortunately i cannot shift to depth ai v3 driver for compatibility issues, because of the ros package configuration changes. At the moment, i have to stick to depth ai v2, so a compatible image aspect ratio, is the only option i see at the moment. Thank you for supporting me.

Oskar-Luxonis

Robotik123 ah I see. Happy to help.
Best,
Oskar