Hi Everyone, I am currently experimenting and exploring the OAK-FFC-4P board. Below is my setup: ## **Hardware** Host: Jetson Orin Nano (8GB) Board: OAK-FFC-4P Cameras: * 2 x OAK-FFC OV9282 (Mono cameras -127° HFOV) connected to sockets B and C * 2 x OAK-FFC OV9782 (Color Cameras - 127° HFOV) connected to sockets A and D ## Calibration I calibrated the cameras in the stereo modes i.e. Mono cameras at B and C as a stereo pair and Color cameras at A and D as another stereo camera. ## Software I am running the cameras on **ROS 2 Humble** using the **`depthai_ros_driver`** -> https://github.com/luxonis/depthai-ros/tree/humble/depthai_ros_driver I have added a **custom pipeline** to the ROS driver to enable running cameras and algorithms based on our requirements. Below is a summary of the pipeline configurations tested and the issues observed. #### Pipeline Configuration 1 **Setup** * Stereo depth enabled using **CAM_B and CAM_C** at **720p** * **CAM_A and CAM_D** running independently (no stereo depth) at **720p** * FPS: **30** * `low_bandwidth_mode`: **enabled** * All ROS topics enabled for publishing **Observed issues** * **CAM_A and CAM_D** publish images at the expected **30 FPS** * **CAM_B, CAM_C, and the depth image topics** publish at approximately \~**18 FPS** Based on the documentation ([https://docs.luxonis.com/software/depthai-components/nodes#Nodes-Inputs and outputs-Node input](https://docs.luxonis.com/software/depthai-components/nodes#Nodes-Inputs%20and%20outputs-Node%20input)), if input blocking is disabled, then even if the stereo depth node cannot keep up with 30 FPS, **CAM_B and CAM_C should still publish their raw images at 30 FPS**, with only the depth output being rate-limited. However, this is not the observed behavior. #### Pipeline Configuration 2 **Setup** * Stereo depth enabled using **CAM_B and CAM_C** at **400p** * **CAM_A and CAM_D** running independently at **720p** * FPS: **30** * `low_bandwidth_mode`: **enabled** * All ROS topics enabled for publishing **Observed issues** * The pipeline **consistently crashes** with the following error: [upl-image-preview uuid=24d23da0-6dd9-48d5-8563-e3fb2246c321 url=https://discuss.luxonis.com/assets/files/2026-01-15/1768471718-352982-image.png alt={TEXT?}] I would appreciate guidance on: * Relevant **parameter tuning** * Any known **limitations or constraints** when running stereo depth together with multiple color cameras on the same device The goal is to reliably run the above pipelines **without frame rate degradation or crashes**.

Thank you very much for the quick response. I have compiled my observations and understanding based on an examination of the documentation and the source code. I would really appreciate it if you could help clarify and answer my questions. Apologies in advance for the long post. ## Observation In the ROS driver, you mentioned that the `left` and `right` image topics are being published from the StereoDepth processing path. However, according to the diagram in the DepthAI documentation (https://docs.luxonis.com/software-v3/depthai/depthai-components/nodes/stereo_depth/#StereoDepth-Inputs%20and%20Outputs), StereoDepth along with the depth/disparity output also publishes rectified and syncLeft and syncRight image frames. I have the following questions regarding this behavior: #### About Sync The Sync image frame outputs (`syncedLeft` and `syncedRight`) are published only when the `i_sync` parameter is enabled, which performs timestamp-based synchronization using a Sync node. **Is this understanding correct?** #### Output Queues While investigating the DepthAI ROS driver source code, I noticed that two separate output queues are created for the stereo cameras: one queue for publishing raw image topics and another queue for feeding images into the StereoDepth node. Specifically: * Raw images are published when `i_publish_topic = true` https://github.com/luxonis/depthai-ros/blob/humble/depthai_ros_driver/src/dai_nodes/sensors/stereo.cpp#L51 https://github.com/luxonis/depthai-ros/blob/humble/depthai_ros_driver/src/dai_nodes/sensors/mono.cpp#L45 * A separate queue is used to provide images as input to the StereoDepth node https://github.com/luxonis/depthai-ros/blob/humble/depthai_ros_driver/src/dai_nodes/sensors/stereo.cpp#L56 Based on this design, my understanding is that even if the StereoDepth node becomes a bottleneck, the raw camera image topics should ideally continue to be published at the full sensor rate (e.g., 30 FPS). However, this does not seem to be happening in practice. From further reading of the documentation (https://docs.luxonis.com/software/depthai-components/nodes/#Nodes-Inputs%20and%20outputs-Node%20output), it appears that this behavior may be limited by the Output message pool limitations in the driver. **Is this understanding correct?** ## Our Goal We are using an **OAK-FFC-4P** board with **4 cameras** connected, running **2 StereoDepth pipelines** (each using a stereo pair). The desired output is: * **4 raw camera image topics** at a stable **30 FPS** * **2 depth image topics** at **30 FPS** The raw camera images will also be used for **visual odometry**, so it is important that they **consistently run at 30 FPS**, regardless of the performance of the StereoDepth pipeline. To achieve this, I have the following questions: 1. How can we ensure that the **raw camera image topics from all 4 cameras always arrive at 30 FPS**, even if the StereoDepth node becomes a bottleneck? 2. Is it possible to **decouple the algorithm FPS from the camera FPS**? For example, can we run: * 4 cameras at **30 FPS**, and * 2 StereoDepth nodes at **2 FPS**? 3. If this is possible, what is the **recommended way to configure the pipeline or ROS driver** to achieve this behavior?

Hey @"adishesha"#p32802 - we don’t currently have a ready-to-run example for throttling StereoDepth specifically. The closest reference (DepthAI v2) is [script_forward_frames.py](https://docs.luxonis.com/software/depthai/examples/script_forward_frames/). The general approach is to insert a Script node before StereoDepth, count incoming frames, and only forward every Nth left/right pair into StereoDepth (dropping the rest).

OAK-FFC-4P: Stereo depth issues with 2 mono & 2 color cameras in ROS2 driver

adishesha

Hi Everyone,

I am currently experimenting and exploring the OAK-FFC-4P board. Below is my setup:

Hardware

Host: Jetson Orin Nano (8GB)

Board: OAK-FFC-4P

Cameras:

2 x OAK-FFC OV9282 (Mono cameras -127° HFOV) connected to sockets B and C
2 x OAK-FFC OV9782 (Color Cameras - 127° HFOV) connected to sockets A and D

Calibration

I calibrated the cameras in the stereo modes i.e. Mono cameras at B and C as a stereo pair and Color cameras at A and D as another stereo camera.

Software

I am running the cameras on ROS 2 Humble using the depthai_ros_driver -> luxonis/depthai-rostree/humble/depthai_ros_driver I have added a custom pipeline to the ROS driver to enable running cameras and algorithms based on our requirements. Below is a summary of the pipeline configurations tested and the issues observed.

Pipeline Configuration 1

Setup

Stereo depth enabled using CAM_B and CAM_C at 720p
CAM_A and CAM_D running independently (no stereo depth) at 720p
FPS: 30
low_bandwidth_mode: enabled
All ROS topics enabled for publishing

Observed issues

CAM_A and CAM_D publish images at the expected 30 FPS
CAM_B, CAM_C, and the depth image topics publish at approximately ~18 FPS

Based on the documentation
(https://docs.luxonis.com/software/depthai-components/nodes#Nodes-Inputs and outputs-Node input),
if input blocking is disabled, then even if the stereo depth node cannot keep up with 30 FPS, CAM_B and CAM_C should still publish their raw images at 30 FPS, with only the depth output being rate-limited. However, this is not the observed behavior.

Pipeline Configuration 2

Setup

Stereo depth enabled using CAM_B and CAM_C at 400p
CAM_A and CAM_D running independently at 720p
FPS: 30
low_bandwidth_mode: enabled
All ROS topics enabled for publishing

Observed issues

The pipeline consistently crashes with the following error:

I would appreciate guidance on:

Relevant parameter tuning
Any known limitations or constraints when running stereo depth together with multiple color cameras on the same device

The goal is to reliably run the above pipelines without frame rate degradation or crashes.

OskarSonc

Hey adishesha - thanks for the detailed setup, what you’re seeing is usually expected when StereoDepth becomes the bottleneck. In the ROS driver the “left/right” topics are often coming from the StereoDepth path (rectified/synced), so they’ll run at the same FPS as depth (18 FPS) even if the cameras themselves are set to 30. The “non-blocking input” note in the docs applies to the StereoDepth input queue, but upstream camera streams can still get throttled if any downstream queue/sync/publisher path blocks.

For the crash (sys_logger_queue / X_LINK_ERROR), that typically means the device pipeline reset or the link dropped under load (too many enabled topics/queues, bandwidth, or USB/power). As a next step, try publishing only the essentials (raw + depth) and disable extra outputs like rectified/disparity/pointcloud, use small non-blocking queues, and reduce stereo load (disable LR-check/subpixel/extended and avoid aligning depth to 720p unless needed). Also try turning low_bandwidth_mode off (or only enabling it for selected streams) to see if FPS/stability improves.

Thanks,
Oskar

adishesha

Thank you very much for the quick response. I have compiled my observations and understanding based on an examination of the documentation and the source code. I would really appreciate it if you could help clarify and answer my questions. Apologies in advance for the long post.

Observation

In the ROS driver, you mentioned that the left and right image topics are being published from the StereoDepth processing path. However, according to the diagram in the DepthAI documentation (https://docs.luxonis.com/software-v3/depthai/depthai-components/nodes/stereo_depth/#StereoDepth-Inputs%20and%20Outputs), StereoDepth along with the depth/disparity output also publishes rectified and syncLeft and syncRight image frames. I have the following questions regarding this behavior:

About Sync

The Sync image frame outputs (syncedLeft and syncedRight) are published only when the i_sync parameter is enabled, which performs timestamp-based synchronization using a Sync node. Is this understanding correct?

Output Queues

While investigating the DepthAI ROS driver source code, I noticed that two separate output queues are created for the stereo cameras: one queue for publishing raw image topics and another queue for feeding images into the StereoDepth node. Specifically:

Raw images are published when i_publish_topic = true

luxonis/depthai-rosblob/humble/depthai_ros_driver/src/dai_nodes/sensors/stereo.cpp#L51

luxonis/depthai-rosblob/humble/depthai_ros_driver/src/dai_nodes/sensors/mono.cpp#L45
A separate queue is used to provide images as input to the StereoDepth node
luxonis/depthai-rosblob/humble/depthai_ros_driver/src/dai_nodes/sensors/stereo.cpp#L56

Based on this design, my understanding is that even if the StereoDepth node becomes a bottleneck, the raw camera image topics should ideally continue to be published at the full sensor rate (e.g., 30 FPS). However, this does not seem to be happening in practice. From further reading of the documentation (https://docs.luxonis.com/software/depthai-components/nodes/#Nodes-Inputs%20and%20outputs-Node%20output), it appears that this behavior may be limited by the Output message pool limitations in the driver. Is this understanding correct?

Our Goal

We are using an OAK-FFC-4P board with 4 cameras connected, running 2 StereoDepth pipelines (each using a stereo pair). The desired output is:

4 raw camera image topics at a stable 30 FPS
2 depth image topics at 30 FPS

The raw camera images will also be used for visual odometry, so it is important that they consistently run at 30 FPS, regardless of the performance of the StereoDepth pipeline.

To achieve this, I have the following questions:

How can we ensure that the raw camera image topics from all 4 cameras always arrive at 30 FPS, even if the StereoDepth node becomes a bottleneck?
Is it possible to decouple the algorithm FPS from the camera FPS?
For example, can we run:
- 4 cameras at 30 FPS, and
- 2 StereoDepth nodes at 2 FPS?
If this is possible, what is the recommended way to configure the pipeline or ROS driver to achieve this behavior?

OskarSonc

adishesha - Thanks for the detailed write-up, one key clarification is that syncedLeft/syncedRight are StereoDepth outputs (the pair used for depth) and don’t depend on the ROS driver i_sync. That parameter is a separate publish-time sync feature.

Even though the driver creates separate branches (raw publish + feeding StereoDepth), the camera node still has finite pools/queues, so if any downstream path blocks or fills (stereo inputs, XLinkOut, ROS publishing/serialization), you can get backpressure and the camera effectively throttles, which is why “raw” can drop to the depth rate.

To keep raw at 30 FPS while depth runs slower, publish raw directly from the camera outputs with small non-blocking queues, and make the StereoDepth input path small/non-blocking so it drops frames instead of stalling the cameras. If you want something like 30 FPS cameras and 2 FPS depth, throttle/drop frames before StereoDepth (only forward every Nth frame).

adishesha

Thank you for your answers. Do you have any example code showing how to throttle or drop frames before the StereoDepth node, so that only every Nth frame is forwarded?

OskarSonc

Hey adishesha - we don’t currently have a ready-to-run example for throttling StereoDepth specifically. The closest reference (DepthAI v2) is script_forward_frames.py. The general approach is to insert a Script node before StereoDepth, count incoming frames, and only forward every Nth left/right pair into StereoDepth (dropping the rest).