Intermittent OAK-4S Camera Firmware Crash (X_LINK_ERROR / MessageQueue Closed)

AkshayKumar

Hi ,

We are using OAK 4S PoE cameras in our application, with around 20 cameras connected simultaneously.

We are facing an intermittent device crash issue. It does not happen consistently, but occasionally a few cameras crash during operation while the remaining cameras continue to work normally.

We have observed the following logs when the issue occurs:

1. CameraSensor internal error during frame send:

$$
[3804810805] [192.168.200.14] [1551501.272] [Camera(0)] [error]
Exception in send_frame(): Internal error occured. Please report.
commit: | version: 0.0.1 | file: /work/src/camera/CameraSensor.cpp:82
$$

2. XLink communication failure — bitstream:

$$
[2026-06-23 14:59:16.831] [depthai] [error]
Communication exception - possible device error/misconfiguration.
Original message 'Couldn't read data from stream: '__x_1_bitstream' (X_LINK_ERROR)'
$$

3. XLink communication failure — hi-res stream:

$$
[2026-06-23 14:59:16.831] [depthai] [error]
Communication exception - possible device error/misconfiguration.
Original message 'Couldn't read data from stream: '__x_2_hires' (X_LINK_ERROR)'
$$

4. Firmware crash detected:

$$
[2026-06-23 14:59:19.818] [warning]
FW crashed - trying to get out the crash dump
$$

5. Device marked as crashed:

$$
[2026-06-23 14:59:31.930] [depthai] [error]
Device has crashed.
$$

After the crash, when attempting to reuse the camera, we sometimes observe the following state:

$$
[INFO] Device already connected
[INFO] Pipeline already running
[CRITICAL] Queue health check: MessageQueue closed
MessageQueue was closed
$$

At this point, rebuilding and reinitializing the camera pipeline restores normal operation.

We have also noticed that changing the camera resolution or FPS sometimes resolves the issue temporarily, but the same crash may recur after some time.

Could you please advise on the possible root cause of this issue? If needed, we can also provide the generated crash dump files for further analysis.

Thank you.

Oskar-Luxonis

Hey AkshayKumar - could you first update the OAK4S devices to the latest Luxonis OS, or at least share the current OS version?

oakctl self-update
oakctl device update

There have been several OAK4 camera-stack fixes in newer Luxonis OS releases, including fixes related to multi-stream camera crashes, so I’d rule out an old OS first.

The X_LINK_ERROR and MessageQueue was closed messages are likely consequences of the device/firmware crash rather than the original cause. After such a crash, rebuilding the pipeline/device object is expected.

If the issue still happens after updating, please send us the crash dumps, Luxonis OS version, oakctl version, DepthAI version, pipeline details/resolution/FPS/encoding settings, and PoE switch/power details. OAK4 requires PoE+ / 802.3at with enough per-port and total power budget, especially with many cameras running simultaneously.

Best,
Oskar

AkshayKumar

Hi @Oskar-Luxonis ,

Thank you for the quick response and the clarification! We will look into it from our end and get back to you soon.

Best regards,
Akshay Kumar

AkshayKumar

Hi @Oskar-Luxonis

I tried to update the OS for the specific camera from 1.20.5 to 1.33.

Since in production we will not have an internet connection, I wanted to verify the offline/local update process. I ran the following command:

oakctl device update -d 192.168.200.14 --local

Below are the terminal logs:

$$
Device Info:
OS: Luxonis OS RVC4 1.20.5
Agent Version: 0.17.0 (rvc4)
Architecture: linux/arm64
Model: Luxonis, Inc. OAK4-S R9
Serial Number: 3804810805
Platform: rvc4
Connection: 192.168.200.14

Is this the device you want to update? Yes
Do you want to update to the latest version? Yes

✔ OS update downloaded successfully

oakctl command failed:
0: The OS on your device doesn't support this feature.

Suggestion: Please upgrade your OS via 'oakctl device update'.
$$

The update package appears to be downloaded successfully, but the update itself fails with the message that the OS does not support this feature.

Could you please help us understand the cause of this issue? Is local/offline update not supported from OS version 1.20.5, or is there a different procedure we should follow for offline upgrades?

Thank you.

Oskar-Luxonis

Hey AkshayKumar - sorry for late answear. I see you already updated the camera. For previous error, this is expected on Luxonis OS 1.20.5. The oakctl device update --local flow was added later, with the newer oakctl device update --url/--path/--local support introduced around OS 1.31.2 / oak-agent v0.19.1.

Did you update all the devices?

I will take a look at the crash dump you send. Thanks.
Oskar

AkshayKumar

Hi @Oskar-Luxonis ,

We have updated the Luxonis OS from 1.20.5 to 1.33.0 as suggested, however we are still experiencing the same intermittent crash issue across a few cameras.

Here are our current device details:

Luxonis OS version: 1.33.0 (updated from 1.20.5)
oakctl version: 0.23.0
Agent Version: 0.17.0 (rvc4)
Device Model: Luxonis, Inc. OAK4-S R9
Serial Number: 3804810805 (and a few others)
Architecture: linux/arm64 (RVC4)
PoE: Using the official Luxonis PoE injector

Pipeline Configuration:

Resolution: 1920 x 1080 (Full HD)
FPS: 30
Preview Stream Encoding: MJPEG
Rotation: Some cameras use 180° rotation via ImageManip node

The issue does not happen on all cameras simultaneously — only a few cameras crash intermittently while the rest continue to operate normally.
We have attached the crash dump files and test script via the link below. Please note that the Python script is a simple test script written to replicate our production pipeline — it is not the production code itself, but it reproduces the same issue.

🔗 Reference files (crash dumps + test script): https://1drv.ms/f/c/6343b4b2c2e02dcd/IgAJSl757y_6Q4kI3XR67t1eAex311e7u_7DHR4KGwLOafU?e=bqGnc2

Could you please help us investigate and resolve this at the earliest? This is affecting our production environment with 20 cameras running simultaneously.

Thank you.
Best regards,
Akshay Kumar

Oskar-Luxonis

AkshayKumar Agent Version: 0.17.0 (rvc4)

This doesn't look right. Can you check this again? With Luxonis OS 1.33.0, you should have Agent version 0.22.1.
How did you update the camera?
Did you folow this guide?:
https://docs.luxonis.com/software-v3/sw-stack/luxonis-os/#Luxonis%20OS-Updating%20OS

From the dumps, this is a real on-device camera stack crash, not just a host-side queue issue. I see CAMX/camera HAL errors such as IFEFenceUnsignaled / Sensor_Running_State, followed by CameraSensor.cpp:82.

I also noticed the test script requests two full-rate camera outputs from the same sensor at 1920x1080 @ 30 FPS and then encodes one path as MJPEG. As a quick isolation test, could you also try using only one camera.requestOutput(...) and either fan that same output to both branches, or reduce the preview branch resolution/FPS? This will help confirm whether the crash is tied to the dual full-resolution stream configuration.

AkshayKumar

Hi @Oskar-Luxonis ,

We have updated 3-4 cameras to Luxonis OS 1.33.0 for testing and monitoring purposes. However, we are still experiencing the same intermittent crash issue even after the update.
Please do take a look at the crash dumps and let us know your findings. Do let us know if you need any additional information from our side.

Thank you.
Best regards,
Akshay Kumar

AkshayKumar

Hi @Oskar-Luxonis ,
Apologies for the confusion regarding the Agent version — I had accidentally shared the details from before the update. The correct current details after the update are:

Luxonis OS version: 1.33.0
Agent Version: 0.22.1 (rvc4)

We updated the cameras using the offline OTA flow via SCP + SSH as documented in the Luxonis OS update guide.
Regarding the dual full-resolution stream . We will test with a single camera.requestOutput(...) and fan it to both branches, and also try reducing the preview branch resolution/FPS to isolate the issue.
We will get back to you with the results shortly.

Thank u,
Akshay Kumar

Oskar-Luxonis

Thanks for confirming, that version combination looks correct now: Luxonis OS 1.33.0 with oak-agent 0.22.1.

For isolation, could you also run the minimal VideoEncoder example on one of the cameras that previously crashed?

luxonis/depthai-coreblob/develop/examples/python/VideoEncoder/video_encode.py

This uses a single Camera.requestOutput(...) feeding an MJPEG encoder, so it is a good baseline against the current dual-output pipeline. Please run it for roughly the same duration/load where you normally see the crash.

Best,
Oskar