Oak-1 POE keeps crashing with no crash dump

mhinckley

I have an Oak-1 POE camera that keeps crashing but doesn't generate any crash dumps. I have added system logging into the script node to try and see what could be causing it, but everything seems to be running fine up until the camera just crashes/reboots. From the logging there isn't any thread growth, memory usage growth, and CPU usage is nowhere near 100% as it is only running at 5fps. Most of the time it will recover on its own but sometimes it will fail to recover and has to be power cycled. I have also tried having Claude Sonnet 4.5 go through and improve the script node with better exception handling, tracking, garbage collecting, etc. while referencing the depthai docs to try and ensure the script is as stable as possible but it that hasn't helped. The camera is running bootloader v0.0.28 and I have tried flashing it with depthai v2.30 and 2.31 with no noticeable difference.

The script basically runs a yolo11 model and provides the original image, detection results, and a health check response to 3 end points on a self-hosted http server. For logging/debugging purposes it also has a syslogger node that passes the temperature and memory stats to the script node which sends the data to a separate script I have running on my local machine so that I can get the system stats with it running in standalone mode. From the network side it is negotiating 15W on the switch with full duplex 1Gbps and the cable run is well under 100m. Attached is the script I am using to create the pipeline and flash the camera.

single-cam-yololivestream.bin

#!/usr/bin/env python3
"""
Standalone YOLO Inference with HTTP Server - Continuous Livestream Version
Runs YOLO inference on DepthAI device and hosts:
- Continuous camera image livestream (no frame limit)

Error previewing file. It may have been deleted, or the provided file ID is invalid.

OskarSonc

Hey @mhinckley - This kind of reboot with no crash dump usually points to a watchdog/power reset rather than a classic firmware crash. In your case the problem I see is running a threaded http.server + long-lived MJPEG streaming inside the Script node on a PoE unit (especially on LEON_CSS, which also handles networking), which can wedge under backpressure and trigger a reset without leaving a dump. The quickest isolation is to run the same pipeline with the HTTP server removed (or with zero /img clients) for a few hours and if it becomes stable, move the HTTP server off-device and just forward frames/detections to a host/edge box. Also increase SystemLogger way up (e.g., 1–2 Hz) during debugging and, if possible, check switch port logs or try an alternate power path to rule out PoE brownouts.

Thanks,
Oskar

mhinckley

Oskar,

Thanks for the feedback. Claude attempted to put some watchdog kicks in the script to try and keep everything running. Sounds like maybe that isn't effective? I have achieved 8-10 hours runtime with the current script but it isn't consistent. My use-case is running the camera in standalone mode as an edge device. I want to being able to remotely connect to the live feed to grab frames for further model training, be able to send a request to the camera and get the latest NN inference results, and ideally have the camera ping a health check server every so often to have visibility of if it goes down for some reason. If long running http servers to host the result and live feed cause watchdog timeouts, what is the recommended way to achieve my usecase with just a camera in standalone mode? I've looked at the switch logs and have never seen a brownout on camera port, it just looks like a link flap where the camera still has power but just drops off the network from it rebooting itself.

OskarSonc

Watchdog inside Script often won’t save you here, because when the Script/network stack wedges (eg. blocked socket writes, too many threads), your kick code doesn’t run and the device reboots without a crash dump.

For a stable standalone pattern on OAK-1 PoE, avoid a long-lived, threaded MJPEG HTTP stream (ThreadingMixIn + keep-alive) in Script. Instead, use short-lived endpoints like /frame (return one JPEG and close) and /detections (return JSON and close), and push health/telemetry outward (UDP heartbeat or MQTT publish every N seconds). If you need a “live” view, start from the official PoE streaming demos and adapt from there rather than running a full webserver on LEON.

mhinckley

OskarSonc

Thank you! I'll look into the UDP heartbeat. The max thread count I saw was 6, not sure what the camera's max is. On a previous iteration it looked like 10 but I don't know if that is really the case. I'll check y'alls examples again. I originally started with the MJPEG server script example y'all that used a "serve forever" http server to host the feed. From the logs it was sending it looked like it was sending kicks up to the crash with kick logs well under the watchdog timeouts I set in the pipeline but I'll check that again too.