OAK4-D latency and choking

martinakos

I got a new OAK-D 4 camera. I installed oak-viewer and played with it a bit. One thing I noticed is the high latency in the point cloud view — on the order of 1 s. Also, all the streams, including the H.264 stream, seem to freeze for a fraction of a second every 6 s.

I wonder whether this is something related to my system (Ubuntu 24.04), or if this is normal for the oak-viewer. I’ve checked https://docs.luxonis.com/software/depthai/optimizing , but unfortunately the diagnostic scripts (e.g. oak_latency_test.py) are for the V2 API and all crash. I guess that page needs updating.

How can I diagnose the issue, or is this the expected behaviour?

Oskar-Luxonis

Hey @martin181818 - this is not expected, especially the freezing ... Are you using the latest version of oak-viewer and is your camera updated to the latest software?
Also do you have proper 1GbE PoE+ switch/injector and known-good cables?
Thanks,
Oskar

martinakos

I followed this tutorial . I downloaded and installed the .deb package. So it's version 1.6.3
I use a PoE injector that supports up to 36W. I'm connecting the ethernet cable directly to my laptop, instead of a switch. I had to setup the Address: 169.254.10.112 Netmask: 255.255.0.0 as described in the No DHCP section for it to work. I'll double check the cables but I think they are 1GbE PoE. We've used these cables for other PoE cameras before without problems.

martinakos

I didn't update the camera software. is just using the factory version. I'll check that.

One curious thing is that I've also just tested a OAK-D lite with the same oak-viewer 1.6.3 , this time with a USB3 connection, and it also showed this intermittent freezing, and a high latency in the pointcloud view. I thought maybe that was related with the USB3 data cable not supporting enough bandwidth, despite I've used that same cable for other USB3 data devices without problem. Observing the same freezing/latency with the OAK-D 4 that uses PoE is suspicious.

Oskar-Luxonis

martinakos - thanks for the info. That is suspicious, could you try running some of our premade DepthAi examples to see if this is maybe oak-viewer thing?
You can find them here:
https://docs.luxonis.com/software-v3/depthai/examples/

Thanks,
Oskar

martinakos

I tried to update the device software but I hit another problem.

oakctl device update

Device Info:

OS: Luxonis OS RVC4 1.20.5

Architecture: linux/arm64

Model: Luxonis, Inc. OAK4-D R9

Serial Number: 3280103544

Platform: rvc4

Connection: 169.254.59.73

Is this the device you want to update? Yes

Do you want to update to the latest firmware? Yes

✔ Starting firmware update... ✔ Update process initiated. ✔ Device is back online. ✖ Failed to update. Reason: Failed to download update package. Reason: error sending request for url (https://luxonisos-rvc4.fra1.digitaloceanspaces.com/OTA/full_update_luxonis_ext4-1.23.0.zip) Device Info:

OS: Luxonis OS RVC4 1.20.5

Architecture: linux/arm64

Model: Luxonis, Inc. OAK4-D R9

Serial Number: 3280103544

Platform: rvc4

I can, however, download the zip file from a web browser

Oskar-Luxonis

Hey martinakos - so for running oakctl device update, your oak needs access to the internet... I would suggest switching to PoE+ powered switch with internet access also you will need that if you want to build apps.
You can, however update the device from host also using SSH or ADB. You can find how to here, under Update From Device option 2:
https://docs.luxonis.com/software-v3/sw-stack/luxonis-os/

SamiUddin231996

Oskar-Luxonis

720p@120FPS when accessing camera through oakctl interface….

But luxonis claimed in their blog that 720@480 FPS is unlocked now…

But when I specify even 130 FPS in depthai api. then I get errors below why?

OS is latest is what I think because I updated when connected to Hub 2 days ago. Best,

Oskar-Luxonis

Hey SamiUddin231996 - 480 FPS is currently supported only through the HFR mode, which uses fixed configurations: 1280×720 @ 480 FPS or 1920×1080 @ 240 FPS. Arbitrary FPS values such as 130 are not supported in that mode.

Could you try the dedicated HFR example instead of changing the FPS in the existing pipeline? It will also check that the device advertises the required IMX586 HFR mode:

https://docs.luxonis.com/software-v3/depthai/examples/camera/camera_hfr/

The firmware crash in your screenshot is not expected. If the HFR example fails as well, please share the output of oakctl device info and the crash dump/log.

Thanks,
Oskar

SamiUddin231996

Oskar-Luxonis

We are getting yolo26-nano pose, yolo26-nano detection, media-pipe palm detection and Main 720p@120 and every model is performing at 120FPS on EDGE MODE….. Final Product data/analytics from these results, we need to publish 10 sec videos, 1 minute videos, analytics.json, images.mjpeg

Can you suggest any network, database, how should we publish this to our dashboard… we only want to save event videos… not full video…. How should we go about that!?

Oskar-Luxonis

Hey SamiUddin231996 - I’d suggest keeping inference and event recording separate: encode one H.264/H.265 stream on the device, retain only short rolling segments locally, and preserve/upload a clip only when your event logic triggers. This also lets you include a few seconds before the event.

For the dashboard, send small event payloads (device ID, timestamp, detections, confidence, tracking ID, clip/thumbnail URL) over MQTT or HTTPS, store metadata in PostgreSQL, and store videos/images in object storage such as S3. I would use MJPEG only for a live preview, not for archived clips.

Useful examples:

Thanks,
Oskar