Stream OAK-4S (Ethernet) with FastAPI and DepthAI 3.0.0

AkshayKumar

Hi everyone,
I’m trying to stream video from an OAK-4S camera (connected over Ethernet) using FastAPI and DepthAI 3.0.0.

What I want to do:

1.Connect to the camera using its IP address

2.Stream video to web clients using FastAPI

3.Support multiple cameras

4.Clean up resources when clients disconnect or stream stops

Problems I’m facing:

1.How to connect to a device by IP in DepthAI v3?

2.How to build a proper streaming pipeline ?

3.How to send the stream to the browser using FastAPI?

4.How to clean up everything properly when stopping or disconnecting?

5.Any advice for handling multiple cameras?

I’ve checked the v2 to v3 migration guide but still confused. If anyone has working code or tips, please help.

Thanks!

jakaskerl

AkshayKumar

AkshayKumar 1.How to connect to a device by IP in DepthAI v3?

with dai.Pipeline(defaultDevice=dai.Device(dai.DeviceInfo("IP"))) as pipeline:

AkshayKumar 2.How to build a proper streaming pipeline ?

Probably a host node (or threaded host node) is your best bet. Create a webserver using fastapi. luxonis/oak-examplestree/main/streaming/mjpeg-streaming

Thanks,
Jaka

AkshayKumar

jakaskerl

Thank you for the clear and direct answers! The code snippet for connecting to the device by IP is exactly what I was looking for, and pointing me to the mjpg streaming example is very helpful.

I'll start building the pipeline with a host node as you suggested and use that example as a reference.

jindrich-flidr

@AkshayKumar There are several ways to achieve the task you’ve described:

How to connect to a device by IP in DepthAI v3?

There are two main approaches:

Run DepthAI v3-based applications directly on your OAK4 device. You will need oakctl to build the app: https://docs.luxonis.com/software-v3/oak-apps/oakctl/ Configuration details: https://docs.luxonis.com/software-v3/oak-apps/configuration/ Example app: luxonis/oak-examplesblob/main/custom-frontend/raw-stream/oakapp.toml

DepthAI v3 can stream everything over WebSocket, and it even includes a simple frontend to render it: https://docs.luxonis.com/software-v3/depthai/tools/oak-visualizer/ If you’d like to have a custom frontend app with HTTPS (required for encoded streams), see for example: luxonis/oak-examplesblob/main/custom-frontend/open-vocabulary-object-detection/README.md Please note that this last example uses the base image, which also provides connection over WebRTC and removes access via https://hub.luxonis.com/.

Run DepthAI v3 on your PC or server. This is less effective from a bandwidth perspective, but it depends on your pipelines. When DepthAI v3 runs directly on the OAK4 device, typically only encoded streams are transferred over the network, and only when they are in use (e.g. opened in the browser).

How to build a proper streaming pipeline?

You can find an example here: luxonis/oak-examplesblob/main/custom-frontend/raw-stream/main.py#L20 And other streaming examples depending on your needs: luxonis/oak-examplestree/main/streaming

How to send the stream to the browser using FastAPI?

The simplest way is to use the WebSocket provided directly by DepthAI v3 in your frontend app. You could also create a proxy with NGINX or re-expose it via FastAPI, but in most cases it’s easiest to use the WebSocket stream directly with our frontend libraries which are used in examples.

Any advice for handling multiple cameras?

If you run an oakapp on each camera, your frontend (anywhere in your local network) will be able to connect to the WebSocket provided by DepthAI v3 using the device IP, wss://, and typically port 9000 (assign to first oakapp running on device by agent).

AkshayKumar

jindrich-flidr

Thank you so much for such a detailed and comprehensive reply. The two main approaches you outlined are extremely helpful for understanding the architecture.

The first approach of running the application directly on the OAK device sounds very promising. I think starting with oakctl to build the app and using the built-in WebSocket streaming will be the most efficient for my use case, especially regarding bandwidth. The links to the configuration and the raw-stream example are perfect.

I will begin by implementing the first approach. If I run into anything, I'll be sure to get back to you.

I really appreciate you taking the time to explain this so clearly.