multiple pipelines on the same device

ctutum

I need to build a WebRTC server and stream the video from the camera. Additionally, these cameras will be used to capture images every 1 second together with some other metadata (some machine telemetry data). So, one system will be used for live view in case of need, and the other system will always (default) be used for keeping a record of historical progress of a manufacturing system. Do we need to create two separate pipelines for these two applications to run simultaneously? Is it even possible or does it make sense? Or do we need to create a single pipeline and use this for both applications?

Thanks!

jakaskerl

Hi ctutum
You can't run multiple pipelines on the same device, which means one pipeline will need to handle everything you want. Luckily the pipelines are designed to run tasks in parallel and you also have the ability to use script nodes to alter the flow if you wish.

For stream: use a camera you wish and simply output the stream to rtc server (here is an example of WebRTC stream with depthai).

For keeping a record: Use the rgb camera's still output to send frames to the server every second. This can be done by utilizing the script node, sending a "take a picture" signal every second. Example here.

Hope this helps you.

Thanks,
Jaka

ctutum

jakaskerl Thanks for the help! I will look into those resources closer and let you know.

Cem

ctutum

HI @jakaskerl ,

I tried to run the example (depthai-experiments/gen2-webrtc-streaming/main.py) you referenced previously. I first tried in Spyder IDE (conda), but it gave this error:

RuntimeError: Cannot run the event loop while another loop is running

I debugged it and it was being generated after running aiohttp.web.runapp(app, accesslog=None, port=8080)

Then I tried running the main.py in a terminal and got the this output (seemed to work):

======== Running on http://0.0.0.0:8080 ========

(Press CTRL+C to quit)

I opened the URL: http://0.0.0.0:8080/ and then pressed Start button at the upper left corner, then received the following error (attached image). I am using Firefox and I can see that it also gives me "Connection not secure" error next to URL. But, the error window shows an additional message about formatting. Have you experienced this?

Moreover, where do I input the IP of the PoE camera I am using? Does it detect the devices automatically? Can we use multiple cameras?

Thanks,

Cem

ctutum

I got it working, but I don't know what changes helped!

Thanks,

Cem

ctutum

Hi @jakaskerl,

Everything is working good so far, thanks for the help!

I have another possibly dumb question (I am new to WebRTC!): I tried to connect to the server ,which has the OAK camera, from different clients, but it allowed only one client at a time. Unless I stop streaming on one client, I cannot simultaneously stream on another device (computer/phone). Am I missing something here or is there another step on the server side to allow parallel streaming on multiple clients?

Thanks,

Cem

jakaskerl

Hi ctutum
I don't think this is an issue of WebRTC, but depthai and the way the app logic is structured.
When you click on the "start" button, the server sends a createPipeline trigger to the OAK device to boot it up and load the pipeline. Since with dai.Device() won't be able to connect twice, you will get an XLINK error.

Once you close the stream using the "stop" button, the OAK is disconnected and can be connected to again from elswhere.

You would essentially need to modifiy the script to run the device constantly and only serve the stream when clients are connected.

Thanks,
Jaka

ctutum

Thanks Jaka @jakaskerl, for the quick and clear explanation! This is very helpful!

Cem

ctutum

Hi @jakaskerl,

Is there any advantage of using WebRTC vs. RTSP? DepthAI has both examples, but I was wondering if it is worth to go deeper in one of these routes?

Thanks,

Cem

jakaskerl

ctutum
Depends on your usecase.

GPT:

WebRTC (Web Real-Time Communication) and RTSP (Real-Time Streaming Protocol) are both protocols used for streaming audio and video data over the internet, but they serve different purposes and have different characteristics. Here are some advantages of each:

Advantages of WebRTC:

Real-Time Communication: WebRTC is designed for real-time communication with low latency, which is ideal for interactive applications like video conferencing, live gaming, and real-time control systems.
Peer-to-Peer: WebRTC can establish peer-to-peer connections, which can reduce server bandwidth and potentially decrease latency since the data doesn't need to go through a central server.
Encryption: WebRTC mandates encryption (SRTP for media streams), making it a secure choice for sensitive applications.
NAT Traversal: WebRTC includes mechanisms like STUN and TURN to handle NAT traversal, which can be a significant advantage in environments where clients are behind NATs or firewalls.
Browser Support: WebRTC is supported by most modern web browsers, allowing for easy integration into web applications without the need for additional plugins or software.

Advantages of RTSP:

Standard Protocol for Surveillance: RTSP is widely used in surveillance and IP camera systems, making it a good choice if you are integrating with existing surveillance systems.
Control Over the Stream: RTSP allows for control over the media stream, such as pause, play, and rewind, which can be useful for applications that require on-demand access to the stream.
Compatibility with Traditional Media Players: RTSP streams can be consumed by a variety of traditional media players and software, which can make it easier to integrate with existing systems.
Streaming from Files: RTSP is well-suited for streaming from stored media files, not just live feeds, which can be an advantage in certain applications.
Use with RTP/RTCP: RTSP is often used in conjunction with RTP (Real-Time Protocol) and RTCP (Real-Time Control Protocol) for media delivery and control, providing a complete solution for streaming needs.

Choosing Between WebRTC and RTSP:

The choice between WebRTC and RTSP will depend on the specific requirements of your application:

If you need real-time, interactive communication with low latency, or if you are developing a web application that requires browser support without plugins, WebRTC is likely the better choice.
If you are working with surveillance systems, need precise control over the stream, or are integrating with traditional media players, RTSP might be more suitable.

In the context of DepthAI, which is a platform for running real-time spatial AI on edge devices, the decision might come down to whether you need the lowest possible latency and browser integration (WebRTC) or if you are integrating with systems that traditionally use RTSP. If the application is sensitive to latency and requires real-time interaction, WebRTC would be worth exploring further. If the application is more about streaming video feeds that might be recorded or controlled remotely, RTSP could be more appropriate.