Error installing PyGObject in window for running RTSP streaming.
@nikul
You are encoding into H265 (not MJPEG) - and in http response you are writing jpgboundary and in header you have specified image/jpeg
.
This will, of course, not work, and I don't think you can just stream raw h264 bitstream to a browser and it will decode/display it. You would likely need to containerize it (eg mp4), so html5's video player could play the stream. The why
is that h264/h265 is much more complex than mjpeg. Thoughts?
Thanks, Erik
- Edited
erik
Thank you.
I got the RTSP streaming link however it is throwing error and i don't know how to debug this.
Further, can I flash RTSP experiment/demo in to the device so it can stream it on stand alone. if yes how.
Here is the link of experiment
luxonis/depthai-experimentstree/master/gen2-rtsp-streaming
Would really appreciate you expertise in this matter.
Thank you.
- Edited
Hi @nikul ,
Here's step by step tutorial:
- Install WLS 2
- RUn commands
sudo apt-get update sudo apt-get upgrade sudo apt-get install ffmpeg gstreamer-1.0 gir1.2-gst-rtsp-server-1.0 libgirepository1.0-dev gstreamer1.0-plugins-bad gstreamer1.0-plugins-good gstreamer1.0-plugins-base
- Install python requirements
opencv-python numpy depthai==2.24 PyGObject
- I went with standalone approach, so flashing TCP server streaming H265 bitstream to clients that connect to it. You can run this flashing script from Windows (no need for WSL). Note that you need latest bootlaoder flashed on device before flashing this script to the OAK POE camera:
import depthai as dai # Start defining a pipeline pipeline = dai.Pipeline() camRgb = pipeline.createColorCamera() camRgb.setIspScale(2,3) videoEnc = pipeline.create(dai.node.VideoEncoder) videoEnc.setDefaultProfilePreset(30, dai.VideoEncoderProperties.Profile.H265_MAIN) camRgb.video.link(videoEnc.input) script = pipeline.create(dai.node.Script) script.setProcessor(dai.ProcessorType.LEON_CSS) videoEnc.bitstream.link(script.inputs['frame']) script.inputs['frame'].setBlocking(False) script.inputs['frame'].setQueueSize(1) script.setScript(""" import socket import time server = socket.socket(socket.AF_INET, socket.SOCK_STREAM) server.bind(("0.0.0.0", 5000)) server.listen() node.warn("Server up") while True: conn, client = server.accept() node.warn(f"Connected to client IP: {client}") try: while True: pck = node.io["frame"].get() data = pck.getData() ts = pck.getTimestamp() header = f"ABCDE " + str(ts.total_seconds()).ljust(18) + str(len(data)).ljust(8) # node.warn(f'>{header}<') conn.send(bytes(header, encoding='ascii')) conn.send(data) except Exception as e: node.warn("Client disconnected") """) (f, bl) = dai.DeviceBootloader.getFirstAvailableDevice() bootloader = dai.DeviceBootloader(bl) progress = lambda p : print(f'Flashing progress: {p*100:.1f}%') bootloader.flash(progress, pipeline)
- Find IP of your camera that's running the TCP server (I just used device manager), then run this script within the WLS2:
#!/usr/bin/env python3 import threading import gi gi.require_version('Gst', '1.0') gi.require_version('GstRtspServer', '1.0') from gi.repository import Gst, GstRtspServer, GLib import socket import re import cv2 import numpy as np # Enter your own IP! OAK_IP = "192.168.0.102" class RtspSystem(GstRtspServer.RTSPMediaFactory): def __init__(self, **properties): super(RtspSystem, self).__init__(**properties) self.data = None self.launch_string = 'appsrc name=source is-live=true block=true format=GST_FORMAT_TIME ! h265parse ! rtph265pay name=pay0 config-interval=1 name=pay0 pt=96' def send_data(self, data): self.data = data def start(self): t = threading.Thread(target=self._thread_rtsp) t.start() def _thread_rtsp(self): loop = GLib.MainLoop() loop.run() def on_need_data(self, src, length): if self.data is not None: retval = src.emit('push-buffer', Gst.Buffer.new_wrapped(self.data.tobytes())) if retval != Gst.FlowReturn.OK: print(retval) def do_create_element(self, url): return Gst.parse_launch(self.launch_string) def do_configure(self, rtsp_media): self.number_frames = 0 appsrc = rtsp_media.get_element().get_child_by_name('source') appsrc.connect('need-data', self.on_need_data) class RTSPServer(GstRtspServer.RTSPServer): def __init__(self, **properties): super(RTSPServer, self).__init__(**properties) self.rtsp = RtspSystem() self.rtsp.set_shared(True) self.get_mount_points().add_factory("/preview", self.rtsp) self.attach(None) Gst.init(None) self.rtsp.start() def send_data(self, data): self.rtsp.send_data(data) def get_frame(socket, size): bytes = socket.recv(4096) while True: read = 4096 if size-len(bytes) < read: read = size-len(bytes) bytes += socket.recv(read) if size == len(bytes): return bytes if __name__ == "__main__": server = RTSPServer() sock = socket.socket() sock.connect((OAK_IP, 5000)) try: while True: header = str(sock.recv(32), encoding="ascii") chunks = re.split(' +', header) if chunks[0] == "ABCDE": # print(f">{header}<") ts = float(chunks[1]) imgSize = int(chunks[2]) img = get_frame(sock, imgSize) buf = np.frombuffer(img, dtype=np.byte) server.send_data(buf) # print(buf.shape, buf.size) if cv2.waitKey(1) == ord('q'): break except Exception as e: print("Error:", e) sock.close()
- Find IP of your WLS2 (ifconfig) which is now running RTSP server
- ???
7.1 Profit! - I used RTSP PLayer on Windows to verify that my RTSP (on WLS2) works as expected. Img below.
Also if you are going to use ffplay to read from RTSP server, please see flags below to get lowest latency (as my RTSP player has about 1sec latency):
https://discuss.luxonis.com/d/3516-oak-1-poe-low-latency-streaming/4
- Edited
erik
Thank you very much for tutorial it clears lot of concepts for me,
Its streaming : )
:
I am getting following error in the terminal sometimes.
Error: negative buffersize in recv
I have one more question Is it possible to flash everything into the device in way that it connects to the TCP server running on the camera, receives the H265 encoded video frames, and then streams them via an RTSP server. With host as device itself.
for instance: rtsp:://192.168.0.102:8554/preview (Considering sample OAK_IP you have mention in tutorial)
erik
Hi @erik ,
I am getting error while Flashing TCP on to the device.
It is able to gain access of the device however when it trying to flash, it causing error occurs.
same with device manager when I search in device manger I Have all of them available but when i click on it, device not found.
Thank you
Hi @nikul ,
can you ping the device - is it actually accessible? I'd suggest checking POE troubleshooting docs:
https://docs.luxonis.com/projects/hardware/en/latest/pages/guides/getting-started-with-poe/#poe-troubleshooting
erik
Thank you.
I do able to ping all the devices connected to the LAN.
I specified the IP and remove application through GUI. and it worked temporary. i will look more into it.
I have moved everything to lunux. for some reasons, streaming script is not working. By any chance will you be having idea if I may have missed some settings or anything else you want me to check.
Thank you.
My best guess is that the device doesn't have the TCP server running on it.
@jakaskerl can you help nikul debug this? My guess is the incorrect IP (mixing between host IP and WSL2 IP), or failing to set firewall for that port (if accessing from different machine).
nikul I believe IT has TCP server running since can access streaming through WSL but not from linux.
Could you once again try on WSL? Since you have stated:
I specified the IP and remove application through GUI. and it worked temporary. i will look more into it.
Perhaps there is no TCP stream running on the device anymore. Also make sure the device IP didn't change. If it did, you need to change the OAK_IP inside the RTSP server script.
Maybe check the firewall as well: sudo ufw status verbose
. And Gstreamer: gst-launch-1.0 videotestsrc ! autovideosink
Let me know how it goes.
Thanks,
Jaka