• DepthAI-v2
  • Error installing PyGObject in window for running RTSP streaming.

Hi @nikul
Likely something wrong with the headers or data decoding. Also I'd lower the FPS to 10 for now.

Thanks,
Jaka

    jakaskerl
    Thank you for getting back.
    I have lower down FPS to 10. And still not able to debug the code,
    Any help in this matter will be appreciated.

    Thank you.

    Hi @nikul
    I'll have to debug this on monday since I don't have any poe device to test on currently.

    Thanks,
    Jaka

    @nikul
    You are encoding into H265 (not MJPEG) - and in http response you are writing jpgboundary and in header you have specified image/jpeg.
    This will, of course, not work, and I don't think you can just stream raw h264 bitstream to a browser and it will decode/display it. You would likely need to containerize it (eg mp4), so html5's video player could play the stream. The why is that h264/h265 is much more complex than mjpeg. Thoughts?
    Thanks, Erik

      erik
      Thank you.

      I got the RTSP streaming link however it is throwing error and i don't know how to debug this.

      Further, can I flash RTSP experiment/demo in to the device so it can stream it on stand alone. if yes how.

      Here is the link of experiment
      luxonis/depthai-experimentstree/master/gen2-rtsp-streaming

      Would really appreciate you expertise in this matter.
      Thank you.

      Hi @nikul ,
      Here's step by step tutorial:

      1. Install WLS 2
      2. RUn commands
        sudo apt-get update
        sudo apt-get upgrade
        sudo apt-get install ffmpeg gstreamer-1.0 gir1.2-gst-rtsp-server-1.0 libgirepository1.0-dev gstreamer1.0-plugins-bad gstreamer1.0-plugins-good gstreamer1.0-plugins-base
      3. Install python requirements
        opencv-python
        numpy
        depthai==2.24
        PyGObject
      4. I went with standalone approach, so flashing TCP server streaming H265 bitstream to clients that connect to it. You can run this flashing script from Windows (no need for WSL). Note that you need latest bootlaoder flashed on device before flashing this script to the OAK POE camera:
        import depthai as dai
        
        # Start defining a pipeline
        pipeline = dai.Pipeline()
        
        camRgb = pipeline.createColorCamera()
        camRgb.setIspScale(2,3)
        
        videoEnc = pipeline.create(dai.node.VideoEncoder)
        videoEnc.setDefaultProfilePreset(30, dai.VideoEncoderProperties.Profile.H265_MAIN)
        camRgb.video.link(videoEnc.input)
        
        script = pipeline.create(dai.node.Script)
        script.setProcessor(dai.ProcessorType.LEON_CSS)
        
        videoEnc.bitstream.link(script.inputs['frame'])
        script.inputs['frame'].setBlocking(False)
        script.inputs['frame'].setQueueSize(1)
        
        script.setScript("""
        import socket
        import time
        
        server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        server.bind(("0.0.0.0", 5000))
        server.listen()
        node.warn("Server up")
        
        while True:
            conn, client = server.accept()
            node.warn(f"Connected to client IP: {client}")
            try:
                while True:
                    pck = node.io["frame"].get()
                    data = pck.getData()
                    ts = pck.getTimestamp()
                    header = f"ABCDE " + str(ts.total_seconds()).ljust(18) + str(len(data)).ljust(8)
                    # node.warn(f'>{header}<')
                    conn.send(bytes(header, encoding='ascii'))
                    conn.send(data)
        
            except Exception as e:
                node.warn("Client disconnected")
        """)
        
        (f, bl) = dai.DeviceBootloader.getFirstAvailableDevice()
        bootloader = dai.DeviceBootloader(bl)
        progress = lambda p : print(f'Flashing progress: {p*100:.1f}%')
        bootloader.flash(progress, pipeline)
      5. Find IP of your camera that's running the TCP server (I just used device manager), then run this script within the WLS2:
        #!/usr/bin/env python3
        
        import threading
        
        import gi
        gi.require_version('Gst', '1.0')
        gi.require_version('GstRtspServer', '1.0')
        from gi.repository import Gst, GstRtspServer, GLib
        import socket
        import re
        import cv2
        import numpy as np
        
        # Enter your own IP!
        OAK_IP = "192.168.0.102"
        
        class RtspSystem(GstRtspServer.RTSPMediaFactory):
            def __init__(self, **properties):
                super(RtspSystem, self).__init__(**properties)
                self.data = None
                self.launch_string = 'appsrc name=source is-live=true block=true format=GST_FORMAT_TIME ! h265parse ! rtph265pay name=pay0 config-interval=1 name=pay0 pt=96'
        
            def send_data(self, data):
                self.data = data
        
            def start(self):
                t = threading.Thread(target=self._thread_rtsp)
                t.start()
        
            def _thread_rtsp(self):
                loop = GLib.MainLoop()
                loop.run()
        
            def on_need_data(self, src, length):
                if self.data is not None:
                    retval = src.emit('push-buffer', Gst.Buffer.new_wrapped(self.data.tobytes()))
                    if retval != Gst.FlowReturn.OK:
                        print(retval)
        
            def do_create_element(self, url):
                return Gst.parse_launch(self.launch_string)
        
            def do_configure(self, rtsp_media):
                self.number_frames = 0
                appsrc = rtsp_media.get_element().get_child_by_name('source')
                appsrc.connect('need-data', self.on_need_data)
        
        
        class RTSPServer(GstRtspServer.RTSPServer):
            def __init__(self, **properties):
                super(RTSPServer, self).__init__(**properties)
                self.rtsp = RtspSystem()
                self.rtsp.set_shared(True)
                self.get_mount_points().add_factory("/preview", self.rtsp)
                self.attach(None)
                Gst.init(None)
                self.rtsp.start()
        
            def send_data(self, data):
                self.rtsp.send_data(data)
        
        def get_frame(socket, size):
            bytes = socket.recv(4096)
            while True:
                read = 4096
                if size-len(bytes) < read:
                    read = size-len(bytes)
                bytes += socket.recv(read)
                if size == len(bytes):
                    return bytes
        
        if __name__ == "__main__":
            server = RTSPServer()
        
            sock = socket.socket()
            sock.connect((OAK_IP, 5000))
        
            try:
                while True:
                    header = str(sock.recv(32), encoding="ascii")
                    chunks = re.split(' +', header)
                    if chunks[0] == "ABCDE":
                        # print(f">{header}<")
                        ts = float(chunks[1])
                        imgSize = int(chunks[2])
                        img = get_frame(sock, imgSize)
                        buf = np.frombuffer(img, dtype=np.byte)
                        server.send_data(buf)
                        # print(buf.shape, buf.size)
                    if cv2.waitKey(1) == ord('q'):
                        break
            except Exception as e:
                print("Error:", e)
        
            sock.close()
      6. Find IP of your WLS2 (ifconfig) which is now running RTSP server
      7. ???
        7.1 Profit!
      8. I used RTSP PLayer on Windows to verify that my RTSP (on WLS2) works as expected. Img below.

        erik
        Thank you very much for tutorial it clears lot of concepts for me,

        Its streaming : )

        :

        I am getting following error in the terminal sometimes.
        Error: negative buffersize in recv

        I have one more question Is it possible to flash everything into the device in way that it connects to the TCP server running on the camera, receives the H265 encoded video frames, and then streams them via an RTSP server. With host as device itself.

        for instance: rtsp:://192.168.0.102:8554/preview (Considering sample OAK_IP you have mention in tutorial)

        Hi @nikul ,
        Unfortunately, running RTSP on the device itself wouldn't be possible, as this feature hasn't (and likely won't) be implemented on the RVC2-based devices.
        BR, Erik

          erik
          Thank you for getting back @erik.
          I have one more question. Since have multiple camera. I want to run them on the same server. will I be able to run them through single script ?. having different endpoints or maybe different ports to access RTSP streaming.

          • erik replied to this.

            nikul Yep that should work, are you experiencing any issues with that approach?

              erik
              Hi @erik ,
              I am getting error while Flashing TCP on to the device.
              It is able to gain access of the device however when it trying to flash, it causing error occurs.
              same with device manager when I search in device manger I Have all of them available but when i click on it, device not found.

              Thank you

              erik
              Thank you.
              I do able to ping all the devices connected to the LAN.
              I specified the IP and remove application through GUI. and it worked temporary. i will look more into it.

              I have moved everything to lunux. for some reasons, streaming script is not working. By any chance will you be having idea if I may have missed some settings or anything else you want me to check.

              Thank you.

              Hi @nikul ,
              Without any context / screenshots / terminal outputs / logs / anything, we aren't able to help much.

                erik
                Thank you for your prompt attention.

                here the screenshot terminal and code snap.

                Thank you.

                My best guess is that the device doesn't have the TCP server running on it.

                  erik
                  Thank you.
                  I believe IT has TCP server running since can access streaming through WSL but not from linux.

                    @jakaskerl can you help nikul debug this? My guess is the incorrect IP (mixing between host IP and WSL2 IP), or failing to set firewall for that port (if accessing from different machine).