• DepthAI-v2
  • Error installing PyGObject in window for running RTSP streaming.

jakaskerl
Thank you for getting back.
I have lower down FPS to 10. And still not able to debug the code,
Any help in this matter will be appreciated.

Thank you.

Hi @nikul
I'll have to debug this on monday since I don't have any poe device to test on currently.

Thanks,
Jaka

@nikul
You are encoding into H265 (not MJPEG) - and in http response you are writing jpgboundary and in header you have specified image/jpeg.
This will, of course, not work, and I don't think you can just stream raw h264 bitstream to a browser and it will decode/display it. You would likely need to containerize it (eg mp4), so html5's video player could play the stream. The why is that h264/h265 is much more complex than mjpeg. Thoughts?
Thanks, Erik

    erik
    Thank you.

    I got the RTSP streaming link however it is throwing error and i don't know how to debug this.

    Further, can I flash RTSP experiment/demo in to the device so it can stream it on stand alone. if yes how.

    Here is the link of experiment
    luxonis/depthai-experimentstree/master/gen2-rtsp-streaming

    Would really appreciate you expertise in this matter.
    Thank you.

    Hi @nikul ,
    Here's step by step tutorial:

    1. Install WLS 2
    2. RUn commands
      sudo apt-get update
      sudo apt-get upgrade
      sudo apt-get install ffmpeg gstreamer-1.0 gir1.2-gst-rtsp-server-1.0 libgirepository1.0-dev gstreamer1.0-plugins-bad gstreamer1.0-plugins-good gstreamer1.0-plugins-base
    3. Install python requirements
      opencv-python
      numpy
      depthai==2.24
      PyGObject
    4. I went with standalone approach, so flashing TCP server streaming H265 bitstream to clients that connect to it. You can run this flashing script from Windows (no need for WSL). Note that you need latest bootlaoder flashed on device before flashing this script to the OAK POE camera:
      import depthai as dai
      
      # Start defining a pipeline
      pipeline = dai.Pipeline()
      
      camRgb = pipeline.createColorCamera()
      camRgb.setIspScale(2,3)
      
      videoEnc = pipeline.create(dai.node.VideoEncoder)
      videoEnc.setDefaultProfilePreset(30, dai.VideoEncoderProperties.Profile.H265_MAIN)
      camRgb.video.link(videoEnc.input)
      
      script = pipeline.create(dai.node.Script)
      script.setProcessor(dai.ProcessorType.LEON_CSS)
      
      videoEnc.bitstream.link(script.inputs['frame'])
      script.inputs['frame'].setBlocking(False)
      script.inputs['frame'].setQueueSize(1)
      
      script.setScript("""
      import socket
      import time
      
      server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
      server.bind(("0.0.0.0", 5000))
      server.listen()
      node.warn("Server up")
      
      while True:
          conn, client = server.accept()
          node.warn(f"Connected to client IP: {client}")
          try:
              while True:
                  pck = node.io["frame"].get()
                  data = pck.getData()
                  ts = pck.getTimestamp()
                  header = f"ABCDE " + str(ts.total_seconds()).ljust(18) + str(len(data)).ljust(8)
                  # node.warn(f'>{header}<')
                  conn.send(bytes(header, encoding='ascii'))
                  conn.send(data)
      
          except Exception as e:
              node.warn("Client disconnected")
      """)
      
      (f, bl) = dai.DeviceBootloader.getFirstAvailableDevice()
      bootloader = dai.DeviceBootloader(bl)
      progress = lambda p : print(f'Flashing progress: {p*100:.1f}%')
      bootloader.flash(progress, pipeline)
    5. Find IP of your camera that's running the TCP server (I just used device manager), then run this script within the WLS2:
      #!/usr/bin/env python3
      
      import threading
      
      import gi
      gi.require_version('Gst', '1.0')
      gi.require_version('GstRtspServer', '1.0')
      from gi.repository import Gst, GstRtspServer, GLib
      import socket
      import re
      import cv2
      import numpy as np
      
      # Enter your own IP!
      OAK_IP = "192.168.0.102"
      
      class RtspSystem(GstRtspServer.RTSPMediaFactory):
          def __init__(self, **properties):
              super(RtspSystem, self).__init__(**properties)
              self.data = None
              self.launch_string = 'appsrc name=source is-live=true block=true format=GST_FORMAT_TIME ! h265parse ! rtph265pay name=pay0 config-interval=1 name=pay0 pt=96'
      
          def send_data(self, data):
              self.data = data
      
          def start(self):
              t = threading.Thread(target=self._thread_rtsp)
              t.start()
      
          def _thread_rtsp(self):
              loop = GLib.MainLoop()
              loop.run()
      
          def on_need_data(self, src, length):
              if self.data is not None:
                  retval = src.emit('push-buffer', Gst.Buffer.new_wrapped(self.data.tobytes()))
                  if retval != Gst.FlowReturn.OK:
                      print(retval)
      
          def do_create_element(self, url):
              return Gst.parse_launch(self.launch_string)
      
          def do_configure(self, rtsp_media):
              self.number_frames = 0
              appsrc = rtsp_media.get_element().get_child_by_name('source')
              appsrc.connect('need-data', self.on_need_data)
      
      
      class RTSPServer(GstRtspServer.RTSPServer):
          def __init__(self, **properties):
              super(RTSPServer, self).__init__(**properties)
              self.rtsp = RtspSystem()
              self.rtsp.set_shared(True)
              self.get_mount_points().add_factory("/preview", self.rtsp)
              self.attach(None)
              Gst.init(None)
              self.rtsp.start()
      
          def send_data(self, data):
              self.rtsp.send_data(data)
      
      def get_frame(socket, size):
          bytes = socket.recv(4096)
          while True:
              read = 4096
              if size-len(bytes) < read:
                  read = size-len(bytes)
              bytes += socket.recv(read)
              if size == len(bytes):
                  return bytes
      
      if __name__ == "__main__":
          server = RTSPServer()
      
          sock = socket.socket()
          sock.connect((OAK_IP, 5000))
      
          try:
              while True:
                  header = str(sock.recv(32), encoding="ascii")
                  chunks = re.split(' +', header)
                  if chunks[0] == "ABCDE":
                      # print(f">{header}<")
                      ts = float(chunks[1])
                      imgSize = int(chunks[2])
                      img = get_frame(sock, imgSize)
                      buf = np.frombuffer(img, dtype=np.byte)
                      server.send_data(buf)
                      # print(buf.shape, buf.size)
                  if cv2.waitKey(1) == ord('q'):
                      break
          except Exception as e:
              print("Error:", e)
      
          sock.close()
    6. Find IP of your WLS2 (ifconfig) which is now running RTSP server
    7. ???
      7.1 Profit!
    8. I used RTSP PLayer on Windows to verify that my RTSP (on WLS2) works as expected. Img below.

      erik
      Thank you very much for tutorial it clears lot of concepts for me,

      Its streaming : )

      :

      I am getting following error in the terminal sometimes.
      Error: negative buffersize in recv

      I have one more question Is it possible to flash everything into the device in way that it connects to the TCP server running on the camera, receives the H265 encoded video frames, and then streams them via an RTSP server. With host as device itself.

      for instance: rtsp:://192.168.0.102:8554/preview (Considering sample OAK_IP you have mention in tutorial)

      Hi @nikul ,
      Unfortunately, running RTSP on the device itself wouldn't be possible, as this feature hasn't (and likely won't) be implemented on the RVC2-based devices.
      BR, Erik

        erik
        Thank you for getting back @erik.
        I have one more question. Since have multiple camera. I want to run them on the same server. will I be able to run them through single script ?. having different endpoints or maybe different ports to access RTSP streaming.

        • erik replied to this.

          nikul Yep that should work, are you experiencing any issues with that approach?

            erik
            Hi @erik ,
            I am getting error while Flashing TCP on to the device.
            It is able to gain access of the device however when it trying to flash, it causing error occurs.
            same with device manager when I search in device manger I Have all of them available but when i click on it, device not found.

            Thank you

            erik
            Thank you.
            I do able to ping all the devices connected to the LAN.
            I specified the IP and remove application through GUI. and it worked temporary. i will look more into it.

            I have moved everything to lunux. for some reasons, streaming script is not working. By any chance will you be having idea if I may have missed some settings or anything else you want me to check.

            Thank you.

            Hi @nikul ,
            Without any context / screenshots / terminal outputs / logs / anything, we aren't able to help much.

              erik
              Thank you for your prompt attention.

              here the screenshot terminal and code snap.

              Thank you.

              My best guess is that the device doesn't have the TCP server running on it.

                erik
                Thank you.
                I believe IT has TCP server running since can access streaming through WSL but not from linux.

                  @jakaskerl can you help nikul debug this? My guess is the incorrect IP (mixing between host IP and WSL2 IP), or failing to set firewall for that port (if accessing from different machine).

                  nikul I believe IT has TCP server running since can access streaming through WSL but not from linux.

                  Could you once again try on WSL? Since you have stated:

                  I specified the IP and remove application through GUI. and it worked temporary. i will look more into it.

                  Perhaps there is no TCP stream running on the device anymore. Also make sure the device IP didn't change. If it did, you need to change the OAK_IP inside the RTSP server script.

                  Maybe check the firewall as well: sudo ufw status verbose. And Gstreamer: gst-launch-1.0 videotestsrc ! autovideosink

                  Let me know how it goes.

                  Thanks,
                  Jaka