• DepthAI-v2
  • Error installing PyGObject in window for running RTSP streaming.

My best guess is that the device doesn't have the TCP server running on it.

    erik
    Thank you.
    I believe IT has TCP server running since can access streaming through WSL but not from linux.

      @jakaskerl can you help nikul debug this? My guess is the incorrect IP (mixing between host IP and WSL2 IP), or failing to set firewall for that port (if accessing from different machine).

      nikul I believe IT has TCP server running since can access streaming through WSL but not from linux.

      Could you once again try on WSL? Since you have stated:

      I specified the IP and remove application through GUI. and it worked temporary. i will look more into it.

      Perhaps there is no TCP stream running on the device anymore. Also make sure the device IP didn't change. If it did, you need to change the OAK_IP inside the RTSP server script.

      Maybe check the firewall as well: sudo ufw status verbose. And Gstreamer: gst-launch-1.0 videotestsrc ! autovideosink

      Let me know how it goes.

      Thanks,
      Jaka

      Thank you.
      @erik
      @jakaskerl
      I tried on WSL it working fine. And unknowingly it now working in linux too.
      However quality of the video fluctuate some time. Something like this

      Hi @nikul
      Few things to try:

      • firewall: run sudo ufw disable and retry the script. Maybe the firewall is blocking some part of the stream.
      • Make sure both versions of Gstreamer are the same
      • Install necessary plugins if not already installed: sudo apt-get install gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly
      • Ensure you have enough network bandwidth (try using ETH cable).
      • Maybe switching to H264 would be better

      Thanks,
      Jaka

        What you are seeing are h26x artifacts. If one frame gets missed, you will see this strange image until you get another keyframe (you can set frequency of that inside VideoEncoder node).
        So a frame could get lost either from OAK to WSL server, or from WSL server (RTSP) to the RTSP client, best to first find that out, then look for causes on why that could happen (and how to mitigate it).

        jakaskerl
        thank you for getting back.
        I did try everything u mentioned issue is still the same.
        While implementing H265_MAINgetting following error on terminal.

        [rtsp @ 0x7f6524000cc0] method DESCRIBE failed: 503 Service Unavailable

        rtsp://11.11.11.25:8554/OAK60: Server returned 5XX Server Error reply

        Than you
        Nikul

        Hi @nikul
        As erik mentioned, check where the packets are getting lost.

        1. OAK-RTSP server - instead of creating the server, you could just display the stream coming from OAK directly. If there is no packet loss, the error is likely with RTSP server streaming.

        2. RTSP Server-Client, try sending random video/webcam stream with RTSP to see if the stream works as expected - no packet loss. Then you can check where the packet loss occurs with OAK.

        Thanks,
        Jaka

          jakaskerl
          I have figured out that 60 FPS with 1080P is what causing problem. However I can not compromise with 60FPS as it something crucial for the project i am working on. Any thoughts ?

            Hi @nikul

            nikul I have figured out that 60 FPS with 1080P is what causing problem

            So a bandwidth or processing issue? In both cases, can you afford to lower the resolution? Something like 720p should be much less (less than 50%) demanding.

            You would need to set the ISP scaling to 2/3 on the OAK device to achieve that since 720p native sensor resolution is not supported by depthai.

            Thanks,
            Jaka

              jakaskerl So a bandwidth or processing issue? In both cases, can you afford to lower the resolution? Something like 720p should be much less (less than 50%) demanding.

              Thank you @jakaskerl

              I am assuming its not a bandwidth issue since it is connected pretty good industrial network infrastructure.

              If Its processing, Is there any other way then lowering down resolution.

              @erik .

              Hi Erik.
              Apart from the issues with FPS and resolution, I have an additional concern. I have disabled the TCP server on one device, leaving it in a bootable state. I am currently experimenting with gen2-RTSP. My objective is to have multiple scripts running for each camera to stream them individually to different endpoints. However, when I run the script for one device, I encounter difficulties streaming from the other device simultaneously. Below is the code I'm using:

              #!/usr/bin/env python3
              
              import threading
              
              import gi
              gi.require_version('Gst', '1.0')
              gi.require_version('GstRtspServer', '1.0')
              from gi.repository import Gst, GstRtspServer, GLib
              
              
              class RtspSystem(GstRtspServer.RTSPMediaFactory):
                  def __init__(self, **properties):
                      super(RtspSystem, self).__init__(**properties)
                      self.data = None
                      self.launch_string = 'appsrc name=source is-live=true block=true format=GST_FORMAT_TIME ! h265parse ! rtph265pay name=pay0 config-interval=1 name=pay0 pt=96'
              
                  def send_data(self, data):
                      self.data = data
              
                  def start(self):
                      t = threading.Thread(target=self._thread_rtsp)
                      t.start()
              
                  def _thread_rtsp(self):
                      loop = GLib.MainLoop()
                      loop.run()
              
                  def on_need_data(self, src, length):
                      if self.data is not None:
                          retval = src.emit('push-buffer', Gst.Buffer.new_wrapped(self.data.tobytes()))
                          if retval != Gst.FlowReturn.OK:
                              print(retval)
              
                  def do_create_element(self, url):
                      return Gst.parse_launch(self.launch_string)
              
                  def do_configure(self, rtsp_media):
                      self.number_frames = 0
                      appsrc = rtsp_media.get_element().get_child_by_name('source')
                      appsrc.connect('need-data', self.on_need_data)
              
              
              class RTSPServer(GstRtspServer.RTSPServer):
                  def __init__(self, **properties):
                      super(RTSPServer, self).__init__(**properties)
                      self.rtsp = RtspSystem()
                      self.rtsp.set_shared(True)
                      self.get_mount_points().add_factory("/oak63", self.rtsp)
                      self.attach(None)
                      Gst.init(None)
                      self.rtsp.start()
              
                  def send_data(self, data):
                      self.rtsp.send_data(data)
              
              if __name__ == "__main__":
                  import depthai as dai
              
                  server = RTSPServer()
              
                  pipeline = dai.Pipeline()
              
                  FPS = 60
                  colorCam = pipeline.create(dai.node.ColorCamera)
                  colorCam.setResolution(dai.ColorCameraProperties.SensorResolution.THE_720_P)
                  colorCam.setInterleaved(False)
                  colorCam.setColorOrder(dai.ColorCameraProperties.ColorOrder.BGR)
                  # colorCam.setIspScale(2,3)
                  # colorCam.setFps(FPS)
              
                  videnc = pipeline.create(dai.node.VideoEncoder)
                  videnc.setDefaultProfilePreset(FPS, dai.VideoEncoderProperties.Profile.H265_MAIN)
                  colorCam.video.link(videnc.input)
              
                  veOut = pipeline.create(dai.node.XLinkOut)
                  veOut.setStreamName("encoded")
                  videnc.bitstream.link(veOut.input)
              
                  desired_ip = "00.00.00.00
                  device_infos = dai.DeviceBootloader.getAllAvailableDevices()
                  device_info = [device_info for device_info in device_infos if device_info.name == desired_ip][0]
                  if device_info:
                      if device_info.protocol != dai.XLinkProtocol.X_LINK_USB_VSC:
                          print("Running RTSP stream may be unstable due to connection... (protocol: {})".format(device_info.protocol))
              
                      with dai.Device(pipeline, device_info) as device:
                          encoded = device.getOutputQueue("encoded", maxSize=30, blocking=True)
                          print("Setup finished, RTSP stream available under \"rtsp://localhost:8554/oak63\"")
                          while True:
                              data = encoded.get().getData()
                              server.send_data(data)
                  else
                      print('No device found')

              Further I haven't been able to figure out where am I loosing packages.
              need your suggestion how can I achieve desired outcome which is having 60FPS with 1080 Resolution and running different script on different end point in single host.

              Thank you very much.
              Nikul

              Hi @nikul
              Try lowering the bitrate on the VideoEncoder node as it will lower the bandwidth requirements. Hopefully to the point it works, without losing to much quality.

              videoEnc.setBitrateKbps(10000) (20k by default I think).

              Thanks,
              Jaka

              jakaskerl
              @erik
              Hi
              Thank you for prompt response appreciate it.
              Apologies it seems like it was bandwidth issue, since it is working for now with lower bit rate.


              Now I just want to run multiple script on single host to run multiple RTSP server for different streaming. I somewhat tried running one script with different end point for streamings but that may cause problem in the future since i want other cameras to stream parallel while I am experimenting with one.

                Hi @nikul
                It's great to hear that lowering the bitrate resolved the bandwidth issue. To run multiple RTSP servers on a single host for different streams from multiple cameras, you'll need to ensure that each RTSP server instance is running on a unique port and that each stream has its own endpoint. Here's how you can achieve this:

                1. Unique Ports: Each RTSP server must listen on a different port. For example, the first one could be on 8554, the second on 8555, and so on.

                2. Unique Endpoints: Each RTSP server can have its own endpoint, such as /oak63, /oak64, etc.

                3. Multiple Instances: Run each script in a separate process, either manually or automated via a script. Make sure each script is configured with different port numbers and endpoints.

                4. Network Configuration: Ensure your network infrastructure can handle the simultaneous streams. This includes not just the bandwidth but also any firewalls, routers, or switches that need to be configured to allow the traffic.

                5. Hardware Resources: Verify that your host machine has sufficient CPU, memory, and network capabilities to handle multiple video streams without dropping frames.

                Here's an example of how you can modify your script for two cameras:

                # RTSPServer for the first camera on port 8554
                class RTSPServer1(GstRtspServer.RTSPServer):
                    # ... existing code ...
                
                server1 = RTSPServer1()
                server1.rtsp.set_shared(True)
                server1.get_mount_points().add_factory("/oak63", server1.rtsp)
                server1.attach(None)
                server1.rtsp.start()
                
                # RTSPServer for the second camera on port 8555
                class RTSPServer2(GstRtspServer.RTSPServer):
                    # ... existing code ...
                
                server2 = RTSPServer2()
                server2.rtsp.set_shared(True)
                server2.get_mount_points().add_factory("/oak64", server2.rtsp)
                server2.attach(None)
                server2.rtsp.start()
                
                # ... existing code for setting up the pipeline and device ...
                
                # Run the servers in separate threads or processes

                For each camera, you will run a separate instance of the above script (or a script modified to handle multiple cameras) with the relevant IP, port, and endpoint configuration.

                If you need to start and manage multiple RTSP servers programmatically, consider using a supervisor process or a process manager like systemd, supervisord, or Docker containers for better isolation and management.

                Thanks,
                Jaka

                  jakaskerl
                  @erik
                  thank you. for getting back.
                  I am facing connection establishment issue while running the script I dont really knwo whats wrong in here. It happens in second script i run. To summarize i can only stream one camera at a time.

                  #!/usr/bin/env python3
                  
                  import threading
                  
                  import gi
                  gi.require_version('Gst', '1.0')
                  gi.require_version('GstRtspServer', '1.0')
                  from gi.repository import Gst, GstRtspServer, GLib
                  import depthai as dai
                  
                  class RtspSystem(GstRtspServer.RTSPMediaFactory):
                      def __init__(self, **properties):
                          super(RtspSystem, self).__init__(**properties)
                          self.data = None
                          self.launch_string = 'appsrc name=source is-live=true block=true format=GST_FORMAT_TIME ! h265parse ! rtph265pay name=pay0 config-interval=1 name=pay0 pt=96'
                  
                      def send_data(self, data):
                          self.data = data
                  
                      def start(self):
                          t = threading.Thread(target=self._thread_rtsp)
                          t.start()
                  
                      def _thread_rtsp(self):
                          loop = GLib.MainLoop()
                          loop.run()
                  
                      def on_need_data(self, src, length):
                          if self.data is not None:
                              retval = src.emit('push-buffer', Gst.Buffer.new_wrapped(self.data.tobytes()))
                              if retval != Gst.FlowReturn.OK:
                                  print(retval)
                  
                      def do_create_element(self, url):
                          return Gst.parse_launch(self.launch_string)
                  
                      def do_configure(self, rtsp_media):
                          self.number_frames = 0
                          appsrc = rtsp_media.get_element().get_child_by_name('source')
                          appsrc.connect('need-data', self.on_need_data)
                  
                  
                  class RTSPServer(GstRtspServer.RTSPServer):
                      def __init__(self, port, endpoint, device_ip, **properties):
                          super(RTSPServer, self).__init__(**properties)
                          self.rtsp = RtspSystem()
                          self.rtsp.set_shared(True)
                          self.get_mount_points().add_factory(endpoint, self.rtsp)
                          self.attach(None)
                          Gst.init(None)
                          self.rtsp.start()
                  
                      def send_data(self, data):
                          self.rtsp.send_data(data)
                  
                  if __name__ == "__main__":
                      port1 = 8554
                      endpoint1 = "/oak61"
                      device_ip1 = "10.10.10.61"
                  
                      server1 = RTSPServer(port1, endpoint1, device_ip1)
                  
                      pipeline1 = dai.Pipeline()
                  
                      FPS = 60
                      colorCam1 = pipeline1.create(dai.node.ColorCamera)
                      colorCam1.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1080_P)
                      colorCam1.setInterleaved(False)
                      colorCam1.setColorOrder(dai.ColorCameraProperties.ColorOrder.BGR)
                      colorCam1.setFps(FPS)
                      colorCam1.initialControl.setManualFocus(185) # 0..255
                  
                      videnc1 = pipeline1.create(dai.node.VideoEncoder)
                      videnc1.setDefaultProfilePreset(FPS, dai.VideoEncoderProperties.Profile.H265_MAIN)
                      videnc1.setBitrateKbps(8000)
                      colorCam1.video.link(videnc1.input)
                  
                      veOut1 = pipeline1.create(dai.node.XLinkOut)
                      veOut1.setStreamName("encoded")
                      videnc1.bitstream.link(veOut1.input)
                  
                      device_infos = dai.DeviceBootloader.getAllAvailableDevices()
                      device_info1 = [device_info for device_info in device_infos if device_info.name == desired_ip][0]
                      if device_info1:
                          if device_info1.protocol != dai.XLinkProtocol.X_LINK_USB_VSC:
                              print("Running RTSP stream may be unstable due to connection... (protocol: {})".format(device_info1.protocol))
                  
                          with dai.Device(pipeline1, device_info1) as device1:
                              encoded1 = device1.getOutputQueue("encoded", maxSize=30, blocking=True)
                              print("Setup finished, RTSP stream available under \"rtsp://localhost:{}/oak61\"".format(port1))
                              while True:
                                  data1 = encoded1.get().getData()
                                  server1.send_data(data1)
                      else:
                          print('No device found')

                  Further, about h26x artifacts, I now have doubt about bandwidth too. Since I have come to know about the ports are running on 1GBPS. and we have monitored traffic and its way less then capacity. So likely something else seems to be the issue. Over more, CPU also has good performance so processing should also not be the issue.
                  will share performance if it helps.

                    nikul if name == "main":
                    port1 = 8554
                    endpoint1 = "/oak61"
                    device_ip1 = "10.10.10.61"

                    I see you are hardcoding the endpoints. These need to differ from one script to the other, otherwise you will get conflicts.

                    nikul Since I have come to know about the ports are running on 1GBPS. and we have monitored traffic and its way less then capacity.

                    Which machine is this for? I suspected a bandwidth issue on the machine that both receives a stream from the OAK device as well as serve another stream via RTSP.

                    nikul Apologies it seems like it was bandwidth issue, since it is working for now with lower bit rate.

                    Can this be reproducible? Increasing the bitrate induces stream errors?

                    Thanks,
                    Jaka

                      jakaskerl I see you are hardcoding the endpoints. These need to differ from one script to the other, otherwise you will get conflicts.

                      end points and ports are different in each script.

                      Demo1.py
                      port = 8555

                      endpoint = '/oak63'

                      server = RTSPServer(port, endpoint)

                      Demo2.py
                      port = 8554

                      endpoint = '/oak62'

                      server = RTSPServer(port, endpoint)

                      jakaskerl Which machine is this for? I suspected a bandwidth issue on the machine that both receives a stream from the OAK device as well as serve another stream via RTSP.

                      This one is linux machine, running the script which gets the stream from OAK device and serves through RTSP.

                      jakaskerl Can this be reproducible? Increasing the bitrate induces stream errors?

                      Reducing the bit rate makes the streaming better(Not the best), That seems to be consistent so far.

                      Hi @nikul ,
                      We provide free-tier support with pointers/suggestions and any depthai/OAK issues you might have.
                      The questions/challenges you are having are about general development (encoding/servers/message dropping), which exceeds our free-tier support. If you are interested in paid support, please reach out to us via email (support@luxonis.com), and we can help you further.
                      Thank you for understanding.