• DepthAI-v2
  • Error installing PyGObject in window for running RTSP streaming.

jakaskerl
thank you for getting back.
I did try everything u mentioned issue is still the same.
While implementing H265_MAINgetting following error on terminal.

[rtsp @ 0x7f6524000cc0] method DESCRIBE failed: 503 Service Unavailable

rtsp://11.11.11.25:8554/OAK60: Server returned 5XX Server Error reply

Than you
Nikul

Hi @nikul
As erik mentioned, check where the packets are getting lost.

  1. OAK-RTSP server - instead of creating the server, you could just display the stream coming from OAK directly. If there is no packet loss, the error is likely with RTSP server streaming.

  2. RTSP Server-Client, try sending random video/webcam stream with RTSP to see if the stream works as expected - no packet loss. Then you can check where the packet loss occurs with OAK.

Thanks,
Jaka

    jakaskerl
    I have figured out that 60 FPS with 1080P is what causing problem. However I can not compromise with 60FPS as it something crucial for the project i am working on. Any thoughts ?

      Hi @nikul

      nikul I have figured out that 60 FPS with 1080P is what causing problem

      So a bandwidth or processing issue? In both cases, can you afford to lower the resolution? Something like 720p should be much less (less than 50%) demanding.

      You would need to set the ISP scaling to 2/3 on the OAK device to achieve that since 720p native sensor resolution is not supported by depthai.

      Thanks,
      Jaka

        jakaskerl So a bandwidth or processing issue? In both cases, can you afford to lower the resolution? Something like 720p should be much less (less than 50%) demanding.

        Thank you @jakaskerl

        I am assuming its not a bandwidth issue since it is connected pretty good industrial network infrastructure.

        If Its processing, Is there any other way then lowering down resolution.

        @erik .

        Hi Erik.
        Apart from the issues with FPS and resolution, I have an additional concern. I have disabled the TCP server on one device, leaving it in a bootable state. I am currently experimenting with gen2-RTSP. My objective is to have multiple scripts running for each camera to stream them individually to different endpoints. However, when I run the script for one device, I encounter difficulties streaming from the other device simultaneously. Below is the code I'm using:

        #!/usr/bin/env python3
        
        import threading
        
        import gi
        gi.require_version('Gst', '1.0')
        gi.require_version('GstRtspServer', '1.0')
        from gi.repository import Gst, GstRtspServer, GLib
        
        
        class RtspSystem(GstRtspServer.RTSPMediaFactory):
            def __init__(self, **properties):
                super(RtspSystem, self).__init__(**properties)
                self.data = None
                self.launch_string = 'appsrc name=source is-live=true block=true format=GST_FORMAT_TIME ! h265parse ! rtph265pay name=pay0 config-interval=1 name=pay0 pt=96'
        
            def send_data(self, data):
                self.data = data
        
            def start(self):
                t = threading.Thread(target=self._thread_rtsp)
                t.start()
        
            def _thread_rtsp(self):
                loop = GLib.MainLoop()
                loop.run()
        
            def on_need_data(self, src, length):
                if self.data is not None:
                    retval = src.emit('push-buffer', Gst.Buffer.new_wrapped(self.data.tobytes()))
                    if retval != Gst.FlowReturn.OK:
                        print(retval)
        
            def do_create_element(self, url):
                return Gst.parse_launch(self.launch_string)
        
            def do_configure(self, rtsp_media):
                self.number_frames = 0
                appsrc = rtsp_media.get_element().get_child_by_name('source')
                appsrc.connect('need-data', self.on_need_data)
        
        
        class RTSPServer(GstRtspServer.RTSPServer):
            def __init__(self, **properties):
                super(RTSPServer, self).__init__(**properties)
                self.rtsp = RtspSystem()
                self.rtsp.set_shared(True)
                self.get_mount_points().add_factory("/oak63", self.rtsp)
                self.attach(None)
                Gst.init(None)
                self.rtsp.start()
        
            def send_data(self, data):
                self.rtsp.send_data(data)
        
        if __name__ == "__main__":
            import depthai as dai
        
            server = RTSPServer()
        
            pipeline = dai.Pipeline()
        
            FPS = 60
            colorCam = pipeline.create(dai.node.ColorCamera)
            colorCam.setResolution(dai.ColorCameraProperties.SensorResolution.THE_720_P)
            colorCam.setInterleaved(False)
            colorCam.setColorOrder(dai.ColorCameraProperties.ColorOrder.BGR)
            # colorCam.setIspScale(2,3)
            # colorCam.setFps(FPS)
        
            videnc = pipeline.create(dai.node.VideoEncoder)
            videnc.setDefaultProfilePreset(FPS, dai.VideoEncoderProperties.Profile.H265_MAIN)
            colorCam.video.link(videnc.input)
        
            veOut = pipeline.create(dai.node.XLinkOut)
            veOut.setStreamName("encoded")
            videnc.bitstream.link(veOut.input)
        
            desired_ip = "00.00.00.00
            device_infos = dai.DeviceBootloader.getAllAvailableDevices()
            device_info = [device_info for device_info in device_infos if device_info.name == desired_ip][0]
            if device_info:
                if device_info.protocol != dai.XLinkProtocol.X_LINK_USB_VSC:
                    print("Running RTSP stream may be unstable due to connection... (protocol: {})".format(device_info.protocol))
        
                with dai.Device(pipeline, device_info) as device:
                    encoded = device.getOutputQueue("encoded", maxSize=30, blocking=True)
                    print("Setup finished, RTSP stream available under \"rtsp://localhost:8554/oak63\"")
                    while True:
                        data = encoded.get().getData()
                        server.send_data(data)
            else
                print('No device found')

        Further I haven't been able to figure out where am I loosing packages.
        need your suggestion how can I achieve desired outcome which is having 60FPS with 1080 Resolution and running different script on different end point in single host.

        Thank you very much.
        Nikul

        Hi @nikul
        Try lowering the bitrate on the VideoEncoder node as it will lower the bandwidth requirements. Hopefully to the point it works, without losing to much quality.

        videoEnc.setBitrateKbps(10000) (20k by default I think).

        Thanks,
        Jaka

        jakaskerl
        @erik
        Hi
        Thank you for prompt response appreciate it.
        Apologies it seems like it was bandwidth issue, since it is working for now with lower bit rate.


        Now I just want to run multiple script on single host to run multiple RTSP server for different streaming. I somewhat tried running one script with different end point for streamings but that may cause problem in the future since i want other cameras to stream parallel while I am experimenting with one.

          Hi @nikul
          It's great to hear that lowering the bitrate resolved the bandwidth issue. To run multiple RTSP servers on a single host for different streams from multiple cameras, you'll need to ensure that each RTSP server instance is running on a unique port and that each stream has its own endpoint. Here's how you can achieve this:

          1. Unique Ports: Each RTSP server must listen on a different port. For example, the first one could be on 8554, the second on 8555, and so on.

          2. Unique Endpoints: Each RTSP server can have its own endpoint, such as /oak63, /oak64, etc.

          3. Multiple Instances: Run each script in a separate process, either manually or automated via a script. Make sure each script is configured with different port numbers and endpoints.

          4. Network Configuration: Ensure your network infrastructure can handle the simultaneous streams. This includes not just the bandwidth but also any firewalls, routers, or switches that need to be configured to allow the traffic.

          5. Hardware Resources: Verify that your host machine has sufficient CPU, memory, and network capabilities to handle multiple video streams without dropping frames.

          Here's an example of how you can modify your script for two cameras:

          # RTSPServer for the first camera on port 8554
          class RTSPServer1(GstRtspServer.RTSPServer):
              # ... existing code ...
          
          server1 = RTSPServer1()
          server1.rtsp.set_shared(True)
          server1.get_mount_points().add_factory("/oak63", server1.rtsp)
          server1.attach(None)
          server1.rtsp.start()
          
          # RTSPServer for the second camera on port 8555
          class RTSPServer2(GstRtspServer.RTSPServer):
              # ... existing code ...
          
          server2 = RTSPServer2()
          server2.rtsp.set_shared(True)
          server2.get_mount_points().add_factory("/oak64", server2.rtsp)
          server2.attach(None)
          server2.rtsp.start()
          
          # ... existing code for setting up the pipeline and device ...
          
          # Run the servers in separate threads or processes

          For each camera, you will run a separate instance of the above script (or a script modified to handle multiple cameras) with the relevant IP, port, and endpoint configuration.

          If you need to start and manage multiple RTSP servers programmatically, consider using a supervisor process or a process manager like systemd, supervisord, or Docker containers for better isolation and management.

          Thanks,
          Jaka

            jakaskerl
            @erik
            thank you. for getting back.
            I am facing connection establishment issue while running the script I dont really knwo whats wrong in here. It happens in second script i run. To summarize i can only stream one camera at a time.

            #!/usr/bin/env python3
            
            import threading
            
            import gi
            gi.require_version('Gst', '1.0')
            gi.require_version('GstRtspServer', '1.0')
            from gi.repository import Gst, GstRtspServer, GLib
            import depthai as dai
            
            class RtspSystem(GstRtspServer.RTSPMediaFactory):
                def __init__(self, **properties):
                    super(RtspSystem, self).__init__(**properties)
                    self.data = None
                    self.launch_string = 'appsrc name=source is-live=true block=true format=GST_FORMAT_TIME ! h265parse ! rtph265pay name=pay0 config-interval=1 name=pay0 pt=96'
            
                def send_data(self, data):
                    self.data = data
            
                def start(self):
                    t = threading.Thread(target=self._thread_rtsp)
                    t.start()
            
                def _thread_rtsp(self):
                    loop = GLib.MainLoop()
                    loop.run()
            
                def on_need_data(self, src, length):
                    if self.data is not None:
                        retval = src.emit('push-buffer', Gst.Buffer.new_wrapped(self.data.tobytes()))
                        if retval != Gst.FlowReturn.OK:
                            print(retval)
            
                def do_create_element(self, url):
                    return Gst.parse_launch(self.launch_string)
            
                def do_configure(self, rtsp_media):
                    self.number_frames = 0
                    appsrc = rtsp_media.get_element().get_child_by_name('source')
                    appsrc.connect('need-data', self.on_need_data)
            
            
            class RTSPServer(GstRtspServer.RTSPServer):
                def __init__(self, port, endpoint, device_ip, **properties):
                    super(RTSPServer, self).__init__(**properties)
                    self.rtsp = RtspSystem()
                    self.rtsp.set_shared(True)
                    self.get_mount_points().add_factory(endpoint, self.rtsp)
                    self.attach(None)
                    Gst.init(None)
                    self.rtsp.start()
            
                def send_data(self, data):
                    self.rtsp.send_data(data)
            
            if __name__ == "__main__":
                port1 = 8554
                endpoint1 = "/oak61"
                device_ip1 = "10.10.10.61"
            
                server1 = RTSPServer(port1, endpoint1, device_ip1)
            
                pipeline1 = dai.Pipeline()
            
                FPS = 60
                colorCam1 = pipeline1.create(dai.node.ColorCamera)
                colorCam1.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1080_P)
                colorCam1.setInterleaved(False)
                colorCam1.setColorOrder(dai.ColorCameraProperties.ColorOrder.BGR)
                colorCam1.setFps(FPS)
                colorCam1.initialControl.setManualFocus(185) # 0..255
            
                videnc1 = pipeline1.create(dai.node.VideoEncoder)
                videnc1.setDefaultProfilePreset(FPS, dai.VideoEncoderProperties.Profile.H265_MAIN)
                videnc1.setBitrateKbps(8000)
                colorCam1.video.link(videnc1.input)
            
                veOut1 = pipeline1.create(dai.node.XLinkOut)
                veOut1.setStreamName("encoded")
                videnc1.bitstream.link(veOut1.input)
            
                device_infos = dai.DeviceBootloader.getAllAvailableDevices()
                device_info1 = [device_info for device_info in device_infos if device_info.name == desired_ip][0]
                if device_info1:
                    if device_info1.protocol != dai.XLinkProtocol.X_LINK_USB_VSC:
                        print("Running RTSP stream may be unstable due to connection... (protocol: {})".format(device_info1.protocol))
            
                    with dai.Device(pipeline1, device_info1) as device1:
                        encoded1 = device1.getOutputQueue("encoded", maxSize=30, blocking=True)
                        print("Setup finished, RTSP stream available under \"rtsp://localhost:{}/oak61\"".format(port1))
                        while True:
                            data1 = encoded1.get().getData()
                            server1.send_data(data1)
                else:
                    print('No device found')

            Further, about h26x artifacts, I now have doubt about bandwidth too. Since I have come to know about the ports are running on 1GBPS. and we have monitored traffic and its way less then capacity. So likely something else seems to be the issue. Over more, CPU also has good performance so processing should also not be the issue.
            will share performance if it helps.

              nikul if name == "main":
              port1 = 8554
              endpoint1 = "/oak61"
              device_ip1 = "10.10.10.61"

              I see you are hardcoding the endpoints. These need to differ from one script to the other, otherwise you will get conflicts.

              nikul Since I have come to know about the ports are running on 1GBPS. and we have monitored traffic and its way less then capacity.

              Which machine is this for? I suspected a bandwidth issue on the machine that both receives a stream from the OAK device as well as serve another stream via RTSP.

              nikul Apologies it seems like it was bandwidth issue, since it is working for now with lower bit rate.

              Can this be reproducible? Increasing the bitrate induces stream errors?

              Thanks,
              Jaka

                jakaskerl I see you are hardcoding the endpoints. These need to differ from one script to the other, otherwise you will get conflicts.

                end points and ports are different in each script.

                Demo1.py
                port = 8555

                endpoint = '/oak63'

                server = RTSPServer(port, endpoint)

                Demo2.py
                port = 8554

                endpoint = '/oak62'

                server = RTSPServer(port, endpoint)

                jakaskerl Which machine is this for? I suspected a bandwidth issue on the machine that both receives a stream from the OAK device as well as serve another stream via RTSP.

                This one is linux machine, running the script which gets the stream from OAK device and serves through RTSP.

                jakaskerl Can this be reproducible? Increasing the bitrate induces stream errors?

                Reducing the bit rate makes the streaming better(Not the best), That seems to be consistent so far.

                Hi @nikul ,
                We provide free-tier support with pointers/suggestions and any depthai/OAK issues you might have.
                The questions/challenges you are having are about general development (encoding/servers/message dropping), which exceeds our free-tier support. If you are interested in paid support, please reach out to us via email (support@luxonis.com), and we can help you further.
                Thank you for understanding.

                  erik
                  Thank you for getting back.
                  Will let you know whatever we make decision .