• DepthAI-v2
  • Error installing PyGObject in window for running RTSP streaming.

erik
Thank you.
I do able to ping all the devices connected to the LAN.
I specified the IP and remove application through GUI. and it worked temporary. i will look more into it.

I have moved everything to lunux. for some reasons, streaming script is not working. By any chance will you be having idea if I may have missed some settings or anything else you want me to check.

Thank you.

Hi @nikul ,
Without any context / screenshots / terminal outputs / logs / anything, we aren't able to help much.

    erik
    Thank you for your prompt attention.

    here the screenshot terminal and code snap.

    Thank you.

    My best guess is that the device doesn't have the TCP server running on it.

      erik
      Thank you.
      I believe IT has TCP server running since can access streaming through WSL but not from linux.

        @jakaskerl can you help nikul debug this? My guess is the incorrect IP (mixing between host IP and WSL2 IP), or failing to set firewall for that port (if accessing from different machine).

        nikul I believe IT has TCP server running since can access streaming through WSL but not from linux.

        Could you once again try on WSL? Since you have stated:

        I specified the IP and remove application through GUI. and it worked temporary. i will look more into it.

        Perhaps there is no TCP stream running on the device anymore. Also make sure the device IP didn't change. If it did, you need to change the OAK_IP inside the RTSP server script.

        Maybe check the firewall as well: sudo ufw status verbose. And Gstreamer: gst-launch-1.0 videotestsrc ! autovideosink

        Let me know how it goes.

        Thanks,
        Jaka

        Thank you.
        @erik
        @jakaskerl
        I tried on WSL it working fine. And unknowingly it now working in linux too.
        However quality of the video fluctuate some time. Something like this

        Hi @nikul
        Few things to try:

        • firewall: run sudo ufw disable and retry the script. Maybe the firewall is blocking some part of the stream.
        • Make sure both versions of Gstreamer are the same
        • Install necessary plugins if not already installed: sudo apt-get install gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly
        • Ensure you have enough network bandwidth (try using ETH cable).
        • Maybe switching to H264 would be better

        Thanks,
        Jaka

          What you are seeing are h26x artifacts. If one frame gets missed, you will see this strange image until you get another keyframe (you can set frequency of that inside VideoEncoder node).
          So a frame could get lost either from OAK to WSL server, or from WSL server (RTSP) to the RTSP client, best to first find that out, then look for causes on why that could happen (and how to mitigate it).

          jakaskerl
          thank you for getting back.
          I did try everything u mentioned issue is still the same.
          While implementing H265_MAINgetting following error on terminal.

          [rtsp @ 0x7f6524000cc0] method DESCRIBE failed: 503 Service Unavailable

          rtsp://11.11.11.25:8554/OAK60: Server returned 5XX Server Error reply

          Than you
          Nikul

          Hi @nikul
          As erik mentioned, check where the packets are getting lost.

          1. OAK-RTSP server - instead of creating the server, you could just display the stream coming from OAK directly. If there is no packet loss, the error is likely with RTSP server streaming.

          2. RTSP Server-Client, try sending random video/webcam stream with RTSP to see if the stream works as expected - no packet loss. Then you can check where the packet loss occurs with OAK.

          Thanks,
          Jaka

            jakaskerl
            I have figured out that 60 FPS with 1080P is what causing problem. However I can not compromise with 60FPS as it something crucial for the project i am working on. Any thoughts ?

              Hi @nikul

              nikul I have figured out that 60 FPS with 1080P is what causing problem

              So a bandwidth or processing issue? In both cases, can you afford to lower the resolution? Something like 720p should be much less (less than 50%) demanding.

              You would need to set the ISP scaling to 2/3 on the OAK device to achieve that since 720p native sensor resolution is not supported by depthai.

              Thanks,
              Jaka

                jakaskerl So a bandwidth or processing issue? In both cases, can you afford to lower the resolution? Something like 720p should be much less (less than 50%) demanding.

                Thank you @jakaskerl

                I am assuming its not a bandwidth issue since it is connected pretty good industrial network infrastructure.

                If Its processing, Is there any other way then lowering down resolution.

                @erik .

                Hi Erik.
                Apart from the issues with FPS and resolution, I have an additional concern. I have disabled the TCP server on one device, leaving it in a bootable state. I am currently experimenting with gen2-RTSP. My objective is to have multiple scripts running for each camera to stream them individually to different endpoints. However, when I run the script for one device, I encounter difficulties streaming from the other device simultaneously. Below is the code I'm using:

                #!/usr/bin/env python3
                
                import threading
                
                import gi
                gi.require_version('Gst', '1.0')
                gi.require_version('GstRtspServer', '1.0')
                from gi.repository import Gst, GstRtspServer, GLib
                
                
                class RtspSystem(GstRtspServer.RTSPMediaFactory):
                    def __init__(self, **properties):
                        super(RtspSystem, self).__init__(**properties)
                        self.data = None
                        self.launch_string = 'appsrc name=source is-live=true block=true format=GST_FORMAT_TIME ! h265parse ! rtph265pay name=pay0 config-interval=1 name=pay0 pt=96'
                
                    def send_data(self, data):
                        self.data = data
                
                    def start(self):
                        t = threading.Thread(target=self._thread_rtsp)
                        t.start()
                
                    def _thread_rtsp(self):
                        loop = GLib.MainLoop()
                        loop.run()
                
                    def on_need_data(self, src, length):
                        if self.data is not None:
                            retval = src.emit('push-buffer', Gst.Buffer.new_wrapped(self.data.tobytes()))
                            if retval != Gst.FlowReturn.OK:
                                print(retval)
                
                    def do_create_element(self, url):
                        return Gst.parse_launch(self.launch_string)
                
                    def do_configure(self, rtsp_media):
                        self.number_frames = 0
                        appsrc = rtsp_media.get_element().get_child_by_name('source')
                        appsrc.connect('need-data', self.on_need_data)
                
                
                class RTSPServer(GstRtspServer.RTSPServer):
                    def __init__(self, **properties):
                        super(RTSPServer, self).__init__(**properties)
                        self.rtsp = RtspSystem()
                        self.rtsp.set_shared(True)
                        self.get_mount_points().add_factory("/oak63", self.rtsp)
                        self.attach(None)
                        Gst.init(None)
                        self.rtsp.start()
                
                    def send_data(self, data):
                        self.rtsp.send_data(data)
                
                if __name__ == "__main__":
                    import depthai as dai
                
                    server = RTSPServer()
                
                    pipeline = dai.Pipeline()
                
                    FPS = 60
                    colorCam = pipeline.create(dai.node.ColorCamera)
                    colorCam.setResolution(dai.ColorCameraProperties.SensorResolution.THE_720_P)
                    colorCam.setInterleaved(False)
                    colorCam.setColorOrder(dai.ColorCameraProperties.ColorOrder.BGR)
                    # colorCam.setIspScale(2,3)
                    # colorCam.setFps(FPS)
                
                    videnc = pipeline.create(dai.node.VideoEncoder)
                    videnc.setDefaultProfilePreset(FPS, dai.VideoEncoderProperties.Profile.H265_MAIN)
                    colorCam.video.link(videnc.input)
                
                    veOut = pipeline.create(dai.node.XLinkOut)
                    veOut.setStreamName("encoded")
                    videnc.bitstream.link(veOut.input)
                
                    desired_ip = "00.00.00.00
                    device_infos = dai.DeviceBootloader.getAllAvailableDevices()
                    device_info = [device_info for device_info in device_infos if device_info.name == desired_ip][0]
                    if device_info:
                        if device_info.protocol != dai.XLinkProtocol.X_LINK_USB_VSC:
                            print("Running RTSP stream may be unstable due to connection... (protocol: {})".format(device_info.protocol))
                
                        with dai.Device(pipeline, device_info) as device:
                            encoded = device.getOutputQueue("encoded", maxSize=30, blocking=True)
                            print("Setup finished, RTSP stream available under \"rtsp://localhost:8554/oak63\"")
                            while True:
                                data = encoded.get().getData()
                                server.send_data(data)
                    else
                        print('No device found')

                Further I haven't been able to figure out where am I loosing packages.
                need your suggestion how can I achieve desired outcome which is having 60FPS with 1080 Resolution and running different script on different end point in single host.

                Thank you very much.
                Nikul

                Hi @nikul
                Try lowering the bitrate on the VideoEncoder node as it will lower the bandwidth requirements. Hopefully to the point it works, without losing to much quality.

                videoEnc.setBitrateKbps(10000) (20k by default I think).

                Thanks,
                Jaka

                jakaskerl
                @erik
                Hi
                Thank you for prompt response appreciate it.
                Apologies it seems like it was bandwidth issue, since it is working for now with lower bit rate.


                Now I just want to run multiple script on single host to run multiple RTSP server for different streaming. I somewhat tried running one script with different end point for streamings but that may cause problem in the future since i want other cameras to stream parallel while I am experimenting with one.

                  Hi @nikul
                  It's great to hear that lowering the bitrate resolved the bandwidth issue. To run multiple RTSP servers on a single host for different streams from multiple cameras, you'll need to ensure that each RTSP server instance is running on a unique port and that each stream has its own endpoint. Here's how you can achieve this:

                  1. Unique Ports: Each RTSP server must listen on a different port. For example, the first one could be on 8554, the second on 8555, and so on.

                  2. Unique Endpoints: Each RTSP server can have its own endpoint, such as /oak63, /oak64, etc.

                  3. Multiple Instances: Run each script in a separate process, either manually or automated via a script. Make sure each script is configured with different port numbers and endpoints.

                  4. Network Configuration: Ensure your network infrastructure can handle the simultaneous streams. This includes not just the bandwidth but also any firewalls, routers, or switches that need to be configured to allow the traffic.

                  5. Hardware Resources: Verify that your host machine has sufficient CPU, memory, and network capabilities to handle multiple video streams without dropping frames.

                  Here's an example of how you can modify your script for two cameras:

                  # RTSPServer for the first camera on port 8554
                  class RTSPServer1(GstRtspServer.RTSPServer):
                      # ... existing code ...
                  
                  server1 = RTSPServer1()
                  server1.rtsp.set_shared(True)
                  server1.get_mount_points().add_factory("/oak63", server1.rtsp)
                  server1.attach(None)
                  server1.rtsp.start()
                  
                  # RTSPServer for the second camera on port 8555
                  class RTSPServer2(GstRtspServer.RTSPServer):
                      # ... existing code ...
                  
                  server2 = RTSPServer2()
                  server2.rtsp.set_shared(True)
                  server2.get_mount_points().add_factory("/oak64", server2.rtsp)
                  server2.attach(None)
                  server2.rtsp.start()
                  
                  # ... existing code for setting up the pipeline and device ...
                  
                  # Run the servers in separate threads or processes

                  For each camera, you will run a separate instance of the above script (or a script modified to handle multiple cameras) with the relevant IP, port, and endpoint configuration.

                  If you need to start and manage multiple RTSP servers programmatically, consider using a supervisor process or a process manager like systemd, supervisord, or Docker containers for better isolation and management.

                  Thanks,
                  Jaka

                    jakaskerl
                    @erik
                    thank you. for getting back.
                    I am facing connection establishment issue while running the script I dont really knwo whats wrong in here. It happens in second script i run. To summarize i can only stream one camera at a time.

                    #!/usr/bin/env python3
                    
                    import threading
                    
                    import gi
                    gi.require_version('Gst', '1.0')
                    gi.require_version('GstRtspServer', '1.0')
                    from gi.repository import Gst, GstRtspServer, GLib
                    import depthai as dai
                    
                    class RtspSystem(GstRtspServer.RTSPMediaFactory):
                        def __init__(self, **properties):
                            super(RtspSystem, self).__init__(**properties)
                            self.data = None
                            self.launch_string = 'appsrc name=source is-live=true block=true format=GST_FORMAT_TIME ! h265parse ! rtph265pay name=pay0 config-interval=1 name=pay0 pt=96'
                    
                        def send_data(self, data):
                            self.data = data
                    
                        def start(self):
                            t = threading.Thread(target=self._thread_rtsp)
                            t.start()
                    
                        def _thread_rtsp(self):
                            loop = GLib.MainLoop()
                            loop.run()
                    
                        def on_need_data(self, src, length):
                            if self.data is not None:
                                retval = src.emit('push-buffer', Gst.Buffer.new_wrapped(self.data.tobytes()))
                                if retval != Gst.FlowReturn.OK:
                                    print(retval)
                    
                        def do_create_element(self, url):
                            return Gst.parse_launch(self.launch_string)
                    
                        def do_configure(self, rtsp_media):
                            self.number_frames = 0
                            appsrc = rtsp_media.get_element().get_child_by_name('source')
                            appsrc.connect('need-data', self.on_need_data)
                    
                    
                    class RTSPServer(GstRtspServer.RTSPServer):
                        def __init__(self, port, endpoint, device_ip, **properties):
                            super(RTSPServer, self).__init__(**properties)
                            self.rtsp = RtspSystem()
                            self.rtsp.set_shared(True)
                            self.get_mount_points().add_factory(endpoint, self.rtsp)
                            self.attach(None)
                            Gst.init(None)
                            self.rtsp.start()
                    
                        def send_data(self, data):
                            self.rtsp.send_data(data)
                    
                    if __name__ == "__main__":
                        port1 = 8554
                        endpoint1 = "/oak61"
                        device_ip1 = "10.10.10.61"
                    
                        server1 = RTSPServer(port1, endpoint1, device_ip1)
                    
                        pipeline1 = dai.Pipeline()
                    
                        FPS = 60
                        colorCam1 = pipeline1.create(dai.node.ColorCamera)
                        colorCam1.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1080_P)
                        colorCam1.setInterleaved(False)
                        colorCam1.setColorOrder(dai.ColorCameraProperties.ColorOrder.BGR)
                        colorCam1.setFps(FPS)
                        colorCam1.initialControl.setManualFocus(185) # 0..255
                    
                        videnc1 = pipeline1.create(dai.node.VideoEncoder)
                        videnc1.setDefaultProfilePreset(FPS, dai.VideoEncoderProperties.Profile.H265_MAIN)
                        videnc1.setBitrateKbps(8000)
                        colorCam1.video.link(videnc1.input)
                    
                        veOut1 = pipeline1.create(dai.node.XLinkOut)
                        veOut1.setStreamName("encoded")
                        videnc1.bitstream.link(veOut1.input)
                    
                        device_infos = dai.DeviceBootloader.getAllAvailableDevices()
                        device_info1 = [device_info for device_info in device_infos if device_info.name == desired_ip][0]
                        if device_info1:
                            if device_info1.protocol != dai.XLinkProtocol.X_LINK_USB_VSC:
                                print("Running RTSP stream may be unstable due to connection... (protocol: {})".format(device_info1.protocol))
                    
                            with dai.Device(pipeline1, device_info1) as device1:
                                encoded1 = device1.getOutputQueue("encoded", maxSize=30, blocking=True)
                                print("Setup finished, RTSP stream available under \"rtsp://localhost:{}/oak61\"".format(port1))
                                while True:
                                    data1 = encoded1.get().getData()
                                    server1.send_data(data1)
                        else:
                            print('No device found')

                    Further, about h26x artifacts, I now have doubt about bandwidth too. Since I have come to know about the ports are running on 1GBPS. and we have monitored traffic and its way less then capacity. So likely something else seems to be the issue. Over more, CPU also has good performance so processing should also not be the issue.
                    will share performance if it helps.