• DepthAI-v2

  • Error installing PyGObject in window for running RTSP streaming.

I have been trying to install PyGObject since long now. I have tried everything MSYS, Chocolatey, gtk-build (github) and Microsoft build tool.
But keep throwing the same error.

Collecting PyGObject
  Downloading PyGObject-3.46.0.tar.gz (723 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 723.4/723.4 kB 7.9 MB/s eta 0:00:00
  Installing build dependencies ... error
  error: subprocess-exited-with-error

  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [48 lines of output]
      Collecting setuptools
        Obtaining dependency information for setuptools from https://files.pythonhosted.org/packages/55/3a/5121b58b578a598b269537e09a316ad2a94fdd561a2c6eb75cd68578cc6b/setuptools-69.0.3-py3-none-any.whl.metadata
        Downloading setuptools-69.0.3-py3-none-any.whl.metadata (6.3 kB)
      Collecting wheel
        Obtaining dependency information for wheel from https://files.pythonhosted.org/packages/c7/c3/55076fc728723ef927521abaa1955213d094933dc36d4a2008d5101e1af5/wheel-0.42.0-py3-none-any.whl.metadata
        Downloading wheel-0.42.0-py3-none-any.whl.metadata (2.2 kB)
      Collecting pycairo
        Downloading pycairo-1.25.1.tar.gz (347 kB)
           ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 347.1/347.1 kB 5.3 MB/s eta 0:00:00
        Installing build dependencies: started
        Installing build dependencies: finished with status 'done'
        Getting requirements to build wheel: started
        Getting requirements to build wheel: finished with status 'done'
        Installing backend dependencies: started
        Installing backend dependencies: finished with status 'done'
        Preparing metadata (pyproject.toml): started
        Preparing metadata (pyproject.toml): finished with status 'done'
      Using cached setuptools-69.0.3-py3-none-any.whl (819 kB)
      Using cached wheel-0.42.0-py3-none-any.whl (65 kB)
      Building wheels for collected packages: pycairo
        Building wheel for pycairo (pyproject.toml): started
        Building wheel for pycairo (pyproject.toml): finished with status 'error'
        error: subprocess-exited-with-error

        × Building wheel for pycairo (pyproject.toml) did not run successfully.
        │ exit code: 1
        ╰─> [12 lines of output]
            running bdist_wheel
            running build
            running build_py
            creating build
            creating build/lib.msys-3.4.10.-x86_64-cpython-311
            creating build/lib.msys-3.4.10.-x86_64-cpython-311/cairo
            copying cairo/__init__.py -> build/lib.msys-3.4.10.-x86_64-cpython-311/cairo
            copying cairo/__init__.pyi -> build/lib.msys-3.4.10.-x86_64-cpython-311/cairo
            copying cairo/py.typed -> build/lib.msys-3.4.10.-x86_64-cpython-311/cairo
            running build_ext
            'pkg-config' not found.
            Command ['pkg-config', '--print-errors', '--exists', 'cairo >= 1.15.10']
            [end of output]

        note: This error originates from a subprocess, and is likely not a problem with pip.
        ERROR: Failed building wheel for pycairo
      Failed to build pycairo
      ERROR: Could not build wheels for pycairo, which is required to install pyproject.toml-based projects

      [notice] A new release of pip is available: 23.2.1 -> 24.0
      [notice] To update, run: pip install --upgrade pip
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

Hi @nikul
Difficult to say what the issue is, especially running on WIN. I checked the pycairo and it looks like it only supports python 3.8+ - https://pypi.org/project/pycairo/. Are you perhaps running on python3.12? It's not explicitly stated on which python versions it works.

Try looking online, this is not a depthai specific problem.

Thanks,
Jaka

5 days later

Following code is running but not streaming not streaming on URL

import depthai as dai
import time

# Start defining a pipeline
pipeline = dai.Pipeline()
# Define a source - color camera
cam = pipeline.create(dai.node.ColorCamera)
cam.initialControl.setManualFocus(0) # 0..255
cam.setFps(60)
cam.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1080_P)

# VideoEncoder
jpeg = pipeline.create(dai.node.VideoEncoder)
jpeg.setDefaultProfilePreset(cam.getFps(), dai.VideoEncoderProperties.Profile.H265_MAIN)

# Script node
script = pipeline.create(dai.node.Script)

script.setProcessor(dai.ProcessorType.LEON_CSS)
script.setScript("""
    import time
    import socket
    import fcntl
    import struct
    from socketserver import ThreadingMixIn
    from http.server import BaseHTTPRequestHandler, HTTPServer

    PORT = 8080

    def get_ip_address(ifname):
        s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
        return socket.inet_ntoa(fcntl.ioctl(
            s.fileno(),
            -1071617759,  # SIOCGIFADDR
            struct.pack('256s', ifname[:15].encode())
        )[20:24])

    class ThreadingSimpleServer(ThreadingMixIn, HTTPServer):
        pass

    class HTTPHandler(BaseHTTPRequestHandler):
        def do_GET(self):
            if self.path == '/':
                self.send_response(200)
                self.end_headers()
                self.wfile.write(b'<h1>[DepthAI] Hello, world!</h1><p>Click <a href="img">here</a> for an image</p>')
            elif self.path == '/img':
                try:
                    self.send_response(200)
                    self.send_header('Content-type', 'multipart/x-mixed-replace; boundary=--jpgboundary')
                    self.end_headers()
                    fpsCounter = 0
                    timeCounter = time.time()
                    while True:
                        jpegImage = node.io['jpeg'].get()
                        self.wfile.write("--jpgboundary".encode())
                        self.wfile.write(bytes([13, 10]))
                        self.send_header('Content-type', 'image/jpeg')
                        self.send_header('Content-length', str(len(jpegImage.getData())))
                        self.end_headers()
                        self.wfile.write(jpegImage.getData())
                        self.end_headers()

                        fpsCounter = fpsCounter + 1
                        if time.time() - timeCounter > 1:
                            node.warn(f'FPS: {fpsCounter}')
                            fpsCounter = 0
                            timeCounter = time.time()
                except Exception as ex:
                    node.warn(str(ex))

    with ThreadingSimpleServer(("", PORT), HTTPHandler) as httpd:
        node.warn(f"Serving at {get_ip_address('re0')}:{PORT}")
        httpd.serve_forever()
""")

# Connections
cam.video.link(jpeg.input)
jpeg.bitstream.link(script.inputs['jpeg'])


device_info = dai.DeviceInfo("0.0.0.0")
with dai.Device(pipeline,device_info) as device:
    while True:
        pass  # Keep the script running

I dont know what the problem is

Hi @nikul
Likely something wrong with the headers or data decoding. Also I'd lower the FPS to 10 for now.

Thanks,
Jaka

    jakaskerl
    Thank you for getting back.
    I have lower down FPS to 10. And still not able to debug the code,
    Any help in this matter will be appreciated.

    Thank you.

      Hi @nikul
      I'll have to debug this on monday since I don't have any poe device to test on currently.

      Thanks,
      Jaka

      @nikul
      You are encoding into H265 (not MJPEG) - and in http response you are writing jpgboundary and in header you have specified image/jpeg.
      This will, of course, not work, and I don't think you can just stream raw h264 bitstream to a browser and it will decode/display it. You would likely need to containerize it (eg mp4), so html5's video player could play the stream. The why is that h264/h265 is much more complex than mjpeg. Thoughts?
      Thanks, Erik

        erik
        Thank you.

        I got the RTSP streaming link however it is throwing error and i don't know how to debug this.

        Further, can I flash RTSP experiment/demo in to the device so it can stream it on stand alone. if yes how.

        Here is the link of experiment
        luxonis/depthai-experimentstree/master/gen2-rtsp-streaming

        Would really appreciate you expertise in this matter.
        Thank you.

          Hi @nikul ,
          Here's step by step tutorial:

          1. Install WLS 2
          2. RUn commands 
            sudo apt-get update
sudo apt-get upgrade
sudo apt-get install ffmpeg gstreamer-1.0 gir1.2-gst-rtsp-server-1.0 libgirepository1.0-dev gstreamer1.0-plugins-bad gstreamer1.0-plugins-good gstreamer1.0-plugins-base
          3. Install python requirements 
            opencv-python
numpy
depthai==2.24
PyGObject
          4. I went with standalone approach, so flashing TCP server streaming H265 bitstream to clients that connect to it. You can run this flashing script from Windows (no need for WSL). Note that you need latest bootlaoder flashed on device before flashing this script to the OAK POE camera: 
            import depthai as dai

# Start defining a pipeline
pipeline = dai.Pipeline()

camRgb = pipeline.createColorCamera()
camRgb.setIspScale(2,3)

videoEnc = pipeline.create(dai.node.VideoEncoder)
videoEnc.setDefaultProfilePreset(30, dai.VideoEncoderProperties.Profile.H265_MAIN)
camRgb.video.link(videoEnc.input)

script = pipeline.create(dai.node.Script)
script.setProcessor(dai.ProcessorType.LEON_CSS)

videoEnc.bitstream.link(script.inputs['frame'])
script.inputs['frame'].setBlocking(False)
script.inputs['frame'].setQueueSize(1)

script.setScript("""
import socket
import time

server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(("0.0.0.0", 5000))
server.listen()
node.warn("Server up")

while True:
    conn, client = server.accept()
    node.warn(f"Connected to client IP: {client}")
    try:
        while True:
            pck = node.io["frame"].get()
            data = pck.getData()
            ts = pck.getTimestamp()
            header = f"ABCDE " + str(ts.total_seconds()).ljust(18) + str(len(data)).ljust(8)
            # node.warn(f'>{header}<')
            conn.send(bytes(header, encoding='ascii'))
            conn.send(data)

    except Exception as e:
        node.warn("Client disconnected")
""")

(f, bl) = dai.DeviceBootloader.getFirstAvailableDevice()
bootloader = dai.DeviceBootloader(bl)
progress = lambda p : print(f'Flashing progress: {p*100:.1f}%')
bootloader.flash(progress, pipeline)
          5. Find IP of your camera that's running the TCP server (I just used device manager), then run this script within the WLS2: 
            #!/usr/bin/env python3

import threading

import gi
gi.require_version('Gst', '1.0')
gi.require_version('GstRtspServer', '1.0')
from gi.repository import Gst, GstRtspServer, GLib
import socket
import re
import cv2
import numpy as np

# Enter your own IP!
OAK_IP = "192.168.0.102"

class RtspSystem(GstRtspServer.RTSPMediaFactory):
    def __init__(self, **properties):
        super(RtspSystem, self).__init__(**properties)
        self.data = None
        self.launch_string = 'appsrc name=source is-live=true block=true format=GST_FORMAT_TIME ! h265parse ! rtph265pay name=pay0 config-interval=1 name=pay0 pt=96'

    def send_data(self, data):
        self.data = data

    def start(self):
        t = threading.Thread(target=self._thread_rtsp)
        t.start()

    def _thread_rtsp(self):
        loop = GLib.MainLoop()
        loop.run()

    def on_need_data(self, src, length):
        if self.data is not None:
            retval = src.emit('push-buffer', Gst.Buffer.new_wrapped(self.data.tobytes()))
            if retval != Gst.FlowReturn.OK:
                print(retval)

    def do_create_element(self, url):
        return Gst.parse_launch(self.launch_string)

    def do_configure(self, rtsp_media):
        self.number_frames = 0
        appsrc = rtsp_media.get_element().get_child_by_name('source')
        appsrc.connect('need-data', self.on_need_data)


class RTSPServer(GstRtspServer.RTSPServer):
    def __init__(self, **properties):
        super(RTSPServer, self).__init__(**properties)
        self.rtsp = RtspSystem()
        self.rtsp.set_shared(True)
        self.get_mount_points().add_factory("/preview", self.rtsp)
        self.attach(None)
        Gst.init(None)
        self.rtsp.start()

    def send_data(self, data):
        self.rtsp.send_data(data)

def get_frame(socket, size):
    bytes = socket.recv(4096)
    while True:
        read = 4096
        if size-len(bytes) < read:
            read = size-len(bytes)
        bytes += socket.recv(read)
        if size == len(bytes):
            return bytes

if __name__ == "__main__":
    server = RTSPServer()

    sock = socket.socket()
    sock.connect((OAK_IP, 5000))

    try:
        while True:
            header = str(sock.recv(32), encoding="ascii")
            chunks = re.split(' +', header)
            if chunks[0] == "ABCDE":
                # print(f">{header}<")
                ts = float(chunks[1])
                imgSize = int(chunks[2])
                img = get_frame(sock, imgSize)
                buf = np.frombuffer(img, dtype=np.byte)
                server.send_data(buf)
                # print(buf.shape, buf.size)
            if cv2.waitKey(1) == ord('q'):
                break
    except Exception as e:
        print("Error:", e)

    sock.close()
          6. Find IP of your WLS2 (ifconfig) which is now running RTSP server
          7. ???
            7.1 Profit!
          8. I used RTSP PLayer on Windows to verify that my RTSP (on WLS2) works as expected. Img below.

            erik
            Thank you very much for tutorial it clears lot of concepts for me,

            Its streaming : )

            :

            I am getting following error in the terminal sometimes.
            Error: negative buffersize in recv

            I have one more question Is it possible to flash everything into the device in way that it connects to the TCP server running on the camera, receives the H265 encoded video frames, and then streams them via an RTSP server. With host as device itself.

            for instance: rtsp:://192.168.0.102:8554/preview (Considering sample OAK_IP you have mention in tutorial)

              Hi @nikul ,
              Unfortunately, running RTSP on the device itself wouldn't be possible, as this feature hasn't (and likely won't) be implemented on the RVC2-based devices.
              BR, Erik

                erik
                Thank you for getting back @erik.
                I have one more question. Since have multiple camera. I want to run them on the same server. will I be able to run them through single script ?. having different endpoints or maybe different ports to access RTSP streaming.

                • erik replied to this.

                    nikul Yep that should work, are you experiencing any issues with that approach?

                        erik
                        Hi @erik ,
                        I am getting error while Flashing TCP on to the device.
                        It is able to gain access of the device however when it trying to flash, it causing error occurs.
                        same with device manager when I search in device manger I Have all of them available but when i click on it, device not found.

                        Thank you

                          erik
                          Thank you.
                          I do able to ping all the devices connected to the LAN.
                          I specified the IP and remove application through GUI. and it worked temporary. i will look more into it.

                          I have moved everything to lunux. for some reasons, streaming script is not working. By any chance will you be having idea if I may have missed some settings or anything else you want me to check.

                          Thank you.

                            Hi @nikul ,
                            Without any context / screenshots / terminal outputs / logs / anything, we aren't able to help much.

                              erik
                              Thank you for your prompt attention.

                              here the screenshot terminal and code snap.

                              Thank you.