Low framerate when playing OAK-D PRO stream on USB2.0

jbalao

Hey guys!

I am trying to get a fluid stream of my OAK-D PRO, but it seems that I always get a slow framerate when using USB2.0.

I am using the example in depthai-python (depthai-python/examples/UVC/uvc_rgb.py ) to test the stream and then running a simple gstreamer pipeline such as:

gst-launch-1.0 v4l2src device=/dev/video0 ! videoconvert ! autovideosink

However, I can only get 30+ fps, when using USB3.0.

I've tried forcing usb2.0 mode and also:

with dai.Device(pipeline, maxUsbSpeed=dai.UsbSpeed.HIGH) as device:

But nothing worked so far. Is there a way to reduce the stream quality (less than 1080p) or bitrate further so that it can be used in USB2.0 with a faster framerate?

erik

@jbalao what's the FPS you expect via usb2 (480mbps)? You can do both, but UVC node requires NV12 frames, and doesn't support encoded frames. It should support lower quality tho, you can specify the size in the code;

    config = dai.Device.Config()
    # config.board.uvc = dai.BoardConfig.UVC()  # enable default 1920x1080 NV12
    config.board.uvc = dai.BoardConfig.UVC(1920, 1080) # Set lower res here and when setting video size
    config.board.uvc.frameType = dai.ImgFrame.Type.NV12
    # config.board.uvc.cameraName = "My Custom Cam"
    pipeline.setBoardConfig(config.board)

(from here: https://docs.luxonis.com/software/depthai/examples/uvc_rgb/#Source%20code )

jbalao

Hi erik, thanks for the help.
25-30fps would be fine for my use case.

I've tried adapting the code indepthai-python/examples/UVC/uvc_rgb.pyto the parameters you provided, but nothing changed in terms of framerate, it's still around 1-2fps (on USB2.0).

I've tried instead to run other examples such as depthai-python/examples/ColorCamera/rgb_preview.py

#!/usr/bin/env python3

import cv2
import depthai as dai

# Create pipeline
pipeline = dai.Pipeline()

# Define source and output
camRgb = pipeline.create(dai.node.ColorCamera)
xoutRgb = pipeline.create(dai.node.XLinkOut)

xoutRgb.setStreamName("rgb")

# Properties
camRgb.setPreviewSize(1280, 720)
camRgb.setInterleaved(False)
camRgb.setColorOrder(dai.ColorCameraProperties.ColorOrder.RGB)

# Linking
camRgb.preview.link(xoutRgb.input)

# Connect to device and start pipeline
with dai.Device(pipeline) as device:

    print('Connected cameras:', device.getConnectedCameraFeatures())
    # Print out usb speed
    print('Usb speed:', device.getUsbSpeed().name)
    # Bootloader version
    if device.getBootloaderVersion() is not None:
        print('Bootloader version:', device.getBootloaderVersion())
    # Device name
    print('Device name:', device.getDeviceName(), ' Product name:', device.getProductName())

    # Output queue will be used to get the rgb frames from the output defined above
    qRgb = device.getOutputQueue(name="rgb", maxSize=4, blocking=False)

    while True:
        inRgb = qRgb.get()  # blocking call, will wait until a new data has arrived

        # Retrieve 'bgr' (opencv format) frame
        cv2.imshow("rgb", inRgb.getCvFrame())

        if cv2.waitKey(1) == ord('q'):
            break

I can get a decent ~25fps, but now I would like to have it encoded over RTSP, so that it can be played in VLC or GStreamer.

Do you have any ideias on how I can do this?

I've noticed you can get encoded data from the camera as shown in depthai-python/examples/rgb_encoding/rgb_encoding.pymaybe there is a way to convert this to RTSP instead of writing to a file?

erik

@jbalao best to check out this example: luxonis/depthai-experimentstree/master/gen2-rtsp-streaming

jbalao

Thanks @eric-soderberg

I will return with updates once I have a chance to test this.