• Edited

Hi,

We are using Luxonis (OAK-D Pro) cameras for robotics applications but are finding the latency to be difficult to deal with. We are using USB-3 and high speed cables, have set maxUsbSpeed=SUPER_PLUS, but as the frame size (set via setPreviewSize()) scales up, the latency does as well, which is unexpected given that there is more than enough bandwidth on the USB bus.

For example, at the default preview size of 300x300, latency is about 31 ms on my machine but at 1280x720, this balloons to 390 ms. Note that 390/31 = 12.6 and ((1280 x 720) / (300 x 300)) = 10.2.

Where in the pipeline is this latency incurred? Is there some sort of processing happening that can be disabled? There must be a way to get a reasonable latency at full frame resolution.

Thank you,

Bart

Here is the code (based on a Luxonis sample):

#!/usr/bin/env python3

import cv2
import depthai as dai

# Create pipeline
pipeline = dai.Pipeline()

pipeline.setXLinkChunkSize(0)

# Define source and output
camRgb = pipeline.create(dai.node.ColorCamera)
xoutRgb = pipeline.create(dai.node.XLinkOut)

xoutRgb.setStreamName("rgb")

# Properties
camRgb.setPreviewSize(1280, 720)
camRgb.setInterleaved(False)
camRgb.setColorOrder(dai.ColorCameraProperties.ColorOrder.RGB)
camRgb.setResolution(dai.ColorCameraProperties.SensorResolution.THE_720_P)

# Linking
camRgb.preview.link(xoutRgb.input)

# Connect to device and start pipeline
with dai.Device(pipeline, maxUsbSpeed=dai.UsbSpeed.SUPER_PLUS) as device:

    print('Connected cameras:', device.getConnectedCameraFeatures())
    # Print out usb speed
    print('Usb speed:', device.getUsbSpeed().name)
    # Bootloader version
    if device.getBootloaderVersion() is not None:
        print('Bootloader version:', device.getBootloaderVersion())
    # Device name
    print('Device name:', device.getDeviceName(), ' Product name:', device.getProductName())

    # Output queue will be used to get the rgb frames from the output defined above
    qRgb = device.getOutputQueue(name="rgb", maxSize=1, blocking=False)

    while True:
        inRgb = qRgb.get()  # blocking call, will wait until a new data has arrived

        latency_ms = (dai.Clock.now() - inRgb.getTimestamp()).total_seconds() * 1000
        print(latency_ms)

        # Retrieve 'bgr' (opencv format) frame
        cv2.imshow("rgb", inRgb.getCvFrame())

        if cv2.waitKey(1) == ord('q'):
            break

    Bart
    Hmm, works fine for me on OAK-D Pro

    36.553000000000004
    36.302
    36.554
    36.422999999999995
    36.278999999999996
    36.401999999999994
    36.365
    36.653

    Try swapping out the cable or replugging the device - SUPER should be more than enough for this usecase.

    Thanks,
    Jaka