Hi,
We are using Luxonis (OAK-D Pro) cameras for robotics applications but are finding the latency to be difficult to deal with. We are using USB-3 and high speed cables, have set maxUsbSpeed=SUPER_PLUS, but as the frame size (set via setPreviewSize()) scales up, the latency does as well, which is unexpected given that there is more than enough bandwidth on the USB bus.
For example, at the default preview size of 300x300, latency is about 31 ms on my machine but at 1280x720, this balloons to 390 ms. Note that 390/31 = 12.6 and ((1280 x 720) / (300 x 300)) = 10.2.
Where in the pipeline is this latency incurred? Is there some sort of processing happening that can be disabled? There must be a way to get a reasonable latency at full frame resolution.
Thank you,
Bart
Here is the code (based on a Luxonis sample):
#!/usr/bin/env python3
import cv2
import depthai as dai
# Create pipeline
pipeline = dai.Pipeline()
pipeline.setXLinkChunkSize(0)
# Define source and output
camRgb = pipeline.create(dai.node.ColorCamera)
xoutRgb = pipeline.create(dai.node.XLinkOut)
xoutRgb.setStreamName("rgb")
# Properties
camRgb.setPreviewSize(1280, 720)
camRgb.setInterleaved(False)
camRgb.setColorOrder(dai.ColorCameraProperties.ColorOrder.RGB)
camRgb.setResolution(dai.ColorCameraProperties.SensorResolution.THE_720_P)
# Linking
camRgb.preview.link(xoutRgb.input)
# Connect to device and start pipeline
with dai.Device(pipeline, maxUsbSpeed=dai.UsbSpeed.SUPER_PLUS) as device:
print('Connected cameras:', device.getConnectedCameraFeatures())
# Print out usb speed
print('Usb speed:', device.getUsbSpeed().name)
# Bootloader version
if device.getBootloaderVersion() is not None:
print('Bootloader version:', device.getBootloaderVersion())
# Device name
print('Device name:', device.getDeviceName(), ' Product name:', device.getProductName())
# Output queue will be used to get the rgb frames from the output defined above
qRgb = device.getOutputQueue(name="rgb", maxSize=1, blocking=False)
while True:
inRgb = qRgb.get() # blocking call, will wait until a new data has arrived
latency_ms = (dai.Clock.now() - inRgb.getTimestamp()).total_seconds() * 1000
print(latency_ms)
# Retrieve 'bgr' (opencv format) frame
cv2.imshow("rgb", inRgb.getCvFrame())
if cv2.waitKey(1) == ord('q'):
break
