Impact of Latency on Practical Applications

Hi YaoHui ,
I haven't been able to repro this issue - because OpenVINO is PITA to work with - and I haven't come to any conclusion. But your depthi tests seems what I would expect as well, and results from openvino are a bit far-fetched, at least for send-to-receive result (so time since when you sent an image to the device to the time when NN results are returned to the host machine).
Thanks, Erik

Hi YaoHui ,
If I understand correctly; instead of computing 3D coordinates on device (with Spatial Detection Network) you are just performing object detection on-device, and stream both results and depth to host and you calculated spatial coords on the host to reduce latency?
What stereo depth modes/postprocessing filters do you have enabled? Usually, they add additional delay.
Thanks, Erik

Hi YaoHui ,
I have tried locally as well, these are my results for 400P LR-check + Subpixel:

FPS: 80.0, Latency: 24.03 ms, Average latency: 23.87 ms, Std: 0.85
FPS: 80.0, Latency: 23.96 ms, Average latency: 23.87 ms, Std: 0.85
FPS: 80.0, Latency: 23.76 ms, Average latency: 23.87 ms, Std: 0.85

I have used this script:

import depthai as dai
import numpy as np
from depthai_sdk import FPSHandler

# Create pipeline
pipeline = dai.Pipeline()
# This might improve reducing the latency on some systems
pipeline.setXLinkChunkSize(0)

FPS = 80
# Define source and output
monoLeft = pipeline.create(dai.node.MonoCamera)
monoLeft.setResolution(dai.MonoCameraProperties.SensorResolution.THE_400_P)
monoLeft.setFps(FPS)
monoLeft.setBoardSocket(dai.CameraBoardSocket.LEFT)

monoRight = pipeline.create(dai.node.MonoCamera)
monoRight.setResolution(dai.MonoCameraProperties.SensorResolution.THE_400_P)
monoRight.setFps(FPS)
monoRight.setBoardSocket(dai.CameraBoardSocket.RIGHT)

stereo = pipeline.create(dai.node.StereoDepth)
# stereo.setDefaultProfilePreset(dai.node.StereoDepth.PresetMode.HIGH_DENSITY)
# stereo.initialConfig.setMedianFilter(dai.MedianFilter.KERNEL_7x7)
stereo.setLeftRightCheck(True)
stereo.setExtendedDisparity(False)
stereo.setSubpixel(True)

# Linking
monoLeft.out.link(stereo.left)
monoRight.out.link(stereo.right)

xout = pipeline.create(dai.node.XLinkOut)
xout.setStreamName('out')
stereo.depth.link(xout.input)

fps = FPSHandler()

# Connect to device and start pipeline
with dai.Device(pipeline) as device:
    print(device.getUsbSpeed())
    q = device.getOutputQueue(name="out", )
    diffs = np.array([])
    while True:
        imgFrame = q.get()
        fps.nextIter()
        # Latency in miliseconds 
        latencyMs = (dai.Clock.now() - imgFrame.getTimestamp()).total_seconds() * 1000
        diffs = np.append(diffs, latencyMs)
        print('FPS: {:.1f}, Latency: {:.2f} ms, Average latency: {:.2f} ms, Std: {:.2f}'.format(fps.fps(), latencyMs, np.average(diffs), np.std(diffs)))

Could you confirm the same on your side?
Thanks, Erik

YaoHui yes that makes sense - what's the latency of your NN model? You could use a similar script, just replace StereoDepth node with NeuralNetwork node.
Thanks, Erik

Hi YaoHui ,
The SpatialLocationCalculatorData does support getTimestamp() (see here), it was added 3 months ago, so it might be that you have an older version of depthai which doesn't support it.
Thanks, Erik

YaoHui Latency would be time between capturing the frame - it's timestamp (see when timstamp is captured here) and when the message has arrived to the host computer. For NN/detections/spatial detections, result message will have the same timestmap/seq num as the frame it was inferenced upon.