depthai 3.9.0 · device OAK-D PoE (RVC2) · host Python 3.14, aarch64 (Jetson)
Gate.input documents that "If the Gate is Closed, messages received here are discarded/dropped". With an ImageManip downstream this holds for a small source and fails for a large one: nothing is discarded, every frame that passed the gate is retained and handed out in order, one per openGate(1), so each open returns a frame further behind real time than the last.
Repro
this repro shows the build up of stale frames for large image inputs. 1920x1200 works fine, 1920x1280 builds up stale frames.
import sys
import time
import depthai as dai
SIZE = (1920, 1280) # (1920, 1200) behaves correctly
CROP = 840
OUTPUT = 448
FPS = 5
OPENS = 6
GAP_S = 4.0
def main() -> None:
ip = sys.argv[1]
pipeline = dai.Pipeline(dai.Device(dai.DeviceInfo(ip)))
camera = pipeline.create(dai.node.Camera).build(dai.CameraBoardSocket.CAM_A)
source = camera.requestOutput(SIZE, dai.ImgFrame.Type.NV12, fps=FPS)
gate = pipeline.create(dai.node.Gate)
gate.initialConfig = dai.GateControl.closeGate()
source.link(gate.input)
control = gate.inputControl.createInputQueue()
manip = pipeline.create(dai.node.ImageManip)
manip.initialConfig.addCrop(
(SIZE[0] - CROP) // 2, (SIZE[1] - CROP) // 2, CROP, CROP
)
manip.initialConfig.setOutputSize(OUTPUT, OUTPUT)
gate.output.link(manip.inputImage)
out = manip.out.createOutputQueue()
pipeline.start()
print(f'depthai {dai.__version__}')
print(
f'source {SIZE[0]}x{SIZE[1]} at {FPS} fps, {SIZE[0] * SIZE[1] * 3 // 2:,} '
f'bytes NV12'
)
print(f'manip crop {CROP} -> {OUTPUT} ({CROP / OUTPUT:.3f}x)')
print(f'{OPENS} opens, {GAP_S:g}s apart, each openGate(1)\n')
print(f'{"open":>4} {"seq":>4} {"behind":>9}')
time.sleep(2.0)
out.tryGetAll()
for index in range(1, OPENS + 1):
requested_at = dai.Clock.now()
control.send(dai.GateControl.openGate(1))
frame = None
while frame is None:
frame = out.tryGet()
time.sleep(0.01)
behind = (requested_at - frame.getTimestamp()).total_seconds()
print(f'{index:>4} {frame.getSequenceNum():>4} {behind:>+8.3f}s')
time.sleep(GAP_S)
print('\nbehind should stay within a sensor period (0.2s) and seq should')
print(f'advance by about {round(GAP_S * FPS)} per open, not by 1.')
if __name__ == '__main__':
main()
Observed
At 5 fps with 4 s between opens, behind is how long before its own open the frame was exposed:
At resolution 1920x1200, everything works fine
depthai 3.9.0
source 1920x1200 at 5 fps, 3,456,000 bytes NV12
manip crop 840 -> 448 (1.875x)
6 opens, 4s apart, each openGate(1)
open seq behind
1 9 +0.159s
2 30 +0.021s
3 50 +0.083s
4 70 +0.145s
5 90 +0.207s
6 111 +0.069s
At resolution 1920x1280 and above, the pipeline builds up stale frames
depthai 3.9.0
source 1920x1280 at 5 fps, 3,686,400 bytes NV12
manip crop 840 -> 448 (1.875x)
6 opens, 4s apart, each openGate(1)
open seq behind
1 9 +0.159s
2 10 +4.020s
3 11 +7.881s
4 12 +11.743s
5 13 +15.604s
6 31 +16.066s
