Running this sample code on oak d pro. I added print(f"Latency is {benchmarkReport.averageLatency}")at the end to print latency and noticed it goes up over time:

FPS is 66.89564514160156
Latency is 1.0084450244903564
FPS is 67.15149688720703
Latency is 2.5158498287200928
FPS is 67.58968353271484
Latency is 4.011687755584717
FPS is 67.43702697753906
Latency is 5.505953788757324
FPS is 67.24176025390625
Latency is 7.0066237449646
FPS is 67.24274444580078
Latency is 8.508944511413574
FPS is 67.22396087646484
Latency is 10.011377334594727
FPS is 67.23606872558594
Latency is 11.513825416564941
FPS is 67.29620361328125
Latency is 13.015336036682129
FPS is 67.15917205810547
Latency is 14.517755508422852
FPS is 67.14630126953125
Latency is 16.022001266479492

So I thought maybe the inputs to the net are piling up and added

neuralNetwork.input.setBlocking(False)
neuralNetwork.input.setMaxSize(1)

But not only did that make it worse, it also drop kicked my FPS:

FPS is 19.345617294311523
Latency is 2.9978113174438477
FPS is 19.30980110168457
Latency is 8.227405548095703
FPS is 19.361085891723633
Latency is 13.449291229248047
FPS is 19.338600158691406
Latency is 18.669588088989258
FPS is 19.33113670349121
Latency is 23.889989852905273
FPS is 19.31963539123535
Latency is 29.11971664428711
FPS is 19.342355728149414
Latency is 34.34376907348633
FPS is 19.331363677978516
Latency is 39.569026947021484
FPS is 19.361083984375
Latency is 44.78590393066406
FPS is 19.369352340698242
Latency is 50.0016975402832

What is happening here?

    Arthur changed the title to Benchmark latency increases over time .

    Arthur
    When you use BenchmarkOut node, only the first frame in the input is memorized by the node. That same frame is then repeatedly sent into the NN node for inference. The timestamp of that frame stays constant - so the latency increases over time.

    Thanks,
    Jaka