Retrieve data from OAK-D

pedro-UCA

I am working on implement yolov8 segmentation in the OAK-D, I have done it and it works but even when the video output is more or less fluid (about 7-10fps) it have a latency of about 7-10s.

I have been trying to make this faster but I think the problem is the getLayerFp16retrieved the data in a list and I need it on a numpy array, the conversion creates a bottleneck on the code. Or at least I think this is the problem.

I have measured the times and post it on stack overflow, with the code I am using here:

https://stackoverflow.com/questions/78153689/numpy-array-slow-with-large-list

But I will post here in case someone can help me.

jakaskerl

pedro-UCA Or at least I think this is the problem.

Try just printing the message when it is received by the host. This will tell you if the latency stems from the device.
Not sure how you ran the trace mode, but try with env variable: DEPTHAI_LEVEL=TRACE. This should also give you the info on how much time each part of the processing took.

Thanks,
Jaka

pedro-UCA

Hi again, I have set this two additional lines and I reduce latency from 7s to 4.5s

pipeline.setXLinkChunkSize(0)

cam_rgb.setResolution(dai.ColorCameraProperties.SensorResolution.THE_720_P)

But I think this is still too high, this is the log of some loops using TRACE mode:

[14442C1031CDA1D000] [2.1.4] [1710405988.469] [host] [trace] Log vector decoded, size: 2

[14442C1031CDA1D000] [2.1.4] [22.741] [system] [trace] EV:1,S:0,IDS:12,IDD:0,TSS:22,TSN:741034818

[14442C1031CDA1D000] [2.1.4] [22.741] [system] [trace] EV:1,S:1,IDS:12,IDD:0,TSS:22,TSN:741090137

[14442C1031CDA1D000] [2.1.4] [1710405988.480] [host] [trace] Log vector decoded, size: 8

[14442C1031CDA1D000] [2.1.4] [22.741] [system] [trace] EV:1,S:0,IDS:13,IDD:0,TSS:22,TSN:741475664

[14442C1031CDA1D000] [2.1.4] [22.741] [system] [trace] EV:1,S:1,IDS:13,IDD:0,TSS:22,TSN:741510695

[14442C1031CDA1D000] [2.1.4] [22.741] [system] [trace] EV:1,S:0,IDS:9,IDD:0,TSS:22,TSN:741485277

[14442C1031CDA1D000] [2.1.4] [22.741] [system] [trace] EV:1,S:1,IDS:9,IDD:0,TSS:22,TSN:741549017

[14442C1031CDA1D000] [2.1.4] [22.747] [system] [trace] EV:0,S:0,IDS:5,IDD:12,TSS:22,TSN:747914290

[14442C1031CDA1D000] [2.1.4] [22.748] [system] [trace] EV:0,S:1,IDS:5,IDD:12,TSS:22,TSN:747990687

[14442C1031CDA1D000] [2.1.4] [22.747] [system] [trace] EV:0,S:0,IDS:5,IDD:9,TSS:22,TSN:747428365

[14442C1031CDA1D000] [2.1.4] [22.747] [system] [trace] EV:0,S:1,IDS:5,IDD:9,TSS:22,TSN:747490112

[14442C1031CDA1D000] [2.1.4] [1710405988.501] [host] [trace] Log vector decoded, size: 3

[14442C1031CDA1D000] [2.1.4] [22.773] [system] [info] Memory Usage - DDR: 91.93 / 333.46 MiB, CMX: 2.39 / 2.50 MiB, LeonOS Heap: 22.23 / 82.56 MiB, LeonRT Heap: 4.56 / 40.68 MiB / NOC ddr: 1276 MB/s

[14442C1031CDA1D000] [2.1.4] [22.773] [system] [info] Temperatures - Average: 33.75C, CSS: 34.40C, MSS 32.97C, UPA: 34.40C, DSS: 33.21C

[14442C1031CDA1D000] [2.1.4] [22.773] [system] [info] Cpu Usage - LeonOS 14.13%, LeonRT: 7.87%

[14442C1031CDA1D000] [2.1.4] [1710405988.618] [host] [trace] Log vector decoded, size: 2

[14442C1031CDA1D000] [2.1.4] [22.890] [system] [trace] EV:0,S:0,IDS:10,IDD:13,TSS:22,TSN:890136011

[14442C1031CDA1D000] [2.1.4] [22.890] [system] [trace] EV:0,S:1,IDS:10,IDD:13,TSS:22,TSN:890193877

Latency: 5306.07 ms, Average latency: 4701.09 ms, Std: 1514.84

----------

[14442C1031CDA1D000] [2.1.4] [1710405988.652] [host] [trace] Log vector decoded, size: 2

[14442C1031CDA1D000] [2.1.4] [22.924] [system] [trace] EV:1,S:0,IDS:12,IDD:0,TSS:22,TSN:924300492

[14442C1031CDA1D000] [2.1.4] [22.924] [system] [trace] EV:1,S:1,IDS:12,IDD:0,TSS:22,TSN:924357795

[14442C1031CDA1D000] [2.1.4] [1710405988.664] [host] [trace] Log vector decoded, size: 8

[14442C1031CDA1D000] [2.1.4] [22.924] [system] [trace] EV:1,S:0,IDS:13,IDD:0,TSS:22,TSN:924740267

[14442C1031CDA1D000] [2.1.4] [22.924] [system] [trace] EV:1,S:1,IDS:13,IDD:0,TSS:22,TSN:924776645

[14442C1031CDA1D000] [2.1.4] [22.924] [system] [trace] EV:1,S:0,IDS:9,IDD:0,TSS:22,TSN:924748867

[14442C1031CDA1D000] [2.1.4] [22.924] [system] [trace] EV:1,S:1,IDS:9,IDD:0,TSS:22,TSN:924813305

[14442C1031CDA1D000] [2.1.4] [22.931] [system] [trace] EV:0,S:0,IDS:5,IDD:12,TSS:22,TSN:931131238

[14442C1031CDA1D000] [2.1.4] [22.931] [system] [trace] EV:0,S:1,IDS:5,IDD:12,TSS:22,TSN:931184422

[14442C1031CDA1D000] [2.1.4] [22.930] [system] [trace] EV:0,S:0,IDS:5,IDD:9,TSS:22,TSN:930652838

[14442C1031CDA1D000] [2.1.4] [22.930] [system] [trace] EV:0,S:1,IDS:5,IDD:9,TSS:22,TSN:930717612

[14442C1031CDA1D000] [2.1.4] [1710405988.804] [host] [trace] Log vector decoded, size: 2

[14442C1031CDA1D000] [2.1.4] [23.075] [system] [trace] EV:0,S:0,IDS:10,IDD:13,TSS:23,TSN:75816025

[14442C1031CDA1D000] [2.1.4] [23.075] [system] [trace] EV:0,S:1,IDS:10,IDD:13,TSS:23,TSN:75871997

Latency: 5291.30 ms, Average latency: 4706.71 ms, Std: 1508.70

----------

[14442C1031CDA1D000] [2.1.4] [1710405988.838] [host] [trace] Log vector decoded, size: 1

[14442C1031CDA1D000] [2.1.4] [23.109] [system] [trace] EV:1,S:0,IDS:12,IDD:0,TSS:23,TSN:109845290

[14442C1031CDA1D000] [2.1.4] [1710405988.850] [host] [trace] Log vector decoded, size: 9

[14442C1031CDA1D000] [2.1.4] [23.109] [system] [trace] EV:1,S:1,IDS:12,IDD:0,TSS:23,TSN:109900065

[14442C1031CDA1D000] [2.1.4] [23.110] [system] [trace] EV:1,S:0,IDS:13,IDD:0,TSS:23,TSN:110283740

[14442C1031CDA1D000] [2.1.4] [23.110] [system] [trace] EV:1,S:1,IDS:13,IDD:0,TSS:23,TSN:110320607

[14442C1031CDA1D000] [2.1.4] [23.110] [system] [trace] EV:1,S:0,IDS:9,IDD:0,TSS:23,TSN:110292884

[14442C1031CDA1D000] [2.1.4] [23.110] [system] [trace] EV:1,S:1,IDS:9,IDD:0,TSS:23,TSN:110359500

[14442C1031CDA1D000] [2.1.4] [23.116] [system] [trace] EV:0,S:0,IDS:5,IDD:12,TSS:23,TSN:116684230

[14442C1031CDA1D000] [2.1.4] [23.116] [system] [trace] EV:0,S:1,IDS:5,IDD:12,TSS:23,TSN:116756088

[14442C1031CDA1D000] [2.1.4] [23.116] [system] [trace] EV:0,S:0,IDS:5,IDD:9,TSS:23,TSN:116196687

[14442C1031CDA1D000] [2.1.4] [23.116] [system] [trace] EV:0,S:1,IDS:5,IDD:9,TSS:23,TSN:116258317

[14442C1031CDA1D000] [2.1.4] [1710405988.989] [host] [trace] Log vector decoded, size: 2

[14442C1031CDA1D000] [2.1.4] [23.261] [system] [trace] EV:0,S:0,IDS:10,IDD:13,TSS:23,TSN:261320618

[14442C1031CDA1D000] [2.1.4] [23.261] [system] [trace] EV:0,S:1,IDS:10,IDD:13,TSS:23,TSN:261379184

Latency: 5324.18 ms, Average latency: 4712.54 ms, Std: 1502.75

----------

[14442C1031CDA1D000] [2.1.4] [1710405989.038] [host] [trace] Log vector decoded, size: 2

[14442C1031CDA1D000] [2.1.4] [23.309] [system] [trace] EV:1,S:0,IDS:12,IDD:0,TSS:23,TSN:309837715

[14442C1031CDA1D000] [2.1.4] [23.309] [system] [trace] EV:1,S:1,IDS:12,IDD:0,TSS:23,TSN:309892408

[14442C1031CDA1D000] [2.1.4] [1710405989.049] [host] [trace] Log vector decoded, size: 8

[14442C1031CDA1D000] [2.1.4] [23.310] [system] [trace] EV:1,S:0,IDS:13,IDD:0,TSS:23,TSN:310264970

[14442C1031CDA1D000] [2.1.4] [23.310] [system] [trace] EV:1,S:1,IDS:13,IDD:0,TSS:23,TSN:310297530

[14442C1031CDA1D000] [2.1.4] [23.310] [system] [trace] EV:1,S:0,IDS:9,IDD:0,TSS:23,TSN:310275695

[14442C1031CDA1D000] [2.1.4] [23.310] [system] [trace] EV:1,S:1,IDS:9,IDD:0,TSS:23,TSN:310340588

[14442C1031CDA1D000] [2.1.4] [23.316] [system] [trace] EV:0,S:0,IDS:5,IDD:12,TSS:23,TSN:316689660

[14442C1031CDA1D000] [2.1.4] [23.316] [system] [trace] EV:0,S:1,IDS:5,IDD:12,TSS:23,TSN:316757430

[14442C1031CDA1D000] [2.1.4] [23.316] [system] [trace] EV:0,S:0,IDS:5,IDD:9,TSS:23,TSN:316203207

[14442C1031CDA1D000] [2.1.4] [23.316] [system] [trace] EV:0,S:1,IDS:5,IDD:9,TSS:23,TSN:316269608

[14442C1031CDA1D000] [2.1.4] [1710405989.190] [host] [trace] Log vector decoded, size: 2

[14442C1031CDA1D000] [2.1.4] [23.462] [system] [trace] EV:0,S:0,IDS:10,IDD:13,TSS:23,TSN:462059922

[14442C1031CDA1D000] [2.1.4] [23.462] [system] [trace] EV:0,S:1,IDS:10,IDD:13,TSS:23,TSN:462119964

Latency: 5309.95 ms, Average latency: 4718.12 ms, Std: 1496.82

----------

[14442C1031CDA1D000] [2.1.4] [1710405989.225] [host] [trace] Log vector decoded, size: 2

[14442C1031CDA1D000] [2.1.4] [23.496] [system] [trace] EV:1,S:0,IDS:12,IDD:0,TSS:23,TSN:496434978

[14442C1031CDA1D000] [2.1.4] [23.496] [system] [trace] EV:1,S:1,IDS:12,IDD:0,TSS:23,TSN:496490548

[14442C1031CDA1D000] [2.1.4] [1710405989.235] [host] [trace] Log vector decoded, size: 8

[14442C1031CDA1D000] [2.1.4] [23.496] [system] [trace] EV:1,S:0,IDS:13,IDD:0,TSS:23,TSN:496858228

[14442C1031CDA1D000] [2.1.4] [23.496] [system] [trace] EV:1,S:1,IDS:13,IDD:0,TSS:23,TSN:496889334

[14442C1031CDA1D000] [2.1.4] [23.496] [system] [trace] EV:1,S:0,IDS:9,IDD:0,TSS:23,TSN:496867618

[14442C1031CDA1D000] [2.1.4] [23.496] [system] [trace] EV:1,S:1,IDS:9,IDD:0,TSS:23,TSN:496930867

[14442C1031CDA1D000] [2.1.4] [23.502] [system] [trace] EV:0,S:0,IDS:5,IDD:9,TSS:23,TSN:502670720

[14442C1031CDA1D000] [2.1.4] [23.502] [system] [trace] EV:0,S:1,IDS:5,IDD:9,TSS:23,TSN:502731420

[14442C1031CDA1D000] [2.1.4] [23.503] [system] [trace] EV:0,S:0,IDS:5,IDD:12,TSS:23,TSN:503156534

[14442C1031CDA1D000] [2.1.4] [23.503] [system] [trace] EV:0,S:1,IDS:5,IDD:12,TSS:23,TSN:503258402

[14442C1031CDA1D000] [2.1.4] [1710405989.375] [host] [trace] Log vector decoded, size: 2

[14442C1031CDA1D000] [2.1.4] [23.646] [system] [trace] EV:0,S:0,IDS:10,IDD:13,TSS:23,TSN:646879148

[14442C1031CDA1D000] [2.1.4] [23.646] [system] [trace] EV:0,S:1,IDS:10,IDD:13,TSS:23,TSN:646938077

Latency: 5321.70 ms, Average latency: 4723.71 ms, Std: 1490.99

----------

[14442C1031CDA1D000] [2.1.4] [1710405989.401] [host] [trace] Log vector decoded, size: 2

[14442C1031CDA1D000] [2.1.4] [23.673] [system] [trace] EV:1,S:0,IDS:12,IDD:0,TSS:23,TSN:673073705

[14442C1031CDA1D000] [2.1.4] [23.673] [system] [trace] EV:1,S:1,IDS:12,IDD:0,TSS:23,TSN:673129168

[14442C1031CDA1D000] [2.1.4] [1710405989.412] [host] [trace] Log vector decoded, size: 8

[14442C1031CDA1D000] [2.1.4] [23.673] [system] [trace] EV:1,S:0,IDS:13,IDD:0,TSS:23,TSN:673511488

[14442C1031CDA1D000] [2.1.4] [23.673] [system] [trace] EV:1,S:1,IDS:13,IDD:0,TSS:23,TSN:673550061

[14442C1031CDA1D000] [2.1.4] [23.673] [system] [trace] EV:1,S:0,IDS:9,IDD:0,TSS:23,TSN:673520740

[14442C1031CDA1D000] [2.1.4] [23.673] [system] [trace] EV:1,S:1,IDS:9,IDD:0,TSS:23,TSN:673585518

[14442C1031CDA1D000] [2.1.4] [23.680] [system] [trace] EV:0,S:0,IDS:5,IDD:12,TSS:23,TSN:679995647

[14442C1031CDA1D000] [2.1.4] [23.680] [system] [trace] EV:0,S:1,IDS:5,IDD:12,TSS:23,TSN:680071327

[14442C1031CDA1D000] [2.1.4] [23.679] [system] [trace] EV:0,S:0,IDS:5,IDD:9,TSS:23,TSN:679509347

[14442C1031CDA1D000] [2.1.4] [23.679] [system] [trace] EV:0,S:1,IDS:5,IDD:9,TSS:23,TSN:679570595

[14442C1031CDA1D000] [2.1.4] [1710405989.502] [host] [trace] Log vector decoded, size: 3

[14442C1031CDA1D000] [2.1.4] [23.774] [system] [info] Memory Usage - DDR: 91.93 / 333.46 MiB, CMX: 2.39 / 2.50 MiB, LeonOS Heap: 22.23 / 82.56 MiB, LeonRT Heap: 4.56 / 40.68 MiB / NOC ddr: 1283 MB/s

[14442C1031CDA1D000] [2.1.4] [23.774] [system] [info] Temperatures - Average: 33.45C, CSS: 34.40C, MSS 33.21C, UPA: 33.45C, DSS: 32.73C

[14442C1031CDA1D000] [2.1.4] [23.774] [system] [info] Cpu Usage - LeonOS 13.28%, LeonRT: 7.27%

[14442C1031CDA1D000] [2.1.4] [1710405989.550] [host] [trace] Log vector decoded, size: 2

[14442C1031CDA1D000] [2.1.4] [23.822] [system] [trace] EV:0,S:0,IDS:10,IDD:13,TSS:23,TSN:822767392

[14442C1031CDA1D000] [2.1.4] [23.822] [system] [trace] EV:0,S:1,IDS:10,IDD:13,TSS:23,TSN:822825018

Latency: 5303.35 ms, Average latency: 4729.03 ms, Std: 1485.17

----------

Where "----------" marks the end of a loop, have someone some clue of what I am doing wrong? I do not understand what are the outputs of the TRACE mode exactly.

pedro-UCA

Thanks for your reply, right now I use this:

with dai.Device(pipeline) as device:
    # From this point, the Device will be in "running" mode and will start sending data via XLink
    device.setLogLevel(dai.LogLevel.TRACE)
    device.setLogOutputLevel(dai.LogLevel.TRACE)

And do you want that I try this?
os.environ['DEPTHAI_LEVEL'] = 'TRACE'
It did not output anything if I coment setLogLevel and setLogOutputLevel

Putting the print of latency in like these returns the same (about 4.2 of latency):

import cv2
import numpy as np
import depthai as dai
import time
from YOLOSeg import YOLOSeg
import os

os.environ['DEPTHAI_LEVEL'] = 'TRACE' 

pathYoloBlob = "./yolov8n-seg.blob"

pipeline = dai.Pipeline()
pipeline.setXLinkChunkSize(0) 
cam_rgb = pipeline.createColorCamera()
cam_rgb.setPreviewSize(640, 640)  
cam_rgb.setInterleaved(False)

nn = pipeline.create(dai.node.NeuralNetwork)
nn.setBlobPath(pathYoloBlob)

cam_rgb.preview.link(nn.input)

xout_rgb = pipeline.createXLinkOut()
xout_rgb.setStreamName("rgb")
cam_rgb.preview.link(xout_rgb.input)

xout_nn_yolo = pipeline.createXLinkOut()
xout_nn_yolo.setStreamName("nn_yolo")
nn.out.link(xout_nn_yolo.input)

# Pipeline is now finished, and we need to find an available device to run our pipeline
# we are using context manager here that will dispose the device after we stop using it
with dai.Device(pipeline) as device:
    # From this point, the Device will be in "running" mode and will start sending data via XLink
    #device.setLogLevel(dai.LogLevel.TRACE)
    #device.setLogOutputLevel(dai.LogLevel.TRACE)

    q_rgb = device.getOutputQueue("rgb")
    q_nn_yolo = device.getOutputQueue("nn_yolo")

    frame = None

    # Since the detections returned by nn have values from <0..1> range, they need to be multiplied by frame width/height to
    # receive the actual position of the bounding box on the image
    def frameNorm(frame, bbox):
        normVals = np.full(len(bbox), frame.shape[0])
        normVals[::2] = frame.shape[1]
        return (np.clip(np.array(bbox), 0, 1) * normVals).astype(int)

    # Main host-side application loop
    diffs = np.array([])
    while True:

        # we try to fetch the data from nn/rgb queues. tryGet will return either the data packet or None if there isn't any
        in_rgb = q_rgb.tryGet()
        in_nn_yolo = q_nn_yolo.tryGet()

        if in_rgb is not None:
            # Latency in miliseconds 
            latencyMs = (dai.Clock.now() - in_rgb.getTimestamp()).total_seconds() * 1000
            diffs = np.append(diffs, latencyMs)
            print('Latency: {:.2f} ms, Average latency: {:.2f} ms, Std: {:.2f}'.format(latencyMs, np.average(diffs), np.std(diffs)))
            # If the packet from RGB camera is present, we're retrieving the frame in OpenCV format using getCvFrame
            frame = in_rgb.getCvFrame()

            if in_nn_yolo is not None:
                # Recuperamos el output de YOLO (tanto el 0 como el 1), newshape se ha sacado de netron.app  
                output0 = np.reshape(in_nn_yolo.getLayerFp16("output0"), newshape=([1, 116, 8400]))
                output1 = np.reshape(in_nn_yolo.getLayerFp16("output1"), newshape=([1, 32, 160, 160]))
    
                # Si tenemos ambos outputs podemos calcular la mascara final
                if( len(output0) > 0 and len(output1) > 0 ):                 
                    yoloseg = YOLOSeg("", conf_thres=0.3, iou_thres=0.5)
                    yoloseg.prepare_input_for_oakd(frame.shape[:2])
                    yoloseg.segment_objects_from_oakd(output0,output1)
                    combined_img = yoloseg.draw_masks(frame.copy())
                    cv2.imshow("Output", combined_img)

            else:
                print("in_nn_yolo EMPTY")

        else:
            print("in_rgb EMPTY")
        print("----------")
        # at any time, you can press "q" and exit the main loop, therefore exiting the program itself
        if cv2.waitKey(1) == ord('q'):
            break

And using "setloglevel" and "setlogoutputlevel" outputs this:

Latency: 4730.48 ms, Average latency: 3477.59 ms, Std: 1686.62
[14442C1031CDA1D000] [2.1.4] [1710409530.053] [host] [trace] Log vector decoded, size: 2
[14442C1031CDA1D000] [2.1.4] [24.622] [system] [trace] EV:1,S:0,IDS:12,IDD:0,TSS:24,TSN:622920181
[14442C1031CDA1D000] [2.1.4] [24.622] [system] [trace] EV:1,S:1,IDS:12,IDD:0,TSS:24,TSN:622965412
[14442C1031CDA1D000] [2.1.4] [1710409530.063] [host] [trace] Log vector decoded, size: 8
[14442C1031CDA1D000] [2.1.4] [24.623] [system] [trace] EV:1,S:0,IDS:13,IDD:0,TSS:24,TSN:623326768
[14442C1031CDA1D000] [2.1.4] [24.623] [system] [trace] EV:1,S:1,IDS:13,IDD:0,TSS:24,TSN:623359947
[14442C1031CDA1D000] [2.1.4] [24.623] [system] [trace] EV:1,S:0,IDS:9,IDD:0,TSS:24,TSN:623335640
[14442C1031CDA1D000] [2.1.4] [24.623] [system] [trace] EV:1,S:1,IDS:9,IDD:0,TSS:24,TSN:623397882
[14442C1031CDA1D000] [2.1.4] [24.629] [system] [trace] EV:0,S:0,IDS:5,IDD:12,TSS:24,TSN:629836060
[14442C1031CDA1D000] [2.1.4] [24.629] [system] [trace] EV:0,S:0,IDS:5,IDD:9,TSS:24,TSN:629336017
[14442C1031CDA1D000] [2.1.4] [24.629] [system] [trace] EV:0,S:1,IDS:5,IDD:12,TSS:24,TSN:629956637
[14442C1031CDA1D000] [2.1.4] [24.629] [system] [trace] EV:0,S:1,IDS:5,IDD:9,TSS:24,TSN:629396888
[14442C1031CDA1D000] [2.1.4] [1710409530.155] [host] [trace] Log vector decoded, size: 3
[14442C1031CDA1D000] [2.1.4] [24.725] [system] [info] Memory Usage - DDR: 91.93 / 333.46 MiB, CMX: 2.39 / 2.50 MiB, LeonOS Heap: 22.23 / 82.56 MiB, LeonRT Heap: 4.56 / 40.68 MiB / NOC ddr: 1343 MB/s
[14442C1031CDA1D000] [2.1.4] [24.725] [system] [info] Temperatures - Average: 35.17C, CSS: 36.30C, MSS 34.64C, UPA: 35.11C, DSS: 34.64C
[14442C1031CDA1D000] [2.1.4] [24.725] [system] [info] Cpu Usage - LeonOS 13.70%, LeonRT: 8.00%
[14442C1031CDA1D000] [2.1.4] [1710409530.204] [host] [trace] Log vector decoded, size: 2
[14442C1031CDA1D000] [2.1.4] [24.774] [system] [trace] EV:0,S:0,IDS:10,IDD:13,TSS:24,TSN:774239601

[14442C1031CDA1D000] [2.1.4] [24.774] [system] [trace] EV:0,S:1,IDS:10,IDD:13,TSS:24,TSN:774297831

Thanks for your time,
Pedro.

jakaskerl

Hi @pedro-UCA
DEPTHAI_LEVEL=TRACE python3 yourscript.py.

pedro-UCA Putting the print of latency in like these returns the same (about 4.2 of latency):

Remove any NN postprocessing on host. Just print the message as it arrives. We are trying to see if the host loop (processing) takes up the time, or the NN on the device itself (due to model being large/complex).

Thanks,
Jaka

pedro-UCA

Hi again,
using DEPTHAI_LEVEL=TRACE python3 yourscript.py do not work, it says that my computer "DEPTHAI_LEVEL is not recognized as an internal or external command" I have tried with python too.

If I comment all NN postprocessing this is the output:

[14442C1031CDA1D000] [2.1.4] [1710441029.530] [host] [trace] Log vector decoded, size: 9
[14442C1031CDA1D000] [2.1.4] [18.372] [system] [trace] EV:0,S:0,IDS:10,IDD:13,TSS:18,TSN:372637887
[14442C1031CDA1D000] [2.1.4] [18.372] [system] [trace] EV:0,S:1,IDS:10,IDD:13,TSS:18,TSN:372690025
[14442C1031CDA1D000] [2.1.4] [18.372] [system] [trace] EV:1,S:1,IDS:13,IDD:0,TSS:18,TSN:372772857
[14442C1031CDA1D000] [2.1.4] [18.372] [system] [trace] EV:1,S:0,IDS:9,IDD:0,TSS:18,TSN:372157498
[14442C1031CDA1D000] [2.1.4] [18.372] [system] [trace] EV:1,S:1,IDS:9,IDD:0,TSS:18,TSN:372202921
[14442C1031CDA1D000] [2.1.4] [18.380] [system] [trace] EV:0,S:0,IDS:5,IDD:12,TSS:18,TSN:380561762
[14442C1031CDA1D000] [2.1.4] [18.380] [system] [trace] EV:0,S:1,IDS:5,IDD:12,TSS:18,TSN:380654825
[14442C1031CDA1D000] [2.1.4] [18.380] [system] [trace] EV:1,S:1,IDS:12,IDD:0,TSS:18,TSN:380821371
[14442C1031CDA1D000] [2.1.4] [18.380] [system] [trace] EV:0,S:0,IDS:5,IDD:9,TSS:18,TSN:380003251

Latency: 200.25 ms, Average latency: 192.92 ms, Std: 24.65

So I guess the problem is in the postprocess, I come back to my original question in stackoverflow, after measuring times, the worst part is convert the nearly million elements list of output0 and output1 into numpy that I get using getLayerFp16, there is some way to retrieved the output as numpy array?

Thanks again,
Pedro.

erik

Hi @pedro-UCA ,
I believe the answers from StackOverflow are correct, you'd have better latency by using C++ instead (or fixing python bindings to return numpy arrarys).

pedro-UCA

erik could you give me some clues on how to do that? (I mean change bindings to return numpy arrays).

Thanks again,
Pedro.

erik

@pedro-UCA GPT4 can provide quite good suggestions:
https://chat.openai.com/share/64b86e9d-44fe-4730-ac53-47d2ead44e8a