YOLOv5 pipeline not working

davidbegr

Hi, this will be a long post but I don't know what to do. I have project with YOLOv5 trained on custom dataset from Roboflow. Training script here.

I was able to get video from camera with this pipeline:

from flask import Flask, render_template, Response
import cv2
import depthai as dai

app = Flask(__name__)

def create_pipeline():
    pipeline = dai.Pipeline()
    cam_rgb = pipeline.createColorCamera()
    cam_rgb.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1080_P)
    cam_rgb.setBoardSocket(dai.CameraBoardSocket.RGB)
    cam_rgb.setIspScale(1, 2)
    xout_rgb = pipeline.createXLinkOut()
    xout_rgb.setStreamName("rgb")
    cam_rgb.preview.link(xout_rgb.input)
    return pipeline

def gen_frames():
    pipeline = create_pipeline()

    with dai.Device(pipeline) as device:
        queue = device.getOutputQueue(name="rgb", maxSize=4, blocking=False)

        while True:
            frame = queue.get().getCvFrame()
            _, buffer = cv2.imencode('.jpg', frame)
            frame = buffer.tobytes()
            yield (b'--frame\r\n'
                   b'Content-Type: image/jpeg\r\n\r\n' + frame + b'\r\n')

@app.route('/')
def index():
    """Video streaming home page."""
    return render_template('index.html')

@app.route('/video_feed')
def video_feed():
    """Video streaming route. Put this in the src attribute of an img tag."""
    return Response(gen_frames(),
                    mimetype='multipart/x-mixed-replace; boundary=frame')

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000, debug=True)

Then to test if detections are happening I added MobileNetSSD pipeline:

def create_pipeline():
    pipeline = dai.Pipeline()

    cam_rgb = pipeline.createColorCamera()
    cam_rgb.setPreviewSize(300, 300)
    cam_rgb.setInterleaved(False)
    cam_rgb.setColorOrder(dai.ColorCameraProperties.ColorOrder.BGR)

    detection_nn = pipeline.createMobileNetDetectionNetwork()
    detection_nn.setBlobPath("/home/pi/depthai-python/examples/models/mobilenet-ssd_openvino_2021.4_6shave.blob")

    xout_rgb = pipeline.createXLinkOut()
    xout_rgb.setStreamName("rgb")

    xout_nn = pipeline.createXLinkOut()
    xout_nn.setStreamName("nn")

    cam_rgb.preview.link(detection_nn.input)
    detection_nn.passthrough.link(xout_rgb.input)
    detection_nn.out.link(xout_nn.input)

    return pipeline

Video and detections were working just fine, then I changed it to YOLOv5 pipeline and no video or detections. Also no error from debug console with pipeline debug on device. Pipeline for YOLOv5:

def create_pipeline():
    pipeline = dai.Pipeline()

    camRgb = pipeline.create(dai.node.ColorCamera)
    detectionNetwork = pipeline.create(dai.node.YoloDetectionNetwork)
    xoutRgb = pipeline.create(dai.node.XLinkOut)
    nnOut = pipeline.create(dai.node.XLinkOut)

    xoutRgb.setStreamName("rgb")
    nnOut.setStreamName("nn")

    camRgb.setPreviewSize(640, 640)
    camRgb.setInterleaved(False)
    camRgb.setColorOrder(dai.ColorCameraProperties.ColorOrder.BGR)
    camRgb.setFps(40)

    detectionNetwork.setConfidenceThreshold(0.5)
    detectionNetwork.setNumClasses(1)
    detectionNetwork.setCoordinateSize(4)
    detectionNetwork.setIouThreshold(0.5)
    detectionNetwork.setBlobPath("/home/pi/bc/2_5_yolov5s_batch16.blob")
    detectionNetwork.setNumInferenceThreads(2)
    detectionNetwork.input.setBlocking(False)

    camRgb.preview.link(detectionNetwork.input)
    detectionNetwork.passthrough.link(xoutRgb.input)
    detectionNetwork.out.link(nnOut.input)

    return pipeline

I think I tried everything and went throught forum and documentation, maybe I missed something or I am too focused on something that is not even problem. I would like any input from anybody that could help me.

lovro

Hi, here are a few things you can check:

Make sure camRgb.setPreviewSize(640, 640) matches the input size of the model. If the blob wasn’t exported with 640x640, it won’t work.
Try running the script with DEPTHAI_DEBUG=1. It might give more insight into what the problem is.

davidbegr

lovro

Is there a way to check blob shape?
DEPTHAI_DEBUG looked like this on monday, but I didn't find anything usefull there. Maybe I just don't know what to look for:

[2025-05-05 14:22:49.143] [depthai] [debug] Python bindings - version: 2.30.0.0 from  build: 2025-03-18 13:32:35 +0000 
[2025-05-05 14:22:49.143] [depthai] [debug] Library information - version: 2.30.0, commit: e0f6b52d048ef7ceac2ecd44deb2e161dbb97699 from 2025-03-18 12:44:00 +0100, build: 2025-03-18 11:51:40 +0000, libusb enabled: true 
[2025-05-05 14:22:49.155] [depthai] [debug] Initialize - finished 
 * Serving Flask app 'basic_object_det_setup' 
 * Debug mode: on 
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead. 
 * Running on all addresses (0.0.0.0) 
 * Running on http://127.0.0.1:5000 
 * Running on http://10.0.0.142:5000 
Press CTRL+C to quit 
 * Restarting with stat 
[2025-05-05 14:22:49.335] [depthai] [debug] Resources - Archive 'depthai-bootloader-fwp-0.0.28.tar.xz' open: 4ms, archive read: 187ms 
[2025-05-05 14:22:49.974] [depthai] [debug] Python bindings - version: 2.30.0.0 from  build: 2025-03-18 13:32:35 +0000 
[2025-05-05 14:22:49.974] [depthai] [debug] Library information - version: 2.30.0, commit: e0f6b52d048ef7ceac2ecd44deb2e161dbb97699 from 2025-03-18 12:44:00 +0100, build: 2025-03-18 11:51:40 +0000, libusb enabled: true 
[2025-05-05 14:22:49.987] [depthai] [debug] Initialize - finished 
 * Debugger is active! 
 * Debugger PIN: 284-617-301 
172.30.141.248 - - [05/May/2025 14:22:50] "GET /detections HTTP/1.1" 200 - 
[2025-05-05 14:22:50.172] [depthai] [debug] Resources - Archive 'depthai-bootloader-fwp-0.0.28.tar.xz' open: 4ms, archive read: 192ms 
[2025-05-05 14:22:50.230] [depthai] [debug] Resources - Archive 'depthai-device-fwp-a62b2ccb0bc493c2fb41694cb81c08887be24c52.tar.xz' open: 4ms, archive read: 1082ms 
172.30.141.248 - - [05/May/2025 14:22:50] "GET /detections HTTP/1.1" 200 - 
[2025-05-05 14:22:51.012] [depthai] [debug] Resources - Archive 'depthai-device-fwp-a62b2ccb0bc493c2fb41694cb81c08887be24c52.tar.xz' open: 4ms, archive read: 1032ms 
172.30.141.248 - - [05/May/2025 14:22:51] "GET /detections HTTP/1.1" 200 - 
172.30.141.248 - - [05/May/2025 14:22:52] "GET /detections HTTP/1.1" 200 - 
172.30.141.248 - - [05/May/2025 14:22:53] "GET /detections HTTP/1.1" 200 - 
172.30.141.248 - - [05/May/2025 14:22:53] "GET / HTTP/1.1" 200 - 
172.30.141.248 - - [05/May/2025 14:22:55] "GET /detections HTTP/1.1" 200 - 
[2025-05-05 14:22:55.157] [host] [debug] Device - OpenVINO version: 2021.4 
[18443010B1D4650F00] [1.1.2] [1746447775.157] [host] [debug] Device - BoardConfig: {"camera":[],"emmc":null,"gpio":[],"imu":null,"logDevicePrints":null,"logPath":null,"logSizeMax":null,"logVerbosity":null,"network":{"mtu":0,"xlinkTcpNoD
elay":true},"nonExclusiveMode":false,"pcieInternalClock":null,"sysctl":[],"uart":[],"usb":{"flashBootedPid":63037,"flashBootedVid":999,"manufacturer":"","maxSpeed":4,"pid":63035,"productName":"","vid":999},"usb3PhyInternalClock":null,"u
vc":null,"watchdogInitialDelayMs":null,"watchdogTimeoutMs":null}  
libnop: 
0000: b9 12 b9 07 81 e7 03 81 3b f6 81 e7 03 81 3d f6 04 bd 00 bd 00 b9 02 00 01 ba 00 be be bb 00 bb 
0020: 00 be be be be be be be 00 bb 00 be be 
172.30.141.248 - - [05/May/2025 14:22:56] "GET /detections HTTP/1.1" 200 - 
[2025-05-05 14:22:56.066] [depthai] [debug] Searching for booted device: DeviceInfo(name=1.1.2, mxid=18443010B1D4650F00, X_LINK_BOOTED, X_LINK_USB_VSC, X_LINK_MYRIAD_X, X_LINK_SUCCESS), name used as hint only 
[18443010B1D4650F00] [1.1.2] [0.836] [system] [info] Memory Usage - DDR: 0.12 / 333.26 MiB, CMX: 2.04 / 2.50 MiB, LeonOS Heap: 7.50 / 81.70 MiB, LeonRT Heap: 2.89 / 39.87 MiB / NOC ddr: 25 MB/s 
[18443010B1D4650F00] [1.1.2] [0.836] [system] [info] Temperatures - Average: 33.75C, CSS: 34.40C, MSS 33.69C, UPA: 33.69C, DSS: 33.21C 
[18443010B1D4650F00] [1.1.2] [0.836] [system] [info] Cpu Usage - LeonOS 77.11%, LeonRT: 1.71% 
[18443010B1D4650F00] [1.1.2] [1746447776.927] [host] [debug] Schema dump: {"connections":[],"globalProperties":{"calibData":null,"cameraTuningBlobSize":null,"cameraTuningBlobUri":"","leonCssFrequencyHz":700000000.0,"leonMssFrequencyHz":
700000000.0,"pipelineName":null,"pipelineVersion":null,"sippBufferSize":18432,"sippDmaBufferSize":16384,"xlinkChunkSize":-1},"nodes":[[0,{"id":0,"ioInfo":[[["","video"],{"blocking":false,"group":"","id":13,"name":"video","queueSize":8,"
type":0,"waitForMessage":false}],[["","still"],{"blocking":false,"group":"","id":11,"name":"still","queueSize":8,"type":0,"waitForMessage":false}],[["","isp"],{"blocking":false,"group":"","id":10,"name":"isp","queueSize":8,"type":0,"wai
tForMessage":false}],[["","preview"],{"blocking":false,"group":"","id":12,"name":"preview","queueSize":8,"type":0,"waitForMessage":false}],[["","raw"],{"blocking":false,"group":"","id":9,"name":"raw","queueSize":8,"type":0,"waitForMessa
ge":false}],[["","frameEvent"],{"blocking":false,"group":"","id":8,"name":"frameEvent","queueSize":8,"type":0,"waitForMessage":false}],[["","inputConfig"],{"blocking":false,"group":"","id":7,"name":"inputConfig","queueSize":8,"type":3,"
waitForMessage":false}],[["","inputControl"],{"blocking":true,"group":"","id":6,"name":"inputControl","queueSize":8,"type":3,"waitForMessage":false}]],"name":"ColorCamera","properties":[185,26,185,32,0,3,0,136,0,0,0,0,0,0,185,3,0,0,0,18
5,5,0,0,0,0,0,185,5,0,0,0,0,0,0,0,0,0,0,0,0,0,0,185,3,0,0,0,185,3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,186,0,255,189,0,255,0,0,0,129,128,2,129,128,2,255,255,255,255,0,136,0,0,32,66,0,136,0,0,128,191,136,0,0,128,191,1,185,4,0,0,0,0,3,3,4,4,4,190]
}],[1,{"id":1,"ioInfo":[[["","out"],{"blocking":false,"group":"","id":5,"name":"out","queueSize":8,"type":0,"waitForMessage":false}],[["","passthrough"],{"blocking":false,"group":"","id":4,"name":"passthrough","queueSize":8,"type":0,"wa
itForMessage":false}],[["","in"],{"blocking":false,"group":"","id":3,"name":"in","queueSize":5,"type":3,"waitForMessage":true}]],"name":"DetectionNetwork","properties":[185,6,130,128,67,220,0,189,12,97,115,115,101,116,58,95,95,98,108,11
1,98,8,2,0,185,7,0,136,0,0,0,63,80,4,186,0,187,0,136,0,0,0,63]}],[2,{"id":2,"ioInfo":[[["","in"],{"blocking":true,"group":"","id":2,"name":"in","queueSize":8,"type":3,"waitForMessage":true}]],"name":"XLinkOut","properties":[185,3,136,0,
0,128,191,189,3,114,103,98,0]}],[3,{"id":3,"ioInfo":[[["","in"],{"blocking":true,"group":"","id":1,"name":"in","queueSize":8,"type":3,"waitForMessage":true}]],"name":"XLinkOut","properties":[185,3,136,0,0,128,191,189,2,110,110,0]}]]} 
[18443010B1D4650F00] [1.1.2] [1746447776.927] [host] [debug] Asset map dump: {"map":{"/node/1/__blob":{"alignment":64,"offset":0,"size":14435200}}} 
[2025-05-05 14:22:57.076] [depthai] [info] Logging disabled 
[18443010B1D4650F00] [1.1.2] [1.002] [system] [info] SIPP (Signal Image Processing Pipeline) internal buffer size '18432'B, DMA buffer size: '16384'B 
[18443010B1D4650F00] [1.1.2] [1.037] [system] [info] NeuralNetwork allocated resources: shaves: [0-12] cmx slices: [0-12]  
ColorCamera allocated resources: no shaves; cmx slices: [13-15]  
 
[18443010B1D4650F00] [1.1.2] [1.052] [DetectionNetwork(1)] [info] Needed resources: shaves: 6, ddr: 14807296  
[18443010B1D4650F00] [1.1.2] [1.057] [DetectionNetwork(1)] [info] Inference thread count: 2, number of shaves allocated per thread: 6, number of Neural Compute Engines (NCE) allocated per thread: 1 
[18443010B1D4650F00] [1.1.2] [1.837] [system] [info] Memory Usage - DDR: 60.79 / 333.26 MiB, CMX: 2.08 / 2.50 MiB, LeonOS Heap: 21.73 / 81.70 MiB, LeonRT Heap: 5.08 / 39.87 MiB / NOC ddr: 287 MB/s 
[18443010B1D4650F00] [1.1.2] [1.837] [system] [info] Temperatures - Average: 35.77C, CSS: 36.30C, MSS 36.06C, UPA: 35.59C, DSS: 35.11C 
[18443010B1D4650F00] [1.1.2] [1.837] [system] [info] Cpu Usage - LeonOS 31.85%, LeonRT: 41.44% 
[18443010B1D4650F00] [1.1.2] [2.838] [system] [info] Memory Usage - DDR: 60.79 / 333.26 MiB, CMX: 2.08 / 2.50 MiB, LeonOS Heap: 21.73 / 81.70 MiB, LeonRT Heap: 5.08 / 39.87 MiB / NOC ddr: 381 MB/s 
[18443010B1D4650F00] [1.1.2] [2.838] [system] [info] Temperatures - Average: 35.41C, CSS: 36.06C, MSS 35.59C, UPA: 35.59C, DSS: 34.40C 
[18443010B1D4650F00] [1.1.2] [2.838] [system] [info] Cpu Usage - LeonOS 16.49%, LeonRT: 2.10% 
[18443010B1D4650F00] [1.1.2] [3.839] [system] [info] Memory Usage - DDR: 60.79 / 333.26 MiB, CMX: 2.08 / 2.50 MiB, LeonOS Heap: 21.73 / 81.70 MiB, LeonRT Heap: 5.08 / 39.87 MiB / NOC ddr: 379 MB/s 
[18443010B1D4650F00] [1.1.2] [3.839] [system] [info] Temperatures - Average: 35.59C, CSS: 36.53C, MSS 35.59C, UPA: 35.35C, DSS: 34.88C 
[18443010B1D4650F00] [1.1.2] [3.839] [system] [info] Cpu Usage - LeonOS 16.61%, LeonRT: 2.11% 
[18443010B1D4650F00] [1.1.2] [4.840] [system] [info] Memory Usage - DDR: 60.79 / 333.26 MiB, CMX: 2.08 / 2.50 MiB, LeonOS Heap: 21.73 / 81.70 MiB, LeonRT Heap: 5.08 / 39.87 MiB / NOC ddr: 379 MB/s 
[18443010B1D4650F00] [1.1.2] [4.840] [system] [info] Temperatures - Average: 35.59C, CSS: 36.53C, MSS 36.06C, UPA: 35.35C, DSS: 34.40C 
[18443010B1D4650F00] [1.1.2] [4.840] [system] [info] Cpu Usage - LeonOS 16.01%, LeonRT: 2.12% 
[18443010B1D4650F00] [1.1.2] [5.841] [system] [info] Memory Usage - DDR: 60.79 / 333.26 MiB, CMX: 2.08 / 2.50 MiB, LeonOS Heap: 21.73 / 81.70 MiB, LeonRT Heap: 5.08 / 39.87 MiB / NOC ddr: 381 MB/s 
[18443010B1D4650F00] [1.1.2] [5.841] [system] [info] Temperatures - Average: 35.71C, CSS: 36.77C, MSS 36.06C, UPA: 35.35C, DSS: 34.64C 
[18443010B1D4650F00] [1.1.2] [5.841] [system] [info] Cpu Usage - LeonOS 19.46%, LeonRT: 2.10% 
[18443010B1D4650F00] [1.1.2] [6.842] [system] [info] Memory Usage - DDR: 60.79 / 333.26 MiB, CMX: 2.08 / 2.50 MiB, LeonOS Heap: 21.73 / 81.70 MiB, LeonRT Heap: 5.08 / 39.87 MiB / NOC ddr: 383 MB/s 
[18443010B1D4650F00] [1.1.2] [6.842] [system] [info] Temperatures - Average: 36.00C, CSS: 37.24C, MSS 35.83C, UPA: 35.83C, DSS: 35.11C 
[18443010B1D4650F00] [1.1.2] [6.842] [system] [info] Cpu Usage - LeonOS 19.58%, LeonRT: 2.11% 
[18443010B1D4650F00] [1.1.2] [7.843] [system] [info] Memory Usage - DDR: 60.79 / 333.26 MiB, CMX: 2.08 / 2.50 MiB, LeonOS Heap: 21.73 / 81.70 MiB, LeonRT Heap: 5.08 / 39.87 MiB / NOC ddr: 383 MB/s 
[18443010B1D4650F00] [1.1.2] [7.843] [system] [info] Temperatures - Average: 36.00C, CSS: 37.24C, MSS 36.30C, UPA: 35.59C, DSS: 34.88C 
[18443010B1D4650F00] [1.1.2] [7.843] [system] [info] Cpu Usage - LeonOS 19.51%, LeonRT: 2.09% 
[18443010B1D4650F00] [1.1.2] [8.847] [system] [info] Memory Usage - DDR: 60.79 / 333.26 MiB, CMX: 2.08 / 2.50 MiB, LeonOS Heap: 21.73 / 81.70 MiB, LeonRT Heap: 5.08 / 39.87 MiB / NOC ddr: 383 MB/s 
[18443010B1D4650F00] [1.1.2] [8.847] [system] [info] Temperatures - Average: 36.00C, CSS: 37.48C, MSS 36.06C, UPA: 35.83C, DSS: 34.64C 
[18443010B1D4650F00] [1.1.2] [8.847] [system] [info] Cpu Usage - LeonOS 19.57%, LeonRT: 2.10% 
^Cterminate called without an active exception

jakaskerl

davidbegr
How did you convert the model to blob? Make sure you use https://tools.luxonis.com/. Also when training the model, we usually suggest you use the training notebooks from luxonis/depthai-ml-trainingtree/master/colab-notebooks. Afaik some additional layers are added / some pruning. cc @jakob on this.

Thanks,
Jaka

davidbegr

jakaskerl

I used https://blobconverter.luxonis.com/ for conversion and I believe I was going according the notebooks you provided, but I used 640 resolution, which shouldn't really be problem, right?

jakob

@davidbegr another thing worth checking is our generic-example that showcases a simple single-stage inference pipeline. It should work out-of-box for YOLOv5 so it might come handy as a quick test of model performance without the need to code the pipeline.

jakaskerl

davidbegr use tools for yolo.

davidbegr

jakaskerl The tool you suggested probably worked, since it was now giving sensible errors such as:
YoloDetectionNetwork is deprecated, use DetectionNetwork instead and RuntimeError: Failed to connect to device, error message: X_LINK_DEVICE_ALREADY_IN_USE. I still dont see video stream from my app and debug doesn't give me any info other then everything is okay.

I managed to solve issues I mention in this comment. But what is the difference between blobconverter tool and tools for YOLO you suggested?

jakaskerl

davidbegr
I think @jakob would know better, but the blobconverter is meant for any AI model conversion using the OpenVINO toolchain, there is more configurability. The tools are used only for yolo and do some scaling/preprocessing that makes the models run on OAK devices out of the box.
I know the naming conventions are a bit odd, we are changing this in the next depthai iteration.

Thanks,
Jaka

jakob

Blobconverter

General library for exporting any model from the listed sources (.pt not being one of them) to .blob. The obtained model should be (more or less) identical to the source.

Tools

YOLO specific library for exporting YOLO models from .pt to .blob. The obtained model is a modified version of the source as some of the nodes from the end of the network are removed (i.e. the model head). DepthAI automatically detects this and runs the missing calculations on host.