YOLOv5 pipeline not working

davidbegr

Hi, this will be a long post but I don't know what to do. I have project with YOLOv5 trained on custom dataset from Roboflow. Training script here.

I was able to get video from camera with this pipeline:

from flask import Flask, render_template, Response
import cv2
import depthai as dai

app = Flask(__name__)

def create_pipeline():
    pipeline = dai.Pipeline()
    cam_rgb = pipeline.createColorCamera()
    cam_rgb.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1080_P)
    cam_rgb.setBoardSocket(dai.CameraBoardSocket.RGB)
    cam_rgb.setIspScale(1, 2)
    xout_rgb = pipeline.createXLinkOut()
    xout_rgb.setStreamName("rgb")
    cam_rgb.preview.link(xout_rgb.input)
    return pipeline

def gen_frames():
    pipeline = create_pipeline()

    with dai.Device(pipeline) as device:
        queue = device.getOutputQueue(name="rgb", maxSize=4, blocking=False)

        while True:
            frame = queue.get().getCvFrame()
            _, buffer = cv2.imencode('.jpg', frame)
            frame = buffer.tobytes()
            yield (b'--frame\r\n'
                   b'Content-Type: image/jpeg\r\n\r\n' + frame + b'\r\n')

@app.route('/')
def index():
    """Video streaming home page."""
    return render_template('index.html')

@app.route('/video_feed')
def video_feed():
    """Video streaming route. Put this in the src attribute of an img tag."""
    return Response(gen_frames(),
                    mimetype='multipart/x-mixed-replace; boundary=frame')

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000, debug=True)

Then to test if detections are happening I added MobileNetSSD pipeline:

def create_pipeline():
    pipeline = dai.Pipeline()

    cam_rgb = pipeline.createColorCamera()
    cam_rgb.setPreviewSize(300, 300)
    cam_rgb.setInterleaved(False)
    cam_rgb.setColorOrder(dai.ColorCameraProperties.ColorOrder.BGR)

    detection_nn = pipeline.createMobileNetDetectionNetwork()
    detection_nn.setBlobPath("/home/pi/depthai-python/examples/models/mobilenet-ssd_openvino_2021.4_6shave.blob")

    xout_rgb = pipeline.createXLinkOut()
    xout_rgb.setStreamName("rgb")

    xout_nn = pipeline.createXLinkOut()
    xout_nn.setStreamName("nn")

    cam_rgb.preview.link(detection_nn.input)
    detection_nn.passthrough.link(xout_rgb.input)
    detection_nn.out.link(xout_nn.input)

    return pipeline

Video and detections were working just fine, then I changed it to YOLOv5 pipeline and no video or detections. Also no error from debug console with pipeline debug on device. Pipeline for YOLOv5:

def create_pipeline():
    pipeline = dai.Pipeline()

    camRgb = pipeline.create(dai.node.ColorCamera)
    detectionNetwork = pipeline.create(dai.node.YoloDetectionNetwork)
    xoutRgb = pipeline.create(dai.node.XLinkOut)
    nnOut = pipeline.create(dai.node.XLinkOut)

    xoutRgb.setStreamName("rgb")
    nnOut.setStreamName("nn")

    camRgb.setPreviewSize(640, 640)
    camRgb.setInterleaved(False)
    camRgb.setColorOrder(dai.ColorCameraProperties.ColorOrder.BGR)
    camRgb.setFps(40)

    detectionNetwork.setConfidenceThreshold(0.5)
    detectionNetwork.setNumClasses(1)
    detectionNetwork.setCoordinateSize(4)
    detectionNetwork.setIouThreshold(0.5)
    detectionNetwork.setBlobPath("/home/pi/bc/2_5_yolov5s_batch16.blob")
    detectionNetwork.setNumInferenceThreads(2)
    detectionNetwork.input.setBlocking(False)

    camRgb.preview.link(detectionNetwork.input)
    detectionNetwork.passthrough.link(xoutRgb.input)
    detectionNetwork.out.link(nnOut.input)

    return pipeline

I think I tried everything and went throught forum and documentation, maybe I missed something or I am too focused on something that is not even problem. I would like any input from anybody that could help me.

lovro

Hi, here are a few things you can check:

Make sure camRgb.setPreviewSize(640, 640) matches the input size of the model. If the blob wasn’t exported with 640x640, it won’t work.
Try running the script with DEPTHAI_DEBUG=1. It might give more insight into what the problem is.

davidbegr

lovro

Is there a way to check blob shape?
DEPTHAI_DEBUG looked like this on monday, but I didn't find anything usefull there. Maybe I just don't know what to look for:

[2025-05-05 14:22:49.143] [depthai] [debug] Python bindings - version: 2.30.0.0 from  build: 2025-03-18 13:32:35 +0000 
[2025-05-05 14:22:49.143] [depthai] [debug] Library information - version: 2.30.0, commit: e0f6b52d048ef7ceac2ecd44deb2e161dbb97699 from 2025-03-18 12:44:00 +0100, build: 2025-03-18 11:51:40 +0000, libusb enabled: true 
[2025-05-05 14:22:49.155] [depthai] [debug] Initialize - finished 
 * Serving Flask app 'basic_object_det_setup' 
 * Debug mode: on 
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead. 
 * Running on all addresses (0.0.0.0) 
 * Running on http://127.0.0.1:5000 
 * Running on http://10.0.0.142:5000 
Press CTRL+C to quit 
 * Restarting with stat 
[2025-05-05 14:22:49.335] [depthai] [debug] Resources - Archive 'depthai-bootloader-fwp-0.0.28.tar.xz' open: 4ms, archive read: 187ms 
[2025-05-05 14:22:49.974] [depthai] [debug] Python bindings - version: 2.30.0.0 from  build: 2025-03-18 13:32:35 +0000 
[2025-05-05 14:22:49.974] [depthai] [debug] Library information - version: 2.30.0, commit: e0f6b52d048ef7ceac2ecd44deb2e161dbb97699 from 2025-03-18 12:44:00 +0100, build: 2025-03-18 11:51:40 +0000, libusb enabled: true 
[2025-05-05 14:22:49.987] [depthai] [debug] Initialize - finished 
 * Debugger is active! 
 * Debugger PIN: 284-617-301 
172.30.141.248 - - [05/May/2025 14:22:50] "GET /detections HTTP/1.1" 200 - 
[2025-05-05 14:22:50.172] [depthai] [debug] Resources - Archive 'depthai-bootloader-fwp-0.0.28.tar.xz' open: 4ms, archive read: 192ms 
[2025-05-05 14:22:50.230] [depthai] [debug] Resources - Archive 'depthai-device-fwp-a62b2ccb0bc493c2fb41694cb81c08887be24c52.tar.xz' open: 4ms, archive read: 1082ms 
172.30.141.248 - - [05/May/2025 14:22:50] "GET /detections HTTP/1.1" 200 - 
[2025-05-05 14:22:51.012] [depthai] [debug] Resources - Archive 'depthai-device-fwp-a62b2ccb0bc493c2fb41694cb81c08887be24c52.tar.xz' open: 4ms, archive read: 1032ms 
172.30.141.248 - - [05/May/2025 14:22:51] "GET /detections HTTP/1.1" 200 - 
172.30.141.248 - - [05/May/2025 14:22:52] "GET /detections HTTP/1.1" 200 - 
172.30.141.248 - - [05/May/2025 14:22:53] "GET /detections HTTP/1.1" 200 - 
172.30.141.248 - - [05/May/2025 14:22:53] "GET / HTTP/1.1" 200 - 
172.30.141.248 - - [05/May/2025 14:22:55] "GET /detections HTTP/1.1" 200 - 
[2025-05-05 14:22:55.157] [host] [debug] Device - OpenVINO version: 2021.4 
[18443010B1D4650F00] [1.1.2] [1746447775.157] [host] [debug] Device - BoardConfig: {"camera":[],"emmc":null,"gpio":[],"imu":null,"logDevicePrints":null,"logPath":null,"logSizeMax":null,"logVerbosity":null,"network":{"mtu":0,"xlinkTcpNoD
elay":true},"nonExclusiveMode":false,"pcieInternalClock":null,"sysctl":[],"uart":[],"usb":{"flashBootedPid":63037,"flashBootedVid":999,"manufacturer":"","maxSpeed":4,"pid":63035,"productName":"","vid":999},"usb3PhyInternalClock":null,"u
vc":null,"watchdogInitialDelayMs":null,"watchdogTimeoutMs":null}  
libnop: 
0000: b9 12 b9 07 81 e7 03 81 3b f6 81 e7 03 81 3d f6 04 bd 00 bd 00 b9 02 00 01 ba 00 be be bb 00 bb 
0020: 00 be be be be be be be 00 bb 00 be be 
172.30.141.248 - - [05/May/2025 14:22:56] "GET /detections HTTP/1.1" 200 - 
[2025-05-05 14:22:56.066] [depthai] [debug] Searching for booted device: DeviceInfo(name=1.1.2, mxid=18443010B1D4650F00, X_LINK_BOOTED, X_LINK_USB_VSC, X_LINK_MYRIAD_X, X_LINK_SUCCESS), name used as hint only 
[18443010B1D4650F00] [1.1.2] [0.836] [system] [info] Memory Usage - DDR: 0.12 / 333.26 MiB, CMX: 2.04 / 2.50 MiB, LeonOS Heap: 7.50 / 81.70 MiB, LeonRT Heap: 2.89 / 39.87 MiB / NOC ddr: 25 MB/s 
[18443010B1D4650F00] [1.1.2] [0.836] [system] [info] Temperatures - Average: 33.75C, CSS: 34.40C, MSS 33.69C, UPA: 33.69C, DSS: 33.21C 
[18443010B1D4650F00] [1.1.2] [0.836] [system] [info] Cpu Usage - LeonOS 77.11%, LeonRT: 1.71% 
[18443010B1D4650F00] [1.1.2] [1746447776.927] [host] [debug] Schema dump: {"connections":[],"globalProperties":{"calibData":null,"cameraTuningBlobSize":null,"cameraTuningBlobUri":"","leonCssFrequencyHz":700000000.0,"leonMssFrequencyHz":
700000000.0,"pipelineName":null,"pipelineVersion":null,"sippBufferSize":18432,"sippDmaBufferSize":16384,"xlinkChunkSize":-1},"nodes":[[0,{"id":0,"ioInfo":[[["","video"],{"blocking":false,"group":"","id":13,"name":"video","queueSize":8,"
type":0,"waitForMessage":false}],[["","still"],{"blocking":false,"group":"","id":11,"name":"still","queueSize":8,"type":0,"waitForMessage":false}],[["","isp"],{"blocking":false,"group":"","id":10,"name":"isp","queueSize":8,"type":0,"wai
tForMessage":false}],[["","preview"],{"blocking":false,"group":"","id":12,"name":"preview","queueSize":8,"type":0,"waitForMessage":false}],[["","raw"],{"blocking":false,"group":"","id":9,"name":"raw","queueSize":8,"type":0,"waitForMessa
ge":false}],[["","frameEvent"],{"blocking":false,"group":"","id":8,"name":"frameEvent","queueSize":8,"type":0,"waitForMessage":false}],[["","inputConfig"],{"blocking":false,"group":"","id":7,"name":"inputConfig","queueSize":8,"type":3,"
waitForMessage":false}],[["","inputControl"],{"blocking":true,"group":"","id":6,"name":"inputControl","queueSize":8,"type":3,"waitForMessage":false}]],"name":"ColorCamera","properties":[185,26,185,32,0,3,0,136,0,0,0,0,0,0,185,3,0,0,0,18
5,5,0,0,0,0,0,185,5,0,0,0,0,0,0,0,0,0,0,0,0,0,0,185,3,0,0,0,185,3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,186,0,255,189,0,255,0,0,0,129,128,2,129,128,2,255,255,255,255,0,136,0,0,32,66,0,136,0,0,128,191,136,0,0,128,191,1,185,4,0,0,0,0,3,3,4,4,4,190]
}],[1,{"id":1,"ioInfo":[[["","out"],{"blocking":false,"group":"","id":5,"name":"out","queueSize":8,"type":0,"waitForMessage":false}],[["","passthrough"],{"blocking":false,"group":"","id":4,"name":"passthrough","queueSize":8,"type":0,"wa
itForMessage":false}],[["","in"],{"blocking":false,"group":"","id":3,"name":"in","queueSize":5,"type":3,"waitForMessage":true}]],"name":"DetectionNetwork","properties":[185,6,130,128,67,220,0,189,12,97,115,115,101,116,58,95,95,98,108,11
1,98,8,2,0,185,7,0,136,0,0,0,63,80,4,186,0,187,0,136,0,0,0,63]}],[2,{"id":2,"ioInfo":[[["","in"],{"blocking":true,"group":"","id":2,"name":"in","queueSize":8,"type":3,"waitForMessage":true}]],"name":"XLinkOut","properties":[185,3,136,0,
0,128,191,189,3,114,103,98,0]}],[3,{"id":3,"ioInfo":[[["","in"],{"blocking":true,"group":"","id":1,"name":"in","queueSize":8,"type":3,"waitForMessage":true}]],"name":"XLinkOut","properties":[185,3,136,0,0,128,191,189,2,110,110,0]}]]} 
[18443010B1D4650F00] [1.1.2] [1746447776.927] [host] [debug] Asset map dump: {"map":{"/node/1/__blob":{"alignment":64,"offset":0,"size":14435200}}} 
[2025-05-05 14:22:57.076] [depthai] [info] Logging disabled 
[18443010B1D4650F00] [1.1.2] [1.002] [system] [info] SIPP (Signal Image Processing Pipeline) internal buffer size '18432'B, DMA buffer size: '16384'B 
[18443010B1D4650F00] [1.1.2] [1.037] [system] [info] NeuralNetwork allocated resources: shaves: [0-12] cmx slices: [0-12]  
ColorCamera allocated resources: no shaves; cmx slices: [13-15]  
 
[18443010B1D4650F00] [1.1.2] [1.052] [DetectionNetwork(1)] [info] Needed resources: shaves: 6, ddr: 14807296  
[18443010B1D4650F00] [1.1.2] [1.057] [DetectionNetwork(1)] [info] Inference thread count: 2, number of shaves allocated per thread: 6, number of Neural Compute Engines (NCE) allocated per thread: 1 
[18443010B1D4650F00] [1.1.2] [1.837] [system] [info] Memory Usage - DDR: 60.79 / 333.26 MiB, CMX: 2.08 / 2.50 MiB, LeonOS Heap: 21.73 / 81.70 MiB, LeonRT Heap: 5.08 / 39.87 MiB / NOC ddr: 287 MB/s 
[18443010B1D4650F00] [1.1.2] [1.837] [system] [info] Temperatures - Average: 35.77C, CSS: 36.30C, MSS 36.06C, UPA: 35.59C, DSS: 35.11C 
[18443010B1D4650F00] [1.1.2] [1.837] [system] [info] Cpu Usage - LeonOS 31.85%, LeonRT: 41.44% 
[18443010B1D4650F00] [1.1.2] [2.838] [system] [info] Memory Usage - DDR: 60.79 / 333.26 MiB, CMX: 2.08 / 2.50 MiB, LeonOS Heap: 21.73 / 81.70 MiB, LeonRT Heap: 5.08 / 39.87 MiB / NOC ddr: 381 MB/s 
[18443010B1D4650F00] [1.1.2] [2.838] [system] [info] Temperatures - Average: 35.41C, CSS: 36.06C, MSS 35.59C, UPA: 35.59C, DSS: 34.40C 
[18443010B1D4650F00] [1.1.2] [2.838] [system] [info] Cpu Usage - LeonOS 16.49%, LeonRT: 2.10% 
[18443010B1D4650F00] [1.1.2] [3.839] [system] [info] Memory Usage - DDR: 60.79 / 333.26 MiB, CMX: 2.08 / 2.50 MiB, LeonOS Heap: 21.73 / 81.70 MiB, LeonRT Heap: 5.08 / 39.87 MiB / NOC ddr: 379 MB/s 
[18443010B1D4650F00] [1.1.2] [3.839] [system] [info] Temperatures - Average: 35.59C, CSS: 36.53C, MSS 35.59C, UPA: 35.35C, DSS: 34.88C 
[18443010B1D4650F00] [1.1.2] [3.839] [system] [info] Cpu Usage - LeonOS 16.61%, LeonRT: 2.11% 
[18443010B1D4650F00] [1.1.2] [4.840] [system] [info] Memory Usage - DDR: 60.79 / 333.26 MiB, CMX: 2.08 / 2.50 MiB, LeonOS Heap: 21.73 / 81.70 MiB, LeonRT Heap: 5.08 / 39.87 MiB / NOC ddr: 379 MB/s 
[18443010B1D4650F00] [1.1.2] [4.840] [system] [info] Temperatures - Average: 35.59C, CSS: 36.53C, MSS 36.06C, UPA: 35.35C, DSS: 34.40C 
[18443010B1D4650F00] [1.1.2] [4.840] [system] [info] Cpu Usage - LeonOS 16.01%, LeonRT: 2.12% 
[18443010B1D4650F00] [1.1.2] [5.841] [system] [info] Memory Usage - DDR: 60.79 / 333.26 MiB, CMX: 2.08 / 2.50 MiB, LeonOS Heap: 21.73 / 81.70 MiB, LeonRT Heap: 5.08 / 39.87 MiB / NOC ddr: 381 MB/s 
[18443010B1D4650F00] [1.1.2] [5.841] [system] [info] Temperatures - Average: 35.71C, CSS: 36.77C, MSS 36.06C, UPA: 35.35C, DSS: 34.64C 
[18443010B1D4650F00] [1.1.2] [5.841] [system] [info] Cpu Usage - LeonOS 19.46%, LeonRT: 2.10% 
[18443010B1D4650F00] [1.1.2] [6.842] [system] [info] Memory Usage - DDR: 60.79 / 333.26 MiB, CMX: 2.08 / 2.50 MiB, LeonOS Heap: 21.73 / 81.70 MiB, LeonRT Heap: 5.08 / 39.87 MiB / NOC ddr: 383 MB/s 
[18443010B1D4650F00] [1.1.2] [6.842] [system] [info] Temperatures - Average: 36.00C, CSS: 37.24C, MSS 35.83C, UPA: 35.83C, DSS: 35.11C 
[18443010B1D4650F00] [1.1.2] [6.842] [system] [info] Cpu Usage - LeonOS 19.58%, LeonRT: 2.11% 
[18443010B1D4650F00] [1.1.2] [7.843] [system] [info] Memory Usage - DDR: 60.79 / 333.26 MiB, CMX: 2.08 / 2.50 MiB, LeonOS Heap: 21.73 / 81.70 MiB, LeonRT Heap: 5.08 / 39.87 MiB / NOC ddr: 383 MB/s 
[18443010B1D4650F00] [1.1.2] [7.843] [system] [info] Temperatures - Average: 36.00C, CSS: 37.24C, MSS 36.30C, UPA: 35.59C, DSS: 34.88C 
[18443010B1D4650F00] [1.1.2] [7.843] [system] [info] Cpu Usage - LeonOS 19.51%, LeonRT: 2.09% 
[18443010B1D4650F00] [1.1.2] [8.847] [system] [info] Memory Usage - DDR: 60.79 / 333.26 MiB, CMX: 2.08 / 2.50 MiB, LeonOS Heap: 21.73 / 81.70 MiB, LeonRT Heap: 5.08 / 39.87 MiB / NOC ddr: 383 MB/s 
[18443010B1D4650F00] [1.1.2] [8.847] [system] [info] Temperatures - Average: 36.00C, CSS: 37.48C, MSS 36.06C, UPA: 35.83C, DSS: 34.64C 
[18443010B1D4650F00] [1.1.2] [8.847] [system] [info] Cpu Usage - LeonOS 19.57%, LeonRT: 2.10% 
^Cterminate called without an active exception

jakaskerl

davidbegr
How did you convert the model to blob? Make sure you use https://tools.luxonis.com/. Also when training the model, we usually suggest you use the training notebooks from luxonis/depthai-ml-trainingtree/master/colab-notebooks. Afaik some additional layers are added / some pruning. cc @jakob on this.

Thanks,
Jaka

davidbegr

jakaskerl

I used https://blobconverter.luxonis.com/ for conversion and I believe I was going according the notebooks you provided, but I used 640 resolution, which shouldn't really be problem, right?

jakob

@davidbegr another thing worth checking is our generic-example that showcases a simple single-stage inference pipeline. It should work out-of-box for YOLOv5 so it might come handy as a quick test of model performance without the need to code the pipeline.