Run YOLO live

jvitordm

Hello there,

I've finished training my custom YOLOv4 algorithm through this tutorial and would like to make it run live.

My goal was to run the code on python on a raspberry pi and for the given moment when the algorithm detects the object(s) on the video I'd like to save a full resolution picture + the results from the YOLO detection.

The way to combine a python code managing outputs from the camera is still not clear to me.
Any comments/ideias will be appreciated, thank you in advance!

erik

Hello jvitordm!
that's awesome to hear. So first thing I would do is to check if the detections are ok by following the steps at the end of the notebook you were using, section Run the blob on DepthAI. Afterwards, since it's the yolo model, it will work with the YoloDetectionNetwork and SpatialYoloDetectionNetwork nodes/example codes. You just have to specify the path to your blob.
For the full-res image, do you plan on using 1080P/4k/12MP? If it's 1080P, you could just stream the 1080P video frames back to the device and when the device detects an object it would save that frame. Note here that this can consume a lot of disk/sd-card space.
Here's an example that uses mobilenet and video stream to display the detections.
Hope this helps! Erik

jvitordm

Dear erik,

First of all thank you for the quick help!
I implemented it on the last couple of weeks and its running fine. 😀

I've been having some troubles regarding real time operation due to the nature of the process I want to monitor. The object is moving quite fast, so on the first trials I haven't been able to capture it well enough (I wonder if it has something to do with the video embedding). I was thinking if it would make sense to capture just a picture for a given moment I know the object(s) is(are) there and then process the image not the video - is it possible? Any other ideas?

About saving and image, I'm simply using cv2.imwrite(str(count)+'.jpg',frame) for the 1080x1080p image, but I think it's still a coarse image (+/- 230kb), how can I save top quality images?

Thanks!

erik

Hello jvitordm,
for the first part, I would say there's a delay due to inferencing time - so the frame and the detections don't match. We are working on the device side sync node, but for now, you can perform syncing on the host, example here. I'ts getting passthrough timestamp to match the frame and detection.

You can send video or isp (used in example linked above) 2160x2160 frames to the host, perform host side syncing and saving the frame in case of detection.

Thanks, Erik