Integrating Deep SORT Tracking Algorithm with the OAK-D PoE CM4 Camera

Babacar

Hello everyone,

I have developed an object detection and tracking application on my PC, utilizing YOLOv8 for detection and Deep SORT for tracking. My application draws its own bounding boxes and works perfectly with my webcam. Now, I am looking to integrate it with my OAK-D PoE CM4 camera.

For detection, I've seen how to convert my YOLOv8 model using your tools, and I plan to use this converted model for object detection with the DepthAI SDK.

However, for tracking, I'm a bit uncertain. I've seen your object tracking example (https://github.com/luxonis/depthai/blob/main/depthai_sdk/examples/NNComponent/object_tracking.py), but it does not utilize Deep SORT. I want to use my own Deep SORT tracking algorithm, but I'm unsure of the best way to integrate it into my application's image processing pipeline.

Is it possible to use Deep SORT as a tracking algorithm with the DepthAI SDK? If so, are there any examples or guides available on how to do this?
If using Deep SORT is not feasible, what would be the best tracking algorithm to use with the DepthAI SDK to accurately obtain the position of tracked objects?
Is it possible to combine the use of the DepthAI SDK for object detection with Deep SORT for tracking? For instance, could I use the DepthAI SDK to detect objects, and then pass these detected objects to Deep SORT for tracking?

I would greatly appreciate any help or guidance you can provide.

Best regards,

Babacar

erik

Hi Babacar ,
We have DeepSORT demo here: https://github.com/luxonis/depthai-experiments/tree/master/gen2-deepsort-tracking
I hope it helps!

Babacar

erik

Thank you very much for pointing me to the DeepSORT demo. This will be immensely helpful for my project.

Best regards!

pedro-UCA

Hi,

I have been working with an OAK-D using YOLOv8 and I want to try the tracking system, is it possible with the actual state of the luxonis library (I do not have problems using the api or the sdk) to implement this without downgrade YOLO to version 6?

Regards,

Pedro.

pedro-UCA

I have done it. For the record and if someone else has the same problem I have check how the code works of the library and you have to change the line:

yolo = oak.create_nn('yolov6nr3_coco_640x352', input=color)

to:

yolo = oak.create_nn(pathYoloJson, input=color, nn_type="yolo")

where "pathYoloJson" is the path to the json created by http://tools.luxonis.com/ but we have to change the model section of the json from this:

"model": {

    "xml": "best.xml",

    "bin": "best.bin"

}

to this (or the path to your model .blob):

"model": {

    "blob": "best_openvino_2022.1_6shave.blob"

}

I think you should put this kind of workflow (changing diferents models of customize nn) more cleary in the documentation or at least in the coments of the functions in the code.

Regards,

Pedro.