Need a Beginner's Guide on Running YOLO/DeepSort using OAK-D/RaspberryPi

zunnash

I'm a beginner in computer vision, and using an OAK-D device with Raspberry Pi (and currently Windows) to run DeepSort/Yolov5 for multi-object tracking on cars/ traffic scenarios as part of my undergrad engineering project. Unlike object detection, I haven't found any beginner-friendly material on performing object tracking using OAK-D with a Windows host machine/ Raspberry Pi. Can anyone share a beginner-friendly explanation on how to go about this? Can we run tracking on OAK or it has to run on the host device, or how do we segregate the part of the algorithm which runs on OAK and which runs on the host device? Is there any tutorial available for this?

I'd be grateful for any helpful response to this,

Thanks,

Zunnash.

erik

Hi zunnash ,

You can run tracking on device; one option is ObjectTracker node, the other is NN-based tracking, example: https://github.com/nikp00/stark-on-depthai

Embeddings are extracted on-device, tracking is done on host: https://github.com/luxonis/depthai-experiments/tree/master/gen2-deepsort-tracking

Thoughts?

Thanks, Erik

zunnash

Thanks for getting back, Erik. I will check those links out and let you know.

zunnash

Hi,

I have implemented object tracking with OAK on Windows computer. But now I'm installing the setup on Raspberry Pi 4 using this tutorial by Luxonis on Youtube ( and the process takes too long and eventually crashes (I can detect the device connected and download the depthai repository but the examples don't run because of the missing packages, and they even crash when I do them manually one by one). Can you tell me what this problem could be related to, and how can I solve it?

Thanks.

erik

Hi zunnash ,
What's the crash that you get on the RPI - or what takes too long? It might be it's trying to get wheels that aren't available for RPi, so it tries to build them (and fails). But it would be helpful to know the actual terminal output.
Thanks, Erik

zunnash

Basically, the installation process (sudo curl -fL https://docs.luxonis.com/install_dependencies.sh | bash) got to 80 percent a few times and then crashed. The error did say something about the wheels, but I'll share the exact output when I try again tomorrow.

In case it's the wheels, what's the way to resolve it?

erik

We will be able to best suggest based on the output itself🙂

zunnash

Hi, so I used the Luxonis RPi OS image which had the dependencies preinstalled, that resolved those errors. But now I'm getting this error (IndexError: map::at) (I'm using my custom models). Can you please tell me what does this mean? I saw on another forum question someone resolved it by downgrading the OpenVino version, but how do you exactly do that?

(I've tried giving both absolute and relative paths)

jakaskerl

Hi zunnash
As @Andrevancalster mentioned in other thread (source), convert your model to ONNX using something like the
torch.onnx.export function. Blobconverter/openvino tool optimizer does not yet support pytorch framework. You should then you will be able to select the openvino version and possibly solve your problem.

Thanks,
Jaka

krishnashravan

zunnash
did u solve the index error mat : :at error ?
if so can u help me with it?

krishnashravan

I'm getting the same issue using OAK-D CM4 PoE,
using the yolov8 nano trained on person detection using the yolov8 custom notebook
detection_nn.setBlobPath(nnPath)
IndexError: map::at

jakaskerl

Hi krishnashravan
Did you try downgrading openvino version as suggested here?

Thanks,
Jaka

krishnashravan

jakaskerl
Yes i have seen the other thread
but the thing is the camera which i'm using has a pi integrated
it doesn't allow u to install openvino using pip
i was checking how to install openvino on the pi but the last working package in the official website was 2020 which is no longer available in the site
if anyone could tell me how to check which version I'm using and how to change it to the desired version on the pi that would be a great help.

erik

krishnashravan Sorry could you provide full terminal output for this error? From my understanding, you uploaded blob to the CM4, and then with depthai specified the path, but it threw error?

zunnash

krishnashravan This error went away with changing the depthai version. It occured with depthai 2.10 I think, but didn't appear while using depthai 2.11 and 2.12

krishnashravan

File "/usr/lib/python3.9/threading.py", line 954, in bootstrap_inner
    self.run()
File "/usr/lib/python3.9/threading.py", line 892, in run
    self.target(*self.args, **self.kwargs)
File "/home/pi/demo/people/modules/VisionModule/main.py", line 209, in detect_and_count
    pipeline = init_pipeline()
File "/home/pi/demo/people/modules/VisionModule/main.py", line 193, in init_pipeline
    detection_nn.setBlobPath(nnPath)
IndexError: map::at
trying to run the yolov8 nano version on the camera
it worked in the oak-d camera but i'm getting that error when running on the CM4

erik

krishnashravan and what's nnPath in this case, could you print it's value and type?

krishnashravan

erik
this is on the pi
do u think its taking the wrong path
because in the oak d that same code is giving the correct path

krishnashravan

if customModel is True:

nnPath = str((parentDir / Path('../../data/' + model)).resolve().absolute())

print(nnPath)

detection_nn.setBlobPath(nnPath)

print("Custom Model" + nnPath + "Size: " + str(sizeX) + "x" + str(sizeY))
thats the code for getting the nnpath
the blob file path is correct

erik

I'd strongly suggest to use latest depthai version (2.22).