Training custom yolo model

niloofarsabouri

When you specify -m yolo it tries to download it from our cloud. You should pass the path to your .blob path there. So, in .zip from tools.luxonis.com you will get the .json (which you are using correctly) and a .blob file. -m should be path to that blobfile.

python3 main_api.py -c best.json -m best_openvino_2022.1_6shave.blob

niloofarsabouri

114 images is barely the minimum. You should try to use more. The quality of annotations will also affect how well the model works.

Hi niloofarsabouri
Do you absolutely need the NN to perform this task? Can you modify the windows in any way?

We had a competition where the goal was to as quickly as possible, fly a drone through a series of hoops. But each hoop had an Aruco marker glued on the pole on which the hoop was mounted. By using a marker, you would know the distance from the hoop center to the drone.

Would this be possible in your case?

Thanks,
Jaka

Hi niloofarsabouri
Not really sure what you mean by non-processing, but you can get frames straight from the device, without any manipulations done to the image.

Thanks,
Jaka

niloofarsabouri

Yeah, there are more way to approach it. If you just use OAK as a camera, and do the processing on the Jetson, you could follow and implement something like this square detector.

On the other hand, neural network should also be able to detect this, given the shape and color is very distinct. In that case you would need a relatively good and diverse dataset. I would say you'd need at least 100 images for the first iteration, then train the model. After training, you should inspect the test results you get from the training on "test" data (data not used for training). If performance is not good enough, you should collect and annotate more images and repeat.

Handler is how the bboxes are postprocessed and visualized. CC @jakaskerl to explain in more details.

For training, how are you training your models? The easiest way to get it working would be to follow one of our tutorials (like YoloV6) here: https://colab.research.google.com/github/luxonis/depthai-ml-training/blob/master/colab-notebooks/YoloV6_training.ipynb. Then you can take the .pt weights and upload them to tools.luxonis.com.

In Inference section, you can see the inference of the trained model. I'd recommend you check there first. If model is working correctly, then try it on OAK (use tools.luxonis.com to make an export). If the predictions are not OK, try to add more images and quality data and retrained the model.

I'll let @jakaskerl answer how to add a Yolo to the OAK GUI.

niloofarsabouri
Can we first confirm the model behaves as expected, before adding handlers to GUI app.
Could you try and modify the script for RGB yolo preview. I assume you will be using YoloDetectionNetwork node (since you are using yolov7), so won't need additional on host decoding.
Anchors and anchor masks are not needed for yolov7 (I think) so you can remove that part. Play around with IOU threshold and confidence.

Thanks,
Jaka

niloofarsabouri
RGB yolo example I sent above will help you find out if the model works properly (pinpointing the issue to either the model or the way the depthai_demo is handling the results).

Use the example as is, but change the blob path to your own model blob file. Change the preview size to match the YOLO input size. Remove

detectionNetwork.setCoordinateSize(4)
detectionNetwork.setAnchors([10, 14, 23, 27, 37, 58, 81, 82, 135, 169, 344, 319])
detectionNetwork.setAnchorMasks({"side26": [1, 2, 3], "side13": [3, 4, 5]})

change other parameters to conform to your NN.

Thanks,
Jaka