Training custom yolo model

Hi niloofarsabouri ,
Have you also tried notebooks with newer yolo versions (yolov6-yolov8)?

Hi niloofarsabouri
I think the voc is used here for ease of use. since the dataset is pre-made. Also it helps show how to convert between formats since many datasets exist in VOC. According to yolov6 training notebook:

If you are using a custom dataset, you will have to prepare your dataset for training YOLOv6 Wiki. Once you have set up YAML and sorted labels and images into the right directories, you can continue with the next step.
Luckily, we are using VOC, for which YAML already exists. If you inspect the YAML. We'll be the following tutorial on how to train YOLOV6 on the VOC data set.

Thanks,
Jaka

Hi niloofarsabouri

wget http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCtrainval_06-Nov-2007.tar -O ./VOCdevkit/VOCtrainval_06-Nov-2007.tar

Let's break it down:

1. wget - This is a command-line utility to download files from the internet. It supports downloading via HTTP, HTTPS, and FTP.
2. http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCtrainval_06-Nov-2007.tar - The URL of the file you want to download, in this case, the PASCAL VOC 2007 training/validation data.
3. -O - An option for wget that allows you to specify the output filename and path.
4. ./VOCdevkit/VOCtrainval_06-Nov-2007.tar - Where you want the downloaded file to be saved. The . indicates the current directory, so it will save in the VOCdevkit subdirectory with the provided filename.

If you wish to substitute your data, just place the data some folder on your machine and point to it later in the tutorial when path for the dataset is used.

Thanks,
Jaka

cc. @Matija

niloofarsabouri

If you've got it trained, path to weights should likely be runs/train/exp5/weights/best.pt. Notice the 5 in exp5 (not just exp). Which number it is should be based on the experiment that was successful. Assuming it was the last run, check what the exp with the highest number is in runs/train folder.

Hi niloofarsabouri
Clone it using https:
git clone https://github.com/luxonis/depthai-experiments.git

What you are doing is cloning it using SSH, for which you need a key.

Hope this helps,
Jaka

niloofarsabouri

When you specify -m yolo it tries to download it from our cloud. You should pass the path to your .blob path there. So, in .zip from tools.luxonis.com you will get the .json (which you are using correctly) and a .blob file. -m should be path to that blobfile.

python3 main_api.py -c best.json -m best_openvino_2022.1_6shave.blob

niloofarsabouri

114 images is barely the minimum. You should try to use more. The quality of annotations will also affect how well the model works.

Hi niloofarsabouri
Do you absolutely need the NN to perform this task? Can you modify the windows in any way?

We had a competition where the goal was to as quickly as possible, fly a drone through a series of hoops. But each hoop had an Aruco marker glued on the pole on which the hoop was mounted. By using a marker, you would know the distance from the hoop center to the drone.

Would this be possible in your case?

Thanks,
Jaka

Hi niloofarsabouri
Not really sure what you mean by non-processing, but you can get frames straight from the device, without any manipulations done to the image.

Thanks,
Jaka

niloofarsabouri

Yeah, there are more way to approach it. If you just use OAK as a camera, and do the processing on the Jetson, you could follow and implement something like this square detector.

On the other hand, neural network should also be able to detect this, given the shape and color is very distinct. In that case you would need a relatively good and diverse dataset. I would say you'd need at least 100 images for the first iteration, then train the model. After training, you should inspect the test results you get from the training on "test" data (data not used for training). If performance is not good enough, you should collect and annotate more images and repeat.

Handler is how the bboxes are postprocessed and visualized. CC @jakaskerl to explain in more details.

For training, how are you training your models? The easiest way to get it working would be to follow one of our tutorials (like YoloV6) here: https://colab.research.google.com/github/luxonis/depthai-ml-training/blob/master/colab-notebooks/YoloV6_training.ipynb. Then you can take the .pt weights and upload them to tools.luxonis.com.

In Inference section, you can see the inference of the trained model. I'd recommend you check there first. If model is working correctly, then try it on OAK (use tools.luxonis.com to make an export). If the predictions are not OK, try to add more images and quality data and retrained the model.

I'll let @jakaskerl answer how to add a Yolo to the OAK GUI.

niloofarsabouri
Can we first confirm the model behaves as expected, before adding handlers to GUI app.
Could you try and modify the script for RGB yolo preview. I assume you will be using YoloDetectionNetwork node (since you are using yolov7), so won't need additional on host decoding.
Anchors and anchor masks are not needed for yolov7 (I think) so you can remove that part. Play around with IOU threshold and confidence.

Thanks,
Jaka

niloofarsabouri
RGB yolo example I sent above will help you find out if the model works properly (pinpointing the issue to either the model or the way the depthai_demo is handling the results).

Use the example as is, but change the blob path to your own model blob file. Change the preview size to match the YOLO input size. Remove

detectionNetwork.setCoordinateSize(4)
detectionNetwork.setAnchors([10, 14, 23, 27, 37, 58, 81, 82, 135, 169, 344, 319])
detectionNetwork.setAnchorMasks({"side26": [1, 2, 3], "side13": [3, 4, 5]})

change other parameters to conform to your NN.

Thanks,
Jaka

Hi @niloofarsabouri
Can't really tell what the issue is here. There looks to be a mistake in folder structure and model config. Could you maybe share the .ipynb with outputs.

Thanks,
Jaka

The Official Guide has some mistake with the folder structure, I came up with the same issue.
I modified the folder structure as following, just like other suggestion from different reource.

dataset/Image/train

dataset/Image/val

dataset/Image/test

dataset/label/train

dataset/label/val

dataset/label/test