Best way to train vegetable crop models for Oak-D PoE

mannem

I am using Oak-D PoE and I want to perform detection of 3 different vegetable crops. namely, kohlrabi, Eisbergsalad and Lollosalad. However, Each vegetable crop has 19 development stages(1 stage is 1 class). and i want to detect all the development stages in one single NN model (total of 57 classes). My question to you is
Q1: Which detection algorithm suites to identify all the development stages effectively?
Q2: If i want convert the model how many shaves should i use ?
And, I need only 10 Fps in runtime over the oak-D.

jakaskerl

mannem
That's difficult to say, you might look at some research papers that did a similar thing. Also make sure the 19 development stages are discernible enough. In any case you would need custom decoding since you are using a classification model as opposed to a detection one. cc @Matija any thoughts?

mannem if i want convert the model how many shaves should i use ?

Depends on the rest of the pipeline. You will probably need to train for 5 or 6 shaves.

The FPS largely depends on input resolution and model complexity.

Thanks,
Jaka

KlemenSkrlj

Hi @mannem ,

Regarding the models: I would advise you use YOLOv6 Nano R2 since it is a good model for object detection that is also optimized for running on the edge. You can check it's performance in this table compared to other models.
The most straightforward way of training would be to have a model with 57 classes (3 species * 19 stages). But I'm not that familiar with this domain and I'm not sure how similar these stages are between eachother.
If this doesn't work well you could also have a 2 stage approach. For this you could train one object detection model which only detects the crop (3 classes) and 3 separate classification models (one for each vegetable type with 19 classes). You would then pass each detection from the first model into correct second model to predict it's development stage.
I would suggest you first try the first option though.

Regards,
Klemen

mannem

Thanks for the reply, for you to understand the difficulty, the development stages are very close to each other you can find images below for the difference. These stages are defined based on BBCH levels.

Q1:Is it possible to train model with such close features (I need 19 stages for each class)?
Q2: Should i use classification or detection ? which is best in my case.
Q3: Do you have any notebook or training example for classification like you have for detection "luxonis/depthai-ml-training"
I am open to any kind of suggestions.

KlemenSkrlj

1/ Honestly I can't say that even I can differentiate between different stages on these images. So I would imagine it would be very hard task for the model as well. The performance also largely depends on the amount of data and based on my intuituion I would say you would need very large amount for the model to learn all 19 classes per crop. Perhaps even more complex, larger model would be needed (e.g. YOLOv6 small or YoloV6 medium). Can you maybe simplify the problem to less, more distinct classes instead of all 19 of them?
2/ I would still first try training a detection model to see if it is able to learn. And if the anwser is no then add classification. But this is also related to the point above.
3/ We don't have a notebook specifically for classification yet. We are activly developing our luxonis-train library though which has support for training also classification models, feel free to check it out.

mannem

Thank you very much. your suggestions make sense to me. I will try training the detection model to see the results (probably i will chose only three BBCH levels of one crop). One last question how many images would you suggest for me to train the per class ? Should i use augmentation techniques or not ?

KlemenSkrlj

probably i will chose only three BBCH levels of one crop
Yeah this seems more reasonable for the starting experiments, you can always add more later.

how many images would you suggest for me to train the per class ?
Rule of thumb for deep learning is that the more quality data you have the better the end model is. By quality data I mean that all annotations are correct and exact, images cover variaty of different environments, backgrounds, etc. And generally you also need more data for more complex tasks. I can't give you an exact number as this depends on a specific problem. You can start with something like 500-1000 images per each grown stage (3 crops * 3 grown stages ≃ 5k-10k images all together).

Should i use augmentation techniques or not ?
Absolutly, augmentations increase the variability of the dataset for free. Just be careful to pick augmentations that are reprezentable of the occurances that could happen in real life. For example for this task you could add some random brightness but it wouldn't make sense to add image warping.