getAllLayerManes is returning fixed values ( [0,1] since am classifying 2 classes )eventhough the model works fine before the conversion.
Could this be a conversion problem ? and how to solve it ?
THank you
YOLO model for classification
I tried to convert the model using this tool: DepthAI Tools (luxonis.com) but it doesn't work.
so I am converting it to openvino or onnx and then pass it to : BlobConverter | Convert model to MyriadX blob (luxonis.com)
That is why probably the problem is related to conversion
Hey @MahdiMseddi ,
Yes, you are correct. We do not support classification models at tools.luxonis.com. But from the graph it looks like the model is correctly exported. My guess would be that the final layer output0 does nothing and is just an output layer. However, you need to be careful when converting the model from ONNX to blob. Since it's from Ultralytics, my guess would be that ONNX expect BGR images in range 0-1, but on OAK we feed BGR images with range 0-255. This means that you need to specify `--scale 255` model optimizer flag when converting ONNX to blob. This should be possible under advanced options.
The output0 should return probabilities of each class. Number at index 0 will be probability for class 0.
Thank you very much for your reply.
I already tried --scale_values=[255,255,255] and --scale=255 but the probabilities are still fixed and the model loses its performance after the conversion.
I also tried to train other tf model like in OpenVINO + OpenCV-OAK - Tensorflow Image Classification.ipynb - Colaboratory (google.com) but the conversion is not working in the notebook ( errors while converting)
I need to implement a classification model . When I am using yolov8-cls it is converted but loses the performance and when I try to convert the tf models the conversion fails.
Thank you again for your help and I will be grateful if you can give me some hints on how to proceed to make one of the solutions work.
Could you try checking the name in the ONNX for the Softmax layer and specify `--output layer_name` flag to model optimizer as well? Just in case that output0 layer does something with it
In case Yolo-Cls expect RGB images and you are feeding BGR from OAK you should also specify the `--reverse_input_channels`.
4 months later
Hi guys martin181818 Matija
I got a similar problem, which is my ResNet18 image classification trained with PyTorch only shows 2 classes (while there are 7 classes in total). And these 2 detected classes always come with a confidence of 100%
Now I am wondering if I should go for another classification model, or tweak my current model's architecture a bit. Any thoughts please?
Cheers,
Austin
Hey,
ResNet18 should work fine. Are you sure:
1/ Your export process is correct? If you use ImageNet normalization PyTorch model will expect normalized images, while images on OAK get passed in unnormalized manner [0-255]. Some more info here and here.
2/ You are correctly post-processing the outputs? Usually Softmax is not a part of the model itself, so you might have to apply it to the outputs you get from the model.
Thank you so much Matija
Your advice is really on spot. I followed your first point and my model just works way better.
furthermore, may I ask more about your second opinion. Usually how could we really extract the detected results from an image classification model when running OAK camera? Are there any examples I could have a look?
The examples about running a classifier with OAK I can find just process like this way:
## I skip the part of creating node ##
def to_tensor_result(packet):
return {
tensor.name: np.array(packet.getLayerFp16(tensor.name)).reshape(tensor.dims)
for tensor in packet.getRaw().tensors
}
def softmax(x):
"""Compute softmax values for each sets of scores in x."""
e_x = np.exp(x - np.max(x))
return e_x / e_x.sum(axis=0)
## Here I just jump to the part of retrieving result ##
def to_planar(arr: np.ndarray, shape: tuple) -> np.ndarray:
resized = cv2.resize(arr, shape)
return resized.transpose(2, 0, 1).flatten()
inDet = qDet.tryGet()
if inDet is not None:
data = softmax(inDet.getFirstLayerFp16())
result_conf = np.max(data)
if result_conf > 0.2:
result = {
"name": labels[np.argmax(data)],
"conf": round(100 * result_conf, 2)
}
else:
result = NoneReally love to know more about your opinions 
Regards,
Austin
Hi YWei ,
We have a few examples with classification models, and usually you do postprocessing on host/Script node (like softmax);
I hope these help!
To add to that, if you use a basic ResNet18 from torchvision for example, it will not contain a Softmax operation. You can take the ONNX and inspect the last node in netron.app.
In those cases you have to use the softmax like in the example you've shared. Feel free to check out other examples shared by Erik as well.
Best,
Matija
- Edited
Hi Majita Matija and Erik erik
Thank you for your explanation, and I tried more today but didn't get any good.
Not very familiar with those concepts mentioned. You said we probably can't use a softmax to retrieve my classified results, so what solutions else I could use to achieve that accurately? i did a bit study then and it seemed like softmax is just a math way to extract the results without really changing any calculated numbers I guess?
Now I have a trained ResNet18 and YOLOV5s-cls model, both trained with PyTorch. To verify that my trained models can actually work well, I ran them on my host to do classification inferences respectively today (so without an OAK D device). They both worked better than on my OAK D, and a part of code is like below:
# Inference with YOLOv5
with torch.no_grad():
results = model(img_tensor)
# Convert the logits to percentages using softmax
probs = F.softmax(results, dim=1)
# Get class with highest confidence
confidences, class_idx = probs.squeeze(0).max(0)
label = f'{class_names[class_idx]} {confidences:.2f}'Is there anything wrong with the configuration or set-up on OAK D(such as the process of resizing images, colour channel order, etc) or I should try another version of model (e.g., YOLOv6 or higher)? I'm very confused now…
Cheers,
Austin
Hi YWei ,
I believe just the softmax layer of the NN can't be executed on the accelerated HW (no support), so it would need to be run elsewhere - perhaps on the host computer, or on-deivce inside the Script node.
Hi Erik erik
Can you be more detailed please (not very familiar to these…)? Since Oak cannot do anything with Softmax by itself, so this operation is basically done by our computer/host (as most of other examples did)?
And that means we cannot really do softmax with a device in a standalone mode?
Am I understanding your opinion correctly?
Cheers,
Austin
Hi YWei ,
Some examples were shared above, where softmax is done on host computer using numpy:
def softmax(x):
"""Compute softmax values for each sets of scores in x."""
e_x = np.exp(x - np.max(x))
return e_x / e_x.sum(axis=0)You can also use pure python to accomplish the same, and run it inside Script node (on-device), so standalone mode would be possible.