YOLO model for classification

Hey @MahdiMseddi ,

Yes, you are correct. We do not support classification models at tools.luxonis.com. But from the graph it looks like the model is correctly exported. My guess would be that the final layer output0 does nothing and is just an output layer. However, you need to be careful when converting the model from ONNX to blob. Since it's from Ultralytics, my guess would be that ONNX expect BGR images in range 0-1, but on OAK we feed BGR images with range 0-255. This means that you need to specify `--scale 255` model optimizer flag when converting ONNX to blob. This should be possible under advanced options.

The output0 should return probabilities of each class. Number at index 0 will be probability for class 0.

MahdiMseddi

Could you try checking the name in the ONNX for the Softmax layer and specify `--output layer_name` flag to model optimizer as well? Just in case that output0 layer does something with it

MahdiMseddi

In case Yolo-Cls expect RGB images and you are feeding BGR from OAK you should also specify the `--reverse_input_channels`.

Matija

I will check that and add both params

Thank you

Hi guys martin181818 Matija

I got a similar problem, which is my ResNet18 image classification trained with PyTorch only shows 2 classes (while there are 7 classes in total). And these 2 detected classes always come with a confidence of 100%

Now I am wondering if I should go for another classification model, or tweak my current model's architecture a bit. Any thoughts please?

Cheers,

Austin

YWei

Hey,

ResNet18 should work fine. Are you sure:

1/ Your export process is correct? If you use ImageNet normalization PyTorch model will expect normalized images, while images on OAK get passed in unnormalized manner [0-255]. Some more info here and here.

2/ You are correctly post-processing the outputs? Usually Softmax is not a part of the model itself, so you might have to apply it to the outputs you get from the model.

Thank you so much Matija

Your advice is really on spot. I followed your first point and my model just works way better.

furthermore, may I ask more about your second opinion. Usually how could we really extract the detected results from an image classification model when running OAK camera? Are there any examples I could have a look?

The examples about running a classifier with OAK I can find just process like this way:

## I skip the part of creating node ##

def to_tensor_result(packet):
    return {
        tensor.name: np.array(packet.getLayerFp16(tensor.name)).reshape(tensor.dims)
        for tensor in packet.getRaw().tensors
    }


def softmax(x):
    """Compute softmax values for each sets of scores in x."""
    e_x = np.exp(x - np.max(x))
    return e_x / e_x.sum(axis=0)


## Here I just jump to the part of retrieving result ##

def to_planar(arr: np.ndarray, shape: tuple) -> np.ndarray:
        resized = cv2.resize(arr, shape)
        return resized.transpose(2, 0, 1).flatten()

        inDet = qDet.tryGet()

        if inDet is not None:
            data = softmax(inDet.getFirstLayerFp16())
            result_conf = np.max(data)
            if result_conf > 0.2:
                result = {
                    "name": labels[np.argmax(data)],
                    "conf": round(100 * result_conf, 2)
                }
            else:
                result = None

Really love to know more about your opinions

Regards,

Austin

Hi YWei ,
We have a few examples with classification models, and usually you do postprocessing on host/Script node (like softmax);

• TensorFlow example
• Image quality assessment
• Fire classification
• EfficientNet classification model

I hope these help!

erik

Thank you Erik, I will definitely read them through!

Hi YWei ,
I believe just the softmax layer of the NN can't be executed on the accelerated HW (no support), so it would need to be run elsewhere - perhaps on the host computer, or on-deivce inside the Script node.

Hi Erik erik

Can you be more detailed please (not very familiar to these…)? Since Oak cannot do anything with Softmax by itself, so this operation is basically done by our computer/host (as most of other examples did)?

And that means we cannot really do softmax with a device in a standalone mode?

Am I understanding your opinion correctly?

Cheers,

Austin

Hi YWei ,
Some examples were shared above, where softmax is done on host computer using numpy:

def softmax(x):
    """Compute softmax values for each sets of scores in x."""
    e_x = np.exp(x - np.max(x))
    return e_x / e_x.sum(axis=0)

You can also use pure python to accomplish the same, and run it inside Script node (on-device), so standalone mode would be possible.

erik

Roger it! I sent a new post for this for convenience link