Converting onnx to blob given 2 image inputs

AileenLiao

I have a custom model that I have converted to onnx file. It has 2 image inputs (right and left camera eyes) each of size 1xheightxwidthx3. Can I convert this to blob? What would the mean and scale values be? Thanks!

jakaskerl

Hi @AileenLiao
Yes you can. Just follow the guide. https://docs.luxonis.com/software/ai-inference/conversion/

The mean and scale values are used to normalize the input images for the model. The values you use for mean and scale depend on how your model was trained. If your model was trained with images normalized to a range of 0 to 1, you would use a mean of 0 and a scale of 255. If your model was trained with images normalized to a range of -1 to 1, you would use a mean of 127.5 and a scale of 127.5. If your model was trained with images normalized to a range of -0.5 to 0.5, you would use a mean of 127.5 and a scale of 255.

Thanks,
Jaka

AileenLiao

Hi @jakaskerl,

Thank you for the reply. The guide does not have an example with multiple inputs though. Would the mean and scale value formats be --mean_values [[B,G,R], [B,G,R]] and --scale_values [[B,G,R], [B,G,R]] ? The format of my flags is wrong and the blob converter throws errors about input size mismatch

justincdavis

Hi @"AileenLiao",

I work a lot with custom models on the OAK and wrote this helper function which handles the PyTorch -> blob conversion process.
justincdavis/oakutilsblob/main/examples/blobs/compile*model.py*
That shows a brief example on how you can define the network with a wrapper class. In order to handle multiple inputs have the input_names method return a list. Something like [("left", InputType.U8), ("right", InputType.U8)] would probably work since you said you are passing the left/right cameras (use FP16 if color data is used).
Then when running the network make sure to pass the inputs through two separate links. In the same repo I have some functions which can help set that up.

As for decoding, that part is harder since you have to read the custom networks outputs as raw data and then shape accordingly. There are also some helper functions I have in that error, but all of this was written to make various work tasks easier so it is far from comprehensive and I am sure Luxonis has examples as well.

The mean and scale values are not currently used in the things I have mentioned (since we use primarily computation represented as PyTorch models) so if you need those I would use the blobconverter tool.

AileenLiao

justincdavis Thank you very much! I will try to use your repo to convert directly from torch to blob!