If you use OpenVino 2021.4 with the fix (change layer 490 to 32 bit if memory serves) it will work with DeepLabV3 networks and you won't have to mess with this mess... it's entirely possible that Intel have messed something up in 2022.1 that's causing your issue, I would avoid spending too much time digging into this.
Issue with model converted with OpenVino 2022.1 and not 2021.4
Hey @meaton ,
Best to try what @stephansturges suggested. I think there is a bug in 2022.1, which should be resolved in 2022.2 (not yet supported in DepthAI). But I think your issue is related to this bug fix:
Fixed scenario when both reverse_input_channel and changing the layout are applied in the same conversion
Best,
Matija
- Edited
Thanks Matija !
Unfortunately, when I tried "DeepLabV3plus_MNV2.ipynb" with my custom dataset (2 classes instead of 21 classes), I was getting an error with the eval.py related to the class size change (I tried all types of fixes I found on the internet with no luck). So, I moved over to using Keras to train the DeepLabV3 model. However, the OpenVino XML file generated from the Keras model has a different structure than the XML file in "DeepLabV3plus_MNV2.ipynb". So, I am not sure if my OpenVino 2021.4 BLOB issue (no results) is related to the XML issue or a completely different problem.
Do you guys have a date of when OpenVino 2022.2 will be supported?
Thanks,
Mike E
The fix is there just for the version from the Colab, which when exporting mixes up some types. In case you're using a different model, then this does not relate to you. The issue is likely that the layout of the model in TF/Keras is NHWC rather than NCHW which is used on OAK.
Could you try using flag --layout NCHW:NHWC when calling model optimizer meaton ?
Unfortunately, when I tried "DeepLabV3plus_MNV2.ipynb" with my custom dataset (2 classes instead of 21 classes), I was getting an error with the eval.py related to the class size change (I tried all types of fixes I found on the internet with no luck)
Did you change the number of classes before generating tfrec files in data_generator.py? That should fix it 
8 days later
The flag --layout NCHW:NHWC gives me the following error
Layout name is invalid (:). Only english letters, digits and _ is allowed
However, it helped to point out that I need to handle the conversion from NHWC to NCHW. I plan to modify the model in the Keras script to handle the conversion.
Thanks!
Mike E
Yeah, I updated the number of classes in data_generator.py prior to generating tfrec files. At this point, I figure I will just focus on using Keras. Thanks for the suggestion..
Matija
Thanks for the tip! This new setting resolved the
[18443010A157641200] [1.1.1.3] [9.855] [NeuralNetwork(2)] [warning] Input image (256x256) does not match NN (3x256)
error when using OpenVino 2022.1. Below, is the updated python code I used in Keras. Unfortunately, my model still does not segment the object correctly (at all), whereas I get proper results with the Keras model. I plan to use the OpenVino Inference Engine to verify the converted model (IR format) still works.
subprocess.call("mo \
--saved_model_dir \"" + model_export_dir + "\" \
--model_name " + model_base_name + " \
--data_type " + data_type + " \
--input_shape [1," + str(IMAGE_SIZE) + "," + str(IMAGE_SIZE) + ",3] \
--reverse_input_channel \
--source_layout nhwc \
--target_layout nchw \
--output_dir \"" + export_vino_dir + "\"", shell=True)
meaton
Great to hear that. And yes, using OpenVINO Inference Engine is usually the best way to debug that. I noticed that you don't have any preprocessing flags when exporting the model.
Images on OAK are usually UINT8 BGR images. If your model in Keras accepts those directly, that is fine. But if your model in Keras takes normalized/standardized images, you might want to use --mean_values and --scale_values flags. If your model takes BGR images as input, then there is also no need for --reverse_input_channel.
Let me know how it goes
6 days later
Matija
Thanks for the mean/scale value suggestion! That lead me down the right path and the results on the OAK-D are better. Now, I can focus on improving the training of the model.
Here is the call to convert the model
subprocess.call("mo \
--saved_model_dir \"" + model_export_dir + "\" \
--model_name " + model_base_name + " \
--data_type " + data_type + " \
--input_shape [1," + str(IMAGE_SIZE) + "," + str(IMAGE_SIZE) + ",3] \
--reverse_input_channel \
--source_layout nhwc \
--target_layout nchw \
--mean_values [104,117,123] \
--scale_values [255,255,255] \
--output_dir \"" + export_vino_dir + "\"", shell=True)
Here is the code for blob converter
blob_path = blobconverter.from_openvino(
xml=xml_file,
bin=bin_file,
data_type=data_type,
shaves=6,
version="2022.1"
)
meaton, glad to hear that! Often models use ImageNet mean and std for normalization. If this will be useful to someone else in the future, this is how flags look like after scaling to 0 - 255 range:
--mean_values [123.675,116.28,103.53] and --scale_values [58.395,57.12,57.375].
Have fun improving the models and using OAK-D
Best,
Matija
- Edited
In Keras, I am using the ResNet50 model that was pre-trained on ImageNet. So, I used --mean_values [123.675,116.28,103.53] and --scale_values [58.395,57.12,57.375] to follow your advice and it improved the results on the OAK-D. Thanks again!
subprocess.call("mo \
--saved_model_dir \"" + model_export_dir + "\" \
--model_name " + model_base_name + " \
--data_type " + data_type + " \
--input_shape [1," + str(IMAGE_SIZE) + "," + str(IMAGE_SIZE) + ",3] \
--reverse_input_channel \
--source_layout nhwc \
--target_layout nchw \
--mean_values [123.675,116.28,103.53] \
--scale_values [58.395,57.12,57.375] \
--output_dir \"" + export_vino_dir + "\"", shell=True)
4 months later
I am still having the same issue with [NeuralNetwork(0)] [warning] Input image (512x512) does not match NN (3x512) using openvino_2022.2 optimizer (onnx->xml) and compiler(xml->blob).
any fix?
