The fix is there just for the version from the Colab, which when exporting mixes up some types. In case you're using a different model, then this does not relate to you. The issue is likely that the layout of the model in TF/Keras is NHWC rather than NCHW which is used on OAK.
Could you try using flag --layout NCHW:NHWC when calling model optimizer meaton ?
Unfortunately, when I tried "DeepLabV3plus_MNV2.ipynb" with my custom dataset (2 classes instead of 21 classes), I was getting an error with the eval.py related to the class size change (I tried all types of fixes I found on the internet with no luck)
Did you change the number of classes before generating tfrec files in data_generator.py? That should fix it 
The flag --layout NCHW:NHWC gives me the following error
Layout name is invalid (:). Only english letters, digits and _ is allowed
However, it helped to point out that I need to handle the conversion from NHWC to NCHW. I plan to modify the model in the Keras script to handle the conversion.
Thanks!
Mike E
Yeah, I updated the number of classes in data_generator.py prior to generating tfrec files. At this point, I figure I will just focus on using Keras. Thanks for the suggestion..
Matija
Thanks for the tip! This new setting resolved the
[18443010A157641200] [1.1.1.3] [9.855] [NeuralNetwork(2)] [warning] Input image (256x256) does not match NN (3x256)
error when using OpenVino 2022.1. Below, is the updated python code I used in Keras. Unfortunately, my model still does not segment the object correctly (at all), whereas I get proper results with the Keras model. I plan to use the OpenVino Inference Engine to verify the converted model (IR format) still works.
subprocess.call("mo \
--saved_model_dir \"" + model_export_dir + "\" \
--model_name " + model_base_name + " \
--data_type " + data_type + " \
--input_shape [1," + str(IMAGE_SIZE) + "," + str(IMAGE_SIZE) + ",3] \
--reverse_input_channel \
--source_layout nhwc \
--target_layout nchw \
--output_dir \"" + export_vino_dir + "\"", shell=True)
meaton
Great to hear that. And yes, using OpenVINO Inference Engine is usually the best way to debug that. I noticed that you don't have any preprocessing flags when exporting the model.
Images on OAK are usually UINT8 BGR images. If your model in Keras accepts those directly, that is fine. But if your model in Keras takes normalized/standardized images, you might want to use --mean_values and --scale_values flags. If your model takes BGR images as input, then there is also no need for --reverse_input_channel.
Let me know how it goes
Matija
Thanks for the mean/scale value suggestion! That lead me down the right path and the results on the OAK-D are better. Now, I can focus on improving the training of the model.
Here is the call to convert the model
subprocess.call("mo \
--saved_model_dir \"" + model_export_dir + "\" \
--model_name " + model_base_name + " \
--data_type " + data_type + " \
--input_shape [1," + str(IMAGE_SIZE) + "," + str(IMAGE_SIZE) + ",3] \
--reverse_input_channel \
--source_layout nhwc \
--target_layout nchw \
--mean_values [104,117,123] \
--scale_values [255,255,255] \
--output_dir \"" + export_vino_dir + "\"", shell=True)
Here is the code for blob converter
blob_path = blobconverter.from_openvino(
xml=xml_file,
bin=bin_file,
data_type=data_type,
shaves=6,
version="2022.1"
)
meaton, glad to hear that! Often models use ImageNet mean and std for normalization. If this will be useful to someone else in the future, this is how flags look like after scaling to 0 - 255 range:
--mean_values [123.675,116.28,103.53] and --scale_values [58.395,57.12,57.375].
Have fun improving the models and using OAK-D
Best,
Matija
In Keras, I am using the ResNet50 model that was pre-trained on ImageNet. So, I used --mean_values [123.675,116.28,103.53] and --scale_values [58.395,57.12,57.375] to follow your advice and it improved the results on the OAK-D. Thanks again!
subprocess.call("mo \
--saved_model_dir \"" + model_export_dir + "\" \
--model_name " + model_base_name + " \
--data_type " + data_type + " \
--input_shape [1," + str(IMAGE_SIZE) + "," + str(IMAGE_SIZE) + ",3] \
--reverse_input_channel \
--source_layout nhwc \
--target_layout nchw \
--mean_values [123.675,116.28,103.53] \
--scale_values [58.395,57.12,57.375] \
--output_dir \"" + export_vino_dir + "\"", shell=True)
I am still having the same issue with [NeuralNetwork(0)] [warning] Input image (512x512) does not match NN (3x512) using openvino_2022.2 optimizer (onnx->xml) and compiler(xml->blob).
any fix?
