Thanks Matija !

Unfortunately, when I tried "DeepLabV3plus_MNV2.ipynb" with my custom dataset (2 classes instead of 21 classes), I was getting an error with the eval.py related to the class size change (I tried all types of fixes I found on the internet with no luck). So, I moved over to using Keras to train the DeepLabV3 model. However, the OpenVino XML file generated from the Keras model has a different structure than the XML file in "DeepLabV3plus_MNV2.ipynb". So, I am not sure if my OpenVino 2021.4 BLOB issue (no results) is related to the XML issue or a completely different problem.

Do you guys have a date of when OpenVino 2022.2 will be supported?

Thanks,
Mike E

      The fix is there just for the version from the Colab, which when exporting mixes up some types. In case you're using a different model, then this does not relate to you. The issue is likely that the layout of the model in TF/Keras is NHWC rather than NCHW which is used on OAK.

      Could you try using flag --layout NCHW:NHWC when calling model optimizer meaton ?

          meaton

          Unfortunately, when I tried "DeepLabV3plus_MNV2.ipynb" with my custom dataset (2 classes instead of 21 classes), I was getting an error with the eval.py related to the class size change (I tried all types of fixes I found on the internet with no luck)

          Did you change the number of classes before generating tfrec files in data_generator.py? That should fix it 🙂

              8 days later

              Matija

              The flag --layout NCHW:NHWC gives me the following error

              Layout name is invalid (:). Only english letters, digits and _ is allowed

              However, it helped to point out that I need to handle the conversion from NHWC to NCHW. I plan to modify the model in the Keras script to handle the conversion.
              Thanks!
              Mike E

                  stephansturges

                  Yeah, I updated the number of classes in data_generator.py prior to generating tfrec files. At this point, I figure I will just focus on using Keras. Thanks for the suggestion..

                    meaton

                    Hm, it seems this was wrongly written in documentation. You can instead also try setting --source_layout nhwc and --target_layout nchw or vice versa.

                        Matija
                        Thanks for the tip! This new setting resolved the

                        [18443010A157641200] [1.1.1.3] [9.855] [NeuralNetwork(2)] [warning] Input image (256x256) does not match NN (3x256)

                        error when using OpenVino 2022.1. Below, is the updated python code I used in Keras. Unfortunately, my model still does not segment the object correctly (at all), whereas I get proper results with the Keras model. I plan to use the OpenVino Inference Engine to verify the converted model (IR format) still works.

                        subprocess.call("mo \
                        --saved_model_dir \"" + model_export_dir + "\" \
                        --model_name " + model_base_name + " \
                        --data_type " + data_type + " \
                        --input_shape [1," + str(IMAGE_SIZE) + "," + str(IMAGE_SIZE) + ",3] \
                        --reverse_input_channel \
                        --source_layout nhwc \
                        --target_layout nchw \
                        --output_dir \"" + export_vino_dir + "\"", shell=True)

                            meaton
                            Great to hear that. And yes, using OpenVINO Inference Engine is usually the best way to debug that. I noticed that you don't have any preprocessing flags when exporting the model.

                            Images on OAK are usually UINT8 BGR images. If your model in Keras accepts those directly, that is fine. But if your model in Keras takes normalized/standardized images, you might want to use --mean_values and --scale_values flags. If your model takes BGR images as input, then there is also no need for --reverse_input_channel.

                            Let me know how it goes 🙂

                                6 days later

                                Matija
                                Thanks for the mean/scale value suggestion! That lead me down the right path and the results on the OAK-D are better. Now, I can focus on improving the training of the model.

                                Here is the call to convert the model
                                subprocess.call("mo \
                                --saved_model_dir \"" + model_export_dir + "\" \
                                --model_name " + model_base_name + " \
                                --data_type " + data_type + " \
                                --input_shape [1," + str(IMAGE_SIZE) + "," + str(IMAGE_SIZE) + ",3] \
                                --reverse_input_channel \
                                --source_layout nhwc \
                                --target_layout nchw \
                                --mean_values [104,117,123] \
                                --scale_values [255,255,255] \
                                --output_dir \"" + export_vino_dir + "\"", shell=True)

                                Here is the code for blob converter
                                blob_path = blobconverter.from_openvino(
                                xml=xml_file,
                                bin=bin_file,
                                data_type=data_type,
                                shaves=6,
                                version="2022.1"
                                )

                                    meaton, glad to hear that! Often models use ImageNet mean and std for normalization. If this will be useful to someone else in the future, this is how flags look like after scaling to 0 - 255 range:
                                    --mean_values [123.675,116.28,103.53] and --scale_values [58.395,57.12,57.375].

                                    Have fun improving the models and using OAK-D 🙂

                                    Best,
                                    Matija

                                        Matija

                                        In Keras, I am using the ResNet50 model that was pre-trained on ImageNet. So, I used --mean_values [123.675,116.28,103.53] and --scale_values [58.395,57.12,57.375] to follow your advice and it improved the results on the OAK-D. Thanks again!

                                        subprocess.call("mo \
                                        --saved_model_dir \"" + model_export_dir + "\" \
                                        --model_name " + model_base_name + " \
                                        --data_type " + data_type + " \
                                        --input_shape [1," + str(IMAGE_SIZE) + "," + str(IMAGE_SIZE) + ",3] \
                                        --reverse_input_channel \
                                        --source_layout nhwc \
                                        --target_layout nchw \
                                        --mean_values [123.675,116.28,103.53] \
                                        --scale_values [58.395,57.12,57.375] \
                                        --output_dir \"" + export_vino_dir + "\"", shell=True)

                                            4 months later

                                            I am still having the same issue with [NeuralNetwork(0)] [warning] Input image (512x512) does not match NN (3x512) using openvino_2022.2 optimizer (onnx->xml) and compiler(xml->blob).

                                            any fix?