• DepthAI

  • Easiest way to load custom model onto Oak?

I met the similar problem, i use yolov5 model trained with pytorch. Following the steps, 1convert to onnx, 2change the top output layer, 3convert openvino, 4convert to blob, 5add model *.json. The result is wrong. I can not find what happened after these converting steps.

Hi @liu and @barney2074 ,

I apologize for the inconvenience. Due to different ML frameworks and frequent updates, tools and models can slightly vary, so it's hard to keep a consistent stack at all times. That said, our goal is to build a platform that will make this as easy as possible and convenient for the user, and most importantly, follow the motto "robotic vision made simple". We partially started this with https://tools.luxonis.com/, where you can generate a blob and JSON for YoloV5 model directly from the .pt weights. We want to develop more such tools that will require little to no coding from the customer, to make this process easier for customer as well as for us. This is still in alpha version, but feel free to give it a try @liu . Any feedback is more than welcome 🙂

For custom models, conversion from Pytorch is typically easier, as you always have to go Pytorch -> ONNX -> OpenVINO IR -> blob. With TF, you can go directly to OpenVINO IR, but it depends what version of TF you're using. There are plenty of options for saving/exporting TF models as well, which makes this a bit more complicated. If you are using TF1, you need to generate a frozen .pb file or use a saved model. Then you can directly use model optimizer to generate the IR. For TF2, you are required to used the SavedModel format. For better explanation you can check OpenVINO 2021.4 docs - note that 2022.1 is not yet supported. The easiest way would be to pip install openvino-dev==2021.4.2, generating IR by providing --saved_model_dir path/to/saved/dir where dir includes .pb, assets, and variables. After obtaining IR, you can use blobconverter to compile a blob from OpenVINO IR. If the model is in NHWC format, then you'll most likely have to provide --layout "nhwc->nchw" flag to model optimizer as well.

Colab tutorials are designed in a way to show you the whole process - setting up the environment, training, and deployment. We are trying to keep them updated, but as new versions of libraries are being released, it's hard to keep them updated at times. That said, if you find a certain bug, please bring it up so we can fix it. Saying "it fails for one reason or another" does not help us improve documentation, tutorials, or tools in any way.

We are also trying to do our best helping customers with the whole export and compilation process here on Discuss, Discord, and even via email. But to do so, we need to know the model you are trying to convert, version of the tools you are using, what commands are you calling, what are the various errors you are getting, ... So, please share the exact problems you are having @barney2074 so that we can look into it, fix it, and make it easier for everyone 🙂

Best,
Matija

  • liu replied to this.

    Matija
    Hi Matija,
    Thank you very much for your detailed reply. Yes, i understand the complexity of different DL framework and their versions. Thanks.
    And i tried the https://tools.luxonis.com/. it's great and convenient.
    And maybe 'input shape' need certain input format? please check it, Thank you very much!


        Hi liu , yes input shape requires you to input an integer that is divisible by 32. I'd recommend 416 or 608. I think those are default values used for image size for nano and S model in training, respectively. In the next version we'll also add support for non-squared images.

        • liu replied to this.

            Matija
            Hi Matija, great. So quick 🙂
            I can not send the error screenshots (maybe network problem), the error is "An error occurred, requests failed with status code" (Red character). And my 'input shape' input is ' 1,3,640,640 ', is that correct format? My pretrained model input size is 640, its M model (about 40 MB).

              @liu , just provide 640 as input shape. This will be better explained by the tooltip in the next versions 🙂 What FPS are you looking for? If realtime, I'd suggest a nano model, I think M will reach around ~ 5 FPS, and you can expect ~ 15 FPS with S model.

              OK, thank. I first try to pass the blob convert phase, than i will re-train a S model, thanks for your advice.

              Great, if you run into some conversion issues and want to send an image, feel free to send it directly to me (matija@luxonis.com) 🙂

              Thank you. The online converting passed! great.
              But the detection result is not ok. I will send you some result later

              18 days later