• DepthAI-v2
  • Confuguring Camera and ImageManip nodes for Pytorch model

Let's say I have converted a model from PyTorch (with ONNX export) to Myriad X blob.

Given that PyTorch models expect an input in the format (B, C, H, W) with RGB color order, in DepthAI Python API how should I initialize the Camera Node and the Image Manip node and link each other in order to feed images to the NN node?

First Assuming a 224x224 image shape and then a bigger a shape like 400x400

(I ask this question because my camera node always provides frames in (H, W, C) shape despite changing the parameter setInterleaved. Then I am unsure of how handling optimally shapes which are greater than the maximum preview size in the camera node)

    Hi @brick
    When converting the model to .blob, you can specify the layout to be something different if you have issue with it.
    Docs: https://docs.luxonis.com/en/latest/pages/model_conversion/#model-layout-parameter

    Also make sure to set color order to RGB. cam.setColorOrder().

    brick Then I am unsure of how handling optimally shapes which are greater than the maximum preview size in the camera node

    I'm not sure I follow. Please elaborate.

    Thanks,
    Jaka

      jakaskerl

      Thanks for your response. I am referring to the common use case, I have a neural network in Pytorch which expects inputs of shape (batch_size, 3, 224, 224), input data are preprocessed with the pytorch transformations

      ToTensor(),

      Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])

      I export the model with torch.onnx.export, then what are the correct parameters to use on the online blobconverter? Lastly, what is the proper configuration for the Camera Node on DepthAI Python API in order to feed images the NN node?

        Hi @brick
        Make sure you set the mean and the scale values so the RGB values fall in the same range (explained here).

        brick Lastly, what is the proper configuration for the Camera Node on DepthAI Python API in order to feed images the NN node?

        link the preview output to the NN node and make sure to set the size to (setPreviewSize()) to 224, 224.

        Thanks,
        Jaka