Question about device preprocessing of image

heesono

Hello,

I am using a yolov4 tiny model that has an input shape of [1,800,800,3] and BGR input. The output shape is [1,50,50,30] and [1,25,25,30]. When I converted my .onnx file to .blob, I used "--reverse_input_channels", "--layout=nhwc".

This does however not change the layout immediately. I guess the default layout is nchw. That leads to following error messages:

I am not able to convert my input shape immediately with layout=nhwc->nchw because you can't combine a layout switch with --reverse_input_channels as far as I know so the warning in the screenshot will persist. However, I thought that when the image gets sent from the camera to the neural network, the shape of the image gets changed before being inputted into the neural network. Is that not correct?

Also, I am not exactly sure how to solve the second error message. It sees this output shape [1,50,50,30] as width 30 even though thats just number of anchors * bounding box infos and class prob.

My anchor settings are:

detectionNetwork.setAnchors([23, 27, 37, 58, 81, 82, 81, 82, 135, 169, 344, 319])

detectionNetwork.setAnchorMasks({"side50": [0, 1, 2], "side25": [3, 4, 5]})

Thanks for the help!

jakaskerl

heesono However, I thought that when the image gets sent from the camera to the neural network, the shape of the image gets changed before being inputted into the neural network. Is that not correct?

Image doesn't change by itself. The model stays static as well. Maybe try omitting the --reverse_input_channels flag and switching RGB/BGR inside ColorCamera node with setColorOrder.

heesono Also, I am not exactly sure how to solve the second error message. It sees this output shape [1,50,50,30] as width 30 even though thats just number of anchors * bounding box infos and class prob.

From my experience, this fixes itself when proper input is provided.

Thanks,
Jaka

heesono

jakaskerl From my experience, this fixes itself when proper input is provided.

Thanks for the tip with setColorOrder. I have now converted the input layout to nchw and the warning disappeared, but the mask error still persists. Do you have any idea for that one?

Maybe it has something to do with the blob output being [1,50,50,30] and [1,25,25,30], Output layer names: StatefulPartitionedCall:0, StatefulPartitionedCall:1 and depthai.node.YoloDetectionNetwork not accepting the blob output? I think YoloDetectionNetwork wants the blob output to be in the shape of [1, 30, 50, 50] and [1, 30, 25, 25]. Is there a way to reshape it before it gets processed further in YoloDetectionNetwork? My detections list is also empty.

qDet = device.getOutputQueue(name="nn", maxSize=4, blocking=False)
while True:
    inDet = qDet.tryGet()
    if inDet is not None:
    detections = inDet.detections
    counter += 1

jakaskerl

Hi @heesono
You are probably right. Could you try setting the output layout as well? https://docs.openvino.ai/archive/2022.1/openvino_docs_MO_DG_Additional_Optimization_Use_Cases.html#how-to-change-layout-of-a-model-inputs-and-outputs

Thanks,
Jaka