CHW to HWC and RBG to BGR order transforms for models

Tim

Can you explain two discrepancies with the depthai hello_world.py tutorial example please?

1) On line 65 in hello_world.py the oak-d camera frame is transformed from CHW to HWC order. However, the openvino toolkit documentation states that the required order for the modified mobilenet-ssd model is CHW.
2) Also I don't see in this same hello_world.py example where the frame RGB is reversed to BGR as required?

These two steps do appear to have been performed in this example that the conversion is HWC -> CHW (line 152) and RBG -> BGR (line 154) when using recorded video.

Thanks in advance for your help.

Tim

erik

Hello Tim,
1) It is set on line 13 with cam_rgb.setInterleaved(False), meaning the output will be planar (CHW).
2) You can set it with cam_rgb.setColorOrder(dai.ColorCameraProperties.ColorOrder.RGB), but BGR colour order is set by default.
And apologies for this not being clear, I have added it on todo to add this info to docs as well, so it's easier to find.
Thanks, Erik

Tim

Thanks for the clarification Erik! Huge help!

Still a little confused, however.

As you said, Line 13 specifies CHW and then on line 65 we change it to HWC. That's fine. However, it says in the openvino docs that the mobilenet-ssd model used by the hello_world.py example requires CHW:

Converted model
Image, name - prob, shape - 1,3,300,300, format is B,C,H,W where:

B - batch size
C - channel
H - height
W - width
Channel order is BGR

erik

Hello Tim, exactly, on line 13 it's CHW, it sends the CHW frames both to the mobilenet-ssd model and to the host, and on the host it converts them to HWC.

Note that it's a bit low-level tutorial, we could just use in_rgb.getCvFrame() instead of lines 62-66 and use mobilenetDetectionNetwork instead of NeuralNetwork so it would decode the result on the host.

Tim

Got it finally. Thank you for your patience. I did not catch that the CHW to HWC conversion was actually on the host side.

erik

Tim No worries, I'm glad I could help🙂