Deploying a pre-trained model for pose estimation onto Oak camera

SrijayanthSaravanan

How would I approach deploying a newer pretrained model for pose estimation onto the camera ( the inference must happen only in the camera and not on inference servers). What is best way to achieve this?

jakaskerl

Hi @SrijayanthSaravanan
You won't be able to achieve full on-device computing since that only supports Yolo and MobileNetSSD at the moment. You can still run the inference for your custom model, by converting it to .blob, but you will have to do custom decoding on host side (PC side).

Unless the output is really simple, in which case you can use the Script node to perform light arithmetic.

Thanks,
Jaka

SrijayanthSaravanan

@jakaskerl Thank you so much for the reply, can you elaborate on custom decoding on host side?

jakaskerl

@SrijayanthSaravanan
Well, the NeuralNetwork node will output a NNData message, which carries tensors of number type data which has to be decoded first, before anything meaningful can be retrieved.

These are mathematical operations than can not be directly run on the VPU (unless they are YOLO or MBNet type decodings which can be HW accelerated with depthai), so they have to be run on the PC side.

Example - this is YOLO, which can be done on device as well, but just for showcase:
luxonis/depthai-experimentsblob/master/gen2-yolo/host-decoding/main.py#L154-L165

Hope it clears things up.

Thanks,
Jaka

SrijayanthSaravanan

Thanks @jakaskerl , your replies have been very helpful.