Using Host GPU for inference with DepthAI and OAK-D

man_hat

As a newbie, what I have understood and read about DepthAi and OAK-D. The DepthAI passes the *.blob file to the built-in OAK-D CPU for inference.

However, my question is it possible to perform the inference on the host GPU (RTX 3090) while only receiving the data (images) from the camera? This question might be silly considering why DepthAI is being developed.

erik

Hello man_hat ,
I agree with liu. The OpenVINO's Inference Engine (IE in short) uses model in IR format (.bin/.xml) which are used to create .blob. So you could convert model into IR format and use the IE to run the model either on VPU, CPU, GPU, etc.
Thanks, Erik

liu

A suggestion
1 get the image data with depthai api as the link https://docs.luxonis.com/projects/api/en/v2.8.0.0/samples/rgb_preview/

2 you develop your own inference code with certain open framework and language in your GPU host

3 use the image data (np format) from step 1 and send to your own inference module.

yes, you just use a little ability of oak-D, but sometimes we need this combination in our project.