Questions about the inference in gen2-nanosam

Stn73

Hi, I am interested in the gen2-nanosam project that was recently added to depthai-experiments.

In this project, the encoder (or embedder) is deployed on the OAK device, whereas the decoder is not and is run on the CPU instead. I don't quite understand how the decoder is executed. Is it run on the CPU of the OAK-D's raspberry CM4, or on a computer it is connected to?

If the decoder is deployed on the CM4, how can I install the onnxruntime on it, as a simple pip install fails for me.

jakaskerl

Hi Stn73
The decoder is run on the same host, on which the code is executed. If this is done on CM4, then the CPU used will be RPIs.
here <-- the call to decoder

What error are you getting when trying to install the onnxruntime?

Thanks,
Jaka

Stn73

Hi @jakaskerl thanks for the reply!

I get the error that there is no installation candidate. It think the onnxruntime is not available on pip for arm32 targets, no? I've also tried installing pytorch on the CM4 and got similar issues, which I suppose is due to the RaspiOS 32-bit.

Didn't you face these issues? Have you tried with the OAK-D CM4?

jakaskerl

Hi Stn73
Looks like that is the case, yes.

You can install the onnxruntime using prebuilt 32bit wheels: nknytk/built-onnxruntime-for-raspberrypi-linux
Use the 64bit RPI CM4 OS: https://drive.google.com/drive/u/1/folders/1O50jPpGj_82jkAokdrsG--k9OBQfMXK5

Thanks,
Jaka