How to run SAM model on OAK D CM4 POE Camera

GurdeepakSidhu

Hi Luxonis team,

I had a question on how I can use Objection Detection + SAM model on my OAK-D-CM4 POE camera.
Do you have any documentation or examples on how to set up a pipeline that supports Object Detection and SAM model?

Thank you and I look forward to hearing back from you.

Have a great day and best regards,

jakaskerl

Hi @GurdeepakSidhu
I think FastSAM s was converted for the RVC2 as well so it should be possible. In any case, SAM model will need custom decoding to work as depthai API doesn't natively support it.

Model:
https://hub.luxonis.com/ai/models/4af2416c-2ba4-4c85-97d0-fd26f089fc69?view=page

Thanks,
Jaka

GurdeepakSidhu

Hi Jaka,

Thank you for getting back with this information, good to know that FastSAM works for RVC2.

I just wanted to follow up on whether their example codes in which FastSAM is used for an RVC2 and that also includes custom decoding?
This would be helpful, for I am not sure where/how to start.

Thank you and appreciate it.

Have a great day and best regards.

jakaskerl

GurdeepakSidhu

The example at HubAI uses this repo / decoding:
luxonis/depthai-nodesblob/main/depthai_nodes/message/segmentation.py

It is designed to work with depthai v3 so you might need to change it a bit.

Thanks,
Jaka