What is Maximum Models that can be on device?

AdamPolak

Is there a limit based on shaves or inference threads? Can 4 different models be loaded on the device?

jakaskerl

Hi AdamPolak
AFAIK there is no hardcoded limit for the number of models. There are of course issues regarding the processing power (queue saturation/freezing) with would limit you from adding to many models. I believe you should be able to run 4 different models (let me know if this is not the case) if you correctly divide them across the available resources, though the FPS will largely depend on the complexity of the models.

Thanks,
Jaka

AdamPolak

jakaskerl

Could you elaborate on what you mean by dividing them across available resources? Does that mean shaves?

jakaskerl

Yes, shaves. Also, I tested with 6 models running at the same time (and same shaves) and despite the obviously low fps, it seems to work fine.

Also getting the:
[system] [info] NeuralNetwork allocated resources: shaves: [0-12] cmx slices: [0-12]

Hope this helps,
Jaka