• What is Maximum Models that can be on device?

Is there a limit based on shaves or inference threads? Can 4 different models be loaded on the device?

    Hi AdamPolak
    AFAIK there is no hardcoded limit for the number of models. There are of course issues regarding the processing power (queue saturation/freezing) with would limit you from adding to many models. I believe you should be able to run 4 different models (let me know if this is not the case) if you correctly divide them across the available resources, though the FPS will largely depend on the complexity of the models.

    Thanks,
    Jaka

        jakaskerl

        Could you elaborate on what you mean by dividing them across available resources? Does that mean shaves?

          Yes, shaves. Also, I tested with 6 models running at the same time (and same shaves) and despite the obviously low fps, it seems to work fine.

          Also getting the:
          [system] [info] NeuralNetwork allocated resources: shaves: [0-12] cmx slices: [0-12]

          Hope this helps,
          Jaka