It would be very helpful for validating software built against depthai if we had a software-only simulator that spoke xlink and could connect to depthai as a virtual camera device. The super ideal would be something that hooked into something like Gazebo, but even a simple simulator that could just verify that depthai applications will work on real devices would be very helpful for development, CI, etc.

Does Luxonis have anything like this internally that could be reworked to be a public project? Or has anyone started to implement this on their own? If I were to try to implement something like this, how would I start writing something that could talk XLink, or accept pipelines?

  • erik replied to this.

    Hi aposhian ,
    We agree it would be very useful, but there's no simple way to simulate the hardware accelerator blocks, especially 1:1 simulation. One could approximate the real device using eg. OpenVINO's inference engine for AI, opencv for CV and stereo depth (somewhat), but it would be very challenging to create exact replica (even if you have FW source code). That's why we just have 50+ OAKs connected to a server(s) as our "OAK farm" (for QA testing purposes) instead🙂
    Thanks, Erik

      I see what you are saying, but even if the simulator wasn't feature complete, but simply could accept the parameters being set over XLink, and then publish a dummy result, that would be very helpful for verifying basic streaming pipelines.

        Hi aposhian ,
        Due to the technical efforts needed, we likely won't be pursuing this ourselves anytime soon. But feel free to create a feature request on eg. depthai repo for trackability.
        Thanks, Erik

          Is there XLink API documentation somewhere?

          Does the firmware side depend on the same XLink library for communication, or does it use a different library internally? i.e., should I be able to use that XLink library as the building blocks for a simulator?

          • erik replied to this.

            Could you share any hints or code as to how to connect as an XLink device, or could you link to an example for a movidius firmware that does this?

            Is the XLink protocol essentially symmetric: in that each end thinks of the other side as a "device"? Or is there a client-server relationship?

            • erik replied to this.

              Hi aposhian ,
              I think it would be best to create issue on XLink repo, as I am not that familiar with it.
              Thanks, Erik