• Has any work been done on spatial-detection-fusion for non-overlayed devices?

This is a great example of how to get the relative position between sensors using the checkerboard

https://github.com/luxonis/depthai-experiments/tree/master/gen2-multiple-devices/spatial-detection-fusion

The issue is that all 3 sensors see the same checkerboard.

If there are many sensors covering many rooms not all devices would see the same checkerboard. It would be a graph structure linking overlapping sensors.

Has there been any work done on this area?

    Hi AdamPolak
    AFAIK not, but it should be quite straightforward if you know the relative positions of the checkerboards. Just make sure no device sees two boards at the same time.

    Regards,
    Jaka

        jakaskerl

        Each one would need a different checkboard though no? Or are you saying to stitch the extrinsics together one at a time?

        • erik replied to this.

            Hi AdamPolak , in theory it would be possible to create such a calibration script that would calibrate multiple devices, if all cameras have overlapping with at least 1 other camera, like on the img below. It would also be possible to run this dynamically, so to have only 1 calib board and cameras would look for it continously. We do plan on abstracting the multic-cam usage on depthai SDK (along with other abstractions), and such script could be a part of it. Thoughts?
            Thanks, Erik

                erik

                Right now I import stills to colmap along with a video of me walking around the space. Once it solves I then have to try and export that into extrinsics of the devices, and it does not work well as it is not what colmap was meant to do and it is missing a degree of freedom as it does not support depth.

                If we could print out different checkboards and lay them around the area and have a script that automatically finds the relative position between sensors, that would be a game changer.

                It seems what you are suggesting would be even better. If we had live feeds from all the devices, and I could move a single checkboard around until we have the graph of relative positions between all the sensors. Currently, that is my #1 pain point when setting up sensors in indoor spaces, finding the relative position between sensors for accurate fusion of data.

                  @AdamPolak yep, I agree, it's something on our roadmap for the SDK! Feel free to also create feature request for SDK on depthai repo for this feature, so we can close it once it is done.
                  Thanks, Erik

                  19 days later