• DepthAI-v2
  • multi-view acquisition with multiple OAK-Ds

Hello, I run in a machine learning/AI lab oriented towards medical application. We would like to create our own high-resolution dataset for the purpose of characterizing the movements of patients of all ages in a hospital environment. We can't replicate this adequately in a lab environment using labeled motion capture.

As a second-best option, we would like to acquire multi-view video from 3 cameras with distinct views on the hospital room. Originally, we were planning to do this with conventional cameras, but it seems that a set of the Oak-D cameras (OakD pro wide) might be able to give us even more data for the same number of enclosures placed.

We would love to get both the 1080p@60fps and the depth estimate from the cameras and have IR capability for night-vision and stream the results to host computer(s) for post-acquisition analysis.

I wanted to see if anyone has done this? If so what sort of problems need to be overcome.

Questions that occur to me:

  • do I need a separate device to synchronize the cameras
  • do the IR laser dot projectors interfere with each other or improve things by giving a denser "dot texture"
  • can one host computer handled the I/O stream.

Our hope is that once we have our dataset, we would acquire more OAK cameras to run patient/hospital tuned models.

Many thanks in advance
-Chris

    Hi ChrisLee-Messer !
    We have multi-devce calibration demo on which you could build on top. And your projecct actually aligns nicely with multi-cam tracking feature request - feel free to add any suggestions/requests on there๐Ÿ™‚

    • The example above uses software (timestamp) syncing, which is accurate enough in most cases (at 60FPS => 16ms/2 => below 8ms difference). You could also use hardware syncing.
    • That might be the case, but dots might also overlap which wouldn't imrpove things.
    • I think it really depends on the bandwidth. For PoE that's usually 1gbps, for USB3 it's usually 5gbps, so you can calculate how many streams a host can receive based on required FPS, resolution, and encoding.

    Thoughts?
    Thanks, Erik

    a month later