Depth perception 360 degrees

I appreciate your reply and all the hard work it must take to bring these products to market.
I own the OAK-FFC-IMX378-W (Camera sensor) (Qty: 6) OAK-FFC-6P (Qty: 1) Although other cameras system sold by Luxonis may do the job, I would really like to try and get the system I own working. I really think I made the best choice for this in particular task.
It is very interesting that using setting the camera's at a 72 degree angle may give me a 360 view as this would leave a camera to set perpendicular to the 360 view. (straight up or down)
As you were saying, about mounting the camera's at an angle, depth perception is currently not supported. Why not? It just hasn't been done yet or is there a deeper reason I'm missing? For depth perception I thought what I needed was field of view overlap. I'm trying to keep up, but you guys at Luxonis are advancing this technology quickly. It fact I guess my lack of understand is astounding because I don't understand why I cant also mount these cameras at a 45 degree angle additionally and still get depth perception. I thought I understood this stuff, I have been doing it for 8 years (dedicating 4 month a year), please educate me, what am I not getting?
Think of the future and viewing things in VR where you can look around anywhere, the whole 360 view must be capture and recorded, for playback. It's not live stereo vision with the vr headset moving a live stereo camera. We are taking about a whole new way to make video recordings (the whole 360 view, all at once) and I believe with a little thought you will have the market cornered because there are lot's of other applications, like a new form of SLAM, employee monitoring (for safety of course), boating (above (for navigation) and below the water), crowd monitoring, the sky's the limit.
Hopefully you are excited as I am to try at advancing the technology in this direction.

Hi codejocky ,

It just hasn't been done yet or is there a deeper reason I'm missing?

It hasn't been done with our cameras/platforms afaik. So likely you would need to hack the calibration process and rectification matrix. But I am not familiar enough with either to be able to provide any additional pointers. Also note that if you do mount 2 camera pairs at 60° angle (that have 85° HFOV), you would get a small portion of an angle (25°/2) that's seen from both stereo cams and can run disparity matching on.

For NERF, that's a bit of a different solution - it's closer to SfM than to stereo depth.
Thoughts?
Thanks, Erik

Well baby steps right. With stereo depth perception, on ether side of the FOV there is an area that the other camera in the pair does not see. Is that a 25 degree angle? I guess we are back to our original problem. But we may be further ahead; using the cameras at a 72 degree angle will the entire 360 view be covered by two cameras and at what distance? What about the 60 degree angle and at what distance from the camera system?
This is a new camera system and questions like these are worth investigating.
I must say, I love this! Here we are, doing such exciting things with our lives, pushing the technology forward, making a difference. In my life I have seen countless people live and die without affecting the world in the least bit. Be joyful in the fact that we made a difference. We were not lost in the sands of time but have become the "ghost in the machine" and our ingenuity will be the foundation our descendant build upon. Thank you, for your service to the science of computer vision.

I had better go through this documentation more thoroughly and see what answers I can come up with.
https://docs.luxonis.com/projects/hardware/en/latest/pages/DM3390.html
https://docs.luxonis.com/projects/api/en/latest/components/nodes/stereo_depth/#max-stereo-depth-distance
https://docs.luxonis.com/projects/hardware/en/latest/pages/DM3399.html
I'll get to it right away.

Hi codejocky ,
My understanding of how you thought of mounting the cameras:

Where beta is rectified/depth HFOV. Having cameras tilted even more (72deg) would reduce the overlap. But let me know if I am understanding this idea incorrectly.
And we totally agree on technology, we love being on the "bleeding edge" as we are the ones that are pioneering these fields! And please reach out to erik@luxonis.com, we have a small gift for you for sharing our beliefs and evangelizing this area of technology
Regarding the links - you got it all figured out I see. Let us know if you have any further questions.
Thanks, Erik

I had this thought.
using depth perception I could pick out objects and give them bounding boxes, then take the frame of the bounding box and reduce the size of the picture in the bounding box to fix the A.I. models picture input size say 250x250 (as it is a common size to train models) then feed these pics into the A.I. only saving the image search from being the inter picture and now only a few boxes. This should save some time so the A.I. could use that time to scan the 250x250 for an increased library of items it's trying to identify. Has somebody else thought of that?

I meant to say picture size is 224x224 so I could use IMAGENET. for a large library of object to detect.
plus depending on the size of object detected there should be a way to adjust bias toward that type of image identification. like an elephant is bigger than a mouse so they should be a bias toward elephant.

codejocky
Answering my own question, doing more research...
[https://www.stereolabs.com/blog/performance-of-yolo-v5-v7-and-v8/#:~:text=In%20conclusion%2C%20all%20three%20versions,performance%20out%20of%20the%20three.]
This article talks about yolo8 and a depth camera for situational awareness. not exactly what I was hoping for as both are computed separately. I wanted depth perception to create a bounding box and only these feed into yolo8. additionally I wanted the depth perception detection based on the previous frame and only rescan the frame for new objects say 0.2 seconds or have a thread dedicated to this task alone.