Stereo vision to 3D reconstruction

OneWorld

Hi,

I'm a University of Surrey Computer Vision MSc graduate, specialising in Deep Learning for Video Interpolation or Novel View Synthesis (I graduated a while back now).

I'm embarking on a small hobby project to see how I can project an image onto a 3D object using Oak-D
The first part of this problem is in the generation of a 3D Model (a reconstruction) from the two 2D images.

From a quick literature review I have seen this paper:-
Toward 3D Object Reconstruction from Stereo Images - https://arxiv.org/pdf/1910.08223.pdf

MeshStereo: A Global Stereo Model with Mesh Alignment Regularization for
View Interpolation - http://chizhang.me/MeshStereo.pdf

I then saw this link https://github.com/timzhang642/3D-Machine-Learning

And realised that there's actually quite a lot of research in the field.

Where would you suggest I start with something like this ?

Happy to share my dev work.

OneWorld

dhruvsheth-ai

This is an intriguing topic to work on using spatial perception in OAK-D for 3D reconstruction. There might be lot of resources out there, where data captured through mono format can be used to construct point clouds, or the recent implemented example by Tensorflow for 3D Semantic segmentation. There are many varied cases, and some of the implemented and tested ones are for Point Cloud Estimation using depth or using RGB input. Here -

https://github.com/luxonis/depthai-experiments/tree/master/point-cloud-projection

https://github.com/luxonis/depthai-experiments/tree/master/pcl-projection-rgb

https://github.com/luxonis/depthai-experiments/tree/master/gen2-camera-demo

Would be interested in learning what you are planning to implement =D. Thanks!