gen2 mega depth is it possible to set to higher resolution?

jpickens

I'm using depthai_experiments main.py script to generate aligned RGBD images.
The blob file /depthai-experiments/gen2-mega-depth/models/megadepth_192x256_openvino_2021.4_6shave.blob
seems to be set to 192X256 pixels. Is it possible to change this file to the full resolution of the depth camera pair (640X480)?

I tried the command:
NN_WIDTH, NN_HEIGHT = 480, 640
into the script, and it gives an error:
[18443010510F8E1200] [333.752] [NeuralNetwork(0)] [warning] Input image (480x640) does not match NN (256x192)

I assume that means that even though the NN_WIDTH command is used to specify width, that the blob file is preloaded with the lower resolution parameters.

I'm using an Oak-D lite, and its stated depth camera resolution is 640 X 480.

Any ideas how to get to the stated resolution? There are significant object edge artifacts in the RGBD image, which may be caused by mapping the much lower resolution depth map to the 1080 X 1440 color image which is 31 times less dense. The 480X640 resolution would be lower by a factor of 5, which would improve things greatly.

erik

Hello jpickens,
The input image resolution (in your case 256x192) depends on the model itself. The model was trained & exported for 256x196 resolution, so you can't change the input resolution dynamically, but would need to retrain the model with higher-res imgs. If you are familiar with ML, you could try checking the model source code and retrain it. Thoughts?
Thanks, Erik

jpickens

Erik,
Thanks for your response. Your explanation sounds reasonable.
I guess what I'm looking for is more tied to procedural Left/Right depth mapping from stereo pairs than ML from single images. I've browsed through the example scripts in the depthai sections, and haven't found what I'm looking for.

My ideal I/O would be a 640 x 480 pixel stereo pair derived depth map used to create a 4:3 RGBD still image at the Oak-D Lite maximum color camera resolution of 3840 x 2880.

Anyone with suggestions as to how to achieve this would be greatly appreciated.

erik

Hello @jpickens ,
I would suggest looking at the RGB-depth alignment example. Depth map will get scaled to the RGB resolution, so if your color cam is 4k, the depth map should get upscaled from 480P to 4k. Would that work?
Thanks, Erik