Camera Calibration for 12MP (OAK-D Pro W)

vkmNuller

Hi,

Currently, I am trying to use the full resolution (12MP) and undistort the color frames using the ImgMnip node. However, the calibration I get from the EEPROM is for 4K, and I cannot be sure if the undistortion is correct or if it will align with the depth I generate.

Is it possible to create a calibration for 12MP?

jakaskerl

Hi @vkmNuller
You can check this

Hope it helps.

Thanks,
Jaka

vkmNuller

Hi @jakaskerl,

Thank you for the information.

I'm using the OAK-D Pro W, and distortion in the color frame is an issue. I want to undistort the color frame (at 12MP) and align it with the depth frame. We want this, because this data is going to be use for NN input.

In the example you provided, I don't see you undistorting the color frame. Is this required for better alignment?

I have used this example for color undistortion:
luxonis/depthai-pythonblob/main/examples/ColorCamera/rgb_undistort.py

What is the best combination for correctly aligning the images?
Also, in this example, there is no crop after the undistortion.

In this example from OpenCV they crop after the undistortion: https://docs.opencv.org/4.x/dc/dbb/tutorial_py_calibration.html

How can I integrate the crop as well?

Thanks

jakaskerl

Hi @vkmNuller
Generally, the alignment script assumes that both frames are undistorted, so it will work better if they actually are. This is a problem on WFOV cameras since the distortion is higher.

I don't think the current HW is powerful enough to support 12MP warping.

vkmNuller How can I integrate the crop as well?

Crop refers to the alpha value of the undistortion. You can set it to 0, the frame will be cropped to ensure no black pixels are present. Alpha=1 will show all the pixels from the distorted image, but there will be a black edge around the frame (depends on the undistortion amount).
The issue is that by setting the getOptimalNewCameraMatrix(), the image is zoomed in, thus the images are not aligned anymore. But you can use the same crop bbox for the depth image to fix that.

Thanks,
Jaka

vkmNuller

Hi @jakaskerl

Thank you for the information, will test it out.

Sorry for repeating my self , but I'm still uncertain whether the calibration on the device, which is set for 4K resolution (3840x2160) with a 16:9 aspect ratio, will properly undistort a 12MP frame. The aspect ratio differs, and there's more information involved.

Thanks

jakaskerl

Hi @vkmNuller
Should be fine as long as you specify the resize dimensions. The new aspect ration will be taken into account when rescaling the intrinsic matrix.luxonis/depthai-coreblob/879bedb1460dea7d078f559cda4a85b234233525/src/device/CalibrationHandler.cpp#L204-L278

Thanks,
Jaka