Focal length incorrect in camera calibration

Tom

Hello!

I am using several of the IMX577 M12 modules, which come with a 75 deg H-FoV lens. I am having error in my camera calibration, specifically in the focal length. Although the error isn't giant, it is very significant for my application.

Based off the resolution of (4056, 3040), the pixel size of 1.55 umand the H-FoV of 75 degrees, I can calculate my expected focal length to be 4.096558636989045 mm, or 2642.9410561219647 px. I have also experimentally determined the focal length by setting up a chaurco board perpendicular to my camera, measuring the Z-distance, and looking at the FoV. I also get roughly this number (within 10 px or so) doing this. So, I am fairly confident that this is the result I am looking for.

Now, when I perform my calibration, I get a focal length of something closer to 2530 px . This is across several cameras, and I have gone through great effort to get the highest quality calibration possible.

I get good re-projection error (~0.7 px).
I am using a high quality board from calib.io. I have tried two separate boards from them, as well as a printed out board on paper, and I always get within a few pixels of this result.
I am mounting my camera to a tripod, and making sure there is no motion what so ever.
I have good lighting with no glare.
My lens is focused well.
I am taking data mainly at my expected working distance (which is where the lens is focused).
I have lots of varied data, and I have many examples of foreshortening, which I know is important for focal length.
I am using lots of images. I have tried as few as 10-15 and as many as 100, and always get roughly this result.

My distortion parameters come out great, but the intrinsic matrix is always slightly wrong. Just to confirm I wasn't going crazy, I have also ran the images collected through Matlab, as well as my own OpenCV implementation, and I get the exact same results.

What could be going on here?

jakaskerl

Tom
What script are you using for calibration? Make take a note at which resolution the calibration is performed (4K or 12MP). There is some cropping of image from 12MP to 4K which could cause the issue.

Thanks,
Jaka

Tom

jakaskerl

I am using the main branch's calibrate.py.

I also updated this dictionary to use 12MP for the calibration:

camToRgbRes = {
    "IMX378": dai.ColorCameraProperties.SensorResolution.THE_4_K,
    "IMX214": dai.ColorCameraProperties.SensorResolution.THE_4_K,
    "OV9782": dai.ColorCameraProperties.SensorResolution.THE_800_P,
    "IMX582": dai.ColorCameraProperties.SensorResolution.THE_12_MP,
    "AR0234": dai.ColorCameraProperties.SensorResolution.THE_1200_P,
    "IMX577": dai.ColorCameraProperties.SensorResolution.THE_12_MP,
}

The images I am getting are (4056, 3040), so it seems image capture is correct.

jakaskerl

Tom
Just to clarify, which results are different. The matlab and opencv, or depthai calibration results?

Thanks
Jaka

Tom

All three are very similar, but the result I am getting from all is quite a bit off from expected - roughly a focal length of 2530 px.

jakaskerl

Tom
And the expected result is calculated where and how? I would expect all three approaches to be quite similar, but what is the ground truth you are using?

Thanks,
Jaka

Tom

jakaskerl

To get the expected result, I set up the camera perpendicular to the charuco board, and measured the distance from the sensor to the board. Removing 4.1 mm from this measurement, I get (within a mm or two) the depth of the pinhole camera (board to image plane).

Then, I took an image, and measured the pixels of a length of the board using the charuco corners. I also know the size in mm of this length based off the square size. Then, I can get focal length from the pinhole model as follows:

f_px = (size_px * depth_mm) / size_mm

This gives me 2633 pixels. I am aware this test is not very accurate, but it is accurate enough for me to know that I should be probably within 20-30 pixels of this figure. I was having some measurement issues using the intrinsic matrix from calibration, and using this new focal length my accuracy has improved, but I would like to rely on the actual calibration to further improve accuracy.

jakaskerl

Tom
This is expected, since you are also counting in the distortion which affects the end image (number of pixels in the end).

You can try manually setting all CV2_FLAGS, but this won't apply distortion correction.

Further:

https://docs.opencv.org/4.x/d9/d0c/group__calib3d.html#gga89a5ea2c97ef26165c47e792380b96bca564091bdcc5ab14785f185e16430ff62

Thanks,
Jaka