Intrinsics of preview

Bryan

I have an OAK D Pro:

sensor resolution set to 4k
preview resolution 640x480
setPreviewKeepAspectRatio is True

I am working with a preview derived image stream and I require the camera intrinsics. Currently I am reading them from calibration with a custom resolution specified:
rgbd_intrinsics = calibData.getCameraIntrinsics(dai.CameraBoardSocket.RGB, resizeWidth=640, resizeHeight=480, keepAspectRatio=True)
Which returns [514.22424316, 0., 324.875, 0., 514.22424, 243.36540222, 0., 0., 1. ]

My problem is that objects located in the scene (apriltags and checkerboards) are being found closer to the camera than they actually are, by a ratio of 23/30 (₀.76). If I multiply out the f_x and f_y terms of the intrinsic then i get pretty good distance measurements out to 1 meter.

I assume I'm not setting something appropriately to get the correct intrinsics - what am I doing wrong? Complete code can be found here: https://github.com/Taiga-Robotics/luxonis_support/blob/main/scripts/oak_d_node.py

jakaskerl

Hi Bryan
Have you tried getting the camera intrinsics without resizeWidth and resizeHeight? I'm not entirely sure they are necessary.

Thanks,
Jaka

Bryan

Thanks for the reply, I just tried that and im getting reported depths of my apriltag that are about 5x reality, here's the intrinsics without resizing:

[3.08534521e+03 0.00000000e+00 1.94924988e+03
 0.00000000e+00 3.08534521e+03 1.10019238e+03 
 0.00000000e+00 0.00000000e+00 1.00000000e+00]

Of note: my initially calculated correction factor is very close to the ratio of cropped to uncropped pixels after the 4k image is scaled to 640 by 480 with 1:1 aspect ratio:
2160/480 = 4.5
3840/44.5 = 853.3
853.3/640 = 0.75
and the difference between 0.75 and 0.76 is well within my experimental tolerance here on the desk.

jakaskerl

Hi Bryan
Thanks, that is very useful information. The sensor resolution of imx378 is 4056x3040 which is 4/3 - the same as 640:480. When you use 4K resolution, the FOV gets cropped on both width and height to conform to 4K's 16:9 aspect ratio.
I see see you are also using the keepAspectRatio=True. I'm not entirely sure which resolution that aspect is kept from, but perhaps the 4K is used which would likely cause the discrepancy you are experiencing.

Thoughts?
Jaka

Bryan

You make a good point - I had somehow assumed that the scaling of f_x and f_y would be related to the 4k/resize ratio, but doing a source dive i see that it's pulling resolution from eeprom - that's going to be the native sensor resolution for sure.

https://github.com/luxonis/depthai-core/blob/main/src/device/CalibrationHandler.cpp#L204

Bryan

I have written my own intrinsic scaling function and done some digging. It looks like getCameraIntrinsics does use the cropped sensor resolution as the optical centre co ordinates are already adjusted to the 4k resolution (1949, 1100 is about half of 3840, 2160). It's not clear to me why the resize+keep aspect part of the depthai algorithm didn't work out - that may be a candidate for review by luxonis.

My revised intrinsic scaling function is now part of my node here:
https://github.com/Taiga-Robotics/luxonis_support/blob/main/scripts/oak_d_node.py#L42

This code now produces correct intrinsics to track my apriltag here on the desk, and I expect my team to report a successful handeye calibration with the checkerboard.
Thanks for the help jakaskerl!