Fields and formatting guide for calibration.json for flashing

cniessl

I'm currently trying to calibrate an OAK 4P FFC using a calibration board and method that is different from the built-in calibrate.py script included in the depthai repository. The procedure I am using is able to generate pinhole fx,fy,cx,cy values, radial and tangential distortion values for each imager. In addition, it also generates the rotation and translation transforms between the left and right imager used in the calibration procedure.

While I've checked the Depth Calibration documentation pages (https://docs.luxonis.com/hardware/platform/depth/calibration?searchTokens=%5B%22calibration%22%2C%22Calibration%22%5D) I was not able to find any documentation relating to what fields are expected or how to edit them in a calibration.json file.

I also did a search through the forum using 'calibration.json' (https://discuss.luxonis.com/?q=calibration.json%20is%3Asolved) but only got results related to 'How to flash' and none on editing the file.

Looking through source I know that flashCalibration2 (https://docs.luxonis.com/software/api/python/#depthai.DeviceBase.flashCalibration2) will flash a dai.CalibrationHandler() object that has been populated, some entries are either missing any documentation, such as SetCameraInstrinsics (https://docs.luxonis.com/software/api/python/#depthai.CalibrationHandler.setCameraIntrinsics) or do not include the order of fields that should be specified in the list, such as SetDistortionCoefficients (https://docs.luxonis.com/software/api/python/#depthai.CalibrationHandler.setDistortionCoefficients)

Any pointers to such a document or guide (if it exists), or a description of what's expected for each of those settings, would be helpful and appreciated.

jakaskerl

cniessl
Random calibration dump. Added comments to help:

{
##### LEAVE THESE
    "batchName": "",
    "batchTime": 1681367328,
    "boardConf": "IR-C00M00-00",
    "boardCustom": "",
    "boardName": "NG9097",
    "boardOptions": 7,
    "boardRev": "R4M2E4", 

###### CHANGE THESE
    "cameraData": [
        [
            0, ## SOCKET ENUM - A, B, C,...
            {
                "cameraType": 0, ## leave 0
                "distortionCoeff": [ ## as the name implies
                    17.491783142089844,
                    -221.29478454589844,
                    -0.000791755854152143,
                    0.0004502780793700367,
                    790.3506469726563,
                    17.052539825439453,
                    -217.1355743408203,
                    776.1067504882813,
                    0.0,
                    0.0,
                    0.0,
                    0.0,
                    0.0,
                    0.0
                ],
                "extrinsics": {
                    "rotationMatrix": [
                        [
                            0.0,
                            0.0,
                            0.0
                        ],
                        [
                            0.0,
                            0.0,
                            0.0
                        ],
                        [
                            0.0,
                            0.0,
                            0.0
                        ]
                    ],
                    "specTranslation": {
                        "x": -0.0,
                        "y": -0.0,
                        "z": -0.0
                    },
                    "toCameraSocket": -1,
                    "translation": {
                        "x": 0.0,
                        "y": 0.0,
                        "z": 0.0
                    }
                },
                "height": 2160, # height at which intrinsics were calibrated
                "intrinsicMatrix": [
                    [
                        3101.561767578125,
                        0.0,
                        1907.5284423828125
                    ],
                    [
                        0.0,
                        3101.561767578125,
                        1096.5872802734375
                    ],
                    [
                        0.0,
                        0.0,
                        1.0
                    ]
                ],
                "lensPosition": 0, # ignore
                "specHfovDeg": 108.0, # ignore
                "width": 3840 # same as height
            }
        ],
        [
            1,
            {
                "cameraType": 0,
                "distortionCoeff": [
                    8.23864459991455,
                    8.331841468811035,
                    0.0002747059625107795,
                    -0.0008555621607229114,
                    -30.809667587280273,
                    7.88591194152832,
                    9.332347869873047,
                    -31.51665687561035,
                    0.0,
                    0.0,
                    0.0,
                    0.0,
                    0.0,
                    0.0
                ],
                "extrinsics": {
                    "rotationMatrix": [
                        [
                            0.999963104724884,
                            0.005203192587941885,
                            -0.006830564234405756
                        ],
                        [
                            -0.005314992740750313,
                            0.9998505115509033,
                            -0.016452839598059654
                        ],
                        [
                            0.006743935868144035,
                            0.016488537192344666,
                            0.9998413324356079
                        ]
                    ],
                    "specTranslation": {
                        "x": -7.5,
                        "y": 0.0,
                        "z": 0.0
                    },
                    "toCameraSocket": 2,
                    "translation": {
                        "x": -7.509305000305176,
                        "y": 0.008042487315833569,
                        "z": -0.05586768686771393
                    }
                },
                "height": 800,
                "intrinsicMatrix": [
                    [
                        804.4434204101563,
                        0.0,
                        621.8860473632813
                    ],
                    [
                        0.0,
                        804.4434204101563,
                        415.5000915527344
                    ],
                    [
                        0.0,
                        0.0,
                        1.0
                    ]
                ],
                "lensPosition": 0,
                "specHfovDeg": 127.0,
                "width": 1280
            }
        ],
        [
            2,
            {
                "cameraType": 0,
                "distortionCoeff": [
                    6.221896648406982,
                    8.844538688659668,
                    -0.0016147504793480039,
                    -0.0005014842026866972,
                    -28.97982406616211,
                    5.919337272644043,
                    9.687726020812988,
                    -29.566320419311523,
                    0.0,
                    0.0,
                    0.0,
                    0.0,
                    0.0,
                    0.0
                ],
                "extrinsics": {
                    "rotationMatrix": [
                        [
                            0.9999995231628418,
                            -0.0005868538282811642,
                            0.0007861829362809658
                        ],
                        [
                            0.0005769522394984961,
                            0.9999212622642517,
                            0.012536052614450455
                        ],
                        [
                            -0.0007934778113849461,
                            -0.01253559347242117,
                            0.9999210834503174
                        ]
                    ],
                    "specTranslation": {
                        "x": 3.75,
                        "y": 0.0,
                        "z": 0.0
                    },
                    "toCameraSocket": 0,
                    "translation": {
                        "x": 3.7460412979125977,
                        "y": -0.017947779968380928,
                        "z": -0.14741647243499756
                    }
                },
                "height": 800,
                "intrinsicMatrix": [
                    [
                        803.9879760742188,
                        0.0,
                        631.4297485351563
                    ],
                    [
                        0.0,
                        803.9879760742188,
                        433.7477111816406
                    ],
                    [
                        0.0,
                        0.0,
                        1.0
                    ]
                ],
                "lensPosition": 0,
                "specHfovDeg": 127.0,
                "width": 1280
            }
        ]
    ],

### KEEP THESE THE SAME
    "deviceName": "",
    "hardwareConf": "F1-FV00-BC001",
    "housingExtrinsics": {
        "rotationMatrix": [],
        "specTranslation": {
            "x": 0.0,
            "y": 0.0,
            "z": 0.0
        },
        "toCameraSocket": -1,
        "translation": {
            "x": 0.0,
            "y": 0.0,
            "z": 0.0
        }
    },

## IMU EXTRINSICS
    "imuExtrinsics": {
        "rotationMatrix": [
            [
                0.0,
                0.0,
                0.0
            ],
            [
                0.0,
                0.0,
                0.0
            ],
            [
                0.0,
                0.0,
                0.0
            ]
        ],
        "specTranslation": {
            "x": 0.0,
            "y": 0.0,
            "z": 0.0
        },
        "toCameraSocket": -1,
        "translation": {
            "x": 0.0,
            "y": 0.0,
            "z": 0.0
        }
    },
    "miscellaneousData": [], ## LEAVE 
    "productName": "OAK-D-PRO-POE-FF", ## LEAVE!!!!!!
  
##### YOU CAN CHANGE THESE IF YOU WISH
    "stereoEnableDistortionCorrection": false, 
    "stereoRectificationData": {
        "leftCameraSocket": 1,
        "rectifiedRotationLeft": [
            [
                0.9999907612800598,
                0.004254900850355625,
                0.0006256530759856105
            ],
            [
                -0.00424960907548666,
                0.9999570846557617,
                -0.008229006081819534
            ],
            [
                -0.0006606398383155465,
                0.008226270787417889,
                0.9999659657478333
            ]
        ],
        "rectifiedRotationRight": [
            [
                0.9999717473983765,
                -0.0010709726484492421,
                0.007439584471285343
            ],
            [
                0.001009725616313517,
                0.9999656081199646,
                0.008231471292674541
            ],
            [
                -0.007448144257068634,
                -0.008223727345466614,
                0.999938428401947
            ]
        ],
        "rightCameraSocket": 2
    },
    "stereoUseSpecTranslation": true,
    "version": 7,
    "verticalCameraSocket": -1 ## LEAVE
}

Thanks,
Jaka

cniessl

This is my question though. What is the distortionCoeff? It's listed as a 14 element tuple, of which 8 are populated.

Typical distortion coefficients are listed as k parameters for radial distortion, and p parameters for tangential distortion.
OpenCV usually provides this as a 5 tuple, ordered (k₁, k₂, p₁, p₂, k₃) where
x'=x(1+k₁r²+k₂r⁴+k₃r⁶) and x'=x+[2p₁xy+p₂(r²+2x²)] and similarly for y.

With the tuple listed, do you allow for a constant offset and odd numbered powers? (k_n for r¹, r³, r⁵)?
Do you allow for independent k parameters for x and y radial distortion? (k_x1, k_x2, k_x3 and k_y1, k_y2, k_y3)?
Or something else? And if so, is the ordering similar to OpenCV with k₁, k₂, p₁, p₂, k₃, ??₁, ??₂, ??₃ or a different ordering?

Thanks,
Chris

cniessl

Found it in GetDistortionCoefficients:
https://docs.luxonis.com/software/api/python/#depthai.CalibrationHandler.getDistortionCoefficients