How to convert Kalibr's T_cam_imu to DepthAI's imuExtrinsics

HiepNguyen

Hi Luxonis experts,

I'm using Kalibr to do the IMU to Camera calibration, and have the result file camchain-imu cam.yaml

cam0:
  T_cam_imu:
    [-0.001827859233177992, -0.999909119974655, 0.013357047657805154, 0.030298330137844887,
    0.0039387461853423456, -0.013364165204938133, -0.9999029379729119, 0.028682372533570984,
    0.9999905725601166, -0.0017750717969459706, 0.003962816044752765, -0.014747523214162677,
    0.0, 0.0, 0.0, 1.0]
...
cam1:
  T_cam_imu:
    [-0.008382154257505742, -0.9999532504652721, 0.00482041221684304, -0.08958472628850654,
    0.006913259577842834, -0.004878415733911984, -0.9999642033102668, 0.028854934277130317,
    0.99994097142382, -0.00834852944320319, 0.006953828019388014, -0.015662499900516994,
    0.0, 0.0, 0.0, 1.0]
  T_cn_cnm1:
    [0.9999420825622907, 0.008510580920464224, -0.006587984015209744, -0.12022256172433968,
    -0.008491233601698718, 0.9999595702862754, 0.0029591797526304676, 0.0004746321347084293,
    0.006612902003644948, -0.0029030682532840716, 0.9999739205208336, -0.001032454297023307,
    0.0, 0.0, 0.0, 1.0]
...

My question is: how do I convert the T_cam_imu matrixes from the Kalibr results to the imuExtrinsics of DepthAI calibration format , and how to flash the result to *EEPROM.
*
I'm looking forward to seeing your response soon. Many thanks in advance!

Best regards,
Hiep Nguyen

jakaskerl

Hi HiepNguyen
device.readCalibration().eepromToJsonFile("./calibration_current.json")

Open the json file and edit the extrinsics. Make sure to set the IMU extrinsics to point to some camera (1, 2, 3.., dont leave it at -1). You might have to compute the inverse since you currently have cam->imu (not entirelly sure).

Once you have set the json, load it and try it out before flashing: https://docs.luxonis.com/projects/api/en/latest/samples/calibration/calibration_load/#calibration-load

When you wish to flash: https://docs.luxonis.com/projects/api/en/latest/samples/calibration/calibration_flash/#calibration-flash

Thanks,
Jaka

HiepNguyen

Hi jakaskerl ,

Thank you for your reply. I'm a bit unclear with your response.

So let's take cam0's T_cam_imu, corresponding to OAK-D Cam left, as an example.

The cam0's T_cam_imu format are :

cam0:
  T_cam_imu:
    [-0.001827859233177992, -0.999909119974655, 0.013357047657805154, 0.030298330137844887,
    0.0039387461853423456, -0.013364165204938133, -0.9999029379729119, 0.028682372533570984,
    0.9999905725601166, -0.0017750717969459706, 0.003962816044752765, -0.014747523214162677,
    0.0, 0.0, 0.0, 1.0]

The default imuExtrinsics matrixes format are :

    "imuExtrinsics": {
        "rotationMatrix": [
            [
                0.0,
                0.0,
                0.0
            ],
            [
                0.0,
                0.0,
                0.0
            ],
            [
                0.0,
                0.0,
                0.0
            ]
        ],
        "specTranslation": {
            "x": 0.0,
            "y": 0.0,
            "z": 0.0
        },
        "toCameraSocket": -1,
        "translation": {
            "x": 0.0,
            "y": 0.0,
            "z": 0.0
        }
    },

The cam0's T_cam_imu is in cam->imu, hence I need to inverse it, to have matrix imu->cam. Am I correct ?

cam0_T_cam_imu = np.array(T_cam_imu)
inverse = np.linalg.inv(cam0_T_cam_imu)

Asume I have the inverse matrix, it is in 4x4 matrix, so I take 3 first columns and fill to rotationMatrix of imuExtrinsics ? Am I correct ? And of course, the toCameraSocket=-1 is change to toCameraSocket=1 to corresponding to OAK-D left camera number.

Please correct me if I'm wrong. Thanks in advance !

Best regards,
Hiep Nguyen

jakaskerl

Hi HiepNguyen
Correct! But you need a 3x3 matrix (rotation) so take the upper left 3x3 square of the inverse matrix. Then set the specTranslation to the last column (first three rows). The bottom row of the inverse matrix should not be used. Then set the toCameraSocket to the index of the cam0 (you will have to check which camera that is).
Hope that makes sense.

Thanks,
Jaka

HiepNguyen

Hi jakaskerl ,

Thank you very much, that makes sense to me.

Best regards,
Hiep Nguyen