BN0086 IMU for kalibr camera-imu calibration

gifvgk

Hi,recently I wanted to convert the point cloud generated by Oak-D-Pro from the camera coordinate system to the world coordinate system with the rotation vector information output by the IMU, but I was using the Windows system and did not calibrate the IMU.I don't know how wrong the IMU data is now, but in fact, the coordinate transformation results are not ideal for me. Pls what should I do, I'm about to collapse.Please help me.

erik

Hi @gifvgk ,
What exactly do you wish to accomplish? What exactly is wrong with the output? Your post doesn't help us much on debugging.

gifvgk

How to calibrate IMU with Oak D Pro on Windows system

chutsu

@gifvgk, erik is asking you want does your Camera-IMU calibration data look like. You've given no data for which to work from. It would be very helpful if you could provide a ROSBAG or similar, or show plots where the IMU data is bad?

gifvgk

Sorry for not describing my problem in detail before.I took the quaternion of the IMU at a certain moment in time and converted it into a rotation matrix.

q=np.array([-0.019775,0.479309,0.003967,0.877441])
def quaternion_to_rotation_matrix(q): # x, y ,z ,w
rot_matrix = np.array(
[[1.0 - 2 * (q[1] * q[1] + q[2] * q[2]), 2 * (q[0] * q[1] - q[3] * q[2]), 2 * (q[3] * q[1] + q[0] * q[2])],
[2 * (q[0] * q[1] + q[3] * q[2]), 1.0 - 2 * (q[0] * q[0] + q[2] * q[2]), 2 * (q[1] * q[2] - q[3] * q[0])],
[2 * (q[0] * q[2] - q[3] * q[1]), 2 * (q[1] * q[2] + q[3] * q[0]), 1.0 - 2 * (q[0] * q[0] + q[1] * q[1])]],
dtype=q.dtype)
return rot_matrix

R = np.eye(4)
R[:3, :3] = quaternion_to_rotation_matrix(q)
print(R)

As a result, it was：

[[ 0.54049429 -0.02591829 0.84097384 0. ]

[-0.01199505 0.99918642 0.03850563 0. ]

[-0.84128763 -0.03089995 0.53974366 0. ]

[ 0. 0. 0. 1. ]]

I think the resulting rotation matrix should be an orthogonal identity matrix, but obviously, R is not.An orthogonal identity matrix can convert the point cloud to the world coordinate system.That's where I'm confused

chutsu

gifvgk

Why should the rotation matrix be Identity? An identity rotation matrix means it has no attitude? (e.g. 0 roll, 0 pitch, 0 yaw)

gifvgk

Sorry, I may have made a mistake with the transform relationship. My camera and IMU are not jointly calibrated, and I shouldn't use the IMU's rotation matrix directly on the camera coordinate system transformation.Maybe that's the key.

chutsu

Even if you jointly calibrated the Camera-IMU sensor system, the rotation matrix between them should never be perfectly at identity (it can be close but never exactly).

What should be Identity (exactly) is if you have transformation matrices between all the sensors and you form a loop, then it should be identity. Example:

Let:

T_LCam_IMU: Left camera to IMU 4x4 Transformation matrix
T_RCam_IMU: Right camera to IMU 4x4 Transformation matrix
T_LCam_RCam: Left camera to Right Camera 4x4 Transformation matrix

If you did

inv(T_LCam_IMU) * T_LCam_RCam *  T_RCam_IMU -> Should be Identity 4x4

gifvgk

Thank you very much for your patience.The problem is that my oak d pro currently only has a joint calibration program on ubuntu, there is no windows, my programs are all on windows. Unable to get the conversion matrix for camera and IMU.