Getting the camera extrinsics afterwards

Aingu7ae

Hi, does anybody know a tool or have code to get the extrinsics afterwards?

erik

Hi Aingu7ae !
You'd need to calibrate the device to some other coordiante system, perhaps easiest would be to use multi-cam calibration script. Then you could transform eg. pointclouds or 3d object detections to your world coordinate, which fits with the image above. Thoughts?
Thanks,. Erik

Aingu7ae

Hello erik,

thank you for taking an interest 🙂 Yeah, I did the multi-cam calibration which gave me a few matrices.

world_to_cam
[[ 0.61411884 0.7890858 -0.01420041 -0.21579439]
[-0.00386166 0.02099733 0.99977207 0.31440464]
[ 0.78920412 -0.61392403 0.01594204 1.05200151]
[ 0. 0. 0. 1. ]]

cam_to_world
[[ 0.61411884 -0.00386166 0.78920412 -0.6965064 ]
[ 0.7890858 0.02099733 -0.61392403 0.80952764]
[-0.01420041 0.99977207 0.01594204 -0.33416839]
[ 0. 0. 0. 1. ]]

trans_vec
[[-0.21579439]
[ 0.31440464]
[ 1.05200151]]

rot_vec
[[-1.43083528]
[-0.71236433]
[-0.70309224]]

intrinsics_mat
[[1546.18994140625, 0.0, 962.3573608398438]
[0.0, 1546.18994140625, 531.37255859375]
[0.0, 0.0, 1.0]]

I guess first I have to manage to create the pointcloud from the video material. I recorded the scene with gen2-record-replay. Is gen2-pointcloud a good starting place to do so?

How is the workflow to transform the world coordinates once I aquired the pointcloud?

I found this tutorial about geometric camera calibration (Camera Calibration with Example in Python). I presume this is another approach? The math looks a little scary. Given 6 points in the image plane and their corresponding world coordinates, it should be possible to reconstruct the extrinsic matrix. The points have to be independent but I don't know what it means in this context. I can get the world coordinates for 6 points in the image, but how do I check for independence? I thought there are no more then 3 independent vectors in 3D space?! I'm feeling a little lost here and don't know what to learn first to fill my blanks. If you or other readers got a pointer that would be nice!

jakaskerl

Hi Aingu7ae
You need to convert pointcloud coordinates (xyz in camera coordinates) to world coordinates.
For that you use the transformation matrix calculated with multi-cam-example script. Multiplying camera to world matrix with position vector for each point in pointcloud should give you the position vector of that same point in the chessboard's frame of reference. You can that use that real world reference frame to find out the position of objects in the room.

The tutorial you mentioned is just another way of doing the same thing but you need a few points and their positions. Points being independent just means that no 3 points are collinear (which is almost impossible to do in 3D space). Then just solve a system of equations as mentioned in the tutorial and you should get the transform matrix coefficients.

Hope this helps,
Jaka

erik

cc @jakaskerl ^

Aingu7ae

Hi jakaskerl, thank you for your explanation, I really appreciate that.

Here is what I have trouble wrapping my mind around. Although I did the chessboard calibration at a different location afterwards and with a completely different camera pose, the process you describe (if I understand you: multiplying the points/vectors of the pointcloud with the cam-to-world matrix of the chessboard calibration) will yield the real world positions of the pointcloud points in relation to the (first) camera at the ceiling which recorded the original video material and that had a different pose than the calibration camera?

jakaskerl

Hi Aingu7ae
The camera needs to be calibrated at the same location and with the same camera position and orientation. The extrinsic matrix describes the conversion from world coordinate system to camera coordinate system which means it also includes the translation in xyz from world to camera. That changes when you move the camera or the chessboard so results are no longer valid. What stays the same are the camera intrinsics. That means you have to place the chessboard at the scene where you recorded otherwise the transformations are completely wrong.

Hope this helps,
Jaka