Camera pose estimation OAK-D

gmaranta

Hello community!

So i've been trying to do a simple task : estimate the camera position.
What have i done? Simply set up a little example environment with an implied coordinate system, and tried to verify the results the algorithm calculated. So initially, i tried the same on a different setup and figured that the values are way off. Now the same problem, but a better verifiable environment. The algorithm finds a "good" solution in terms of error value when back-projecting the points with the estimated transition matrix, but the translation values seem to be wrong.

I think my calculation is right, the only thing i can not really verify is the camera intrinsics matrix, which i got from the calibration reader. I used the cameraRgb.socket and set the resolution to 4k. Maybe i'm reading the wrong matrix or there is some more specification on how to do that - or the values of the camera matrix provided are in different "units".

the intrinsics received from the oak-d are:

[[3096.306640625, 0.0, 1913.822509765625],
[0.0, 3096.306640625, 1084.4405517578125],
[0.0, 0.0, 1.0]]

which i understood as, fx,fy focal lengths in pixel, cx,cy offsets from principal point

K = [fx 0 cx
0 fy cy
0 0 1 ]

i read the camera calibration with :

M_rgb, width, height = calibData.getDefaultIntrinsics(dai.CameraBoardSocket.RGB)
(since it was not recalibrated i think it will be the same as getCameraIntrinsics)

Does anyone have similar issues or managed to define the pose with given points in the world coordinates system properly?

kind regards, G.

luxonis-Sachin

Hello,

M_rgb, width, height = calibData.getDefaultIntrinsics(dai.CameraBoardSocket.RGB)

When you do this it will return the intrinsics at the resolution it was calibrated.
Since you do at 4K. you should use getCameraIntrinsics and pass the 4K resolution to get the intrinsics scaled for that resolution or scale it on your own after the getDefaultIntrinsics

gmaranta

Thank you for your quick response. Ok yeah, i did that - turned out the matrix did not change much, the problem remains the same. Any other ideas? I checked the inputs and they really seem to be ok. I took this super simple example to be sure i'm not doing anything wrong.

I gave the same algorithm 7 points and the almost perfect initial guess of its estimated position and it seemed to converge to a totally different solution translation wise - rotation is ok. The only parameter that could be wrong must be the camera matrix then, right?

erik

Hi gmaranta ,
Working now with our engineers to provide such demo.
Thanks, Erik

gmaranta

ok this is kind of embarrassing - i feel i misunderstood the output of the pose estimation. Which does not reproduce the camera location in the global coordinate system, rather than the translating the detections from the camera frame in the wanted global coordinate system - i guess that would be a very important thing to mention in the demo from where to what it is translating 😃 . Anyway would be cool to have a multiple camera pose estimation as well, where single cameras are cross referenced to a first "master" one. Appreciate the efforts you do in this forum!