Image plane of oak-d-pro PoE

walelignmessele

Does anybody know where the image plane (camera coordinate system) for oak-d-pro poe is?
I appreciate it if anybody found documentation of this somewhere and guide me to it.

walelignmessele

I found two problems a bit confusing here and would love to get some help?

How is the world coordinate defined? I see the coordinate used on https://github.com/luxonis/depthai-experiments/tree/master/gen2-multiple-devices/multi-cam-calibration for calibration and it uses right-hand rule with the x-axis out as shown in the picture, Can you confirm the this, please?
Where is the coordinate frame for the camera defined? I see a discussion for IMU coordinate frame in https://discuss.luxonis.com/d/1044-about-oak-d-pro-w-imu-coordinate-system and there it is pointed out as shown

Is this the same for RGB and mono cameras?

Highly appreciate the help

Thanks!

jakaskerl

Hi walelignmessele
According to the code:

 reprojection = frame_rgb.copy()
        reprojection = cv2.line(reprojection, tuple(p_0[0]), tuple(p_x[0]), (0, 0, 255), 5)
        reprojection = cv2.line(reprojection, tuple(p_0[0]), tuple(p_y[0]), (0, 255, 0), 5)
        reprojection = cv2.line(reprojection, tuple(p_0[0]), tuple(p_z[0]), (255, 0, 0), 5)

Color is passed in BGR format, so x is red, y is green and z is blue (also it's the usual color coding).
Coordinate frame for the camera should be (i believe) defined by z - positive depth, x - width, y - height (left hand).

But they are not dependent on one another - one is defined inside the calibration script, and the other one is arbitrary.

IMU CS is another story and is defined by the IMU component (the MEMS accelerometers i believe) and its orientation. The orientation should be visible on the PCB assembly pictures.

Hope this helps,
Jaka

walelignmessele

Thank you jakaskerl ! The confusion comes from https://docs.luxonis.com/projects/api/en/latest/components/nodes/spatial_location_calculator/ (see the screenshot as well)

. From this the camera coordinate systems uses right hand rule with +ve Z facing the depth. But when I do the camera calibration, it gives a -ve values for all, x, y, z from camera to world. If I follow this definition (the one I referenced), I should be getting +ve z at least, right? Are you certain the camera coordinate is left hand with z facing the depth?

jakaskerl

Hi walelignmessele
Perhaps it would be worth checking the cam_to_world and world_to_cam matrices to see which vectors (and orientations) are being used. I'm not familiar with the details on how exactly these reference coordinate systems are set. Ill look into it when I get some spare time.

Thanks,
Jaka

walelignmessele

jakaskerl Thank you! I will be testing different configurations to find out what's going on. I keep you posted here. From your previous explanation (i.e. Color is passed in BGR format, so x is red, y is green and z is blue (also it's the usual color coding), it appears like the world coordinate is defined in the left-hand rule, do you agree?

erik

Relevant conversation here: https://discuss.luxonis.com/d/1527-spatiallocationcalculator-reference-frame/3

walelignmessele

@jakaskerl @erik Can you please take a look at this and what the coordinate system for the camera should be ?

See the two homogenous transformations:

Camera to world transformation:
[[-0.91024599 -0.35431617 0.21427153 -0.26236268]
[ 0.41406236 -0.77620355 0.4754581 -0.58017882]
[-0.00214417 0.52150561 0.85324516 -1.00511733]
[ 0. 0. 0. 1. ]]
World-to-camera transformation:
[[-9.10245991e-01 4.14062360e-01 -2.14416784e-03 -7.39507557e-04]
[-3.54316166e-01 -7.76203552e-01 5.21505610e-01 -1.91218764e-02]
[ 2.14271533e-01 4.75458104e-01 8.53245159e-01 1.18967907e+00]
[ 0.00000000e+00 0.00000000e+00 0.00000000e+00 1.00000000e+00]]

erik

Hi walelignmessele ,
Spatial location calc / Spatial detection networks use left-handed coordinate system. We will be updating docs shortly.

erik

PR here: luxonis/depthai-python846

walelignmessele

erik Thank you!

So the new coordinate system is with Z pointing away from the depth, right? At least this is what my test shows me

erik

walelignmessele I'm not 100%, but I believe the Y is rotated (as you have drawn), but not the Z - if you go away from the camera, Z will increase.