I found two problems a bit confusing here and would love to get some help?

  1. How is the world coordinate defined? I see the coordinate used on https://github.com/luxonis/depthai-experiments/tree/master/gen2-multiple-devices/multi-cam-calibration for calibration and it uses right-hand rule with the x-axis out as shown in the picture, Can you confirm the this, please?

  2. Where is the coordinate frame for the camera defined? I see a discussion for IMU coordinate frame in https://discuss.luxonis.com/d/1044-about-oak-d-pro-w-imu-coordinate-system and there it is pointed out as shown

    Is this the same for RGB and mono cameras?


    Highly appreciate the help

    Thanks!

    Hi walelignmessele
    According to the code:

     reprojection = frame_rgb.copy()
        reprojection = cv2.line(reprojection, tuple(p_0[0]), tuple(p_x[0]), (0, 0, 255), 5)
        reprojection = cv2.line(reprojection, tuple(p_0[0]), tuple(p_y[0]), (0, 255, 0), 5)
        reprojection = cv2.line(reprojection, tuple(p_0[0]), tuple(p_z[0]), (255, 0, 0), 5)

    Color is passed in BGR format, so x is red, y is green and z is blue (also it's the usual color coding).
    Coordinate frame for the camera should be (i believe) defined by z - positive depth, x - width, y - height (left hand).

    But they are not dependent on one another - one is defined inside the calibration script, and the other one is arbitrary.

    IMU CS is another story and is defined by the IMU component (the MEMS accelerometers i believe) and its orientation. The orientation should be visible on the PCB assembly pictures.

    Hope this helps,
    Jaka

        Thank you jakaskerl ! The confusion comes from https://docs.luxonis.com/projects/api/en/latest/components/nodes/spatial_location_calculator/ (see the screenshot as well)

        . From this the camera coordinate systems uses right hand rule with +ve Z facing the depth. But when I do the camera calibration, it gives a -ve values for all, x, y, z from camera to world. If I follow this definition (the one I referenced), I should be getting +ve z at least, right? Are you certain the camera coordinate is left hand with z facing the depth?

            Hi walelignmessele
            Perhaps it would be worth checking the cam_to_world and world_to_cam matrices to see which vectors (and orientations) are being used. I'm not familiar with the details on how exactly these reference coordinate systems are set. Ill look into it when I get some spare time.

            Thanks,
            Jaka

                jakaskerl Thank you! I will be testing different configurations to find out what's going on. I keep you posted here. From your previous explanation (i.e. Color is passed in BGR format, so x is red, y is green and z is blue (also it's the usual color coding), it appears like the world coordinate is defined in the left-hand rule, do you agree?

                  @jakaskerl @erik Can you please take a look at this and what the coordinate system for the camera should be ?

                  See the two homogenous transformations:

                  Camera to world transformation:
                   [[-0.91024599 -0.35431617  0.21427153 -0.26236268]
                   [ 0.41406236 -0.77620355  0.4754581  -0.58017882]
                   [-0.00214417  0.52150561  0.85324516 -1.00511733]
                   [ 0.          0.          0.          1.        ]]
                  World-to-camera transformation:
                   [[-9.10245991e-01  4.14062360e-01 -2.14416784e-03 -7.39507557e-04]
                   [-3.54316166e-01 -7.76203552e-01  5.21505610e-01 -1.91218764e-02]
                   [ 2.14271533e-01  4.75458104e-01  8.53245159e-01  1.18967907e+00]
                   [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  1.00000000e+00]]

                  • erik replied to this.

                    Hi walelignmessele ,
                    Spatial location calc / Spatial detection networks use left-handed coordinate system. We will be updating docs shortly.

                      erik Thank you!

                      So the new coordinate system is with Z pointing away from the depth, right? At least this is what my test shows me

                      • erik replied to this.

                          walelignmessele I'm not 100%, but I believe the Y is rotated (as you have drawn), but not the Z - if you go away from the camera, Z will increase.