getCameraToImuExtrinsics() same result

Hello @Dong_Liu , so the error means that your device hasn't been calibrated. You can calibrate your device following this tutorial. Just curious, which device do you have (and where did you buy it from)?

    Hi erik, the device is OAK-D, my co-worker purchased it. I'll ask him where did he ordered it.

    But do you mean the device hasn't been calibrated for the IMU to LEFT, RIGHT and RGB cameras or it hasn't been calibrated at all? Because I can get other calibration info, such as camera's intrinsics, left right camera extrinsics etc just fine.

    Thanks!

      I'm actually not sure about the IMU calibration. Asking Sachin for help here. He's from US so will probably respond later.

      Hi erik, another question. The distortion coeff I got from

      calibData.getDistortionCoefficients(dai::CameraBoardSocket::RIGHT);

      is [9.6685 26.4033 0.000109294 -0.00131915 177.747 9.71128 24.7958 178.397 0 0 0 0 0 0]

      Are these right values? they seem too big. And the images captured on the host side seem have no distortion. Actually if I use these value to undistort the images, I got some fisheye results.

      Thanks!

        Dong_Liu We don't have IMU calibration inPlace during factory calibration. Hence it is empty. We will be adding IMU calibration support in the future.

            Dong_Liu Those are fine. Since the equation is the ratio the big numbers don't make any difference.
            We use a rational model from OpenCV and they provide large numbers instead of small floats. Hope this answers your doubt.

                luxonis-Sachin Thanks for the info. Is there any instructions to calibrate IMU? I could not find how to calibrate imu from the video and calibrate.py on luxonis web site.

                Thanks,

                    luxonis-Sachin Yes, I tried it again. The distort coefficients are correct. I didn't realize I need to use the whole vector to call cv.undistot(), I just used the first 4 elements, so it gave me wrong result.

                    Thanks!

                      Dong_Liu We don't have IMU calibration support yet since we had to prioritize over some CV features.
                      But if more users or use-cases request we can increase the priority on this I suppose.

                          luxonis-Sachin We want to use OAK-D to do VIO, it is important to have IMU to camera calibrated.

                          I'm trying to use kalibr to do it myself. But it is quite an old project. I managed to run it under docker. But the test pattern file and some config are hosted in a private google drive now it is closed 🙁

                              luxonis-Sachin Thanks! I was able to print out the AprilTag pattern. Now I'll need to hunt down the config yaml file. I think I saw it somehwere.

                              I'll report back if the calibration works or not.

                              By the way, that python freeze emacs when open it 🙁 Because it contains a VERY long line.

                              Thanks again.

                                  Dong_Liu

                                  By the way, that python freeze emacs when open it 🙁 Because it contains a VERY long line.

                                  In Kalibr ?

                                      luxonis-Sachin No, I tried to open the file using emacs, the emacs was frozen. It was actually running, because it is in python mode, just took forever to parse the very long line. So I had to kill it.

                                        Another question. I set the update rate of IMU to 400HZ. But when I recorded the ros bag, there were only slightly more imu messages than image messages. Wii it be a problem?

                                        $ rosbag info /foo/2021-09-29-13-49-27.bag
                                        path: /home/dliu/foo/2021-09-29-13-49-27.bag
                                        version: 2.0
                                        duration: 1:08s (68s)
                                        start: Sep 29 2021 13:49:27.96 (1632937767.96)
                                        end: Sep 29 2021 13:50:36.62 (1632937836.62)
                                        size: 1006.9 MB
                                        messages: 6195
                                        compression: none [1030/1030 chunks]
                                        types: sensor_msgs/Image [060021388200f6f0f447d0fcd9c64743]
                                        sensor_msgs/Imu [6a62c6daae103f4ff57a132d6f95cec2]
                                        topics: /cam0/image_raw 2059 msgs : sensor_msgs/Image
                                        /cam1/image_raw 2059 msgs : sensor_msgs/Image
                                        /imu0 2077 msgs : sensor_msgs/Imu

                                          luxonis-Sachin which topic? and who publishes it? My code only publish /cam{0|1}/image_raw and /imu0. The depthai ros examples stereo node also doesn't publish anything with hz?

                                            Here is calibrated result of imu with the stereo camera set to 640x400 resolution
                                            cam0:
                                            T_cam_imu:

                                            • [0.03887680448008979, -0.9992016387989338, -0.009202124479843001, -0.0006451104397754213]
                                            • [0.99846239120464, 0.0384805735529119, 0.03990111287861405, -0.0028735167146291523]
                                            • [-0.03951515435032274, -0.010739202976226664, 0.9991612593050765, 0.000392439569927346]
                                            • [0.0, 0.0, 0.0, 1.0]
                                              cam_overlaps: [1]
                                              camera_model: pinhole
                                              distortion_coeffs: [0.026474033567193075, -0.05969556492611341, -0.002817745414182798,
                                              0.004924235269650544]
                                              distortion_model: radtan
                                              intrinsics: [400.0064170380642, 397.2638167232537, 329.5418926134828, 197.04540739777443]
                                              resolution: [640, 400]
                                              rostopic: /cam0/image_raw
                                              timeshift_cam_imu: 0.0008590827955647041
                                              cam1:
                                              T_cam_imu:
                                            • [0.026880135407322037, -0.999628330381034, 0.004545263480889773, -0.07358932785004997]
                                            • [0.9989752544416461, 0.027027644643332516, 0.036303545808673066, -0.0017805337977785335]
                                            • [-0.036412900649807046, 0.003564761515223131, 0.999330472437225, -0.002939578961411555]
                                            • [0.0, 0.0, 0.0, 1.0]
                                              T_cn_cnm1:
                                            • [0.9998334535948544, -0.011446106146888528, 0.014214490024609772, -0.07298279375678171]
                                            • [0.011496829079291606, 0.9999278124509344, -0.0034918201857753366, 0.0011015625375971258]
                                            • [-0.014173496170921451, 0.003654660197938216, 0.9998928719943602, -0.0033306182332533537]
                                            • [0.0, 0.0, 0.0, 1.0]
                                              cam_overlaps: [0]
                                              camera_model: pinhole
                                              distortion_coeffs: [0.016727023328150355, -0.04497168337727961, -0.002975051937495909,
                                              0.005200871168133805]
                                              distortion_model: radtan
                                              intrinsics: [399.1877435307988, 395.58676406318335, 331.88450604453317, 194.85854223018865]
                                              resolution: [640, 400]
                                              rostopic: /cam1/image_raw
                                              timeshift_cam_imu: -0.0009904323881395447

                                            The eprom calibration for the left camera's intrinsics are

                                            421.241 0 317.216
                                            0 421.378 202.988
                                            0 0 1

                                            Little bit different. But eprom left to right extrinsics are

                                            0.999711 -0.0122152 0.0207223 -7.458
                                            0.0123016 0.999916 -0.00405039 0.0796994
                                            -0.0206711 0.00430414 0.999777 0.082101

                                            The number at the end of first row -7.458 seems wrong. That number is the distance of two camera if I understand correctly. 7.458 meters is way to large. I think it should be 0.07458 meters, which is close to value I got 0.07359

                                            Thanks!

                                              Dong_Liu I would suggest ignoring the camera intrinsic from this and use the IMU extrinsics and other information for the usecase without affecting the EEPROM would be best.
                                              As of Why. We calibrate the camera at full resolution in the factory so that it gets scaled well when used at different resolutions for stereo. So it might not be a good idea to mess with calibrated stereo I think.
                                              Thoughts ?