luxonis-Sachin Thanks for the info. Is there any instructions to calibrate IMU? I could not find how to calibrate imu from the video and calibrate.py on luxonis web site.

Thanks,

      luxonis-Sachin Yes, I tried it again. The distort coefficients are correct. I didn't realize I need to use the whole vector to call cv.undistot(), I just used the first 4 elements, so it gave me wrong result.

      Thanks!

        Dong_Liu We don't have IMU calibration support yet since we had to prioritize over some CV features.
        But if more users or use-cases request we can increase the priority on this I suppose.

            luxonis-Sachin We want to use OAK-D to do VIO, it is important to have IMU to camera calibrated.

            I'm trying to use kalibr to do it myself. But it is quite an old project. I managed to run it under docker. But the test pattern file and some config are hosted in a private google drive now it is closed 🙁

                luxonis-Sachin Thanks! I was able to print out the AprilTag pattern. Now I'll need to hunt down the config yaml file. I think I saw it somehwere.

                I'll report back if the calibration works or not.

                By the way, that python freeze emacs when open it 🙁 Because it contains a VERY long line.

                Thanks again.

                    Dong_Liu

                    By the way, that python freeze emacs when open it 🙁 Because it contains a VERY long line.

                    In Kalibr ?

                        luxonis-Sachin No, I tried to open the file using emacs, the emacs was frozen. It was actually running, because it is in python mode, just took forever to parse the very long line. So I had to kill it.

                          Another question. I set the update rate of IMU to 400HZ. But when I recorded the ros bag, there were only slightly more imu messages than image messages. Wii it be a problem?

                          $ rosbag info /foo/2021-09-29-13-49-27.bag
                          path: /home/dliu/foo/2021-09-29-13-49-27.bag
                          version: 2.0
                          duration: 1:08s (68s)
                          start: Sep 29 2021 13:49:27.96 (1632937767.96)
                          end: Sep 29 2021 13:50:36.62 (1632937836.62)
                          size: 1006.9 MB
                          messages: 6195
                          compression: none [1030/1030 chunks]
                          types: sensor_msgs/Image [060021388200f6f0f447d0fcd9c64743]
                          sensor_msgs/Imu [6a62c6daae103f4ff57a132d6f95cec2]
                          topics: /cam0/image_raw 2059 msgs : sensor_msgs/Image
                          /cam1/image_raw 2059 msgs : sensor_msgs/Image
                          /imu0 2077 msgs : sensor_msgs/Imu

                            luxonis-Sachin which topic? and who publishes it? My code only publish /cam{0|1}/image_raw and /imu0. The depthai ros examples stereo node also doesn't publish anything with hz?

                              Here is calibrated result of imu with the stereo camera set to 640x400 resolution
                              cam0:
                              T_cam_imu:

                              • [0.03887680448008979, -0.9992016387989338, -0.009202124479843001, -0.0006451104397754213]
                              • [0.99846239120464, 0.0384805735529119, 0.03990111287861405, -0.0028735167146291523]
                              • [-0.03951515435032274, -0.010739202976226664, 0.9991612593050765, 0.000392439569927346]
                              • [0.0, 0.0, 0.0, 1.0]
                                cam_overlaps: [1]
                                camera_model: pinhole
                                distortion_coeffs: [0.026474033567193075, -0.05969556492611341, -0.002817745414182798,
                                0.004924235269650544]
                                distortion_model: radtan
                                intrinsics: [400.0064170380642, 397.2638167232537, 329.5418926134828, 197.04540739777443]
                                resolution: [640, 400]
                                rostopic: /cam0/image_raw
                                timeshift_cam_imu: 0.0008590827955647041
                                cam1:
                                T_cam_imu:
                              • [0.026880135407322037, -0.999628330381034, 0.004545263480889773, -0.07358932785004997]
                              • [0.9989752544416461, 0.027027644643332516, 0.036303545808673066, -0.0017805337977785335]
                              • [-0.036412900649807046, 0.003564761515223131, 0.999330472437225, -0.002939578961411555]
                              • [0.0, 0.0, 0.0, 1.0]
                                T_cn_cnm1:
                              • [0.9998334535948544, -0.011446106146888528, 0.014214490024609772, -0.07298279375678171]
                              • [0.011496829079291606, 0.9999278124509344, -0.0034918201857753366, 0.0011015625375971258]
                              • [-0.014173496170921451, 0.003654660197938216, 0.9998928719943602, -0.0033306182332533537]
                              • [0.0, 0.0, 0.0, 1.0]
                                cam_overlaps: [0]
                                camera_model: pinhole
                                distortion_coeffs: [0.016727023328150355, -0.04497168337727961, -0.002975051937495909,
                                0.005200871168133805]
                                distortion_model: radtan
                                intrinsics: [399.1877435307988, 395.58676406318335, 331.88450604453317, 194.85854223018865]
                                resolution: [640, 400]
                                rostopic: /cam1/image_raw
                                timeshift_cam_imu: -0.0009904323881395447

                              The eprom calibration for the left camera's intrinsics are

                              421.241 0 317.216
                              0 421.378 202.988
                              0 0 1

                              Little bit different. But eprom left to right extrinsics are

                              0.999711 -0.0122152 0.0207223 -7.458
                              0.0123016 0.999916 -0.00405039 0.0796994
                              -0.0206711 0.00430414 0.999777 0.082101

                              The number at the end of first row -7.458 seems wrong. That number is the distance of two camera if I understand correctly. 7.458 meters is way to large. I think it should be 0.07458 meters, which is close to value I got 0.07359

                              Thanks!

                                Dong_Liu I would suggest ignoring the camera intrinsic from this and use the IMU extrinsics and other information for the usecase without affecting the EEPROM would be best.
                                As of Why. We calibrate the camera at full resolution in the factory so that it gets scaled well when used at different resolutions for stereo. So it might not be a good idea to mess with calibrated stereo I think.
                                Thoughts ?

                                    luxonis-Sachin The imu calibration from IMU to camera needs the cameras intrinsics. I will use the intrinsics from the eprom to calibrate the IMU. But can you confirm the eprom stored left right extrinsics calibration are wrong, that is the decimal points in t1 is wrong?

                                    Thanks.

                                        Dong_Liu DepthAi stores the calibration in cm units. I think it's in meters in Kalibr. you need to change the scale.