Can't calibrate imu and images using Kalibr

Arducam-Ashely

Hello, Luxonis's talented engineers!

I meet two problems when using OAK
The Kalibr tool has been compiled and installed in the catkin_ws workspace and is running on all algorithm nodes.

Then I use this file to do calibration and Kalibr works fine.

I use the following code:
cd ~/catkin_ws source devel/setup.bash rosrun kalibr kalibr_calibrate_cameras --target april_6x6.yaml --models pinhole-fov --topics /cam0/image_raw --bag dynamic.bag --bag-from-to 5 45

The result is here:

I use OAK-D-IoT-75 and rosbag to get RGB data and save it as a bag for calibration tasks.
#rosbag record -O xxx.bag topic-name rosbag record -O /home/zdl/bag-data/oakd-IoT-75_MonoRGB3.bag /oakd/rgb/image_raw

The calibration board uses a checkerboard grid with 11 rows x 8 columns and a size of 20mm.

yaml file is here:

The kalibr_calibrate_cameras node of the Kalibr tool is then called to calibrate the rgb image, running the following command:
rosrun kalibr kalibr_calibrate_cameras \ --target checkboard_11x8.yaml \ --bag oakd-IoT-75_MonoRGB3.bag \ --models pinhole-fov \ --topics /oakd/rgb/image_raw

I meet this problem. The program process is stuck and not moving and cannot be processed correctly.

Arducam-Ashely

Scenario 2： I try to get stereo grayscale image data from the OAKD-IoT-75 camera.
#rosbag record -O xxx.bag topic-name rosbag record -O /home/zdl/bag-data/oakd-IoT-75_StereoGray.bag /oakd/stereo_ir/left/image_raw /oakd/stereo_ir/right/image_raw

I use the following commands to do calibration:
rosrun kalibr kalibr_calibrate_cameras \ --target checkboard_11x8.yaml \ --bag oakd-IoT-75_StereoGray.bag \ --models pinhole-fov pinhole-fov \ --topics /oakd/stereo_ir/left/image_raw /oakd/stereo_ir/right/image_raw

Then I found it cannot detect corner points

In the case that both RGB camera and the stereo camera cannot be calibrated, the joint calibration of IMU and camera cannot be performed.

Arducam-Ashely

I feel that the images taken by the camera and the algorithm cannot detect the corner points and the resulting program is stuck or cannot be calculated.

I don't know if the problem is related to the camera model, the type of calibration board, or the image frame rate.

Much thanks!
Ashely

luxonis-Sachin

Arducam-Ashely can you share the bag with me to take a look ? you can share it over slack or send it to sachin@luxonis.com

zdl

luxonis-Sachin
Dear luxonis-Sachin
We have sent the bag to your email, please note to check.
Much thanks!
Arducam-zdl

luxonis-Sachin

zdl I checked the rgb video.
This looks like a IMX376 AF camera and rolling shutter.
This is what I think is happening

You are moving the camera too fast. Slow it down a little
There is too much sudden movements. The VCM might be shaking giving it jumpy effect. Exampe here for the extreme end of the same.
If it is not set to MF mode. Please do record in Manual focus mode.

Let me know if this helps

zdl

luxonis-Sachin
We set the MF mode of the rgb camera and recorded a new .bag file, but still can't calibrate. In addition, the bag file has been sent to you by email, please take the time to check it.
camRgb->initialControl.setManualFocus(135);

Much thanks!
Arducam-zdl