Hello Everyone, I am using the OAK-D-Pro-POE camera for creating a map with rtabmap with rgbd sensor ans stereo sensor. But when, i create a map, its not looking good, even it shows white spaces very less. so i decided to calibrate my camera but when i am going to calibrate my camera its gives me error" am: rgb and focus: False Cam: left and focus: False Cam: right and focus: False Saving dataset to: dataset Sensor name for rgb is IMX378 Sensor name for left is OV9282 Sensor name for right is OV9282 Starting image capture. Press the [ESC] key to abort. Will take 39 total images. Start capturing... new minimum: {'ts': 0.8028270000000006, 'indicies': {'rgb': 0, 'left': 0, 'right': 0}} min required: 0.2 new minimum: {'ts': 0.0028270000000034656, 'indicies': {'rgb': 1, 'left': 0, 'right': 0}} min required: 0.2 Time stamp of rgb is 5:05:42.311399 Markers count ... 0 Total markers needed -> 17 Status of rgb is False Time stamp of left is 5:05:42.312807 Markers count ... 0 Total markers needed -> 17 Status of left is False Time stamp of right is 5:05:42.312818 Markers count ... 0 Total markers needed -> 17 Status of right is False py: Capture failed, unable to find chessboard! Fix position and press spacebar again Start capturing... new minimum: {'ts': 0.20353500000000224, 'indicies': {'rgb': 0, 'left': 0, 'right': 0}} min required: 0.2 new minimum: {'ts': 0.002836000000002059, 'indicies': {'rgb': 2, 'left': 1, 'right': 1}} min required: 0.2 Time stamp of rgb is 5:05:57.511247 Markers count ... 0 Total markers needed -> 17 Status of rgb is False Time stamp of left is 5:05:57.512660 Markers count ... 0 Total markers needed -> 17 Status of left is False Time stamp of right is 5:05:57.512671 Markers count ... 0 Total markers needed -> 17 Status of right is False py: Capture failed, unable to find chessboard! Fix position and press spacebar again My steps are as follow: downlaod the depthai respository by 1: git clone --recursive https://github.com/luxonis/depthai.git 2: $ python3 install_requirements.py 3: python3 calibrate.py -s 3.0 -ms 2.5 -brd OAK-D-PRO-POE Thank you

@"imran"#p25702 That error would suggest that the chessboard is not present in view of the camera when you press "spacebar" to take an image. Thoughts?

thank you @"jakaskerl"#1112 for your reply. I followed this tutorial "https://luxonis-depthai-hardware.readthedocs-hosted.com/en/latest/pages/guides/calibration/" and complete clicking the 32 pictures but when i am calibrationg it gives me to much error. Can you tell me what should i do in this case.? thank you [upl-image-preview url=https://discuss.luxonis.com/assets/files/2024-09-10/1725957181-447076-screenshot-from-2024-09-10-14-02-11.png]

@"imran"#p25712 Can you give me some example images (for left and right) you took of the board? They should be in the dataset folder. Thanks, Jaka

Here is the one example images: Right: [upl-image-preview url=https://discuss.luxonis.com/assets/files/2024-09-10/1725962405-921024-p0-0.png] [upl-image-preview url=https://discuss.luxonis.com/assets/files/2024-09-10/1725962433-813187-p4-13.png] [upl-image-preview url=https://discuss.luxonis.com/assets/files/2024-09-10/1725962434-99954-p1-3.png] [upl-image-preview url=https://discuss.luxonis.com/assets/files/2024-09-10/1725962434-375349-p0-2.png] [upl-image-preview url=https://discuss.luxonis.com/assets/files/2024-09-10/1725962434-657073-p0-1.png] Left: [upl-image-preview url=https://discuss.luxonis.com/assets/files/2024-09-10/1725962452-436698-p4-14.png] [upl-image-preview url=https://discuss.luxonis.com/assets/files/2024-09-10/1725962452-726872-p1-3.png] [upl-image-preview url=https://discuss.luxonis.com/assets/files/2024-09-10/1725962453-11050-p0-2.png] [upl-image-preview url=https://discuss.luxonis.com/assets/files/2024-09-10/1725962453-295351-p0-1.png] [upl-image-preview url=https://discuss.luxonis.com/assets/files/2024-09-10/1725962453-581150-p0-0.png]

@"imran"#p25720 The lighting is very bad, so the markers likely aren't properly detected. Can you fix this please? Also image 6; the view is too tilted for anything to be recognized. Please follow the [guide](https://docs.luxonis.com/hardware/platform/depth/calibration/#Depth%20Calibration-Calibration-Camera%20Positioning%20During%20Calibration). Thanks, Jaka

Calibration of OAK-D-PRO-POE

imran

Here is the one example images:

Right:

Left:

jakaskerl

imran
The lighting is very bad, so the markers likely aren't properly detected. Can you fix this please? Also image 6; the view is too tilted for anything to be recognized. Please follow the guide.

Thanks,
Jaka

imran

Thank you @"jakaskerl,

I followed the tutorial for the calibration and this time i got the error :

Is it best to flash this calibration on the camera. For more clear understanding my application is to create a 3D and 2D grid map using this camera. So i am using depthai_ros and rtabmap_ros respository for my research. I am using the rgbd and depth sensor to be aligned and perform the 2D mapping. but the map looks very bad before 🇦

Can you please help me regarding this.

Sorry if i don't deliver everything clearly.

Thank You

jakaskerl

imran
You can flash the calib using this example: https://docs.luxonis.com/software/depthai/examples/calibration_flash/

The existing calibration on device will get saved to the PC so if you need to reflash it later, you can.

Thanks,
Jaka

imran

I tried to calibrate using this link byut it alwyas gives me error

Calibration Data on the device is backed up at:

/home/darlab/depthai-python/examples/calibration/depthai_calib_backup.json

Failed flashing calibration: No PROTECTED permissions to override protected EEPROM fields.

I also tried to change the branch but still getting this error.

jakaskerl

imran
Check the top few lines in the json files. They shouldn't differ from one calibration to the other. Protected fields should not be changed.

Thanks,
Jaka

imran

Thank You for your response. I calibrated the camera using the backup json file but now is their any way to test that the camera is calibrated correct or not. what i mean is that how i check the errors of depth and rgbd camera?

jakaskerl

imran
The errors (epipolar and such) are printed at the end of custom calibration using calibrate.py script. The json file does not have this info. You will have to check the quality yourself (visual inspection).

Thanks,
Jaka

imran

@jakaskerl , I again calibrated my camera using the calibration.py script and I got below error .But I am still not sure is it good or not. As, i am using the visual odometry with rtabmap and there i am facing lot of problems with it.

<------------Calibrating rgb ------------>

INTRINSIC CALIBRATION

Reprojection error of rgb: 1.3810906257028916

<------------Calibrating left ------------>

INTRINSIC CALIBRATION

Reprojection error of left: 0.18740472456067955

<------------Calibrating right ------------>

INTRINSIC CALIBRATION

Reprojection error of right: 0.20223205153018825

<-------------Extrinsics calibration of left and right ------------>

Reprojection error is 0.20335454666373534

<-------------Epipolar error of left and right ------------>

Original intrinsics ....

L [[570.22926483 0. 656.31588685]

[ 0. 570.32612851 396.3645663 ]

[ 0. 0. 1. ]]

R: [[578.45103955 0. 654.73629999]

[ 0. 578.81537279 393.81854406]

[ 0. 0. 1. ]]

Intrinsics from the getOptimalNewCameraMatrix/Original ....

L: [[578.45103955 0. 654.73629999]

[ 0. 578.81537279 393.81854406]

[ 0. 0. 1. ]]

R: [[578.45103955 0. 654.73629999]

[ 0. 578.81537279 393.81854406]

[ 0. 0. 1. ]]

Average Epipolar Error is : 0.156279393452089

Displaying Stereo Pair for visual inspection. Press the [ESC] key to exit.

<-------------Extrinsics calibration of right and rgb ------------>

Reprojection error is 0.2240963285194658

<-------------Epipolar error of right and rgb ------------>

Original intrinsics ....

L [[578.45103955 0. 654.73629999]

[ 0. 578.81537279 353.81854406]

[ 0. 0. 1. ]]

R: [[768.9135007 0. 633.20994833]

[ 0. 768.65231559 364.57852953]

[ 0. 0. 1. ]]

Intrinsics from the getOptimalNewCameraMatrix/Original ....

L: [[768.9135007 0. 633.20994833]

[ 0. 768.65231559 364.57852953]

[ 0. 0. 1. ]]

R: [[768.9135007 0. 633.20994833]

[ 0. 768.65231559 364.57852953]

[ 0. 0. 1. ]]

Average Epipolar Error is : 0.18926582336425782

Displaying Stereo Pair for visual inspection. Press the [ESC] key to exit.

Reprojection error threshold -> 3

rgb Reprojection Error: 1.381091

Reprojection error threshold -> 1.1111111111111112

left Reprojection Error: 0.187405

Reprojection error threshold -> 1.1111111111111112

right Reprojection Error: 0.202232

Flashing Calibration data into

EEPROM VERSION being flashed is -> 7

I am waiting for your response.

Thank You

jakaskerl

imran
What kind of problems are you facing.

The calibration seems to be good - the RGB error is a bit higher but the rest looks fine .

Thanks,
Jaka

imran

jakaskerl

I am using the visual odometry with rtabmap and it seems i the rgb images are giving me a kind of barrel distortions which results to bad visual odometry. Thats why i am trying to calibrate my camera. Also, during caibration the rgb pics seems like distored. I am sharing some pictures as well.

Hereis the link of dataset i am using for calibarion. You can check for rgb in this :

https://drive.google.com/drive/folders/1Drpo_Of7HwFJfCkCJNJd4iF5zkrjRcfS?usp=sharing

This is the pifcture which i take from mapping:

and when i am using the stereo camera with (depthaiexamples/stereo_inertial_node depth_aligned:=false). I am getting the error with rtabmap.

The stereo baseline (-0.041895) should be positive (baseline=-Tx/fx). We assume a horizontal left/right stereo setup where the Tx (or P(0,3)) is negative in the right camera info msg

jakaskerl

imran
From the dataset I have noticed that you didn't use the full FOV of the cameras. I suggest you add a few images where the left, right and RGB touch the charuco board borders.

Then run python calibrate.py .. .. .. . . -m process to only run the processing stage.

I'm not to accustomed to ROS, but it thing addding rectify_rgb to arguments should help. Also depth aligned to true.

Thanks,
Jaka

imran

Actually when i am taking the pic while using full FOV during calibration, even my camera is placed stright infronytof board still it shows an inner distortion. so i cannot capture a proper image where the charcuo board borders touches

jakaskerl

imran
Yes, that is expected, I meant you take multiple images so the entire FOV has at least one image where the board is touching one of the borders.

Thanks,
Jaka