Hey guys,
Is there a branch in the main /depthai repo that I can use out of the box to calibrate a CM4-PoE board?
I guess the boards were calibrated when they were built so I'm assuming there is a branch that has the right config...
I'm looking at the branches in https://github.com/luxonis/depthai/branches/active
Cheers,
Hi stephansturges ,
Do you get an error when trying to use main branch? There's the OAK-D-CM4-POE.json in the main branch, so I think it should work without an issue.
Thanks, Erik
erik
That one doesn't work for me out of the box, but I haven't spent much time debugging it. The json file parameters look fine but I think there is something in the calibration.py that is referencing sensor settings that don't exist on the CM4-PoE. I'll fix it and report back when I have some time 
FYI this is the error with the default CM4 JSON and main branch (using -drgb flag because I don't care abour the relative RGB position for the moment)
Starting image processing
~~~~~~~~~~~ POSE ESTIMATION LEFT CAMERA ~~~~~~~~~~~~~
=> Processing image /home/pi/depthai/dataset/left/left_p0_0.png
44 number of Markers corners detected in the above image
=> Processing image /home/pi/depthai/dataset/left/left_p10_10.png
41 number of Markers corners detected in the above image
=> Processing image /home/pi/depthai/dataset/left/left_p11_11.png
44 number of Markers corners detected in the above image
=> Processing image /home/pi/depthai/dataset/left/left_p12_12.png
43 number of Markers corners detected in the above image
=> Processing image /home/pi/depthai/dataset/left/left_p1_1.png
43 number of Markers corners detected in the above image
=> Processing image /home/pi/depthai/dataset/left/left_p2_2.png
44 number of Markers corners detected in the above image
=> Processing image /home/pi/depthai/dataset/left/left_p3_3.png
43 number of Markers corners detected in the above image
=> Processing image /home/pi/depthai/dataset/left/left_p4_4.png
44 number of Markers corners detected in the above image
=> Processing image /home/pi/depthai/dataset/left/left_p5_5.png
44 number of Markers corners detected in the above image
=> Processing image /home/pi/depthai/dataset/left/left_p6_6.png
42 number of Markers corners detected in the above image
=> Processing image /home/pi/depthai/dataset/left/left_p7_7.png
44 number of Markers corners detected in the above image
=> Processing image /home/pi/depthai/dataset/left/left_p8_8.png
38 number of Markers corners detected in the above image
=> Processing image /home/pi/depthai/dataset/left/left_p9_9.png
39 number of Markers corners detected in the above image
=> Processing image /home/pi/depthai/dataset/right/right_p0_0.png
44 number of Markers corners detected in the above image
=> Processing image /home/pi/depthai/dataset/right/right_p10_10.png
44 number of Markers corners detected in the above image
=> Processing image /home/pi/depthai/dataset/right/right_p11_11.png
44 number of Markers corners detected in the above image
=> Processing image /home/pi/depthai/dataset/right/right_p12_12.png
44 number of Markers corners detected in the above image
=> Processing image /home/pi/depthai/dataset/right/right_p1_1.png
44 number of Markers corners detected in the above image
=> Processing image /home/pi/depthai/dataset/right/right_p2_2.png
44 number of Markers corners detected in the above image
=> Processing image /home/pi/depthai/dataset/right/right_p3_3.png
44 number of Markers corners detected in the above image
=> Processing image /home/pi/depthai/dataset/right/right_p4_4.png
44 number of Markers corners detected in the above image
=> Processing image /home/pi/depthai/dataset/right/right_p5_5.png
44 number of Markers corners detected in the above image
=> Processing image /home/pi/depthai/dataset/right/right_p6_6.png
44 number of Markers corners detected in the above image
=> Processing image /home/pi/depthai/dataset/right/right_p7_7.png
43 number of Markers corners detected in the above image
=> Processing image /home/pi/depthai/dataset/right/right_p8_8.png
44 number of Markers corners detected in the above image
=> Processing image /home/pi/depthai/dataset/right/right_p9_9.png
43 number of Markers corners detected in the above image
CAMERA CALIBRATION
(1280, 800)
Camera Matrix initialization.............
[[857.1668 0. 643.9126 ]
[ 0. 856.0823 387.56018]
[ 0. 0. 1. ]]
CAMERA CALIBRATION
(1280, 800)
Camera Matrix initialization.............
[[857.1668 0. 643.9126 ]
[ 0. 856.0823 387.56018]
[ 0. 0. 1. ]]
~~~~~~~~~~~~~RMS error of left~~~~~~~~~~~~~~
52.97476788491616
53.252261804773795
[[3.38222696e+03 0.00000000e+00 1.01255487e+03]
[0.00000000e+00 3.37794772e+03 5.31176027e+02]
[0.00000000e+00 0.00000000e+00 1.00000000e+00]]
[[1.06343040e+03 0.00000000e+00 6.44911877e+02]
[0.00000000e+00 1.06208494e+03 3.23631527e+02]
[0.00000000e+00 0.00000000e+00 1.00000000e+00]]
[[ 2.27090183]
[-1.00709123]
[ 0.01134425]
[ 0.88259389]
[-4.09407228]
[-0.32728291]
[-0.32721684]
[-0.95413647]
[ 0. ]
[ 0. ]
[ 0. ]
[ 0. ]
[ 0. ]
[ 0. ]]
[[ 0.0706821 ]
[ 0.5518226 ]
[-0.00255203]
[ 0.3173163 ]
[-0.77125856]
[ 0.13831589]
[-0.62454185]
[ 0.08547696]
[ 0. ]
[ 0. ]
[ 0. ]
[ 0. ]
[ 0. ]
[ 0. ]]
allIds_l
13
13
allIds_l
allIds_r
~~~~~~~~~~~~~RMS error of L-R~~~~~~~~~~~~~~
55.128191055272524
~~~~~ Starting Stereo Calibration ~~~~~
R1 (left)
[[ 0.087886 0.04421 0.995149]
[-0.105144 0.993846 -0.034866]
[-0.990566 -0.10157 0.091993]]
R2 (right)
[[ 0.364598 0.119135 0.923513]
[-0.068392 0.992529 -0.101037]
[-0.92865 -0.026323 0.370022]]
M1 (left)
[[2224.6582 0. 994.02356]
[ 0. 1697.0712 299.0442 ]
[ 0. 0. 1. ]]
M2 (right)
[[805.6539 0. 614.82684]
[ 0. 836.7781 262.49905]
[ 0. 0. 1. ]]
R
[[ 0.959123 0.042471 0.279784]
[-0.067814 0.994361 0.081529]
[-0.274744 -0.09717 0.956595]]
T
[[-12.226009]
[ -3.994925]
[-30.968037]]
M3 (rgb)
[[0. 0. 0.]
[0. 0. 0.]
[0. 0. 0.]]
\R (rgb)
[[0. 0. 0.]
[0. 0. 0.]
[0. 0. 0.]]
T (rgb)
[0. 0. 0.]
Took 7 seconds to run image processing.
Mesh path
/home/pi/depthai/depthai_helpers
shape of maps
(800, 1280)
(800, 1280)
(800, 1280)
(800, 1280)
<-----------------Epipolar error of LEFT-right camera---------------->
HU IHER
Printing p1 and p2
[[ 1266.924599 0. -43430.029258 0. ]
[ 0. 1266.924599 10045.296232 0. ]
[ 0. 0. 1. 0. ]]
[[ 1266.924599 0. -43430.029258 -42483.629139]
[ 0. 1266.924599 10045.296232 0. ]
[ 0. 0. 1. 0. ]]
Traceback (most recent call last):
File "/home/pi/depthai/calibrate.py", line 672, in <module>
Main().run()
File "/home/pi/depthai/calibrate.py", line 667, in run
self.calibrate()
File "/home/pi/depthai/calibrate.py", line 529, in calibrate
epiploar_error, epiploar_error_rRgb, calibData = cal_data.calibrate(self.dataset_path, self.args.squareSizeCm,
File "/home/pi/depthai/depthai_helpers/calibration_utils.py", line 187, in calibrate
return self.test_epipolar_charuco_lr(filepath), None, self.calib_data
File "/home/pi/depthai/depthai_helpers/calibration_utils.py", line 719, in test_epipolar_charuco_lr
res2_l = cv2.aruco.interpolateCornersCharuco(
cv2.error: OpenCV(4.6.0) /io/opencv_contrib/modules/aruco/src/charuco.cpp:446: error: (-215:Assertion failed) _markerCorners.total() == _markerIds.getMat().total() && _markerIds.getMat().total() > 0 in function '_interpolateCornersCharucoLocalHom'These are the images if you'd like to try for yourself... I did not pay close attention to the location of the targets, this was just to test the pipeline.
https://drive.google.com/drive/folders/1M91NBCwgfbI6kBdIpqV-Uu_26Nh-g4Qz?usp=sharing
File "/home/pi/depthai/depthai_helpers/calibration_utils.py", line 719, in test_epipolar_charuco_lr
res2_l = cv2.aruco.interpolateCornersCharuco(
cv2.error: OpenCV(4.6.0) /io/opencv_contrib/modules/aruco/src/charuco.cpp:446: error: (-215:Assertion failed) _markerCorners.total() == _markerIds.getMat().total() && _markerIds.getMat().total() > 0 in function '_interpolateCornersCharucoLocalHom'
I seem to get a crash at this step with every possible combination of flags, and the image set above... I'm not sure what else to try at this point!
stephansturges Can you share the execution command ?
From memory I was using the flags -ms 2.3 -db -drgb True -brd OAK-CM4-POE
I tried this a few times capturing as I went, then I ran the it with the same flags adding -process to use the last image set captured.
I also tried without specifying the square size to use the default 2.2 in case that made a difference but it failed in the same place.
can you try to specify both the squareSize and markerSize? ex python3 calibrate.py -s 4 -ms 3 ...
I just ran it again with the image set above whith this command:
python3 calibrate.py -s 2.36 -ms 1.87 -db -brd OAK-D-CM4-POE -m process -drgb
--> I got the same crash as above once more.
I've found the issue: calibration fails with the latest arm build of opencv-contrib-python from pip, version 4.6.0.66 ! I started investigating this because the issue seemed to be linked to the charuco markers which are in cv2.contrib
I've rolled back to opencv-contrib-python==4.5.5.64 , and it looks like I have a solution!
I've yet to test saving the results to the device and testing the stereo output, but at least it works.
I guess the requirements.txt file for the OAK-D-CM4-POE needs to specify version 4.5.5.64 for the time being!
