After successful calibration with charuco board, I do see the images in the Dataset folder. However, we contninure to see the following error message. What does it mean and how we can fix it? We used this command for our OAK-D-Lite: python calibrate.py -s 2.35 -brd OAK-D-LITE -db gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) cv2.error: OpenCV(4.5.5) D:\a\opencv-python\opencv-python\opencv\modules\imgproc\src\color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cv::cvtColor'

Hi @"JamesYeh"#p5606 , Which commit are you using of the `depthai` repo (where calibrate.py is located)? Are you using latest? Which depthai version are you using? Thanks, Erik

Here is the command we use: python calibrate.py -s 2.45 -brd OAK-D-LITE -db We eventually got t finish the calibration process, but at the end with an Eipolar number. The screen showed red background image with text "EEprom Write Failed!! 0.144 Try recalibrating!! We tried a few more times for calibration but the same outcome showing EEprom Write error. [URL=https://i.imgur.com/ikt15iM.jpg][IMG]https://i.imgur.com/ikt15iMh.jpg[/IMG][/URL]

@"JamesYeh"#p5626 could you share the images that you took while calibrating? I believe they are written in `/dataset` folder. I would assume images for calibration weren't taken ideally which results in bad epipolar error and failed calibration. Thanks, Erik

Here are 3 image sets for your review. Thanks! [URL=https://i.imgur.com/xIqmXoj.jpg][IMG]https://i.imgur.com/xIqmXojh.jpg[/IMG][/URL] [URL=https://i.imgur.com/G2UZhEE.jpg][IMG]https://i.imgur.com/G2UZhEEh.jpg[/IMG][/URL] [URL=https://i.imgur.com/8rdAW6G.jpg][IMG]https://i.imgur.com/8rdAW6Gh.jpg[/IMG][/URL]

@"JamesYeh"#p5630 Can you share a clear image of the calibration board? Just want to see if it has any defects. Also, can you double-check the square and marker size?

Here are 3 up-to-date scanned images and cardboard image. I tried many times and it still showed EEprom Write failed message. I'm wondering if this is due to hardware setting issues instead of calibration procedure. I used the command in the following for OAK_D Lite and the square size is 2.45 cm. Thanks! python calibrate.py -s 2.45 -brd OAK-D-LITE -db [URL=https://i.imgur.com/UqUx5KE.jpg][IMG]https://i.imgur.com/UqUx5KEh.jpg[/IMG][/URL] [URL=https://i.imgur.com/0kmO0CU.jpg][IMG]https://i.imgur.com/0kmO0CUh.jpg[/IMG][/URL] [URL=https://i.imgur.com/KlbkvwB.jpg][IMG]https://i.imgur.com/KlbkvwBh.jpg[/IMG][/URL] [URL=https://i.imgur.com/nuYNI1p.jpg][IMG]https://i.imgur.com/nuYNI1ph.jpg[/IMG][/URL]

@"JamesYeh"#p5640 Might be the quality of the calibration board (as you glued it to the carton board, some dents may have formed, it's also very glossy), can you try to calibrate using a monitor to show the Calib board? Or to print a better board?

@"JamesYeh"#944 what version of depthai you have installed? Do you have other depthai devices, and if so, do you see the same calibration error for them?

It's the latest version we downloaded from your website recently.

@"JamesYeh"#p5647 in the depthai/resources/ there should be export of the calibration file called . Can you find it and share it with us?

Calibration error

PetrNovota

@"JamesYeh" I would like you to try one little experiment.
in your depthai/ folder, locate the calibrate.py file. open it with any text editor or an IDE like Pycharm.
On lines 611-614
you can locate this code:

if self.args.factoryCalibration:
    self.device.flashFactoryCalibration(calibration_handler)
is_write_succesful = self.device.flashCalibration(calibration_handler)

change it to

if self.args.factoryCalibration:
    self.device.flashFactoryCalibration(calibration_handler)
is_write_succesful = self.device.flashCalibration2(calibration_handler)

The change is int the self.device.flashCalibration2

It can happen that this code piece is not on above mentioned lines, it depends on the depthai branch. Either way yor calibrate.py file should contain those lines so you can find them.

When it is done, run the calibration again, the same way as before. Now, it should end with en Error printed out in the terminal. Send us the error please.

JamesYeh

It's interesting that I don't see the codes you sent me. Here is what I installed for the calibration process in your website https://docs.luxonis.com/en/latest/pages/calibration/. Where did you get the calibrate.py?

git clone https://github.com/luxonis/depthai.git
cd depthai
git checkout lite_calibration

PetrNovota

JamesYeh in the branch lite_calibration that you are using it is in
depthai/calibrate.py
in there, lines 570-576 change them from:

try:
    is_write_succesful = self.device.flashCalibration(calibration_handler)
except:
    print("Writing in except...")
    is_write_succesful = self.device.flashCalibration(calibration_handler)

self.device.flashCalibration2(calibration_handler)

so it will now look like this:

...
...
calib_dest_path = dest_path + '/' + mx_serial_id + '.json'
calibration_handler.eepromToJsonFile(calib_dest_path)
is_write_succesful = False

is_write_succesful = self.device.flashCalibration2(calibration_handler)
if is_write_succesful:
    resImage = create_blank(900, 512, rgb_color=green)
    ...
    ...
    ...

and run calibration again, you should get an Error, send it to us

JamesYeh

Here are the codes at the end of the calibrate.py

resImage = None
if not self.device.isClosed():
dev_info = self.device.getDeviceInfo()
mx_serial_id = dev_info.getMxId()
calib_dest_path = dest_path + '/' + mx_serial_id + '.json'
calibration_handler.eepromToJsonFile(calib_dest_path)
is_write_succesful = False

            try:
                is_write_succesful = self.device.flashCalibration(
                    calibration_handler)
            except:
                print("Writing in except...")
                is_write_succesful = self.device.flashCalibration(
                    calibration_handler)
            if is_write_succesful:
                resImage = create_blank(900, 512, rgb_color=green)
                text = "Calibration Succesful with"
                cv2.putText(resImage, text, (10, 250),
                            font, 2, (0, 0, 0), 2)
                text = "Epipolar error of " + str(epiploar_error)
                cv2.putText(resImage, text, (10, 300),
                            font, 2, (0, 0, 0), 2)
            else:
                resImage = create_blank(900, 512, rgb_color=red)
                text = "EEprom Write Failed!! " + str(epiploar_error)
                cv2.putText(resImage, text, (10, 250),
                            font, 2, (0, 0, 0), 2)
                text = "Try recalibrating !!"
                cv2.putText(resImage, text, (10, 300),
                            font, 2, (0, 0, 0), 2)
        else:
            calib_dest_path = dest_path + '/depthai_calib.json'
            # calibration_handler.eepromToJsonFile(calib_dest_path)
            resImage = create_blank(900, 512, rgb_color=red)
            text = "Calibratin succesful. " + str(epiploar_error)
            cv2.putText(resImage, text, (10, 250), font, 2, (0, 0, 0), 2)
            # text = "Device not found to write to EEPROM"
            # cv2.putText(resImage, text, (10, 300), font, 2, (0, 0, 0), 2)

        if resImage is not None:
            cv2.imshow("Result Image", resImage)
            cv2.waitKey(0)
    except AssertionError as e:
        print("[ERROR] " + str(e))
        raise SystemExit(1)

def run(self):
    if 'capture' in self.args.mode:
        try:
            if Path('dataset').exists():
                shutil.rmtree('dataset/')
            Path("dataset/left").mkdir(parents=True, exist_ok=True)
            Path("dataset/right").mkdir(parents=True, exist_ok=True)
            if not self.args.disableRgb:
                Path("dataset/rgb").mkdir(parents=True, exist_ok=True)
        except OSError:
            traceback.print_exc()
            print("An error occurred trying to create image dataset directories!")
            raise SystemExit(1)
        self.show_info_frame()
        self.capture_images()
    self.dataset_path = str(Path("dataset").absolute())
    if 'process' in self.args.mode:
        self.calibrate()
    print('py: DONE.')

if name == "main":
Main().run()

JamesYeh

I changed the calibrate.py and ran it again. It is still the same. See the screen shots...

PetrNovota

JamesYeh thank you, good job. Now, I want to ask you to do one more thing (consists of two sub tasks 🙂).

With editor of your choice create a new file called get_calibration.py and save it to your depthai folder, so it will look like this: dephtai/get_calibration.py
The content of the get_calibration.py file is:

import depthai as dai

with dai.Device() as device:
    print(device.readCalibration2().eepromToJson())

and in depthai folder run python get_calibration.py
This will print the current device calibration in your terminal. Please copy it and send it to us.

Second task is the same one you have done before. In depthai/resources there is your <device_mx_id(long number).json> file. Please send its contents to us as well.

Thank you for your patience.
Best regards
Petr

JamesYeh

Ok will try it. And, is it possible to run the test again with the sames images in Dataset folder under depthai folder? It would be more consistent for this debug with the same image set. Thanks!

JamesYeh

here is the .json text file.

{
"batchName": "",
"batchTime": 0,
"boardConf": "",
"boardCustom": "",
"boardName": "OAK-D-LITE",
"boardOptions": 0,
"boardRev": "R0M0E0",
"cameraData": [
[
1,
{
"cameraType": 0,
"distortionCoeff": [
-28.19111442565918,
319.1015625,
0.0021235786844044924,
-0.00796385295689106,
-973.7730712890625,
-28.19227409362793,
319.0585021972656,
-973.5380249023438,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0
],
"extrinsics": {
"rotationMatrix": [
[
0.9998503923416138,
-0.0063782683573663235,
-0.016076868399977684
],
[
0.0063193743117153645,
0.9999731183052063,
-0.003711411962285638
],
[
0.01610010862350464,
0.003609261242672801,
0.999863862991333
]
],
"specTranslation": {
"x": -7.5,
"y": 0.0,
"z": 0.0
},
"toCameraSocket": 2,
"translation": {
"x": -7.544513702392578,
"y": -0.06922637671232224,
"z": -0.03127232566475868
}
},
"height": 480,
"intrinsicMatrix": [
[
455.9234619140625,
0.0,
319.5633239746094
],
[
0.0,
456.3455505371094,
230.58319091796875
],
[
0.0,
0.0,
1.0
]
],
"lensPosition": 0,
"specHfovDeg": 72.9000015258789,
"width": 640
}
],
[
2,
{
"cameraType": 0,
"distortionCoeff": [
21.66294288635254,
-312.0931396484375,
0.0010793297551572323,
-0.0007400921895168722,
906.7825927734375,
21.852170944213867,
-314.07861328125,
912.0392456054688,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0
],
"extrinsics": {
"rotationMatrix": [
[
0.9998428821563721,
0.00951849389821291,
-0.014950711280107498
],
[
-0.009514091536402702,
0.9999546408653259,
0.00036551340599544346
],
[
0.014953513629734516,
-0.00022321353026200086,
0.9998881220817566
]
],
"specTranslation": {
"x": 3.75,
"y": 0.0,
"z": 0.0
},
"toCameraSocket": 0,
"translation": {
"x": 3.9650027751922607,
"y": 0.006998621392995119,
"z": -0.5213539600372314
}
},
"height": 480,
"intrinsicMatrix": [
[
455.4236755371094,
0.0,
304.4522399902344
],
[
0.0,
455.9582824707031,
240.3455352783203
],
[
0.0,
0.0,
1.0
]
],
"lensPosition": 0,
"specHfovDeg": 72.9000015258789,
"width": 640
}
],
[
0,
{
"cameraType": 0,
"distortionCoeff": [
30.001792907714844,
-27.810789108276367,
0.0005656384746544063,
0.0028413927648216486,
170.4805908203125,
30.07124900817871,
-31.029281616210938,
174.8137664794922,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0
],
"extrinsics": {
"rotationMatrix": [],
"specTranslation": {
"x": 0.0,
"y": 0.0,
"z": 0.0
},
"toCameraSocket": -1,
"translation": {
"x": 0.0,
"y": 0.0,
"z": 0.0
}
},
"height": 1080,
"intrinsicMatrix": [
[
1493.0745849609375,
0.0,
1028.433837890625
],
[
0.0,
1489.53271484375,
517.1607055664063
],
[
0.0,
0.0,
1.0
]
],
"lensPosition": 135,
"specHfovDeg": 68.7938003540039,
"width": 1920
}
]
],
"hardwareConf": "",
"imuExtrinsics": {
"rotationMatrix": [],
"specTranslation": {
"x": 0.0,
"y": 0.0,
"z": 0.0
},
"toCameraSocket": -1,
"translation": {
"x": 0.0,
"y": 0.0,
"z": 0.0
}
},
"miscellaneousData": [],
"productName": "",
"stereoRectificationData": {
"leftCameraSocket": 1,
"rectifiedRotationLeft": [
[
0.9999244809150696,
0.0028120274655520916,
-0.011965838260948658
],
[
-0.002833560574799776,
0.9999943971633911,
-0.001782975741662085
],
[
0.01196075789630413,
0.001816747011616826,
0.9999268054962158
]
],
"rectifiedRotationRight": [
[
0.9999493360519409,
0.009175259619951248,
0.00414483156055212
],
[
-0.009182704612612724,
0.9999562501907349,
0.0017808872507885098
],
[
-0.004128310363739729,
-0.0018188577378168702,
0.9999898076057434
]
],
"rightCameraSocket": 2
},
"version": 7
}

JamesYeh

From the EEprom data:
PS C:\GG\OAK-Calibrate\OAK-Calibrate\depthai> python get_calibrate.py
{'batchName': '', 'batchTime': 0, 'boardConf': '', 'boardCustom': '', 'boardName': 'OAK-D-LITE', 'boardOptions': 0, 'boardRev': 'R1M1E3', 'cameraData': [[0, {'cameraType': 0, 'distortionCoeff': [-4.465783596038818, 9.97364616394043, -0.00019955226161982864, -0.0010946181137114763, -4.277606964111328, -4.544586181640625, 10.269360542297363, -4.632837295532227, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], 'extrinsics': {'rotationMatrix': [[0.0, 0.0, 0.0], [0.0, 0.0, 0.0], [0.0, 0.0, 0.0]], 'specTranslation': {'x': -0.0, 'y': -0.0, 'z': -0.0}, 'toCameraSocket': -1, 'translation': {'x': 0.0, 'y': 0.0, 'z': 0.0}}, 'height': 2160, 'intrinsicMatrix': [[2969.224609375, 0.0, 2027.668701171875], [0.0, 2969.224609375, 1030.9114990234375], [0.0, 0.0, 1.0]], 'lensPosition': 0, 'specHfovDeg': 68.7938003540039, 'width': 3840}], [1, {'cameraType': 0, 'distortionCoeff': [-5.925610542297363, 13.577345848083496, 0.0005030203028582036, -0.0037203540559858084, 2.8491311073303223, -5.928144454956055, 13.576969146728516, 2.8172836303710938, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], 'extrinsics': {'rotationMatrix': [[0.9999157190322876, -0.005482868291437626, -0.011770736426115036], [0.005498494952917099, 0.9999840259552002, 0.0012956152204424143], [0.011763445101678371, -0.0013602273538708687, 0.9999299049377441]], 'specTranslation': {'x': -7.5, 'y': 0.0, 'z': 0.0}, 'toCameraSocket': 2, 'translation': {'x': -7.459471702575684, 'y': -0.11444893479347229, 'z': -0.08647604286670685}}, 'height': 480, 'intrinsicMatrix': [[456.83233642578125, 0.0, 322.490234375], [0.0, 456.83233642578125, 228.93768310546875], [0.0, 0.0, 1.0]], 'lensPosition': 0, 'specHfovDeg': 72.9000015258789, 'width': 640}], [2, {'cameraType': 0, 'distortionCoeff': [15.70577621459961, -133.31661987304688, -0.0004125449631828815, -0.002012420678511262, 418.70599365234375, 15.833054542541504, -134.20359802246094, 420.8946533203125, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], 'extrinsics': {'rotationMatrix': [[0.9999483227729797, 0.008442151360213757, -0.005667971912771463], [-0.008480061776936054, 0.9999415874481201, -0.0066981068812310696], [0.005611094646155834, 0.00674582552164793, 0.9999614953994751]], 'specTranslation': {'x': 3.75, 'y': 0.0, 'z': 0.0}, 'toCameraSocket': 0, 'translation': {'x': 3.7240123748779297, 'y': 0.07089735567569733, 'z': -0.19739195704460144}}, 'height': 480, 'intrinsicMatrix': [[455.19598388671875, 0.0, 302.0895690917969], [0.0, 455.19598388671875, 237.84361267089844], [0.0, 0.0, 1.0]], 'lensPosition': 0, 'specHfovDeg': 72.9000015258789, 'width': 640}]], 'hardwareConf': '', 'imuExtrinsics': {'rotationMatrix': [[0.0, 0.0, 0.0], [0.0, 0.0, 0.0], [0.0, 0.0, 0.0]], 'specTranslation': {'x': 0.0, 'y': 0.0, 'z': 0.0}, 'toCameraSocket': -1, 'translation': {'x': 0.0, 'y': 0.0, 'z': 0.0}}, 'miscellaneousData': [], 'productName': '', 'stereoRectificationData': {'leftCameraSocket': 1, 'rectifiedRotationLeft': [[0.9999515414237976, 0.009842065162956715, -0.00015885815082583576], [-0.009841952472925186, 0.9999513030052185, 0.0006939409067854285], [0.0001656802196521312, -0.0006923438049852848, 0.9999997615814209]], 'rectifiedRotationRight': [[0.9998151659965515, 0.015339930541813374, 0.011590640991926193], [-0.015331892296671867, 0.999882161617279, -0.0007820441969670355], [-0.011601271107792854, 0.0006041931919753551, 0.9999325275421143]], 'rightCameraSocket': 2}, 'version': 6}
PS C:\GG\OAK-Calibrate\OAK-Calibrate\depthai>

PetrNovota

JamesYeh Thank you very much. We know what the problem is, we can solve it, we just have to discuss what the best possible course of action is. Ill get back to you as soon as I can.

JamesYeh

Thanks! Are we talking about 1-2 days or more? We've tried the calibration in the past week without success...

PetrNovota

JamesYeh Hello James. The fix is done.
Please pull latest changes from lite_calibration branch, i.e. git fetch and git pull.
After that you can do the calibration one more time, this time it should succeed. If not, let me know, but it is very unlikely.

JamesYeh

can you please elaborate more how to proceed with some details? Where is the lite_calibration branch, etc... Thanks!

PetrNovota

JamesYeh You wrote:
git clone https://github.com/luxonis/depthai.git
cd depthai
git checkout lite_calibration

Right now, open your terminal and cd into your deptahi dir. When you are in the depthai dir, you enter commands
git fetch confirm with enter
git pull confirm with enter

PetrNovota

@JamesYeh if you dont want to repeat the image capturing, because you already have those, then you can comment out lines 611-623 in calibrate.py script, i.e. add # in front of each line.

PetrNovota

@JamesYeh then you can run python calibrate.py again, I suggest you comment out lines I mentioned, then capturing images will be skipped. You can do this if you still have captured images from before in deptahi/dataset/ folder

JamesYeh

after fetch and pull, I got this error message

PetrNovota

JamesYeh I am sorry didnt realize we made changes.
Use git restore first, then git fetch then git pull.
Only when its successful, run calibration again

JamesYeh

Then I ran the calibration again but it is still the same error message with EEprom Write failed...

JamesYeh

WE MADE IT!!!

PetrNovota

JamesYeh Nice! good job 🥳. I am sorry for the inconvenience, there was a bug in the script and with your help we were able to find it and resolve it. Thank you

« Previous Page