I have been trying to follow this tutorial: https://docs.luxonis.com/hardware/platform/depth/calibration/#Depth%20Calibration

However, I cannot get the right camera to read in sufficient squares almost none of the time, it is impossible to get even one image at a middle distance, let alone from afar. I am using a 55inch television screen and a 55inch 30x17 board. This is an example of what my terminal output looks like:

python3 calibrate.py -s 3.76 --board OAK-D-PRO-W -nx 30 -ny 17 -cm fisheye

Cam: rgb and focus: False

Cam: left and focus: False

Cam: right and focus: False

Saving dataset to: dataset

Sensor name for rgb is IMX378

Sensor name for left is OV9282

Sensor name for right is OV9282

Starting image capture. Press the [ESC] key to abort.

Will take 39 total images.

Start capturing...

new minimum: {'ts': 0.20283700000000238, 'indicies': {'rgb': 0, 'left': 0, 'right': 0}} min required: 0.2

new minimum: {'ts': 0.20283700000000238, 'indicies': {'rgb': 0, 'left': 0, 'right': 0}} min required: 0.2

new minimum: {'ts': 0.003540999999998462, 'indicies': {'rgb': 3, 'left': 3, 'right': 3}} min required: 0.2

Time stamp of rgb is 5:37:09.535184

Markers count ... 250

Total markers needed -> 102

py: Saved image as: dataset/rgb/p0_0.png

Status of rgb is True

Time stamp of left is 5:37:09.536949

Markers count ... 117

Total markers needed -> 102

py: Saved image as: dataset/left/p0_0.png

Status of left is True

Time stamp of right is 5:37:09.536960

Markers count ... 26

Total markers needed -> 102

Status of right is False

py: Capture failed, unable to find full board! Fix position and press spacebar again

Start capturing...

new minimum: {'ts': 0.002820999999983087, 'indicies': {'rgb': 0, 'left': 0, 'right': 0}} min required: 0.2

Time stamp of rgb is 5:37:41.434374

Markers count ... 231

Total markers needed -> 102

py: Saved image as: dataset/rgb/p0_0.png

Status of rgb is True

Time stamp of left is 5:37:41.435779

Markers count ... 121

Total markers needed -> 102

py: Saved image as: dataset/left/p0_0.png

Status of left is True

Time stamp of right is 5:37:41.435790

Markers count ... 20

Total markers needed -> 102

Status of right is False

py: Capture failed, unable to find full board! Fix position and press spacebar again

Start capturing...

new minimum: {'ts': 0.00282699999999636, 'indicies': {'rgb': 0, 'left': 0, 'right': 0}} min required: 0.2

Time stamp of rgb is 5:38:16.033349

Markers count ... 243

Total markers needed -> 102

py: Saved image as: dataset/rgb/p0_0.png

Status of rgb is True

Time stamp of left is 5:38:16.034757

Markers count ... 118

Total markers needed -> 102

py: Saved image as: dataset/left/p0_0.png

Status of left is True

Time stamp of right is 5:38:16.034768

Markers count ... 23

Total markers needed -> 102

Status of right is False

py: Capture failed, unable to find full board! Fix position and press spacebar again

Start capturing...

new minimum: {'ts': 0.20283699999995974, 'indicies': {'rgb': 0, 'left': 0, 'right': 0}} min required: 0.2

new minimum: {'ts': 0.0035399999999583542, 'indicies': {'rgb': 2, 'left': 2, 'right': 2}} min required: 0.2

Time stamp of rgb is 5:39:15.331228

Markers count ... 241

Total markers needed -> 102

py: Saved image as: dataset/rgb/p0_0.png

Status of rgb is True

Time stamp of left is 5:39:15.332991

Markers count ... 92

Total markers needed -> 102

Status of left is False

Time stamp of right is 5:39:15.333003

Markers count ... 17

Total markers needed -> 102

Status of right is False

py: Capture failed, unable to find full board! Fix position and press spacebar again

Start capturing...

new minimum: {'ts': 0.002810999999979913, 'indicies': {'rgb': 0, 'left': 0, 'right': 0}} min required: 0.2

Time stamp of rgb is 5:39:44.030613

Markers count ... 247

Total markers needed -> 102

py: Saved image as: dataset/rgb/p0_0.png

Status of rgb is True

Time stamp of left is 5:39:44.032014

Markers count ... 123

Total markers needed -> 102

py: Saved image as: dataset/left/p0_0.png

Status of left is True

Time stamp of right is 5:39:44.032025

Markers count ... 19

Total markers needed -> 102

Status of right is False

py: Capture failed, unable to find full board! Fix position and press spacebar again

Start capturing...

new minimum: {'ts': 0.10353499999999372, 'indicies': {'rgb': 0, 'left': 0, 'right': 0}} min required: 0.2

new minimum: {'ts': 0.0028440000000102827, 'indicies': {'rgb': 3, 'left': 3, 'right': 3}} min required: 0.2

Time stamp of rgb is 5:40:19.429817

Markers count ... 247

Total markers needed -> 102

py: Saved image as: dataset/rgb/p0_0.png

Status of rgb is True

Time stamp of left is 5:40:19.431233

Markers count ... 131

Total markers needed -> 102

py: Saved image as: dataset/left/p0_0.png

Status of left is True

Time stamp of right is 5:40:19.431245

Markers count ... 21

Total markers needed -> 102

Status of right is False

py: Capture failed, unable to find full board! Fix position and press spacebar again

^[py: Calibration has been interrupted!

Could this be a hardware issue? What should I try next?

Here are some examples of images I took for left and right:

    isagmz
    From the images you have taken above, it seems like there is to much brightness for the cameras to detect the aruco markers. Seems that on the lower images, that issue is fixed. Could you also try with a smaller board (less markers so they are better seen)?

    Thanks,
    Jaka

      What I am trying to show on the top images is that the left camera detects almost all of the aruco markers when at the same time and location the right camera does not. The images on the bottom are very very close to the board.

      How can it be too bright for the right camera but not too bright for the left and RGB camera?

      In the calibration documentation they say we should take some images close and some far away. But since my right camera is not detecting the aruco markers from even a little bit far away, even though the left one is… I cannot make much progress with the calibration. Since the documentation also suggests that the bigger the charuco board, the better, my first instinct was to try with the biggest board.

      I will try with a smaller board now, but I am still not understanding how it is possible that the left camera and the RGB camera have no problem detecting 131 and 247 markers respectively, and at the same time the right camera detects 21 markers.

      As per this example in the terminal output:

      Time stamp of rgb is 5:40:19.429817

      Markers count ... 247

      Total markers needed -> 102

      py: Saved image as: dataset/rgb/p0_0.png

      Status of rgb is True

      Time stamp of left is 5:40:19.431233

      Markers count ... 131

      Total markers needed -> 102

      py: Saved image as: dataset/left/p0_0.png

      Status of left is True

      Time stamp of right is 5:40:19.431245

      Markers count ... 21

      Total markers needed -> 102

      Status of right is False

      The reason why I am trying to calibrate the device is because I am getting nonsense depth outputs. I am running a maskrcnn_coco model and estimating the depth of objects and these are the outputs I am getting:

      person depth: 14.00 mm

      person depth: 1538.00 mm

      person depth: 14.00 mm

      person depth: 14.00 mm

      person depth: 14.00 mm

      person depth: 14.00 mm

      person depth: 14.00 mm

      person depth: 1538.00 mm

      person depth: 14.00 mm

      person depth: 1538.00 mm

      How is it switching from 14 mm to 1.5 meters in seconds? Furthermore, if I put the camera at an arm's length away, the readings go to zero only to return to the 14 mm to 15 hundred mm reading when I bring it a bit closer to myself.

      person depth: 0.00 mm

      person depth: 0.00 mm

      person depth: 0.00 mm

      person depth: 0.00 mm

      person depth: 14.00 mm

      person depth: 1538.00 mm

      person depth: 1346.00 mm

      person depth: 14.00 mm

      person depth: 15.00 mm

      person depth: 14.00 mm

      person depth: 1346.00 mm

      isagmz python3 calibrate.py -s 3.76 --board OAK-D-PRO-W -nx 30 -ny 17 -cm fisheye

      Not a fisheye, don't use fisheye. Set the -mdmp 0.2 to lower min detected markers.

      Thanks,
      Jaka