Hardware used:

Hardware mounting:

Config board:

  • OAK-FFC-4P.json

I have been trying to calibrate a stereo pair for depth calculation (disparity). However, I cannot get the epipolar error lower than 1.06. I also have noticed that the reprojection error between cameras differ a lot (0.76 against 1.24)

I am using the charuco from the official github in a flat monitor of 27inch (so I adapted the checker size to 3.8cm). I have followed the recommendations from the website but the error is still high.

This is the command I am running to make de calibration: python3 calibrate.py -s 3.8 -nx 15 -ny 8 -brd OAK-FFC-4P.json

Anyway, I flashed the EEPROM and tried to run depth estimation but the results are really bad

¿Do you have any idea/recomendation about what I can do to improve the calibration?

I attach the images used for the calibration and the script I ran for the disparity calculation

https://drive.google.com/drive/folders/1CVXPqhDcZUiZ2mwpr0SuMn01XbnaRlPP?usp=sharing

Kind regards

Adrian

    adrivg
    Try switching the left and right when performing stereo.

    # Linking
monoLeft.out.link(depth.right)
monoRight.out.link(depth.left)
depth.disparity.link(xout.input)

    or switch sockets for left and right.

    Thanks,
    Jaka

      Thanks for the quick answer. It seems to get better results. However, the person standing in this picture is around 180cm from the camera but the working distance I need is around 70cm. How can I fix that?. From the picture it seems that the FOV is around 20/30º when it should be 45º.

      By the way, I am using RGB cameras for stereo, does it affect?

      Left = pipeline.create(dai.node.ColorCamera)

      Right = pipeline.create(dai.node.ColorCamera)

      depth = pipeline.create(dai.node.StereoDepth)

      xout = pipeline.create(dai.node.XLinkOut)

      Left.video.link(depth.right)

      Right.video.link(depth.left)

      depth.disparity.link(xout.input)

      Thanks,

      Adrian

        adrivg
        One thing I notice is that lenses are out of focus. This will have an affect on the end result (error).
        Second thing that is the problem is that the sensors are rolling shutter which is a problem for moving tagrets -- but should be fine if cameras are completely stationary and the image does not move.

        Can you add the calibration dump from the camera?

        Thanks,
        Jaka

          adrivg
          Hey, the extrisics in the calibration dump say -5cm from one camera to the other, but you have specified 2.9cm. Make sure the calibration was actually applied.

          Thanks,
          Jaka

            Hi!

            You were right, The calibration I used to flash the EEPROM was with 5cm between cameras ( I moved the cameras to that distance) but the disparity is almost as bad.

            I attatch another full configuration (images used, dump file, reprojection errors, ffc json conf and disparity error). I have tried many different configurations but I cannot get a lower epipolar error of 2.5 ( I think it is too much). Watching the images, do you have any recommendation or maybe any conf that I could try?

            https://drive.google.com/drive/folders/1sGeHljsd4kI0ILADPgCvLULOnQ3uTewE?usp=drive_link

            Kind regards,

            Adrian

              adrivg
              Don't point the camera at the bright window, it throws off the AE.
              Set lens position manually with -rlp {position 0-255}
              '-rlp', '--rgbLensPosition', nargs='*', action=ParseKwargs, required=False, default={} , help="Set the manual lens position of the camera for calibration. Example -rlp rgb=135 night=135")

              Thanks
              Jaka

                I will try it and give you a respond.

                What do you mean by "set lens position manually"?. I do not understand what that number between 0-255 refers to.

                Thanks

                Adrian

                  adrivg
                  by setting the -rlp argument (when running the calibrate.py), you can set the value of lens position (same value that is changed using autofocus). It can be 0-255, 0 being close to the glass, 255 being the farthest away.

                  It will change the focus of the image.

                  Thanks,
                  Jaka

                    I tried to calibrate again. I used python3 calibrate.py -s 3.9 -nx 15 -ny 8 -brd OAK-FFC-4P.json -dst calib_3_7_C -ep 2 -rlp right=135 left=135.

                    I tested different rlp values to focus a distance of 90/100 cm from the cameras and the example values were the ones that gave me the best results. The reprojection error was reduced a bit but the epipolar error is still too high (1.06)

                    I send you the setup for the new calibration: https://drive.google.com/drive/folders/1wITNra_xuafv5RCj7222uhPM0IfPHbqV?usp=drive_link

                      4 days later

                      adrivg adrivg
                      The calibration looks fine now.
                      What script are you using to create stereo depth? Can you show left and right rectified frames and the depth created from them?

                      Thanks,
                      Jaka

                          adrivg
                          Cameras are not on the same horizontal axis.. Stereo matching won't work in this case.

                          it is also not accounted for in the extrinsics (nether translation, nor rotation).

                          Please keep them on the same axis.

                          Thanks,
                          Jaka

                            the distribution is the one shown in the harware mounting, They should be in the same horizontal... I am going to check if there is any slack

                              adrivg
                              We checked the dump and recreated depth.. works ok. I think the only issue now is that sockets are switched.

                              Though I would advise you to correct the camera positions 🙂

                              Thanks,
                              Jaka

                                I swapped the sockets based on your previous comment:

                                Try switching the left and right when performing stereo.

                                Linking
                                monoLeft.out.link(depth.right)
                                monoRight.out.link(depth.left)
                                depth.disparity.link(xout.input)
                                or switch sockets for left and right.

                                Thanks,
                                Jaka

                                Should I change them back?
                                I am trying to correct camera positions, not sure why I have such slack using OAK-FFC 4P

                                  adrivg
                                  I know, I was suggesting you try switching it in case it improves the depth. Since it did not solve the issue, yes, revert it back.

                                  Thanks,
                                  Jaka