Return to blog overview
Blog

Announcing dramatic improvement to OAK-D depth accuracy

Evolution gave humans, and most all other mammals, two eyes. Human eyes are typically spaced 6 to 7 centimeters (2.4 to 2.8 inches) apart. Remarkably, a substantial portion of the human brain is dedicated to processing visual information, with some studies suggesting a significant investment of the brain's resources in this area (source: University of Rochester). Our brain is able to perceive both how close an object is from us and where that object is heading. When Luxonis set out to create an edge artificial intelligence (AI) depth camera, we used learnings from human eye sight to design our products. This is why the stereo baselines of our cameras are typically 7.5 centimeters (3 inches) apart and a sizable portion of our onboard processing is dedicated towards fusing the left and right cameras together to create a three dimensional map. 

At Luxonis, we are on a long term mission of empowering companies to surpass human perception in every task. To achieve that mission, it requires the accuracy of the depth map to be superior to humans. Fortunately, we have already long since exceeded this because human beings are actually quite poor at estimating distances. If you look at your hands do you really know if they are currently 10, 12, or 14 inches away from your head? There is a reason we all have a tape measure (or laser or smartphone) at home to help us determine distances. 

We have recently been working diligently to improve the accuracy of our depth maps. And we are now proud to announce that starting in March 2024, stereo cameras shipped from Luxonis will be implemented with a next generation calibration procedure that delivers much better accuracy than our old generation. Please note that fortunately the performance is not hardware limited so existing devices can be recalibrated. 

Here are the real world (not theoretical) depth accuracy measurements that you’ll be experiencing with our products come March: 

  • 0.7m - 4m (2’4” to 13’1”): below 2% absolute depth error

  • 4m - 7m (13’1” to 23’): below 4% absolute depth error

  • 7m - 12m (23’ to 39’4”): below 6% absolute depth error

You can learn more about our improved performance here:

https://docs.luxonis.com/projects/hardware/en/latest/pages/guides/depth_accuracy/#p-75mm-baseline-distance-oaks

If you’d like to get your hands on an OAK with improved depth sooner please reach out to us at [support@luxonis.com](mailto:support@luxonis.com).

We'll be releasing competitive benchmark data on depth performance by the end of March.

    Comments (16)

    Evolution gave humans, and most all other mammals, two eyes. Human eyes are typically spaced 6 to 7 centimeters (2.4 to 2.8 inches) apart. Remarkably, a substantial portion of the human brain is dedicated to processing visual information, with some studies suggesting a significant investment of the brain's resources in this area (source: University of Rochester). Our brain is able to perceive both how close an object is from us and where that object is heading. When Luxonis set out to create an edge artificial intelligence (AI) depth camera, we used learnings from human eye sight to design our products. This is why the stereo baselines of our cameras are typically 7.5 centimeters (3 inches) apart and a sizable portion of our onboard processing is dedicated towards fusing the left and right cameras together to create a three dimensional map. 

    At Luxonis, we are on a long term mission of empowering companies to surpass human perception in every task. To achieve that mission, it requires the accuracy of the depth map to be superior to humans. Fortunately, we have already long since exceeded this because human beings are actually quite poor at estimating distances. If you look at your hands do you really know if they are currently 10, 12, or 14 inches away from your head? There is a reason we all have a tape measure (or laser or smartphone) at home to help us determine distances. 

    We have recently been working diligently to improve the accuracy of our depth maps. And we are now proud to announce that starting in March 2024, stereo cameras shipped from Luxonis will be implemented with a next generation calibration procedure that delivers much better accuracy than our old generation. Please note that fortunately the performance is not hardware limited so existing devices can be recalibrated. 

    Here are the real world (not theoretical) depth accuracy measurements that you’ll be experiencing with our products come March: 

    • 0.7m - 4m (2’4” to 13’1”): below 2% absolute depth error

    • 4m - 7m (13’1” to 23’): below 4% absolute depth error

    • 7m - 12m (23’ to 39’4”): below 6% absolute depth error

    You can learn more about our improved performance here:

    https://docs.luxonis.com/projects/hardware/en/latest/pages/guides/depth_accuracy/#p-75mm-baseline-distance-oaks

    If you’d like to get your hands on an OAK with improved depth sooner please reach out to us at [support@luxonis.com](mailto:support@luxonis.com).

    We'll be releasing competitive benchmark data on depth performance by the end of March.

      Hello,

      Is the software to recalibrate the cameras already available? and will there be a guide on how to do this?
      Would it also be possible to extend the depth performance test to include extended disparity. A range between 35cm and 1m is missing from this graph.

      Hi @Tsjarly
      This refers to the factory calibration which has been improved. The calibration procedure should still be the same can be done by following the guide here.

      okay perfect!
      I just noticed that the graph in this post is the same one as already posted in the main docs. How much of an improvement are we talking about with this new calibration, compared to the older version?

      This is very interesting. We've seen some anecdotal data that indicates our cameras are not all calibrated the same (in some cases egregiously in the 4-10m range). How would one determine if their camera would benefit from a manual calibration? Also, if the factory calibration improved so much how does one take advantage of it?

      I happen to have a screenshot of the reported depth accuracy from the end of October, which I would assume is using the calibration as before your update. I see here that the error at 4m is around 1.7%, and in your new data it seems to lie around 0.7% (with the peak being conveniently at the bottom at that mark). How is this an improvement of 5X?
      Also, is there an explanation for this oscillating effect?
      And indeed, how would this new calibration effect usage of the cameras. Is it purely disparity to depth computation that is better, or is there also improvement in the general quality of the depth map?

      Hi @Tsjarly ,
      The graph you have screenshotted is from the first unit that came off of the new calibration rig. After that, we tested 10 factory-calibrated cameras (with new rig we setup at factory), we tested all of them and took the median depth accuracy for what we advertise as our depth (instead of just taking the best one).
      We tested 10 cameras that were calibrated with previous system, and the average was about 20% of error at 4m, and now it's well below 1/5 of that (median is 2% below 4m).
      Regarding the oscillation effect - I'll be writing docs about it, essentially it's disparity pixel thing. so "error valley" (where accuracy is spot-on) at 6.6m would be likely be equate to 10 disparity pixels, while valley at 7.5m would be 9pix disparity (so it was 9 pixels difference between left/right img, which corresponds to 7.5m depth).
      Here's the theoretical calculations for an OAK-D (it will vary camera to camera due to diff in focal len):

      purely disparity to depth computation

      This as well, but major thing is cam intrinsic estimation and distortion model estimation (especially for wide fov cameras). So disparity itself will come out different already.
      Thoughts?
      Thanks, Erik


        erik

        Thanks for the explanation!
        If I understand correctly, the screenshot that I attached already used the new calibration rig.
        We had some issues with the calibration of the cameras, and back then it was advised to recalibrate the stereo camera, which we did (this was on october 27th 2023) which also seemed to slightly improve our own results. We used the depthai calibrate.py script to do so.
        Does the current update differ much from back then? As in would it be worth it to recalibrate the cameras again?

        Regarding the oscillating effect, that sounds as if it makes sense 🙂

          6 days later

          Hi @erik, what about the previously purchased cameras? Will the calibrate.py script recalibrate the camera to perform in the same manner as new cameras shipped from March? Thank you.

          Hi @marlu ,
          calibrate.py should improve calibration compared to old calibration approach, but will still fall short to the current rail calibration approach. Perhaps ask on support@Luxonis.com if you could send in your device to our EU office (Ljubljana/Prague) to get your camera recalibrated with the new rail calibration approach.
          Thanks, Erik

          19 days later

          BradleyDillon

          I have an OAK-D Pro PoE FF product.

          The casting date is March 7th, 2024 and the delivery date is March 8th.

          Installed flashed bootloader version is 0.0.22, but newest is 0.0.26.
          - Should I update my bootloader?

          - What additional work will be needed on the calibration data if I update it?

          The MxID is 184430108145A20F00. Has improved calibration been done for this product?

          Is there any way to check this?

            erik

            1. Is there any way to check new calibrated state?

            1. you say "we are now proud to announce that starting in March 2024, stereo cameras shipped from Luxonis will be implemented with a next generation calibration procedure that delivers much better accuracy than our old generation. Please note that fortunately the performance is not hardware limited so existing devices can be recalibrated."

              So, can you replace the device with a freshly calibrated one?

            2. Is there any difference between the reference site you suggested for manual recalibration and the site below?
              https://docs.luxonis.com/projects/hardware/en/latest/pages/guides/calibration/

              @OseongKwon ,

              1. No, not really
              2. Yes, send an email to support@luxonis.com and they will handle it
              3. Yes - factory calibration (and calibration in our offices) is better, as it's always identical and (close to) ideal WRT the distance/angles/FOV coverage.