Ever wished your robot had the vision of an eagle? Well, meet the OAK-D Long Range, a smart 3D camera that's bringing super sight to projects worldwide. Let’s dive into how it achieves this, especially with its 15cm stereo baseline.

The 15cm Stereo Baseline: A Further Perspective

With a 15cm stereo baseline, the OAK-D Long Range can perceive further objects more accurately. In the Configuring Stereo Depth documentation, we mentioned that baseline distance (linearly) correlates with accuracy, so if we increase the baseline distance by 2x, you will get 2x less error at the same distance.

Depth Accuracy

On our Depth Accuracy documentation page, we have more in-depth (pun intended) information about each device's accuracy, along with graphs and explanations on why the accuracy graphs are the way they are. Here's the depth accuracy (with relation to the distance) of our OAK-D Long Range:

This accuracy was obtained with the default lens with 82° horizontal FOV. We'd get better long-range accuracy if we reduced FOV (a different lens or a central-cropping frame instead of ISP downscaling).

Pointclouds!

Graphs are great, but they're useless if the camera isn't applicable in real-world applications. We have recorded some footage on a vibration-heavy tractor (a 30 years old IMT 549), and unfiltered pointclouds look quite decent.
Using rerun.io (well, our flavour of it) we visualized RGB-D frames we got directly from the device and there was no on-host postprocessing (khm.. like RealSense™ cameras operate). So we'd get even smoother surfaces if we added host-compute-intensive pointcloud processing.

Reproduce: The Code

We're using depthai-sdk for the pipeline construction and replaying, and the depthai-viewer for visualization. We could also use native Rerun (here's an example with Rerun), but pointclouds wouldn't be colorized. Before running the code you'll need to install the requirements:

python3 -mpip install depthai-sdk==1.14 depthai-viewer==0.2.2

After installing the requirements and connecting an OAK device to your computer (which does stereo reconstruction), you can run the following code:

from depthai_sdk import OakCamera
import depthai as dai
import depthai_viewer as viewer
import subprocess
import sys
import cv2

# Download and run the tractor recording
with OakCamera(replay="tractor-oak-d-lr") as oak:

    # Run & initialize the depthai_viewer
    try:
        subprocess.Popen([sys.executable, "-m", "depthai_viewer", "--viewer-mode"], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
    except subprocess.TimeoutExpired:
        pass
    viewer.init("Depthai Viewer")
    viewer.connect()

    oak.replay.set_fps(10)
    cam_b = oak.create_camera('CAM_B')
    cam_c = oak.create_camera('CAM_C')
    # Downscale the 1920x1200 frames to 1280x800
    oak.replay.resize('cam_c', (1280, 800))
    oak.replay.resize('cam_b', (1280, 800))

    nn = oak.create_nn('mobilenet-ssd', cam_c)

    stereo = oak.create_stereo(left=cam_b, right=cam_c)
    stereo.node.setOutputSize(640, 400)
    stereo.config_stereo(confidence=215, lr_check=True, extended=True, subpixel=True, subpixel_bits=5)
    stereo.config_stereo(align=cam_c)
    stereo.node.setDepthAlign(dai.CameraBoardSocket.CAM_C)

    # On-device post processing for stereo depth
    config = stereo.node.initialConfig.get()
    stereo.node.setPostProcessingHardwareResources(4, 4)
    config.postProcessing.speckleFilter.enable = True
    config.postProcessing.speckleFilter.speckleRange = 50
    config.postProcessing.temporalFilter.enable = False
    config.postProcessing.spatialFilter.enable = True
    config.postProcessing.spatialFilter.holeFillingRadius = 1
    config.postProcessing.spatialFilter.numIterations = 1
    config.postProcessing.thresholdFilter.minRange = 400
    config.postProcessing.thresholdFilter.maxRange = 15000
    config.postProcessing.brightnessFilter.maxBrightness = 255
    stereo.node.initialConfig.set(config)

    q = oak.queue([
        stereo.out.depth.set_name('depth'),
        stereo.out.rectified_right.set_name('rr'),
        nn.out.main.set_name('nn'),
    ]).configure_syncing(enable_sync=True, threshold_ms=250).get_queue()
    # oak.show_graph()
    oak.start()

    calibData = oak.device.readCalibration()
    intrinsics = calibData.getCameraIntrinsics(dai.CameraBoardSocket.CAM_C, dai.Size2f(640, 400))
    viewer.log_pinhole("CAM_C/transform/Color", child_from_parent=intrinsics, width=640, height=400, timeless=True)
    viewer.log_rigid3("CAM_C/transform", child_from_parent=([0, 0, 0], [1, 0, 0, 0]), xyz="RDF", timeless=True)

    while oak.running():
        packets = q.get()
        depth = packets['depth']
        viewer.log_image("Right", packets['rr'].frame)

        color = packets['nn'].frame
        color = cv2.pyrDown(color) # Downscale to match 640x400 depth frame

        viewer.log_image("CAM_C/transform/Color/Image", color[:, :, ::-1]) # BGR to RGB
        viewer.log_depth_image("CAM_C/transform/Color/Depth", depth.frame, meter=1e3)

        oak.poll()

Note that on the first run, depthai-sdk will download the tractor-oak-d-lr recording, which is about 300MB:

Let us know your thoughts in the comments!

Hi oscarthorn ,
RVC2 can do stereo on max 1280 pixel width frames, and AR0234 (sensors in OAK-D LR) are 1920 pixel in width. So you can either downscale by 2/3, or crop (however you like, I assume central crop is the most relevant).
So by center cropping you lose 33% of the FOV, but you'd achieve better long-range accuracy, as you have "more pixels" of the distant objects. You could use formulas from config stereo depth do determine the accuracy change that'd make (cropping vs downscaling).
Thanks, Erik