OAK-D Lite Noisy Mono Camera

@jakaskerl I used the Mono Camera Control demo from the documentation, turned the ISO all the way down, and the exposure time to 20500. Looks better, but still seeing noise, especially in the corners, and especially on the left sensor.

Maybe its also just some thermal noise? The thermal pad on my camera does not seem very tacky/sticky, so maybe I just need to replace that to pull heat away from the chip so it does not propagate to the cameras?

No. Not really much to blame on thermal issues. I checked the IC temperature, and the output does seem to line up with what the documentation says I should see. So I'm guessing its just the general result of the OV7251's sensor noise.

I decided to do some experiments though, because why not? I captured lots of frames from both mono cameras with both 100 ISO, and 20500 exposure, and them summed them up to see what "hot spots" were generated. Most frames had no hot spots at all, but some did.

The left camera was a bit "hotter' than the right camera. I'm think that one is closer to the Myriad X, so its probably just a little bit more thermal noise.

Repeating the same test, but with the highest ISO setting gives similar results, but with many more bright pixels. This makes sense, as high ISO is going to be more sensor gain, and thus a nosier picture.

Here's the code for grabbing all the frames. I just manually stopped it around 700-1000 frames

#!/usr/bin/env python3

from pathlib import Path
import cv2
import depthai as dai
import time
import shutil

# Create pipeline
pipeline = dai.Pipeline()


# Define source and output
monoRight = pipeline.create(dai.node.MonoCamera)
monoLeft = pipeline.create(dai.node.MonoCamera)
xoutRight = pipeline.create(dai.node.XLinkOut)
xoutLeft = pipeline.create(dai.node.XLinkOut)

xoutRight.setStreamName("right")
xoutLeft.setStreamName("left")

# Properties
monoRight.setCamera("right")
monoLeft.setCamera("left")
monoRight.setResolution(dai.MonoCameraProperties.SensorResolution.THE_480_P)
monoLeft.setResolution(dai.MonoCameraProperties.SensorResolution.THE_480_P)


# Linking
monoRight.out.link(xoutRight.input)
monoLeft.out.link(xoutLeft.input)


# Image control
controlIn = pipeline.create(dai.node.XLinkIn)
controlIn.setStreamName('control')
controlIn.out.link(monoRight.inputControl)
controlIn.out.link(monoLeft.inputControl)

# Connect to device and start pipeline
with dai.Device(pipeline) as device:

    # Output queue will be used to get the grayscale frames from the output defined above
    qRight = device.getOutputQueue(name="right", maxSize=4, blocking=False)
    qLeft = device.getOutputQueue(name="left", maxSize=4, blocking=False)

    # Queue for sending sensor config
    controlQueue = device.getInputQueue(controlIn.getStreamName())

    dirName_l = "mono_data_l"
    dirName_r = "mono_data_r"
    shutil.rmtree(dirName_l)
    shutil.rmtree(dirName_r)
    Path(dirName_l).mkdir(parents=True, exist_ok=True)
    Path(dirName_r).mkdir(parents=True, exist_ok=True)

    while True:
        ctrl = dai.CameraControl()
        ctrl.setManualExposure(20500, 1600)
        controlQueue.send(ctrl)

        inRight = qRight.get()  # Blocking call, will wait until a new data has arrived
        inLeft = qLeft.get()
        # Data is originally represented as a flat 1D array, it needs to be converted into HxW form
        # Frame is transformed and ready to be shown
        cv2.imshow("right", inRight.getCvFrame())
        cv2.imshow("left", inLeft.getCvFrame())

        # After showing the frame, it's being stored inside a target directory as a PNG image
        cv2.imwrite(f"{dirName_r}/{int(time.time() * 1000)}.png",
                    inRight.getFrame())
        cv2.imwrite(f"{dirName_l}/{int(time.time() * 1000)}.png",
                    inLeft.getFrame())

        if cv2.waitKey(1) == ord('q'):
            break

And here's the script that summed up the data.

from pathlib import Path
import cv2
import numpy as np
from PIL import Image


def stack_dir(directory: str) -> np.array:
    frame = np.zeros((480, 640), dtype=np.float32)
    files = list(Path(directory).glob("*.png"))
    print(f"{directory}: {len(files)}")
    for image in files:
        loaded = np.asarray(Image.open(image), np.float32)
        frame += loaded
    return frame.astype(np.float32)


leftData = stack_dir("mono_data_l")
rightData = stack_dir("mono_data_r")
print(leftData.min(), leftData.max())
print(rightData.min(), rightData.max())
cv2.imshow("left", leftData)
cv2.imshow("right", rightData)
cv2.waitKey()

The noise seems fairly random in how its distributed, so I'm thinking if I really wanted to try it (and frankly, the depth data is pretty good, so this is all somewhat academic at this point), the simplest way might be to average out a few frames (likely using some logic running on the SHAVE cores), then pass that average into the depth calculations. Is that possible, or does the StereoDepth node require the original images?