Adapt quickly to changing light conditions at low frame rate.

WouterOddBot

Hi,

We're using your Oak-D pro camera's at a 0.5 Hz frame rate. It only produces depth images. Does a camera adapt slower to changing light conditions when it's working at a lower frame rate? And if so, what is the recommended way to make the camera adapt quicker at a low frame rate?

We're still using depthai v2.

Thanks in advance,

Wouter

jakaskerl

WouterOddBot
3A speed is set to the same FPS as the sensor. It can't be faster, only slower (if you set it via 3AFps setting). If you need fast changes, set the FPS to something like 10, but discard all but every 5th frame.

Thanks,
Jaka

WouterOddBot

Ok we made something like:

from datetime import timedelta

from depthai import CameraBoardSocket
from depthai import Device
from depthai import ImgFrame
from depthai import Pipeline
from depthai.node import MessageDemux
from depthai.node import Sync


SYNC_STREAM_LEFT = "left"
SYNC_STREAM_RIGHT = "right"

DOWNSAMPLE_SCRIPT_STREAM_IN = "DOWNSAMPLE_IN"
DOWNSAMPLE_SCRIPT_STREAM_OUT = "DOWNSAMPLE_OUT"

RIGHT_STREAM_NAME = "RIGHT"
DEPTH_STREAM_NAME = "DEPTH"


def create_downsample_script(stream_in: str, stream_out: str, modulo: int) -> str:
    return f"""
    stream_in = node.io['{stream_in}']
    stream_out = node.io['{stream_out}']

    i = 0
    while True:
        frame = stream_in.get()
        i += 1
        i %= {modulo}
        if i == 0:
            stream_out.send(frame)
    """


frame_rate = 1.0
pipeline = Pipeline()
left = pipeline.createMonoCamera()
left.setBoardSocket(CameraBoardSocket.LEFT)
left.setFps(10.0 * frame_rate)
right = pipeline.createMonoCamera()
right.setBoardSocket(CameraBoardSocket.RIGHT)
right.setFps(10.0 * frame_rate)
sync = pipeline.create(Sync)
sync.setSyncThreshold(timedelta(milliseconds=50))
downsample_script = pipeline.createScript()
downsample_script.setScript(create_downsample_script(DOWNSAMPLE_SCRIPT_STREAM_IN, DOWNSAMPLE_SCRIPT_STREAM_OUT, 10))
demux = pipeline.create(MessageDemux)
depth = pipeline.createStereoDepth()
right_out = pipeline.createXLinkOut()
right_out.setStreamName(RIGHT_STREAM_NAME)
depth_out = pipeline.createXLinkOut()
depth_out.setStreamName(DEPTH_STREAM_NAME)

left.out.link(sync.inputs[SYNC_STREAM_LEFT])
right.out.link(sync.inputs[SYNC_STREAM_RIGHT])
sync.out.link(downsample_script.inputs[DOWNSAMPLE_SCRIPT_STREAM_IN])
downsample_script.outputs[DOWNSAMPLE_SCRIPT_STREAM_OUT].link(demux.input)
demux.outputs[SYNC_STREAM_LEFT].link(depth.left)
demux.outputs[SYNC_STREAM_RIGHT].link(depth.right)
depth.depth.link(depth_out.input)
depth.rectifiedRight.link(right_out.input)


def get_right_image(frame: ImgFrame):
    print(f"right image {frame.getTimestamp()}")


def get_depth_image(frame: ImgFrame):
    print(f"depth image {frame.getTimestamp()}")


with Device(pipeline) as device:
    right_queue = device.getOutputQueue(name=RIGHT_STREAM_NAME, maxSize=1, blocking=False)
    depth_queue = device.getOutputQueue(name=DEPTH_STREAM_NAME, maxSize=1, blocking=False)
    right_queue.addCallback(get_right_image)
    depth_queue.addCallback(get_depth_image)
    while True:
        pass

It works, but is this also the recommended way?

WouterOddBot

With the above pipeline we actually get a crash dump every once in a while. For some reason I'm not allowed to attach the crash dump json file. And it's too big to copy it into this post. But perhaps these screenshots may help:

and the bottom part:

We used the downsample script for a long time now in another camera setup and there it works fine.