• DepthAI-v2

  • Automatic Exposure of Multiple Cameras in FFC3

Hi!
I feel like this has been answered somewhere but could not find any information. Could you point me to some resources or help me out with the following:

I have two cams attached to an USB Oak board FFC3. However, the lighting for the two cameras is different. I want both of them to have auto-exposure, but independently from another. In my current implementation, I believe that one cam controls the auto-exposure for both. How would I change that?

class OakBoard_live(Thread):
def __init__(self):
Thread.__init__(self)# Create pipeline
self.pipeline = dai.Pipeline()
self.exception = None# Define source and output
`camRgb2 = self.pipeline.create(dai.node.ColorCamera)
camRgb3 = self.pipeline.create(dai.node.ColorCamera)
imu = self.pipeline.create(dai.node.IMU)
imu.enableIMUSensor(dai.IMUSensor.ACCELEROMETER_RAW, 360)
imu.enableIMUSensor(dai.IMUSensor.ARVR_STABILIZED_ROTATION_VECTOR, 400)
imu.setBatchReportThreshold(10)
imu.setMaxBatchReports(10)

    xoutVideo2 = self.pipeline.create(dai.node.XLinkOut)
    xoutVideo3 = self.pipeline.create(dai.node.XLinkOut)
    imuOut = self.pipeline.create(dai.node.XLinkOut)

    xoutVideo2.setStreamName("video2")
    xoutVideo3.setStreamName("video3")
    imuOut.setStreamName("imu")

    `*`# Properties`*

    `camRgb2.setBoardSocket(dai.CameraBoardSocket.CAM_B)
    camRgb2.setResolution(dai.ColorCameraProperties.SensorResolution.THE_2024X1520)
    camRgb2.setPreviewKeepAspectRatio(False)
    camRgb2.setIspScale((1, 1))
    camRgb3.setBoardSocket(dai.CameraBoardSocket.CAM_C)
    camRgb3.setResolution(dai.ColorCameraProperties.SensorResolution.THE_2024X1520)
    camRgb3.setPreviewKeepAspectRatio(False)
    camRgb3.setIspScale((1, 1))

    xoutVideo2.input.setBlocking(False)
    xoutVideo2.input.setQueueSize(1)
    xoutVideo3.input.setBlocking(False)
    xoutVideo3.input.setQueueSize(1)
    imuOut.input.setQueueSize(1)
    imuOut.input.setBlocking(False)

    `*`# Linking`*
    `camRgb2.video.link(xoutVideo2.input)
    camRgb3.video.link(xoutVideo3.input)
    imu.out.link(imuOut.input)
    self.msg_rgb = None
    self.msg_rgb2 = None
    self.msg_rgb3 = None
    self.msg_imu = None
    self.looping = True
    self.stop = Event()

def run(self):
    self.running = True
    try:
        with dai.Device(self.pipeline) as device:
            `*`#video = device.getOutputQueue(name="video", maxSize=1, blocking=False)`*
            `video2 = device.getOutputQueue(name="video2", maxSize=1, blocking=False)
            video3 = device.getOutputQueue(name="video3", maxSize=1, blocking=False)
            next_msg_imu = device.getOutputQueue('imu', maxSize=1, blocking=False)
            while True:
                if not self.looping:
                    break
                videoIn2 = video2.get()
                videoIn3 = video3.get()
                imunext = next_msg_imu.get()

                self.msg_rgb2 = cv2.resize(videoIn2.getCvFrame(),(1920,1080))
                x, y, w, h = 52, 220, 1920, 1080
                self.msg_rgb3 = videoIn3.getCvFrame()[y:y+h, x:x+w]

                if imunext is not None:
                    self.msg_imu = imunext

    except Exception as e:
        self.exception = e
        self.running = False
        return`
    6 days later

    If I'm showing only one feed at a time, a workaround/compromise would be to only stream one cam at a time. However, I would like to record all the camera streams and changing the cams has some overhead this way..

      Hi BenjaminKiefer
      Only streaming one camera at a time would mean you would have to restart the device each time you wanted to switch which is not a great alternative imo. Could you perhaps connect one of the cameras to cam_a if not already populated?

      I have still not received any response from the team on the potential solutions for your problem.

      Thanks,
      Jaka

        I can use CAM_A for one of the cam, yes. But, that would control the exposure of the other cams then also?

        Thanks!

          BenjaminKiefer
          B and C sockets are hardware-synced, cameras connected to these two sockets are expected to be used for stereo, this is why they are given the same 3a processing, since the stereo matching would not work otherwise. This is not the case for CAM_A as the 4 mipi lane socket is usually intended for rgb camera, but it doesn't really matter if you use it with mono.

          Thanks,
          Jaka

            Ok, as a temporary workaround, I will put the main cam on CAM_A and the less important ones/less frequently used ones on CAM_B and CAM_C.

            Which socket drives the other? CAM_B drives CAM_C? Or What is meant by hardware-synced?

            Thanks!

              a month later

              Hi again,

              I was just wondering if there is any update on this.

              Is there a way to control the exposure for each of the cams individually? I have an oakffc3 USB and an oakffc4poe board. I have an IMX462 and two IMX477 plugged in, but each cam should have an individual exposure.

              Thanks!

              Hi @BenjaminKiefer
              Sadly, no. This would require both HW and FW change on the 3P. Haven't actually received a response from the dev team, assuming it's low prio right now.

              Thanks,
              Jaka