We recently noticed a weird behavior on our stereo rig, which is composed of :

  • OAK-FFC-4P
  • 2 IMX296 sensors plugged on CAM_B and CAM_C

Sometimes, I can't exactly explain the exact conditions when this happens, one camera becomes very dark and the other either stays the same or becomes very bright.

Normal :

Broken

I can pretty reliably make this behavior happen after some time (a few minutes) by placing my hand on the cameras and removing it, over and over. Below is an accelerated video.

output.mp4
9MB

Once it gets in this state, it won't go back and we have to re launch the pipeline.

Any idea of what's going on ?

Thanks !

    apirrone
    Left and right (if connected to B and C) share the same exposure setting, where left drives the right. So it is expected if left is covered (increase in exposure) the right camera will at the same exposure look very bright.

    Was not able to repro the issue on OV9282, but it is very strange the AE just breaks. cc @Luxonis-Alex any ideas?

    Thanks,
    Jaka

      13 days later

      Hi @apirrone
      Sorry for the delay... This is a strange issue. Unfortunately I don't have 2x IMX296 to try to replicate it.

      I noticed in the video that when the camera view in right window is covered, then the exposure/ISO in left starts increasing. Just to confirm, are the cameras "swapped" in your setup (i.e. left is mapped to CAM_C)? It should be no problem if so, just to understand the behavior.

      It may be worth experimenting with the new option "3a-follow" available on camera_controls_misc
      luxonis/depthai-python986 (we'll also merge this today to develop):

      camNodeB.initialControl.setMisc("3a-follow", dai.CameraBoardSocket.CAM_C)
camNodeC.initialControl.setMisc("3a-follow", dai.CameraBoardSocket.CAM_C)

      (by default it would be the opposite, CAM_B 3A driving CAM_C as well)

      or each camera's exposure to be set by its own ISP statistics:

      camNodeB.initialControl.setMisc("3a-follow", dai.CameraBoardSocket.CAM_B)
camNodeC.initialControl.setMisc("3a-follow", dai.CameraBoardSocket.CAM_C)

      however when shared-I2C ports B and C are used, independent exposures cannot be set, the exposure set on CAM_C is applied to both (and exposure set to CAM_B is ignored).

      Hi @Luxonis-Alex

      Yes, "left" is on "CAM_C" and "right" is on "CAM_B" in our setup (it just makes more sense for us because of the way we mount the FFC-4P on our rig).

      So with this 3a-follow option, I can choose which socket drives the exposure right ? I can try to swap it and see if we observe the same behavior again.

      Thanks !

      Sounds good!

      Yes, it can be selected which socket drives the exposure, or the independent option but unfortunately that doesn't work for same cameras on B and C, due to I2C bus sharing. Still the independent option could be tried for this issue.

      Unfortunately there were no Python prebuilts for the latest commit on camera_controls_misc, but I just merged it to develop, build progress: luxonis/depthai-pythonactions/runs/10994759427
      Anyway, at any time python3 examples/install_requirements.py can build from sources.

      a month later

      Hi @Luxonis-Alex

      We tried using the other camera to drive the auto exposure, but it didn't fix the issue. It basically didn't change anything.

      Any other ideas ?

      Thanks

      Hi @apirrone , I'll need to get a second camera to replicate this, currently have a single RPi GS module. Will post when I have new findings.

        25 days later

        Hi Luxonis-Alex,
        Any news on this? (I work with Antoine.) This is a major issue for us. It happens very frequently, although it is hard to trigger voluntarily. However, it seems that rolling back to depthai v2.24.0 fixes the issue (this is the version we used some time ago when we had no problems). For sure, depthai v2.27.0 has the problem. Hopefully, this will help you narrow down the issue.

          17 days later

          @Luxonis-Alex @jakaskerl Right now we are kind of stuck, we need an older version of depthai (v2.24.0) to use the imx296 cameras, but a more recent version to operate the ToF sensor properly. Any news on this issue would be very mush appreciated