Auto exposure failure on stereo rig

apirrone

We recently noticed a weird behavior on our stereo rig, which is composed of :

OAK-FFC-4P
2 IMX296 sensors plugged on CAM_B and CAM_C

Sometimes, I can't exactly explain the exact conditions when this happens, one camera becomes very dark and the other either stays the same or becomes very bright.

Normal :

Broken

I can pretty reliably make this behavior happen after some time (a few minutes) by placing my hand on the cameras and removing it, over and over. Below is an accelerated video.

output.mp4

9MB

Once it gets in this state, it won't go back and we have to re launch the pipeline.

Any idea of what's going on ?

Thanks !

jakaskerl

apirrone
Left and right (if connected to B and C) share the same exposure setting, where left drives the right. So it is expected if left is covered (increase in exposure) the right camera will at the same exposure look very bright.

Was not able to repro the issue on OV9282, but it is very strange the AE just breaks. cc @Luxonis-Alex any ideas?

Thanks,
Jaka

apirrone

Hi @jakaskerl @Luxonis-Alex

Any ideas on this issue ?

Thanks !

Luxonis-Alex

Hi @apirrone
Sorry for the delay... This is a strange issue. Unfortunately I don't have 2x IMX296 to try to replicate it.

I noticed in the video that when the camera view in right window is covered, then the exposure/ISO in left starts increasing. Just to confirm, are the cameras "swapped" in your setup (i.e. left is mapped to CAM_C)? It should be no problem if so, just to understand the behavior.

It may be worth experimenting with the new option "3a-follow" available on camera_controls_misc
luxonis/depthai-python986 (we'll also merge this today to develop):

camNodeB.initialControl.setMisc("3a-follow", dai.CameraBoardSocket.CAM_C)
camNodeC.initialControl.setMisc("3a-follow", dai.CameraBoardSocket.CAM_C)

(by default it would be the opposite, CAM_B 3A driving CAM_C as well)

or each camera's exposure to be set by its own ISP statistics:

camNodeB.initialControl.setMisc("3a-follow", dai.CameraBoardSocket.CAM_B)
camNodeC.initialControl.setMisc("3a-follow", dai.CameraBoardSocket.CAM_C)

however when shared-I2C ports B and C are used, independent exposures cannot be set, the exposure set on CAM_C is applied to both (and exposure set to CAM_B is ignored).

apirrone

Hi @Luxonis-Alex

Yes, "left" is on "CAM_C" and "right" is on "CAM_B" in our setup (it just makes more sense for us because of the way we mount the FFC-4P on our rig).

So with this 3a-follow option, I can choose which socket drives the exposure right ? I can try to swap it and see if we observe the same behavior again.

Thanks !

Luxonis-Alex

Sounds good!

Yes, it can be selected which socket drives the exposure, or the independent option but unfortunately that doesn't work for same cameras on B and C, due to I2C bus sharing. Still the independent option could be tried for this issue.

Unfortunately there were no Python prebuilts for the latest commit on camera_controls_misc, but I just merged it to develop, build progress: luxonis/depthai-pythonactions/runs/10994759427
Anyway, at any time python3 examples/install_requirements.py can build from sources.

apirrone

Hi @Luxonis-Alex

We tried using the other camera to drive the auto exposure, but it didn't fix the issue. It basically didn't change anything.

Any other ideas ?

Thanks

Luxonis-Alex

Hi @apirrone , I'll need to get a second camera to replicate this, currently have a single RPi GS module. Will post when I have new findings.

FabPoll

Hi Luxonis-Alex,
Any news on this? (I work with Antoine.) This is a major issue for us. It happens very frequently, although it is hard to trigger voluntarily. However, it seems that rolling back to depthai v2.24.0 fixes the issue (this is the version we used some time ago when we had no problems). For sure, depthai v2.27.0 has the problem. Hopefully, this will help you narrow down the issue.