Amir

For anyone coming to this issue - important note that this is not really a legitimate fix. This tuning blob will crash the camera firmware if you are running the camera with stereo output enabled… it is only usable for color only applications.

Hi @jakaskerl @erik -

Is there any update on this issue? Have you had a chance to discuss internally?

Any guidance here will be very appreciated - there seems to be an issue in the OAKD firmware when the FSYNC signal is being used (and FrameSync is set to input).

Hi @erik ,

Just checking in to see if you heard anything back - I notice there is an identical issue mentioned in this Github issue:
luxonis/depthai-core784

Based on that Github issue and some recent discussions in this forum, and a Discord thread in the #synchronization channel from Nov 2022, I printed out the host timestamps for each packet (rgb, left, right, depth output) for each OAK I am syncing. I noticed the sync pulse is triggering the depth data at twice the sync pulse frequency!! So I modified my setup slightly as follows:
- we have a sync pulse that stays high (12V) and pulses low (0V) for 1ms at a frequency of 5Hz
- set rgb, left, right to FrameSyncMode INPUT as previously described
- the sync pulse is actually triggering the depth twice! so to compensate I set fps as follows (for a 5Hz pulse):
camRgb.setFps(5.0)
left.setFps(2.5)
right.setFps(2.5)

So now I am getting much more reliable sync operation, the timestamps from rgb and depth stream from all of my OAKD cameras are within 1ms of each other… but the mono camera exposure flickering which causes bad depth data still persists! So there is still a very annoying firmware bug where the FSYNC signal is causing the left/right cameras to flicker/reset their exposure times which corrupts the depth data.

See here, with no FSYNC and normal operation, three OAKDs produce good depth data, but the timestamps are not sync'd at all (as expected), so while depth is good here, the unsync'd cameras are not usable for my application:

Here, the FSYNC is syncing the rgb,left,right of each OAK, and the timestamps of all all packets are within 1ms of each other - which is great! But the depth data is now unusable:

Important side note:
Not to distract from the main critical bug, but there is a secondary bug where I setup output queues for left, right, and depth streams while using FSYNC , and the depthai driver (or perhaps OAKD firmware?) crashes after 39 messages received. It is very odd but very consistent behavior. I work around this by only reading rgb and depth stream and not using left/right raw camera streams at the same time as the depth stream.

So back to my main issues:
1) Why does the depth image trigger at twice the FPS of the FSYNC pulse?
2) Why does the FSYNC pulse cause the left/right mono cameras to flicker their exposure and corrupt the depth data?

The Github issue is unanswered, and this forum is full of questions about this topic also unanswered - please provide some insight here. I would love to schedule a call with the firmware team to discuss in more detail if that would be helpful.

Cheers,
-Amir

@erik Following up on this - is there someone on the Firmware team who may have insight into why the FSYNC pulse could be affecting (reseting) the exposure settings on the left/right mono cameras?

We are attempting to synchronize multiple OAK-D Pro W POE cameras with IMX378 sensors.

We are generating an external 'active low' sync pulse at 5Hz (12V high, then pulses down to 0V for 1 ms) and are connecting this signal via the M8 connector to all four of our OAK-D cameras.

The Python test script is very straightforward, we are setting the FSYNC to input:
camRgb.initialControl.setFrameSyncMode(dai.CameraControl.FrameSyncMode.INPUT)
left.initialControl.setFrameSyncMode(dai.CameraControl.FrameSyncMode.INPUT)
right.initialControl.setFrameSyncMode(dai.CameraControl.FrameSyncMode.INPUT)

Annoyingly, in order to get some sort of output, I need to set the FPS to 5.1 (not 5.0 to match the input FSYNC):
camRgb.setFps(5.1)
left.setFps(5.1)
right.setFps(5.1)

So my issue is that this setup … does not work. It looks like the exposure settings on the OAK-D left+right cameras is resetting every pulse. For example, I point the OAKD at a wall/desk:

When I run the OAK in normal mode (no FSYNC) the depth looks as expected:

When I enable FSYNC, the depth never "settles" into place, and constantly resets to this:

It looks like the exposure for left/right mono cameras is constantly being reset, and the depth algorithm can never settle down and get good results.

Please advise !!!

I am having a very similar problem with OAK-D PRO POE - can someone please provide some guidance?