Using F-SYNCH while recording

Clevy

Hello,

I have two OAK-D-PRO-POE cameras which I an recording from and saving the six video streams. We need the recording to be synched with each other below a 33ms variation. I have been able to use the FSYNCH to trigger a start recording, but it is not lining the recordings up. There is a disparity of between 40- 500 ms between the streams coming the two devices. The way the code works, the cameras must disconnect and reconnect after the synch signal is received. Ideally the cameras are synched with a pulse sent through the F-SYNCH, but once the recording has started I am not able to read the synch port anymore. This is the code I am currently running. Is there a way I could record and synch with the F-SYNCH simultaneously?

Code used: synch Code

Thanks!!

jakaskerl

Hi Clevy
Most likely the problem is because of the bootup time of the device.
Maybe try using the SDK's trigger action to achieve recording when triggered. (https://docs.luxonis.com/projects/sdk/en/latest/features/conditional-actions/#triggers)

Alternatively, you can keep track of the time on your host machine and apply software synchronization with timestamps and device time.

Thoughts?
Jaka

Clevy

Thank you for your response, I would like to use timestamp synching, I assume that this should this be implemented at the beginning of the recording to align the start of the videos, then continue monitoring throughout the recording. If so how would the additional frames be discarded?

clevy

erik

Hi Clevy ,
How did you wire up the devices (FSYNC)? Regarding timestamp syncing, you can use this demo to achieve multi-device timestamp syncing: https://github.com/luxonis/depthai-experiments/blob/master/gen2-syncing/host-multiple-OAK-sync.py

Clevy

hello,

That code was helpful to look at and gave me ideas as to how to edit my working code. however, I was not able to get the cameras to synch and record. the recording code cycles through one device at a time and save the frame, using the checkSynch function to verify that all the frames in one device are in synch. I have tried to save all queues to an array and send that to the checkSynch function but I am getting an error I don't entirely understand in this context.

Could you show an example of how the https://github.com/luxonis/depthai-experiments/blob/master/gen2-syncing/host-multiple-OAK-sync.py and https://github.com/luxonis/depthai/blob/main/apps/record/main.py appended together?

Thank you for your support

erik

Hi Clevy ,
The record app should already handle multi-device syncing and recording - does it not work?

Regarding the error, could you please copy the full terminal output?

Thanks, Erik

Clevy

no- it is only synching the cameras on one device, not the two devices.

as you can see from the ms displayed in the VLC, there is a difference between the two video streams.

This is the full terminal:

erik

Hi Clevy ,
You are right, my bad. I'll try to implement something over the weekend, or start of the next week.

Clevy

Hello erik

Thank you for taking this on, could I get an update on the project?

erik

Hi Clevy ,
Sorry about the delay, adding multi-device support to SDK seems a bit more complicated than I initially thought and it will take some time to implement. It would be faster to update record.py's syncing part with (host) timestamp syncing, maybe taking the one implemented here: https://github.com/luxonis/depthai/blob/develop/depthai_sdk/src/depthai_sdk/oak_outputs/syncing.py#L52-L96
Thougths?

Clevy

Hello, it seems that the code there has only capability to synch one device as the 'self' argument, are you recommending adding a second camera argument to that code?

My ideal solution is using an external FSYNCH signal to adjust the frame capture within the 33 ms time window of capture. I would like to use frame synch mode _`<`camera.initialControl.setFrameSyncMode(dai.CameraControl.FrameSyncMode.INPUT)~

There is documentation https://docs.luxonis.com/projects/hardware/en/latest/pages/guides/sync_frames/ which mentions using synch mode to synch multiple devices but I am having difficulties.

Is there example code using this mode to synch two cameras during recording, all I have found is code used to detect a trigger in order to take a snapshot. This is not too helpful im my case because I must record and use FSYNCH to synch frames at the same time.

The FSYNCH code available that I have found uses the older depthai SDK, where a new queue is made and checks if there is an element in the queue. Form what I understand the queue should have an element arriving for each frame, so checking (wait(lambda: oakQueue.has() == True) should always return true while recording. Is there another way to access the information from the FSYNCH?

Thank you!