Hi! I'm trying to hardware sync 2 x OAK-FFC OV9282, 1 x OAK-FFC IMX378 in a OAK-FFC 4P board. I use latest DepthAi version v2.21.2, bootloader on device is on 0.24, latest. Cables between devices are ok, and all devices works ok under depthai_demo. I take as reference, [hardware sync](https://docs.luxonis.com/projects/hardware/en/latest/pages/guides/sync_frames.html) from documentation and a [post](https://discuss.luxonis.com/d/934-ffc-4p-hardware-synchronization/3) on this forum. This is my code in c++ : ``` #include "depthai/depthai.hpp" #include int main(int argc, char* argv[]) { dai::Pipeline pipeline; auto odispa = pipeline.create(); auto ocolor = pipeline.create(); ocolor->setStreamName("rgb"); odispa->setStreamName("disparity"); auto color = pipeline.create(); color->setInterleaved(true); color->setResolution(dai::ColorCameraProperties::SensorResolution::THE_1080_P); color->setFps(30); color->setBoardSocket(dai::CameraBoardSocket::RGB); color->setColorOrder( dai::ColorCameraProperties::ColorOrder::RGB); color->initialControl.setFrameSyncMode( dai::CameraControl::FrameSyncMode::OUTPUT ); auto left = pipeline.create(); left->setResolution(dai::MonoCameraProperties::SensorResolution::THE_720_P); left->setBoardSocket(dai::CameraBoardSocket::LEFT); left->setFps(30); left->initialControl.setFrameSyncMode(dai::CameraControl::FrameSyncMode::INPUT); auto right = pipeline.create(); right->setResolution(dai::MonoCameraProperties::SensorResolution::THE_720_P); right->setBoardSocket(dai::CameraBoardSocket::RIGHT); right->setFps(30); right->initialControl.setFrameSyncMode(dai::CameraControl::FrameSyncMode::INPUT); auto stereo = pipeline.create(); stereo->initialConfig.setConfidenceThreshold(254); stereo->initialConfig.setMedianFilter(static_cast(0)); if(false){ stereo->initialConfig.setBilateralFilterSigma(65000); } stereo->setRectifyEdgeFillColor(0); stereo->setLeftRightCheck(true); stereo->setExtendedDisparity(false); stereo->setSubpixel(true); stereo->setDepthAlign(static_cast(0)); left->out.link(stereo->left); right->out.link(stereo->right); stereo->disparity.link(odispa->input); color->isp.link(ocolor->input); pipeline.getDeviceConfig().board.gpio[6] = dai::BoardConfig::GPIO(dai::BoardConfig::GPIO::Direction::OUTPUT, dai::BoardConfig::GPIO::Level::HIGH); dai::Device device(pipeline); device.setLogLevel(dai::LogLevel::DEBUG); device.setLogOutputLevel(dai::LogLevel::DEBUG); auto dispq = device.getOutputQueue("disparity",1, false); auto rgbq = device.getOutputQueue("rgb",1, false); while (true) { try{ auto dispf = dispq->get(); auto rgbf = rgbq->get(); if(dispf && rgbf){ int64_t cft = rgbf->getTimestampDevice().time_since_epoch().count(); int64_t dft = dispf->getTimestampDevice().time_since_epoch().count(); std::cout initialControl.setFrameSyncMode(dai::CameraControl::FrameSyncMode::INPUT); color->initialControl.setFrameSyncMode(dai::CameraControl::FrameSyncMode::OUTPUT); ``` ``` DIFF: 425 DIFF: 418 DIFF: 406 DIFF: 404 DIFF: 407 DIFF: 407 DIFF: 67063 DIFF: 133719 [19443010F117E01200] [1.2.3] [2.309] [system] [info] Memory Usage - DDR: 75.98 / 340.34 MiB, CMX: 2.11 / 2.50 MiB, LeonOS Heap: 59.82 / 77.23 MiB, LeonRT Heap: 5.01 / 41.14 MiB [19443010F117E01200] [1.2.3] [2.309] [system] [info] Temperatures - Average: 43.96C, CSS: 45.32C, MSS 43.73C, UPA: 43.27C, DSS: 43.50C [19443010F117E01200] [1.2.3] [2.309] [system] [info] Cpu Usage - LeonOS 59.31%, LeonRT: 23.73% DIFF: 167044 DIFF: 200368 DIFF: 267023 DIFF: 300349 DIFF: 333677 DIFF: 366998 DIFF: 400326 DIFF: 433651 DIFF: 466985 DIFF: 500303 DIFF: 533625 DIFF: 566953 DIFF: 533617 DIFF: 533616 ``` but as you can see frame timestamp difference reach 600ms and no sync at all. I try to reduce fps but no luck, other important thing is this line: ``` pipeline.getDeviceConfig().board.gpio[6] = dai::BoardConfig::GPIO(dai::BoardConfig::GPIO::Direction::OUTPUT, dai::BoardConfig::GPIO::Level::HIGH); dai::Device device(pipeline); ``` I try to change order, but nothing happends. Can anyone point me? Thanks

Hi @"thebambuproject"#p8344 , It doesn't look like you are syncing frames on the host side by timestamps - could you try that as well?

Hardware sync problems

thebambuproject

I’m trying hardware sync because software sync its very problematic on long term running and high frame rate. I discuss about it a lot at discord. For me it’s not a valid aproach as far as I test it.

thebambuproject

Well, going a little bit further seems that set input/output its not usefull and not work properly.

if you not set this value for all sensors, works under a ms in this case us:

[18443010C14DCC0F00] [1.2.3] [2.806] [system] [info] Memory Usage - DDR: 47.76 / 340.34 MiB, CMX: 2.10 / 2.50 MiB, LeonOS Heap: 48.29 / 77.23 MiB, LeonRT Heap: 4.54 / 41.14 MiB
[18443010C14DCC0F00] [1.2.3] [2.806] [system] [info] Temperatures - Average: 41.09C, CSS: 41.66C, MSS 40.97C, UPA: 40.97C, DSS: 40.74C
[18443010C14DCC0F00] [1.2.3] [2.806] [system] [info] Cpu Usage - LeonOS 32.00%, LeonRT: 11.33%
DIFF: -24
DIFF: -24
DIFF: -25
DIFF: -24
DIFF: -24
DIFF: -25
DIFF: -24
DIFF: -25
DIFF: -24

Well any sensor must be master, i suspect master in this case is one Global shutter L or R, if i set RGB as INPUT, no works at all. Same behaviour if i set L & R to INPUT and RGB to OUTPUT. I tested it on 3 different OAKs and sensors, Its not a hardware problem. Seems that depth sync is protected anywhere in your code.

Setting INPUT/OUPUT seems broken at least in this version or not match with your documentation and spected behaviours. Hardware sync its very important for custom boards.

erik

Hi thebambuproject ,
I am not saying that you should do software syncing, but even when all cameras are HW synced, they arrive to the host at different times, due to eg. longer exposure times, takes longer for transfer, etc. That's why even if you HW sync, you still need to make sure that you are comparing timestamps of the same seq. number. Thoughts?

erik

(especially when you are also running stereo pipeline - that can take quite some time before it's sent to the host)

thebambuproject

Sure! Hope that I undestand you ok, sorry if not. The code posted is a reduced sample. Seems, as far as i tested, if FPS are not high near limits ( RGB 29 -30, i use 15 for security in this case), hardware sync works as expected and frames came back host well aligned. This is why i have small differences on device timestamps. I wrote this minimal example to remark the unexpected behaviour of INPUT/OUTPUT.

erik

thebambuproject could you still share the minimal repro example, but that it's still synced on host (by seq. numbers or timestamps)?

thebambuproject

erik

Sorry for delay but problem exists, and its a random problem .

I use script on host to keep streams synced, only 2 streams, in random points its output this:

[184430108140BE0F00] [1.2.2] [11018.015] [Script(0)] [info] Synced frame!
[184430108140BE0F00] [1.2.2] [11018.016] [Script(0)] [info] disp, ts: 3:03:37.966405, seq 110174
[184430108140BE0F00] [1.2.2] [11018.017] [Script(0)] [info] rgb, ts: 3:03:37.966417, seq 110174
[184430108140BE0F00] [1.2.2] [11018.087] [Script(0)] [info] rgb, ts: 3:03:38.066399
[184430108140BE0F00] [1.2.2] [11018.115] [Script(0)] [info] disp, ts: 3:03:38.066386
[184430108140BE0F00] [1.2.2] [11018.115] [Script(0)] [info] Synced frame!
[184430108140BE0F00] [1.2.2] [11018.116] [Script(0)] [info] disp, ts: 3:03:38.066386, seq 110175
[184430108140BE0F00] [1.2.2] [11018.117] [Script(0)] [info] rgb, ts: 3:03:38.066399, seq 110175
[184430108140BE0F00] [1.2.2] [11018.187] [Script(0)] [info] rgb, ts: 3:03:38.166344
[184430108140BE0F00] [1.2.2] [11018.215] [Script(0)] [info] disp, ts: 3:03:38.166377
 STORED  56265 [85 ms ]
[184430108140BE0F00] [1.2.2] [11018.215] [Script(0)] [info] Synced frame!
[184430108140BE0F00] [1.2.2] [11018.216] [Script(0)] [info] disp, ts: 3:03:38.166377, seq 110176
[184430108140BE0F00] [1.2.2] [11018.217] [Script(0)] [info] rgb, ts: 3:03:38.166344, seq 110176
[184430108140BE0F00] [1.2.2] [11018.287] [Script(0)] [info] rgb, ts: 3:03:38.266330
[184430108140BE0F00] [1.2.2] [11018.315] [Script(0)] [info] disp, ts: 3:03:38.266361
[184430108140BE0F00] [1.2.2] [11018.315] [Script(0)] [info] Synced frame!
[184430108140BE0F00] [1.2.2] [11018.316] [Script(0)] [info] disp, ts: 3:03:38.266361, seq 110177
[184430108140BE0F00] [1.2.2] [11018.317] [Script(0)] [info] rgb, ts: 3:03:38.266330, seq 110177
[184430108140BE0F00] [1.2.2] [11018.387] [Script(0)] [info] rgb, ts: 3:03:38.366301
[184430108140BE0F00] [1.2.2] [11018.415] [Script(0)] [info] disp, ts: 3:03:38.366332
 STORED  56266 [87 ms ]
[184430108140BE0F00] [1.2.2] [11018.415] [Script(0)] [info] Synced frame!
[184430108140BE0F00] [1.2.2] [11018.416] [Script(0)] [info] disp, ts: 3:03:38.366332, seq 110178
[184430108140BE0F00] [1.2.2] [11018.417] [Script(0)] [info] rgb, ts: 3:03:38.366301, seq 110178
[184430108140BE0F00] [1.2.2] [11018.490] [Script(0)] [info] rgb, ts: 3:03:38.466287
[184430108140BE0F00] [1.2.2] [11018.518] [Script(0)] [info] disp, ts: 3:03:38.466319
[184430108140BE0F00] [1.2.2] [11018.518] [Script(0)] [info] Synced frame!
[184430108140BE0F00] [1.2.2] [11018.519] [Script(0)] [info] disp, ts: 3:03:38.466319, seq 110179
[184430108140BE0F00] [1.2.2] [11018.520] [Script(0)] [info] rgb, ts: 3:03:38.466287, seq 110179
[184430108140BE0F00] [1.2.2] [11018.587] [Script(0)] [info] rgb, ts: 3:03:38.566280
[184430108140BE0F00] [1.2.2] [11018.615] [Script(0)] [info] disp, ts: 3:03:38.566311
[184430108140BE0F00] [1.2.2] [11018.615] [Script(0)] [info] Synced frame!
[184430108140BE0F00] [1.2.2] [11018.616] [Script(0)] [info] disp, ts: 3:03:38.566311, seq 110180
[184430108140BE0F00] [1.2.2] [11018.617] [Script(0)] [info] rgb, ts: 3:03:38.566280, seq 110180
 STORED  56267 [90 ms ]
[184430108140BE0F00] [1.2.2] [11018.690] [Script(0)] [info] rgb, ts: 3:03:38.666266
[184430108140BE0F00] [1.2.2] [11018.787] [Script(0)] [info] rgb, ts: 3:03:38.766270
[184430108140BE0F00] [1.2.2] [11018.858] [system] [info] Memory Usage - DDR: 59.63 / 340.34 MiB, CMX: 2.10 / 2.50 MiB, LeonOS Heap: 48.30 / 77.23 MiB, LeonRT Heap: 7.58 / 41.14 MiB
[184430108140BE0F00] [1.2.2] [11018.858] [system] [info] Temperatures - Average: 67.01C, CSS: 67.88C, MSS 67.06C, UPA: 66.86C, DSS: 66.24C
[184430108140BE0F00] [1.2.2] [11018.858] [system] [info] Cpu Usage - LeonOS 24.35%, LeonRT: 18.94%
[184430108140BE0F00] [1.2.2] [11019.859] [system] [info] Memory Usage - DDR: 59.63 / 340.34 MiB, CMX: 2.10 / 2.50 MiB, LeonOS Heap: 48.30 / 77.23 MiB, LeonRT Heap: 7.58 / 41.14 MiB
[184430108140BE0F00] [1.2.2] [11019.859] [system] [info] Temperatures - Average: 66.65C, CSS: 67.88C, MSS 65.63C, UPA: 67.06C, DSS: 66.04C
[184430108140BE0F00] [1.2.2] [11019.859] [system] [info] Cpu Usage - LeonOS 6.55%, LeonRT: 8.14%
[184430108140BE0F00] [1.2.2] [11020.860] [system] [info] Memory Usage - DDR: 59.63 / 340.34 MiB, CMX: 2.10 / 2.50 MiB, LeonOS Heap: 48.30 / 77.23 MiB, LeonRT Heap: 7.58 / 41.14 MiB
[184430108140BE0F00] [1.2.2] [11020.860] [system] [info] Temperatures - Average: 66.14C, CSS: 67.27C, MSS 65.83C, UPA: 65.83C, DSS: 65.63C

As you can see at any point seems that disparity stream fails to get frame, and then no more sync its available

Thanks

erik

Hi thebambuproject ,
It seems FPS is not the same for RGB and for disparity, otherwise timestamps/syncing looks fine. What's the FPS on mono cameras - is it the same as on color camera? Are you running postprocessing filters on stereo? As those can take a lot of time, and reduce stereo FPS.

thebambuproject

erik , near a month later, I go a little far with this… I think this is a disaster, an unstable solution…

Lets take an optimal solution for frame syncing:

https://github.com/luxonis/depthai-core/blob/main/examples/mixed/frame_sync.cpp

First of all, if you simply change in this example to 4K resolution, example dies, well… I suppose FPS are high and 4K images are heavy, camRgb->setVideoNumFramesPool(10), its big for 4K images. Lets try with 5 on pool and 20 on FPS. Stops in less than 10 seconds… Stoping means that frames not arriving to python script and no sync...

Disparity aligned frames with 4K seems heavier than Color ones.

[19443010E156DF1200] [1.2.2] [2.669] [Script(4)] [debug] RECV STREAM: rgb TS: 0:00:02.470649 SEQ:2
[19443010E156DF1200] [1.2.2] [2.670] [Script(4)] [debug]  SYNC S:disp TS: 0:00:02.471298 SEQ:2
[19443010E156DF1200] [1.2.2] [2.672] [Script(4)] [debug]  SYNC S:rgb TS: 0:00:02.470649 SEQ:2
[19443010E156DF1200] [1.2.2] [2.679] [Script(4)] [debug] RECV STREAM: rgb TS: 0:00:02.537083 SEQ:3
[19443010E156DF1200] [1.2.2] [2.684] [Script(4)] [debug] RECV STREAM: rgb TS: 0:00:02.603561 SEQ:4

RECV is when i recieve frame on host, into sync python script, SYNC is where there is a match.

As you can see rgb frames quick start misaligning.

Lets try 10 frames, 5 on pool. it start on relative good sync, but wait…

[19443010E156DF1200] [1.2.2] [30.427] [Script(4)] [debug] RECV STREAM: disp TS: 0:00:30.058952 SEQ:277
[19443010E156DF1200] [1.2.2] [30.428] [Script(4)] [debug]  SYNC S:disp TS: 0:00:30.058952 SEQ:277
[19443010E156DF1200] [1.2.2] [30.429] [Script(4)] [debug]  SYNC S:rgb TS: 0:00:30.057370 SEQ:277
[19443010E156DF1200] [1.2.2] [30.461] [Script(4)] [debug] RECV STREAM: rgb TS: 0:00:30.257445 SEQ:279
[19443010E156DF1200] [1.2.2] [30.470] [Script(4)] [debug] RECV STREAM: disp TS: 0:00:30.158988 SEQ:278
[19443010E156DF1200] [1.2.2] [30.471] [Script(4)] [debug]  SYNC S:disp TS: 0:00:30.158988 SEQ:278
[19443010E156DF1200] [1.2.2] [30.473] [Script(4)] [debug]  SYNC S:rgb TS: 0:00:30.157021 SEQ:278
[19443010E156DF1200] [1.2.2] [30.575] [Script(4)] [debug] RECV STREAM: rgb TS: 0:00:30.357096 SEQ:280
[19443010E156DF1200] [1.2.2] [30.580] [Script(4)] [debug] RECV STREAM: disp TS: 0:00:30.259023 SEQ:279
[19443010E156DF1200] [1.2.2] [30.581] [Script(4)] [debug]  SYNC S:disp TS: 0:00:30.259023 SEQ:279
[19443010E156DF1200] [1.2.2] [30.583] [Script(4)] [debug]  SYNC S:rgb TS: 0:00:30.257445 SEQ:279
[19443010E156DF1200] [1.2.2] [30.685] [Script(4)] [debug] RECV STREAM: rgb TS: 0:00:30.457522 SEQ:281

In some cases, streams start to missalign between, sync didnt find candidates, then cameras on trace log reports that frames continue arriving, but script didnt sync anything, an nothing comes to it. Cameras remains in a unknow state, not closed state.

[19443010E156DF1200] [1.2.2] [2.608] [Script(4)] [debug] RECV STREAM: disp TS: 0:00:02.346922 SEQ:0
[19443010E156DF1200] [1.2.2] [2.609] [Script(4)] [debug] RECV STREAM: rgb TS: 0:00:02.415929 SEQ:0
[19443010E156DF1200] [1.2.2] [2.611] [Script(4)] [debug] RECV STREAM: disp TS: 0:00:02.446951 SEQ:1
[19443010E156DF1200] [1.2.2] [2.612] [Script(4)] [debug] RECV STREAM: rgb TS: 0:00:02.515579 SEQ:1
[19443010E156DF1200] [1.2.2] [2.615] [Script(4)] [debug] RECV STREAM: disp TS: 0:00:02.546984 SEQ:2
[19443010E156DF1200] [1.2.2] [2.662] [Script(4)] [debug] RECV STREAM: rgb TS: 0:00:02.615229 SEQ:2
[19443010E156DF1200] [1.2.2] [2.762] [Script(4)] [debug] RECV STREAM: rgb TS: 0:00:02.713880 SEQ:3
[19443010E156DF1200] [1.2.2] [2.816] [system] [info] Memory Usage - DDR: 157.02 / 340.34 MiB, CMX: 2.47 / 2.50 MiB, LeonOS Heap: 46.78 / 77.23 MiB, LeonRT Heap: 8.03 / 41.14 MiB
[19443010E156DF1200] [1.2.2] [2.816] [system] [info] Temperatures - Average: 54.97C, CSS: 55.51C, MSS 54.64C, UPA: 55.07C, DSS: 54.64C
[19443010E156DF1200] [1.2.2] [2.816] [system] [info] Cpu Usage - LeonOS 21.90%, LeonRT: 76.51%
[19443010E156DF1200] [1.2.2] [2.861] [Script(4)] [debug] RECV STREAM: rgb TS: 0:00:02.813531 SEQ:4
[19443010E156DF1200] [1.2.2] [3.817] [system] [info] Memory Usage - DDR: 157.02 / 340.34 MiB, CMX: 2.47 / 2.50 MiB, LeonOS Heap: 46.78 / 77.23 MiB, LeonRT Heap: 8.03 / 41.14 MiB
[19443010E156DF1200] [1.2.2] [3.817] [system] [info] Temperatures - Average: 54.26C, CSS: 54.86C, MSS 53.33C, UPA: 54.64C, DSS: 54.21C
[19443010E156DF1200] [1.2.2] [3.817] [system] [info] Cpu Usage - LeonOS 10.14%, LeonRT: 9.38%

DepthAi have not a Reset pipeline or device reset, you must reinit pipeline, after this error let device unclosed, next time to try open pipeline it generates a XLINK_NO_DEVICE and on the next try it starts working.

As a resume, you may try lower FPS till something tolerable, but cameras at any point start to missaligned and stop, this is an erratic behaviour, hard to find, and hard to atack in a production device robust and ready to work for hours…

Can you give me and advise to make this more stable? Im using later API version and Firmware. Its a way to reset camera? Its a way to know when camera dies, when syncing stops...

As a reference with problem

luxonis/depthai238

thebambuproject

As far as i know, disparity fps can only be controled with left-right cameras fps? All cameras are setted with same fps 10, no postfilters. depth aligned with RGB and left-rigth check enabled and subpixel.

erik

Hi thebambuproject ,
With a bit of debugging (node.warn inside script), you can quickly see that Script node doesn't receive all frames (sequence numbers start to skip), then list size increases to 9, which is also the pool size (10), so camera can't create new frames (no RAM, Script node "hogs" frames in the list), and pipeline freezes:

[warning] Added disp to queue, seq 13, list size 2
[warning] Added rgb to queue, seq 39, list size 9
[warning] Added disp to queue, seq 14, list size 3
[info] Synced frame!
[info] disp, ts: 0:00:01.683729, seq 14
[info] rgb, ts: 0:00:01.683191, seq 14
[warning] Added disp to queue, seq 15, list size 1
[warning] Added disp to queue, seq 18, list size 2
[warning] Added rgb to queue, seq 42, list size 9
[warning] Added disp to queue, seq 23, list size 3
[info] Synced frame!
[info] disp, ts: 0:00:01.983891, seq 23
[info] rgb, ts: 0:00:01.983184, seq 23
[warning] Added rgb to queue, seq 45, list size 7
[warning] Added disp to queue, seq 25, list size 1
[warning] Added disp to queue, seq 28, list size 2
[warning] Added rgb to queue, seq 49, list size 8
[warning] Added rgb to queue, seq 50, list size 9
[warning] Added disp to queue, seq 31, list size 3
[info] Memory Usage - DDR: 216.35 / 334.18 MiB, CMX: 2.47 / 2.50 MiB, LeonOS Heap: 46.77 / 82.86 MiB, LeonRT Heap: 7.77 / 40.93 MiB
[info] Temperatures - Average: 34.22C, CSS: 34.88C, MSS 33.93C, UPA: 34.16C, DSS: 33.93C
[info] Cpu Usage - LeonOS 34.32%, LeonRT: 18.16%
[info] Memory Usage - DDR: 216.35 / 334.18 MiB, CMX: 2.47 / 2.50 MiB, LeonOS Heap: 46.77 / 82.86 MiB, LeonRT Heap: 7.77 / 40.93 MiB
[info] Temperatures - Average: 33.98C, CSS: 35.35C, MSS 33.45C, UPA: 33.93C, DSS: 33.21C
[info] Cpu Usage - LeonOS 22.01%, LeonRT: 7.92%
# After this you just get info readings of the device

Pipeline here:

In such case, where there are sequence number skippings, I would advise against using sequence numbers for frame syncing, but instead use timestamps, and some conservative threshold value. But ideally you wouldn't do on-device syncing, as that takes a bunch of RAM, as you need to save many previous messages.
Thoughts?
Thanks, Erik

thebambuproject

erik I take as reference your own code! Your approach to sync frames available as reference in your samples. As far as I know, script uses Timestamps not sequences in order to sync.
Its any sample to sync frames on host c++, something stable and reliable? An what is the best aproach? declare a queue with, i.e 10 frames (getOutputQueue) and query it? use only last ones (getOutputQueue 1)?

Thanks!

erik

Hi thebambuproject ,
I'll address in points:

My bad, you are right, in pipeline above we are using timestamps to sync frames. In that case, it might be required to increase the MS_THRESHOL variable, as some frames are skipping.
Host timestamp syncing example would be a good starting point, and you can use GPT4 to directly convert python to c++, as depthai API are 1:1.
On host, queue size/blocking behavior shouldn't matter, as you would/should be reading frames from it fast enough, as otherwise, you will have even more lost frames.

thebambuproject

Ok thanks @erik! More info. Ive got two versions of boards R4M1E4 and R7M1E7. In R7 revision FSYNC seems that camera works as expected, frames in order, no buffering problems... setting this:

color->initialControl.setFrameSyncMode( dai::CameraControl::FrameSyncMode::OUTPUT )
left->initialControl.setFrameSyncMode(dai::CameraControl::FrameSyncMode::INPUT);
right->initialControl.setFrameSyncMode(dai::CameraControl::FrameSyncMode::INPUT);

pipeline.getDeviceConfig().board.gpio[6] = dai::BoardConfig::GPIO(dai::BoardConfig::GPIO::Direction::OUTPUT, dai::BoardConfig::GPIO::Level::HIGH);

Works in R7 but in R4 frames didn't arrive to queue! We are full of R4 boards (+50 boards of R4 revision). Its any hardware change?
How can i solve this?

jakaskerl

Hi thebambuproject
In previous revisions (5 and lower) the cam_a and cam_d were hard-synced together and so were the cam_b and cam_c. But the pairs were not synced to each other, though you could achieve synchronization by flipping on a switch to create a connection between them (the code above).
In revision 7, this was changed so a, b, c and d are now hard-synced together out of the box.

thebambuproject

Thanks @jakaskerl

But i can`t syncing R4 with posted sync code, not works. There is no way to sync color and mono sensors on this version!

erik

Hi thebambuproject ,
You are correct, we don't do hardware syncing between global+rolling shutter sensors, we are doing soft-syncing between color (rolling shutter) and stereo pair (global shutter, which are hardware synced). Please see docs on supported FSYNC triggering here:
https://docs.luxonis.com/projects/hardware/en/latest/pages/guides/sync_frames/#sensor-fsync-support

That's why I suggested above to use timestamp syncing between stereo pair and color sensors.
Thanks, Erik

thebambuproject

Sorry @erik but this is not a solution, looks like a patch. Timestamps start to derive as soon is conected, in low FPS configuration, this problem with buffers take more time but the end its the same, unmatched frames, in host, in camera or wherever you do it. Please there are any solution for R4?

erik

Hi thebambuproject ,
Are you sure you are applying the GPIO configuration correctly? It's a bit different than what's documented here.

Here's also schematics of R4:

And schematics of R7, where it looks like FSIN_MODE_SELECT is set to HIGH by default (see the 1V8 line):

Could you try to set gpio6 to HIGH using documented way or perhaps via Script node (as that's executed after firmware does its default GPIO configuration)?
Thanks, Erik

thebambuproject

As you can see in other coments i try to apply it that way:

pipeline.getDeviceConfig().board.gpio[6] = dai::BoardConfig::GPIO(dai::BoardConfig::GPIO::Direction::OUTPUT, dai::BoardConfig::GPIO::Level::HIGH);

I script node are you talking about something like this:

script.setScript(f"""
        import GPIO
        MX_PIN = 6
        GPIO.setup(MX_PIN, GPIO.OUT, GPIO.PULL_DOWN)
        GPIO.write(MX_PIN, GPIO.HIGH)
""");