Hi! I'm trying to hardware sync 2 x OAK-FFC OV9282, 1 x OAK-FFC IMX378 in a OAK-FFC 4P board. I use latest DepthAi version v2.21.2, bootloader on device is on 0.24, latest. Cables between devices are ok, and all devices works ok under depthai_demo. I take as reference, [hardware sync](https://docs.luxonis.com/projects/hardware/en/latest/pages/guides/sync_frames.html) from documentation and a [post](https://discuss.luxonis.com/d/934-ffc-4p-hardware-synchronization/3) on this forum. This is my code in c++ : ``` #include "depthai/depthai.hpp" #include int main(int argc, char* argv[]) { dai::Pipeline pipeline; auto odispa = pipeline.create(); auto ocolor = pipeline.create(); ocolor->setStreamName("rgb"); odispa->setStreamName("disparity"); auto color = pipeline.create(); color->setInterleaved(true); color->setResolution(dai::ColorCameraProperties::SensorResolution::THE_1080_P); color->setFps(30); color->setBoardSocket(dai::CameraBoardSocket::RGB); color->setColorOrder( dai::ColorCameraProperties::ColorOrder::RGB); color->initialControl.setFrameSyncMode( dai::CameraControl::FrameSyncMode::OUTPUT ); auto left = pipeline.create(); left->setResolution(dai::MonoCameraProperties::SensorResolution::THE_720_P); left->setBoardSocket(dai::CameraBoardSocket::LEFT); left->setFps(30); left->initialControl.setFrameSyncMode(dai::CameraControl::FrameSyncMode::INPUT); auto right = pipeline.create(); right->setResolution(dai::MonoCameraProperties::SensorResolution::THE_720_P); right->setBoardSocket(dai::CameraBoardSocket::RIGHT); right->setFps(30); right->initialControl.setFrameSyncMode(dai::CameraControl::FrameSyncMode::INPUT); auto stereo = pipeline.create(); stereo->initialConfig.setConfidenceThreshold(254); stereo->initialConfig.setMedianFilter(static_cast(0)); if(false){ stereo->initialConfig.setBilateralFilterSigma(65000); } stereo->setRectifyEdgeFillColor(0); stereo->setLeftRightCheck(true); stereo->setExtendedDisparity(false); stereo->setSubpixel(true); stereo->setDepthAlign(static_cast(0)); left->out.link(stereo->left); right->out.link(stereo->right); stereo->disparity.link(odispa->input); color->isp.link(ocolor->input); pipeline.getDeviceConfig().board.gpio[6] = dai::BoardConfig::GPIO(dai::BoardConfig::GPIO::Direction::OUTPUT, dai::BoardConfig::GPIO::Level::HIGH); dai::Device device(pipeline); device.setLogLevel(dai::LogLevel::DEBUG); device.setLogOutputLevel(dai::LogLevel::DEBUG); auto dispq = device.getOutputQueue("disparity",1, false); auto rgbq = device.getOutputQueue("rgb",1, false); while (true) { try{ auto dispf = dispq->get(); auto rgbf = rgbq->get(); if(dispf && rgbf){ int64_t cft = rgbf->getTimestampDevice().time_since_epoch().count(); int64_t dft = dispf->getTimestampDevice().time_since_epoch().count(); std::cout initialControl.setFrameSyncMode(dai::CameraControl::FrameSyncMode::INPUT); color->initialControl.setFrameSyncMode(dai::CameraControl::FrameSyncMode::OUTPUT); ``` ``` DIFF: 425 DIFF: 418 DIFF: 406 DIFF: 404 DIFF: 407 DIFF: 407 DIFF: 67063 DIFF: 133719 [19443010F117E01200] [1.2.3] [2.309] [system] [info] Memory Usage - DDR: 75.98 / 340.34 MiB, CMX: 2.11 / 2.50 MiB, LeonOS Heap: 59.82 / 77.23 MiB, LeonRT Heap: 5.01 / 41.14 MiB [19443010F117E01200] [1.2.3] [2.309] [system] [info] Temperatures - Average: 43.96C, CSS: 45.32C, MSS 43.73C, UPA: 43.27C, DSS: 43.50C [19443010F117E01200] [1.2.3] [2.309] [system] [info] Cpu Usage - LeonOS 59.31%, LeonRT: 23.73% DIFF: 167044 DIFF: 200368 DIFF: 267023 DIFF: 300349 DIFF: 333677 DIFF: 366998 DIFF: 400326 DIFF: 433651 DIFF: 466985 DIFF: 500303 DIFF: 533625 DIFF: 566953 DIFF: 533617 DIFF: 533616 ``` but as you can see frame timestamp difference reach 600ms and no sync at all. I try to reduce fps but no luck, other important thing is this line: ``` pipeline.getDeviceConfig().board.gpio[6] = dai::BoardConfig::GPIO(dai::BoardConfig::GPIO::Direction::OUTPUT, dai::BoardConfig::GPIO::Level::HIGH); dai::Device device(pipeline); ``` I try to change order, but nothing happends. Can anyone point me? Thanks

Hi @"thebambuproject"#p8344 , It doesn't look like you are syncing frames on the host side by timestamps - could you try that as well?

I’m trying hardware sync because software sync its very problematic on long term running and high frame rate. I discuss about it a lot at discord. For me it’s not a valid aproach as far as I test it.

Well, going a little bit further seems that set input/output its not usefull and not work properly. if you **not set this value** for all sensors, works under a ms in this case us: ``` [18443010C14DCC0F00] [1.2.3] [2.806] [system] [info] Memory Usage - DDR: 47.76 / 340.34 MiB, CMX: 2.10 / 2.50 MiB, LeonOS Heap: 48.29 / 77.23 MiB, LeonRT Heap: 4.54 / 41.14 MiB [18443010C14DCC0F00] [1.2.3] [2.806] [system] [info] Temperatures - Average: 41.09C, CSS: 41.66C, MSS 40.97C, UPA: 40.97C, DSS: 40.74C [18443010C14DCC0F00] [1.2.3] [2.806] [system] [info] Cpu Usage - LeonOS 32.00%, LeonRT: 11.33% DIFF: -24 DIFF: -24 DIFF: -25 DIFF: -24 DIFF: -24 DIFF: -25 DIFF: -24 DIFF: -25 DIFF: -24 ``` Well any sensor must be master, i suspect master in this case is one Global shutter L or R, if i set **RGB as INPUT, no works** at all. Same behaviour if i set **L & R to INPUT and RGB to OUTPUT**. I tested it on 3 different OAKs and sensors, Its not a hardware problem. Seems that depth sync is protected anywhere in your code. **Setting INPUT/OUPUT seems broken at least in this version or not match with your documentation and spected behaviours**. Hardware sync its very important for custom boards.

Hi @"thebambuproject"#p8376 , I am not saying that you should do software syncing, but even when all cameras are HW synced, they arrive to the host at different times, due to eg. longer exposure times, takes longer for transfer, etc. That's why even if you HW sync, you still need to make sure that you are comparing timestamps of the same seq. number. Thoughts?

(especially when you are also running stereo pipeline - that can take quite some time before it's sent to the host)

Sure! Hope that I undestand you ok, sorry if not. The code posted is a reduced sample. Seems, as far as i tested, if FPS are not high near limits ( RGB 29 -30, i use 15 for security in this case), hardware sync works as expected and frames came back host well aligned. This is why i have small differences on device timestamps. I wrote this minimal example to remark the unexpected behaviour of INPUT/OUTPUT.

@"thebambuproject"#p8394 could you still share the minimal repro example, but that it's still synced on host (by seq. numbers or timestamps)?

@"erik"#p8396 Sorry for delay but problem exists, and its a random problem . I use script on host to keep streams synced, only 2 streams, in random points its output this: ``` [184430108140BE0F00] [1.2.2] [11018.015] [Script(0)] [info] Synced frame! [184430108140BE0F00] [1.2.2] [11018.016] [Script(0)] [info] disp, ts: 3:03:37.966405, seq 110174 [184430108140BE0F00] [1.2.2] [11018.017] [Script(0)] [info] rgb, ts: 3:03:37.966417, seq 110174 [184430108140BE0F00] [1.2.2] [11018.087] [Script(0)] [info] rgb, ts: 3:03:38.066399 [184430108140BE0F00] [1.2.2] [11018.115] [Script(0)] [info] disp, ts: 3:03:38.066386 [184430108140BE0F00] [1.2.2] [11018.115] [Script(0)] [info] Synced frame! [184430108140BE0F00] [1.2.2] [11018.116] [Script(0)] [info] disp, ts: 3:03:38.066386, seq 110175 [184430108140BE0F00] [1.2.2] [11018.117] [Script(0)] [info] rgb, ts: 3:03:38.066399, seq 110175 [184430108140BE0F00] [1.2.2] [11018.187] [Script(0)] [info] rgb, ts: 3:03:38.166344 [184430108140BE0F00] [1.2.2] [11018.215] [Script(0)] [info] disp, ts: 3:03:38.166377 STORED 56265 [85 ms ] [184430108140BE0F00] [1.2.2] [11018.215] [Script(0)] [info] Synced frame! [184430108140BE0F00] [1.2.2] [11018.216] [Script(0)] [info] disp, ts: 3:03:38.166377, seq 110176 [184430108140BE0F00] [1.2.2] [11018.217] [Script(0)] [info] rgb, ts: 3:03:38.166344, seq 110176 [184430108140BE0F00] [1.2.2] [11018.287] [Script(0)] [info] rgb, ts: 3:03:38.266330 [184430108140BE0F00] [1.2.2] [11018.315] [Script(0)] [info] disp, ts: 3:03:38.266361 [184430108140BE0F00] [1.2.2] [11018.315] [Script(0)] [info] Synced frame! [184430108140BE0F00] [1.2.2] [11018.316] [Script(0)] [info] disp, ts: 3:03:38.266361, seq 110177 [184430108140BE0F00] [1.2.2] [11018.317] [Script(0)] [info] rgb, ts: 3:03:38.266330, seq 110177 [184430108140BE0F00] [1.2.2] [11018.387] [Script(0)] [info] rgb, ts: 3:03:38.366301 [184430108140BE0F00] [1.2.2] [11018.415] [Script(0)] [info] disp, ts: 3:03:38.366332 STORED 56266 [87 ms ] [184430108140BE0F00] [1.2.2] [11018.415] [Script(0)] [info] Synced frame! [184430108140BE0F00] [1.2.2] [11018.416] [Script(0)] [info] disp, ts: 3:03:38.366332, seq 110178 [184430108140BE0F00] [1.2.2] [11018.417] [Script(0)] [info] rgb, ts: 3:03:38.366301, seq 110178 [184430108140BE0F00] [1.2.2] [11018.490] [Script(0)] [info] rgb, ts: 3:03:38.466287 [184430108140BE0F00] [1.2.2] [11018.518] [Script(0)] [info] disp, ts: 3:03:38.466319 [184430108140BE0F00] [1.2.2] [11018.518] [Script(0)] [info] Synced frame! [184430108140BE0F00] [1.2.2] [11018.519] [Script(0)] [info] disp, ts: 3:03:38.466319, seq 110179 [184430108140BE0F00] [1.2.2] [11018.520] [Script(0)] [info] rgb, ts: 3:03:38.466287, seq 110179 [184430108140BE0F00] [1.2.2] [11018.587] [Script(0)] [info] rgb, ts: 3:03:38.566280 [184430108140BE0F00] [1.2.2] [11018.615] [Script(0)] [info] disp, ts: 3:03:38.566311 [184430108140BE0F00] [1.2.2] [11018.615] [Script(0)] [info] Synced frame! [184430108140BE0F00] [1.2.2] [11018.616] [Script(0)] [info] disp, ts: 3:03:38.566311, seq 110180 [184430108140BE0F00] [1.2.2] [11018.617] [Script(0)] [info] rgb, ts: 3:03:38.566280, seq 110180 STORED 56267 [90 ms ] [184430108140BE0F00] [1.2.2] [11018.690] [Script(0)] [info] rgb, ts: 3:03:38.666266 [184430108140BE0F00] [1.2.2] [11018.787] [Script(0)] [info] rgb, ts: 3:03:38.766270 [184430108140BE0F00] [1.2.2] [11018.858] [system] [info] Memory Usage - DDR: 59.63 / 340.34 MiB, CMX: 2.10 / 2.50 MiB, LeonOS Heap: 48.30 / 77.23 MiB, LeonRT Heap: 7.58 / 41.14 MiB [184430108140BE0F00] [1.2.2] [11018.858] [system] [info] Temperatures - Average: 67.01C, CSS: 67.88C, MSS 67.06C, UPA: 66.86C, DSS: 66.24C [184430108140BE0F00] [1.2.2] [11018.858] [system] [info] Cpu Usage - LeonOS 24.35%, LeonRT: 18.94% [184430108140BE0F00] [1.2.2] [11019.859] [system] [info] Memory Usage - DDR: 59.63 / 340.34 MiB, CMX: 2.10 / 2.50 MiB, LeonOS Heap: 48.30 / 77.23 MiB, LeonRT Heap: 7.58 / 41.14 MiB [184430108140BE0F00] [1.2.2] [11019.859] [system] [info] Temperatures - Average: 66.65C, CSS: 67.88C, MSS 65.63C, UPA: 67.06C, DSS: 66.04C [184430108140BE0F00] [1.2.2] [11019.859] [system] [info] Cpu Usage - LeonOS 6.55%, LeonRT: 8.14% [184430108140BE0F00] [1.2.2] [11020.860] [system] [info] Memory Usage - DDR: 59.63 / 340.34 MiB, CMX: 2.10 / 2.50 MiB, LeonOS Heap: 48.30 / 77.23 MiB, LeonRT Heap: 7.58 / 41.14 MiB [184430108140BE0F00] [1.2.2] [11020.860] [system] [info] Temperatures - Average: 66.14C, CSS: 67.27C, MSS 65.83C, UPA: 65.83C, DSS: 65.63C ``` As you can see at any point seems that disparity stream fails to get frame, and then no more sync its available Thanks

Hi @"thebambuproject"#p11402 , It seems FPS is not the same for RGB and for disparity, otherwise timestamps/syncing looks fine. What's the FPS on mono cameras - is it the same as on color camera? Are you running postprocessing filters on stereo? As those can take a lot of time, and reduce stereo FPS.

As far as i know, disparity fps can only be controled with left-right cameras fps? All cameras are setted with same fps 10, no postfilters. depth aligned with RGB and left-rigth check enabled and subpixel.

Hardware sync problems

erik

Hi thebambuproject ,
Are you sure you are applying the GPIO configuration correctly? It's a bit different than what's documented here.

Here's also schematics of R4:

And schematics of R7, where it looks like FSIN_MODE_SELECT is set to HIGH by default (see the 1V8 line):

Could you try to set gpio6 to HIGH using documented way or perhaps via Script node (as that's executed after firmware does its default GPIO configuration)?
Thanks, Erik

thebambuproject

As you can see in other coments i try to apply it that way:

pipeline.getDeviceConfig().board.gpio[6] = dai::BoardConfig::GPIO(dai::BoardConfig::GPIO::Direction::OUTPUT, dai::BoardConfig::GPIO::Level::HIGH);

I script node are you talking about something like this:

script.setScript(f"""
        import GPIO
        MX_PIN = 6
        GPIO.setup(MX_PIN, GPIO.OUT, GPIO.PULL_DOWN)
        GPIO.write(MX_PIN, GPIO.HIGH)
""");

« Previous Page