FSYNC ERROR

thebambuproject

Hi!

I´ve got a OAK-FFP-4P

1 IMX386 on CAM-A (Color)
1 OV9282 on CAM-B (Left Mono)
1 OV9282 on CAM-C (Right Mono)

When i try to do this:

camA.initialControl.setFrameSyncMode(dai.CameraControl.FrameSyncMode.INPUT)
camB.initialControl.setFrameSyncMode(dai.CameraControl.FrameSyncMode.OUTPUT)
camC.initialControl.setFrameSyncMode(dai.CameraControl.FrameSyncMode.INPUT)

No frames arrives at host, same with camC, If i let intact works.

Of course, GPIO 6 open.

jakaskerl

Hi thebambuproject
Could you add a MRE code?
Also is IMX386 correct? https://docs.luxonis.com/projects/hardware/en/latest/pages/articles/supported_sensors/#already-built-ccms

Thanks,
Jaka

thebambuproject

Here is the MRE: https://github.com/codesian/depthai/tree/main

Thanks jakaskerl

thebambuproject

Sorry its IMX-378-FF!

jakaskerl

Hi thebambuproject
That's not a "minimal" reproducible example. You have a bunch of functions that might be the actual cause of the problem. I suggest you only keep the minimal -> remove anything that isn't colorcamera, monocamera and Xlinkouts for the pipeline side. Just to make sure you are actually not receiving any frames.

Thanks,
Jaka

thebambuproject

Sorry, this bunch of functions are syncing script an one that does nothing, now code is cleaned, sorry. My output its this:


[18443010C175CB0F00] [1.2.1] [2.428] [StereoDepth(5)] [info] Depth baseline: '11.09005' cm, focal length: '802.50854'
[18443010C175CB0F00] [1.2.1] [2.428] [StereoDepth(5)] [info] Depth horizontal FOV: '63.276093', vertical FOV: '38.274208'
[18443010C175CB0F00] [1.2.1] [2.428] [StereoDepth(5)] [info] Depth intrinsics: 
3116.277,    0.000, 1861.253, 
   0.000, 3112.334, 1068.435, 
   0.000,    0.000,    1.000, 
[18443010C175CB0F00] [1.2.1] [2.474] [XLinkOut(0)] [info] Increasing 'disp' stream size from 5242880B to 8345600B
[18443010C175CB0F00] [1.2.1] [3.163] [system] [info] Memory Usage - DDR: 118.98 / 334.18 MiB, CMX: 2.10 / 2.50 MiB, LeonOS Heap: 46.77 / 82.79 MiB, LeonRT Heap: 4.63 / 40.87 MiB
[18443010C175CB0F00] [1.2.1] [3.163] [system] [info] Temperatures - Average: 51.91C, CSS: 53.55C, MSS 51.80C, UPA: 50.47C, DSS: 51.80C
[18443010C175CB0F00] [1.2.1] [3.163] [system] [info] Cpu Usage - LeonOS 21.63%, LeonRT: 9.43%
[18443010C175CB0F00] [1.2.1] [4.164] [system] [info] Memory Usage - DDR: 118.98 / 334.18 MiB, CMX: 2.10 / 2.50 MiB, LeonOS Heap: 46.77 / 82.79 MiB, LeonRT Heap: 4.63 / 40.87 MiB
[18443010C175CB0F00] [1.2.1] [4.164] [system] [info] Temperatures - Average: 52.29C, CSS: 53.12C, MSS 52.46C, UPA: 52.02C, DSS: 51.58C
[18443010C175CB0F00] [1.2.1] [4.164] [system] [info] Cpu Usage - LeonOS 13.26%, LeonRT: 2.88%
[18443010C175CB0F00] [1.2.1] [5.165] [system] [info] Memory Usage - DDR: 118.98 / 334.18 MiB, CMX: 2.10 / 2.50 MiB, LeonOS Heap: 46.77 / 82.79 MiB, LeonRT Heap: 4.63 / 40.87 MiB
[18443010C175CB0F00] [1.2.1] [5.165] [system] [info] Temperatures - Average: 52.40C, CSS: 53.55C, MSS 52.68C, UPA: 52.02C, DSS: 51.36C
[18443010C175CB0F00] [1.2.1] [5.165] [system] [info] Cpu Usage - LeonOS 13.40%, LeonRT: 2.88%

Camera firmware is on 0.26 and DepthaAI core version 2.23.0

erik

Hi thebambuproject ,
Which revision of the OAK-FFC-4P do you have? I have just updated docs here, and we have a similar debate at https://discuss.luxonis.com/d/2303-please-share-the-status-of-hw-sync-for-oak-ffc-4p-ar0234-based-stereo-pair/4 (see schematics of R6 or newer revision of the board).

thebambuproject

thanks Erik!

We are talking about R4M1E4, seems on R7 sync, and setFrameSyncMode works. But we have more than 100 R4 modules. Documentation seems same to me... Only GPIO6...

thebambuproject

You should check, sync on Depthai core. Have no much sense... In your docs seems that IMX386 can be only input . I try this combinations and sorry but this is a chaos, of course GPIO pin HIGH;

RGB: OUTPUT LEFT: INPUT RIGHT: INPUT RESULT: NOT WORK
RGB: INPUT LEFT: OUTPUT RIGHT: INPUT RESULT: NOT WORK
RGB: INPUT LEFT: INPUT RIGHT: OUTPUT RESULT: NOT WORK

Bizarrous combinations:

RGB: NONE LEFT: NONE RIGHT: NONE RESULT: SEEMS TO WORKS
RGB: OUTPUT LEFT: INPUT RIGHT: OUTPUT RESULT: SEEMS TO WORKS (frames on same sequence timestamp diference goes from -100ms to 0ms then to 100ms and then stabilizes at 2 -3 ms diference)
RGB: OUTPUT LEFT: OUTPUT RIGHT: INPUT RESULT: SEEMS TO WORKS Same as previous

Any conclusion, seems that GPIO does nothing, tests have same results, with GPIO HIGH, LOW or whatever...

Thanks

erik

cc @Luxonis-Alex - is this as expected? IIRC we do soft syncing between imx378 and ov9282.

Luxonis-Alex

@thebambuproject Sorry for delay. I checked again with DepthAI 2.23.0, and FSYNC should work, with one of LEFT or RIGHT OV9282 set as OUTPUT. IMX378 doesn't have the output capability for now.

I noticed few issues with your code, regarding the GPIO configuration. Pushed the changes on a fork here:
alex-luxonis/tmp-fsync-testae681eb

DeviceConfig/BoardConfig to be set before creating the Device object (as it's done as early init, before the pipeline is loaded. Attempting to set GPIO6 didn't have effect before
GPIO41 from RVC2 SoC to not be touched, or configured as input (default). It is interconnected to camera FSYNC/FSIN/XVS signals, and if driven (high or low), would create bus contention, with higher drive strength from SoC side which would cancel out the sensor-generated signal. Would likely result in no frames from IMX378, but OV9282 would still stream at configured FPS even in sync INPUT mode with missing signal (sensor particularity). For the given OAK-FFC-4P hardware revision (up to R5, inclusive), only GPIO6 needs to be driven high to interconnect A+D and B+C sync lines on baseboard.

Relevant changes:

// Important for device/board config to be tweaked before sending the pipeline to Device constructor.
// Also in case the pipeline is sent separately (startPipeline after Device creation), then config must be passed to Device constructor instead
dai::Device::Config c = pipeline.getDeviceConfig();
c.board.gpio[6] = dai::BoardConfig::GPIO(dai::BoardConfig::GPIO::Direction::OUTPUT, dai::BoardConfig::GPIO::Level::HIGH);
// c.board.gpio[41] = dai::BoardConfig::GPIO(dai::BoardConfig::GPIO::Direction::INPUT, dai::BoardConfig::GPIO::Level::LOW);
pipeline.setBoardConfig(c.board);
dai::Device device(pipeline);

When hardware sync works, the capture timestamps should get very close together (less than 0.2ms between OV9282 and IMX378). Sequence numbers may be different, we'll try to fix that in FW. But timestamps should be good (and that's also what StereoDepth uses internally to sync left and right). With the extra code to print timestamps, I'm getting:

DISP 0 ts-offset: 4.110202 device_ts: 2.493684
RGB  0 ts-offset: 4.265498 device_ts: 2.648979
DISP 1 ts-offset: 4.235205 device_ts: 2.618687

RGB  1 ts-offset: 4.390498 device_ts: 2.773979
DISP 3 ts-offset: 4.390339 device_ts: 2.773820

RGB  2 ts-offset: 4.515498 device_ts: 2.898979
DISP 4 ts-offset: 4.515337 device_ts: 2.898818

RGB  3 ts-offset: 4.640498 device_ts: 3.023979
DISP 5 ts-offset: 4.640338 device_ts: 3.023819

RGB  4 ts-offset: 4.765498 device_ts: 3.148979
DISP 6 ts-offset: 4.765336 device_ts: 3.148817

RGB  5 ts-offset: 4.890498 device_ts: 3.273979
DISP 7 ts-offset: 4.890337 device_ts: 3.273818

RGB  6 ts-offset: 5.015497 device_ts: 3.398979
DISP 8 ts-offset: 5.015336 device_ts: 3.398817

RGB  7 ts-offset: 5.140498 device_ts: 3.523979
DISP 9 ts-offset: 5.140336 device_ts: 3.523817

RGB  8 ts-offset: 5.265497 device_ts: 3.648979
DISP 10 ts-offset: 5.265336 device_ts: 3.648818

RGB  9 ts-offset: 5.390497 device_ts: 3.773979
DISP 11 ts-offset: 5.390335 device_ts: 3.773816

RGB  10 ts-offset: 5.515497 device_ts: 3.898979
DISP 12 ts-offset: 5.515337 device_ts: 3.898818

RGB  11 ts-offset: 5.640497 device_ts: 4.023979
DISP 13 ts-offset: 5.640336 device_ts: 4.023817

If things still don't work, possible to post a picture with your setup? We'll confirm if the camera modules used are good for sync as-is.

thebambuproject

Thanks Luxonis-Alex for extra work, let me check all. I think it's important for your documentation, make a paragraph on Sync or in samples, detailing in c++ you need to initialize GPIO on this way. I can't find this procedure in c++ examples, documentation... and its totally different in python.