Hello, I was looking for some help or guidance with a setup. We need to create a stereo system where camera B is located several meters away from camera A and rotated 90 degrees to track an object in a contained 3d space. The original thought was to use the OAK-FFC 4P with two OAK-FFC OV9782 M12 (22 pin) cameras attached to the board but the longest available flat camera cables are unable to reach the required distance. Perhaps there are longer ones available? Regardless, the next thought was to use two OAK-FFC 4P boards with one camera attached to each which would then allow us to connect both to a single computer through USB-C. This introduces two questions: Syncing Frames Following the instructions [here](https://docs.luxonis.com/projects/api/en/latest/tutorials/message_syncing/#software-syncing), we used software syncing with timestamps but this introduces a fair bit of latency which we would like to keep low. My understanding is that sequence syncing would not be possible across the FFC boards. This leaves hardware syncing as the next option. How would one go about implementing hardware syncing between two FFC boards (if possible without the port available on other camera modules). Calibrating Cameras The documentation provides a calibration script which I works for a stereo setup on a single FFC board. Does it work, or can it be modified to accommodate cameras on two separate boards?

Setup Help/Syncing and Calibrating two OAK-FFC 4P

kcancio

Hello, I was looking for some help or guidance with a setup. We need to create a stereo system where camera B is located several meters away from camera A and rotated 90 degrees to track an object in a contained 3d space. The original thought was to use the OAK-FFC 4P with two OAK-FFC OV9782 M12 (22 pin) cameras attached to the board but the longest available flat camera cables are unable to reach the required distance. Perhaps there are longer ones available?

Regardless, the next thought was to use two OAK-FFC 4P boards with one camera attached to each which would then allow us to connect both to a single computer through USB-C. This introduces two questions:

Syncing Frames
Following the instructions here, we used software syncing with timestamps but this introduces a fair bit of latency which we would like to keep low. My understanding is that sequence syncing would not be possible across the FFC boards. This leaves hardware syncing as the next option. How would one go about implementing hardware syncing between two FFC boards (if possible without the port available on other camera modules).

Calibrating Cameras
The documentation provides a calibration script which I works for a stereo setup on a single FFC board. Does it work, or can it be modified to accommodate cameras on two separate boards?

jakaskerl

kcancio Perhaps there are longer ones available?

FFC cables have maximum length of about 50cm.

You should be able to hardware sync two ffc-4ps together.
As far as I understand:

Connecting the FSIN1 from one device to the other, then setting one device to have all inputs (link to what i mean)
and the other (master) to one camera as output (has to be OV9282/9782 or something that can drive the fsync signal) should allow the master to control when the slave takes frames.

kcancio The documentation provides a calibration script which I works for a stereo setup on a single FFC board. Does it work, or can it be modified to accommodate cameras on two separate boards?

No, that is single device only. For multiple devices you are on your own. The disparity matching will have to be done on host side (using opencv). You will need camera intrinsics and device extrinsics to use that.

Thanks,
Jaka

Nearpoint

jakaskerl @erik

I have two OAK-FFC-4P boards and each has two IMX477 color cameras connected to CAM_B and CAM_C. I am using two OAKS because I am limited by the bandwidth of USB running at 60fps.

I believe CAM_B and CAM_C are already hardware synced and I believe this is true even if the IMX477 color cameras I am using are the arducam ones in which the FSIN line does not reach the OAK since it uses the 22pin to 26pin cable. I also believe this hold true even though the IMX477 can not generate a frame sync output signal, only can receive it.

So if the above is true and the IMX477 arducam sensor without FSIN line connected to the left and right ports of the OAK-FFC-4P module is indeed hardware synced, then connected both the OAK-FFC-4P modules should then sync all four of the cameras. And the FrameSyncMode for all four imx477 sensors would be set to INPUT mode only since the OAK-FFC-4P itself would be responsible for driving all the cameras together since the imx477 is not capable of frame sync OUTPUT.

So if all the above is true my real question is can you please clarify how to hardware sync dual OAK-FFC-4P devices. As far as I read it is by connecting the FSIN1 from one device to the other. How exactly is that done, I do not understand. Thank you!

kcancio

Thanks this was really helpful!

Elusive

I also have the same problem, that is, because of frame rate requirements, need to choose two OAK-FFC-4P for four camera control, each board to install two cameras, currently facing the biggest problem is how to synchronise the cameras on these two boards to achieve the effect of low latency?

erik

@Nearpoint

I believe CAM_B and CAM_C are already hardware synced

They share same I2C bus, so when chip sends "start frame exposure", both of them start at the same time.

As IMX477 doesn't provide FSYNC signal by itself, you'd need to either drive it externally or via GPIO (not sure if it's precise enough, would need to test), then connect all 4 IMX477 sensors togeter.

erik

@Elusive

how to synchronise the cameras on these two boards to achieve the effect of low latency?

If TIME_DIFF < (0.5/FPS) is good enough for you (in other words, if frame diff between one cam and another is <16ms at 30fps), then you can use timestamp syncing. Otherwise you'd need to use hardware syncing. As FFC-OV9782-M12 doesn't passthrough FSYNC line, you'd need to solder it manually (connect all sensors).

Elusive

I'm currently using an IMX577 camera with the resolution set to 1080p, and I'm asking if and how much the frame rate will drop if I use timestamps for software synchronisation. Also, for the OAK-FFC-4P board, out of the USB cable to the computer, are there any other cables to the computer that will increase the frame rate?

jakaskerl

Hi @Elusive
Not frames should be dropped unless TIME_DIFF < (0.5/FPS).

Elusive Also, for the OAK-FFC-4P board, out of the USB cable to the computer, are there any other cables to the computer that will increase the frame rate?

Usually the bottleneck comes from the device itself, not the USB cable. Only improvement you will achieve with different cable (USB 3 gen2) is higher bandwidth which will increase FPS, if your current requirements exceed the bandwidth of the cable you are using (making cable the bottleneck), which is unlikely.

Thanks,
Jaka

Elusive

jakaskerl

I'd like to ask if the equipment mentioned here refers to computers? I was triggered to ask a question by seeing this reply of yours, maybe I didn't understand correctly what you expressed, would you please explain again how I can increase the frame rate. When I use the OAK-FFC-4P to connect 4 IMX577 cameras with the resolution set to 1080P, and connect it to my computer via USB 3.0, I would like to ask what is the theoretical frame rate that can be reached?

jakaskerl

Elusive
I wrote this before I realized there is a bunch of overhead when running the USB protocol. That's why I thought the USB controller on your side is not capable of handling the speeds.

luxonis/depthai-experimentstree/e0bd127018f375847bec5b3e8a434f842796796e/random-scripts#oak-bandwidth-test here check this, we made some tests on 10Gbps connection. Max speed we were able to achieve for SUPER_PLUS:

Downlink 3362.2 mbps
Uplink 2729.7 mbps

This means you would effectively be able to send data at that speed (calculations). Sorry for the confusion.

Thanks,
Jaka

Elusive

I would like to ask what is the theoretical maximum frame rate an OAK-FFC-4P driving 4 imx577's can achieve at a resolution of 1080P?

erik

Hi @Elusive ,
One bottleneck would be ISP; max 500megapixels/sec. 1080P is about 2MP, so about 250FPS, or 60FPS @ 4 cams.
Other bottleneck could be device CPU, which does the 3A computation (docs here).
And another one would be bandwidth (docs here). 1080P frames (vide/isp output) would be about 25Mbits, so at 3gbps that would be.. 120fps in total, or 30fps per camera. You could use on-device videoEncoder to encode frames to reduce bandwidth usage.
So my estimation would be between 30 and 60 fps.

Elusive

But my 4 cameras only work at a maximum of 20fps, I'm running it on cam_test_modify_user.py, how can I still make improvements? Could you please be more specific about how to encode the frames?