IMX477 FSIN and FSYNC hardware syncing

Nearpoint

Is my understanding correct that due to a bug in the firmware of the IMX477 sensor it can not produce an FSync Output as I read here:

https://docs.luxonis.com/projects/hardware/en/latest/pages/guides/sync_frames/#hardware-message-syncing

If it didn't have this bug and could produce an FSync Output then you would not need to do this:

Which is to connect all the FSIN pads together for all cameras.

If the bug was fixed and the IMX477 could produce an FSync output then you could synchronize all four imx477 color cameras attached to a OAK-FFC-4P without needed to modify the hardware at all, just using the following code?

I find doing all this soldering and connecting the header pins cumbersome and I really do not want to do it, but I do want to sync the cameras so I will do it if I have too. But if I am correct it just sucks that this is due to a bug. Maybe I could check if the bug has been fixed by now? Would it even be possible to update the firmware on the sensors.

Nearpoint

Actually since the IMX477 can not produce a FSYNC output even connecting the FSIN of all four cameras will not work. The only way is to drive the cameras externally I believe is this correct?

Or if I wanted to just sync 2 color cameras, I could connect a OVA Mono sensor to the 2 lane MIPI on an OAK-FFC-4P and 2 color IMX477 cameras to 4 lane MIPI Cam A and Cam D. Then using the code I can hardware sync the color cameras based on the output of the OVA Mono camera which can produce an Fsync output

Is this correct as well?

Nearpoint

On the back of the arducam sensor board I see four metal pads each labeled with a letters O, F, X, G

is the F the FSIN pin that I would connect each together?

Nearpoint

Ok I am getting some interesting results without doing any hardware modification.

I am using the OAK-FFC-4p and have the OVA mono cams connected to cam_b and cam_c and the imx477 connected to cam_a and cam_d. These were all purchased from the luxonis store and utilize the arducam 796 cable where the fsin and strobe are put as test point on the cable.

Now in the code if I add this:

cam_A.initialControl.setFrameSyncMode(dai.CameraControl.FrameSyncMode.INPUT) # 4LANE
cam_B.initialControl.setFrameSyncMode(dai.CameraControl.FrameSyncMode.OUTPUT) # 2LANE
cam_C.initialControl.setFrameSyncMode(dai.CameraControl.FrameSyncMode.INPUT) # 2LANE
cam_D.initialControl.setFrameSyncMode(dai.CameraControl.FrameSyncMode.INPUT) # 4LANE

# AND importantly to tie the FSIN signals of A+D and B+C pairs, by setting a GPIO:
# OAK-FFC-4P requires driving MXIO6 high (FSIN_MODE_SELECT) to connect together
# the A+D FSIN group (4-lane pair) with the B+C group (2-lane pair)
config = dai.Device.Config()
config.board.gpio[6] = dai.BoardConfig.GPIO(dai.BoardConfig.GPIO.OUTPUT,
                                            dai.BoardConfig.GPIO.Level.HIGH)

with dai.Device(config) as device:
    device.startPipeline(pipeline)

It does not working and just waits for the frame and never gets it. the .get() method on the queue does not deliver any frames. But if I simply comment out the cam_b frame sync output

cam_A.initialControl.setFrameSyncMode(dai.CameraControl.FrameSyncMode.INPUT) # 4LANE
# cam_B.initialControl.setFrameSyncMode(dai.CameraControl.FrameSyncMode.OUTPUT)
cam_C.initialControl.setFrameSyncMode(dai.CameraControl.FrameSyncMode.INPUT) # 2LANE
cam_D.initialControl.setFrameSyncMode(dai.CameraControl.FrameSyncMode.INPUT) # 4LANE

Then the frames are delivered. So I can not set the frame sync mode to OUTPUT even for the OVA cams. This may make sense because that is what was used in the forum post used as an example.

cam_b and cam_c seem to be hardware syncing as there is no noticeable latency. Maybe this default but they are synced even without me adding any frame sync code, they are synced by default.

here is the weird thing. now i have added the frame sync code for all four cameras to my python script but with cam_b commented out and not set to output like so:

cam_A.initialControl.setFrameSyncMode(dai.CameraControl.FrameSyncMode.INPUT) # 4LANE
# cam_B.initialControl.setFrameSyncMode(dai.CameraControl.FrameSyncMode.OUTPUT) # 2LANE
cam_C.initialControl.setFrameSyncMode(dai.CameraControl.FrameSyncMode.INPUT) # 2LANE
cam_D.initialControl.setFrameSyncMode(dai.CameraControl.FrameSyncMode.INPUT) # 4LANE

# AND importantly to tie the FSIN signals of A+D and B+C pairs, by setting a GPIO:
# OAK-FFC-4P requires driving MXIO6 high (FSIN_MODE_SELECT) to connect together
# the A+D FSIN group (4-lane pair) with the B+C group (2-lane pair)
config = dai.Device.Config()
config.board.gpio[6] = dai.BoardConfig.GPIO(dai.BoardConfig.GPIO.OUTPUT,
                                            dai.BoardConfig.GPIO.Level.HIGH)

with dai.Device(config) as device:
    device.startPipeline(pipeline)

And with this in the code and without any hardware modification it seems like the color cams are utilizing hardware syncing too. How can this be possible? Before there was noticeable latency when I just had the 2 color cameras attached to the 4P but when I have the 2 mono cams and the 2 color cams, with the code above (notice no camera is set to output) the cameras including the color cameras are all hardware synced and have no noticeable latency at all. Again there was quite a bit of noticeable latency when I just had two color cams connected to the OAK-FFC-4P even if they were connected to the 4 MIPI lanes or 2 MIPI lanes just having two color cameras did not hardware frame sync. But when I add the OVA mono cam and without in the code setting any camera to output it still seems to be hardware frame synced.

I am puzzled can someone clue me in what is going on here.

jakaskerl

Hi Nearpoint
You shouldn't need any hardware modifications. The fact that 477 are input-only, only means they cannot drive the signal when created inside the pipeline. They are normally driven by the board.
OV mono cameras support both, so you can make sure then when one camera takes an image, it also creates a signal, which can be routed to the input of some other camera, to trigger it at the same time.

I think this works because C-B line is still triggered. By setting the gpio, you are linking both C-B and A-D together which gives the A-D line an external trigger.

Try commenting out CAM_C and put B as input, see if it makes a difference.

Thanks,
Jaka

Nearpoint

jakaskerl

Check this out, so I was able to make the hardware modifications described in the forum post for the OVA camera by attaching the header pins and then connecting the F header pin to the FSIN on the arducam 796 cable. Now for cam_b setting it to output works correctly! and does not stop the program.

But the weird thing is now when I set cam_a or cam_d to INPUT is does not work and the program just halts waiting for the frame to return from the queue.get() operation but it never returns any frames. both cam_a and cam_d are IMX477 sensors and the hardware modification I made to those is similar to the OVA but it was not possible to use header pins in this situation because there were only testpoint on the IMX477. So I had soldered a small wire from the FSIN to the F testpoint on the IMX477.

Is this correct because now setting the frame sync for cam_a or cam_d to INPUTS prevents it from returning frames. But I can set cam_b to OUTPUT and it works if only that line of code is set, but if I uncomment the frame sync lines of code that are set to INPUT the program halts. I only have the 3 cameras attached cam_b and cam_a and cam_d. I set cam_b to output hoping to drive cam_a and cam_d. Now OUTPUT for mono cam_b works but the INPUT for color cam_a and cam_d halts the program and does not return frames.

My best guess is that the F testpoint on the back of the IMX477 is not really the FSIN or I don't know what else could it be? Why would setting it to input work before I made this hardware change to the IMX477 sensors? Before I could set it to INPUT and it would not halt the program but when I soldered the wire from FSIN on the cable to test point F on the circuit board now the IMX477 halts if you set it to frame sync INPUT.

Nearpoint

jakaskerl

but I am using the arducam module, so I thought there was no way to drive the cameras with a fsync signal until you connect the fsin with the fsync on the camera module for the arducam cameras only that use the 796 cable

although i did purchase these from the luxonis store so I am not sure if these particular ones have a modification from the original arducam ones

Nearpoint

I am starting to think the FSIN on the board is X not F 😢

F might be strobe.

That sucks, can anyone verify this? I emailed arducam what the testpoints labels mean as they do not list it anywhere.

Nearpoint

So I changed the solder job to connect FSIN to the X instead of F but still if I use the camera as INPUT is halts but if I set the color camera to OUTPUT it works. this is so puzzling