I think this would be the ideal setup but I want to see if it makes any sense.

So currently I have the OAK-FFC-4P connected to a desktop linux PC with Nvidia GPU. We really do need a powerful GPU to handle the kind of real time high quality image processing we are aiming to do. It runs best when the oak is attached to my linux desktop computer. But we need to pack this compute into a product.

Here is what I was thinking

https://docs.luxonis.com/projects/hardware/en/latest/pages/DM1097/#oak-d-cm4

We want something like this but using the raspberry pi 5 not the 4. Then we want to attach a desktop Nvidia GPU to the raspberry pi 5. Or something with the equivalent type of power. Our code heavily utilizes CUDA for image processing performance.

Firstly would attaching a desktop GPU make sense or is there another way to empower our image stitching code done on tensor images to achieve the maximum fps? we need like 4k 60fps, or as close to it. If not that then 1080p 60fps, 4k 30fps.

What do you think about such a setup? How would we go about actually piecing this together?

Hi @Nearpoint ,
Instead of RPi5 + GPU, you'd likely be better off by just using a beefy Jetson Orin/Xavier - as then that's all-in-one solution, that you can connect the FFC-4P directly to.
Thoughts?
Thanks, Erik

That actually does make the most sense. My thought process initially was that there are no Jetsons that have a GPU to rival a desktop GPU like the geforce 3060 but i can see that the higher end modules do offer comparable performance for the type of image processing we are doing. Also been checking out buying the SOM and seperate carrier board.

As for the luxonis setup, i guess i would be looking at using 2 oak-ffc-3p. since the left and right sockets are already hardware synced I would just need to then worry about hardware syncing both oaks. i wonder if i can connect a wire between the two oaks somehow such that they are hardware synced?

then the ToF sensor would just go into one of the oaks.

I mean it just seems janky way of doing it though. Honestly the best setup would be to utilize the OAK SOM Pro with the Nvidia Jetson. Basically put the depthAI hardware into the jetson by connecting the OAK SOM and then connect all four color cameras to and ToF sensor the OAK SOM PRO which is already hard connect to the Nidia Jetson.

We would just need to develop a carrier board for the SOM that allows us to connect 4 color cameras and the ToF Sensor to the SOM and the SOM to the Jetson. And we use a powerful Jetson and this should be the ideal setup, is my thinking making sense?

    Nearpoint i wonder if i can connect a wire between the two oaks somehow such that they are hardware synced?

    You can. https://discuss.luxonis.com/d/3119-setup-helpsyncing-and-calibrating-two-oak-ffc-4p/2

    Nearpoint We would just need to develop a carrier board for the SOM that allows us to connect 4 color cameras and the ToF Sensor to the SOM and the SOM to the Jetson. And we use a powerful Jetson and this should be the ideal setup, is my thinking making sense?

    cc @erik That'd be pretty difficult I assume.

    Thanks,
    Jaka

        Hi,
        I'd say it would be much more straight forward to either:

        • Go with 2x OAK-FFC-4P to connect all of these sensors, or
        • Connect all color cams to one FFC-4P, and get a different TOF (eg. some USB one), instead of our FFC-ToF

        Thoughts?
        Thanks, Erik

        I am leaning towards 2x OAK-FFC-4P. I have used dual oak devices before my main concern is hardware syncing.

        Firstly i think for hardware syncing to work at least one of the cameras has to drive the output. I have all IMX477 sensors and your docs say that they can not output a signal they can only be set to input. Even when just 2 x IMX477 cameras are connecting only to the left and right ports, I still don't think they are hardware synced. So I think the way to get the IMX477 or any sensors that does not support frame syncing output is that you have to use an external trigger. Also the IMX477 is a nightmare to modify to Connect the FSIN on the cable to the XVS test point on the actual back of the board.

        I think the IMX477 is also not ideal for our situation and now exploring the global color cameras in order to get higher frame rate and better imaging.

        So I think if we can 100 percent get hardware syncing fully working between all four cameras with dual oaks and then also have the ToF sensor synced as well that is Ideal. Because the ToF sensor has to be synced to the color cameras so it seems the dual oaks is actually the best strategy.

        The ToF camera can go into one of the extra ports on the 4P. Also why ther 4P and not the 3P? I can connect the color cameras to the left and right ports on each 3P and then the ToF camera to any one of the color ports any one of them. So would dual 3P work just as well at the dual 4P.? Can you connect color cameras to the left and right ports? I mean I have and it works fine. But I do not really run them at 4k just 1080P.

        So all in all I like the dual OAK strategy. But now I need a global color camera with an interchangeable lens. I want the 2MP global color AR0234 that is literally perfect for our application if it could have an interchangeable lens. Why does the lens have to be soldered to the board. Can you please change that so we can swap the lens for a higher focal length lens?

        Oh well maybe its just the module that's soldered to the board but the actual lens inside the holder can be swapped out?

        @erik The AR0234 also is just fsync input only.

        I am really curious about the precise answer to this question because so far I feel like it's been more of a trial let's see does it work like this kind of thing.

        if you are using camera sensors, and all of them are unable to drive an F sync output is it possible to hardware sync all the cameras without an external trigger? I assume this would be done by the Oak board itself using its own trigger to sync, the left and right cameras, unless the left and light cameras are dependent on having a cam that can drive in F sync output.

        That brings me to this question does the left and right camera hardware syncing depend on having camera's attached or at least one of them that can drive an fsync output?

        So again, if none of the cameras, attached to the oak, 4P module can drive an F sync output that means you hundred percent need an external trigger to do the hardware syncing?

        but not only that if you are using the Arducam camera modules, they don't have the Fsync lines built in that goals from Camera to board. The F sync lines are left out and you have to manually hook it up by utilizing the fsync test point. it looks like at least with the AR 0234 that you can at least solder header pins and connect the fsync output of all four cameras to each other via the header pins.

        Does the time of flight camera expose its fsync header pin? and will that also need to be wired to the other four global shutter cameras that have their fsync header pins wired together?

        wiring the Fsync header pins together is not an issue for me that can be done fairly easily but on the IMX 477 it was a big issue because it doesn't even support header pins.

        anyways, I think I am on track to the correct hardware selection. I just need to make sure of the hardware syncing as it is critically important for us.

        Hi @Nearpoint ,
        That's a lot of text - perhaps try to make your messages more concise.

        is it possible to hardware sync all the cameras without an external trigger?

        Yes, eg. on OAK-D-LR we drive FSYNC with the GPIO. Or have an external trigger, like a MCU, to provide signal.

        on having camera's attached

        What do you mean by attached here?

          erik

          Sorry about that, it is confusingly written.

          Is the Oak module itself capable of generating a fsync signal that can be used to drive four cameras which all only have fsync input capabilities?

            Hi @Nearpoint ,
            Yes, FSYNC signal is nothing special, it's just a pulse that you can achieve with a GPIO. Just note that if you'd use GPIO inside Script node, it might not be precise enough (python interpreter on embedded system and all), but it would be best to check.
            Thanks, Erik

              erik

              I am trying to figure out if I need an external mechanism, a pulse signal generator to use for fsync, that I need to connect to the GIPO of the oak module?

              Or is this mechanism built in and utilized automatically to sync left and right sensors that have no fsync output capability?

              Because typically you have OVA cams connected to the left and right and those can drive fsync output signal, so I am wondering if the fsync output of the camera sensor is used for left and right cam syncing with the oak or is the oak board itself capable of generating this signal?

              If not then I would need to get a pulse generator to hook up to the oak because none of the sensors I am using have fsync output capabilities to be used as the base signal to sync the cameras

                Yes, it can, and I think it does it automatically. I just connected 2x AR0234 (updated PCBA, so it exposes FSYNC) to the FFC-4P and frames are hardware synced. See terminal output below.

                Left 1:25:11.878866, Right 1:25:11.878881
Left 1:25:11.912199, Right 1:25:11.912213
Left 1:25:11.945531, Right 1:25:11.945546
Left 1:25:11.978863, Right 1:25:11.978879
Left 1:25:12.012195, Right 1:25:12.012210
Left 1:25:12.045527, Right 1:25:12.045542
Left 1:25:12.078859, Right 1:25:12.078876
Left 1:25:12.112192, Right 1:25:12.112208

                Code here:

                from depthai_sdk import OakCamera

with OakCamera() as oak:
    left = oak.create_camera('CAM_B')
    right = oak.create_camera('CAM_C')

    oak.callback([left, right], lambda packets:
                 print(f"Left {packets['CAM_B_video'].msg.getTimestamp()}, Right {packets['CAM_C_video'].msg.getTimestamp()}")
                 ).configure_syncing(enable_sync=True)

    oak.start(blocking=True)

                  erik

                  Ah correct that reminds me I should separate issues here. One is about the cabling, about fsync even being able to reach the OAK and the other is about which device is responsible for generating an fsync output.

                  As far as cabling and fsync, is that a luxonis carrier board for the AR0234 with an M12 lens mount? Can I get some of those? Otherwise I have the arducam one and it may have header pins for fsync but I do not want to deal with all the soldering work needed to get fsync to reach the oak.

                  As far as the oak itself generating a fsync pulse, now if you add 2 more AR0234 to CAMA and CAMD so you have four AR0234 cameras attached to the OAK-4P are they all four hardware synced? If you can verify this for me that would help sort out a lot of confusion.

                    Hi @Nearpoint ,
                    1 ) Cabling - Yes, we just got initial PCBAs that have mounting holes for the M12 cam module (for the screw). I think we'll have AR0234 and IMX462 with this PCBA soon on the store.
                    2) FSYNC signaling - Yes, should be, you'd just need to toggle GPIO so 2lane mipi and 4lane mipi fsync signals are connected together (more info on ffc-4p docs).

                    Thanks, Erik

                      erik

                      i see makes sense, i think i understand this a lot better now. I think my issue in my testing was faulty soldering connections and thus is was hard for me to really verify the behavior. Eager to get the new luxonis AR0234 sensors to avoid soldering work for the arducam cameras since the fsync can't get to the oak. Getting hardware syncing working with those will be very nice.