FFC-4p hardware synchronization

erik · Apr 21, 2023

Hi Asa , I don't think IMU/frames are hardware synced. But having IMU at higher frequency (eg 5x higher), even timestamp syncing could be good enough. More information here. Thoughts?

Asa · Apr 24, 2023

State estimation systems like to have nice hardware synchronization. If you can trigger the imu at multiples of the images this would help build these kind of tightly coupled systems.

thebambuproject · Apr 25, 2023

How to control GPIO with C++ API? I´m talking about this part:

config.board.gpio[6] = dai.BoardConfig.GPIO(dai.BoardConfig.GPIO.OUTPUT, dai.BoardConfig.GPIO.Level.HIGH)

Thanks!

erik · Apr 25, 2023

thebambuproject Would it not be the same? C++/Python API should be 1:1, as Python just has bindings over.

thebambuproject · Apr 25, 2023

Sorry but its not trivial to me

dai::Device device(pipeline);
dai::Device::Config c = pipeline.getDeviceConfig();
c.board.gpio[6] = dai::BoardConfig::GPIO(dai::BoardConfig::GPIO::OUTPUT, dai::BoardConfig::GPIO::Level::HIGH);

Can somebody point me, its ok? config seems to be private… Thanks

jakaskerl · Apr 25, 2023

Hi thebambuproject

I'd change it to:

dai::Device device(pipeline);
dai::Device::Config c = pipeline.getDeviceConfig();
c.board.gpio[6] = dai::BoardConfig::GPIO(dai::BoardConfig::GPIO::Direction::OUTPUT, dai::BoardConfig::GPIO::Level::HIGH);

Otherwise it looks good (i'm not familiar with c++).

Hope it works,
Jaka

Aaiyangsky · Mar 16, 2024

@Luxonis-Alex Hi, I also tried to sync 4 cameras in FFC-4P, but only camera B and camera C synced successfully anyway.The camera uses a UC-788 module with Arducam OV9282 and the FSIN signal is Connect correctly.

Here's my code

#include "depthai/depthai.hpp"
#include <chrono>
#include <opencv2/opencv.hpp>
#include <time.h>
#include <iostream>
#include <thread>

int main()
{
    // Create pipeline
    dai::Pipeline pipeline;

    // Define sources and outputs
    // cam_a
    auto camA = pipeline.create<dai::node::MonoCamera>();
    camA->setBoardSocket(dai::CameraBoardSocket::CAM_A);
    camA->setResolution(dai::MonoCameraProperties::SensorResolution::THE_400_P);
    camA->setFps(30);
    camA->initialControl.setFrameSyncMode(dai::CameraControl::FrameSyncMode::OUTPUT);

    auto AOut = pipeline.create<dai::node::XLinkOut>();
    AOut->setStreamName("CAM_A");
    camA->out.link(AOut->input);

    // cam_d
    auto camD = pipeline.create<dai::node::MonoCamera>();
    camD->setBoardSocket(dai::CameraBoardSocket::CAM_D);
    camD->setResolution(dai::MonoCameraProperties::SensorResolution::THE_400_P);
    camD->setFps(30);
    camD->initialControl.setFrameSyncMode(dai::CameraControl::FrameSyncMode::INPUT);

    auto DOut = pipeline.create<dai::node::XLinkOut>();
    DOut->setStreamName("CAM_D");
    camD->out.link(DOut->input);

    // cam_b
    auto camB = pipeline.create<dai::node::MonoCamera>();
    camB->setBoardSocket(dai::CameraBoardSocket::CAM_B);
    camB->setResolution(dai::MonoCameraProperties::SensorResolution::THE_400_P);
    camB->setFps(30);
    camB->initialControl.setFrameSyncMode(dai::CameraControl::FrameSyncMode::INPUT);

    auto BOut = pipeline.create<dai::node::XLinkOut>();
    BOut->setStreamName("CAM_B");
    camB->out.link(BOut->input);

    // cam_c
    // auto camC = pipeline.create<dai::node::MonoCamera>();
    // camC->setBoardSocket(dai::CameraBoardSocket::CAM_C);
    // camC->setResolution(dai::MonoCameraProperties::SensorResolution::THE_400_P);
    // camC->initialControl.setFrameSyncMode(dai::CameraControl::FrameSyncMode::INPUT);

    // auto COut = pipeline.create<dai::node::XLinkOut>();
    // COut->setStreamName("CAM_C");
    // camC->out.link(COut->input);

    dai::BoardConfig Config = pipeline.getBoardConfig();
    Config.gpio[6] = dai::BoardConfig::GPIO(dai::BoardConfig::GPIO::OUTPUT, dai::BoardConfig::GPIO::Level::HIGH);
    pipeline.setBoardConfig(Config);

    dai::Device device(pipeline);

    auto lastTime = std::chrono::duration_cast<std::chrono::nanoseconds>(std::chrono::system_clock::now().time_since_epoch()).count();
    auto currTime = std::chrono::duration_cast<std::chrono::nanoseconds>(std::chrono::system_clock::now().time_since_epoch()).count();

    cv::Mat AMat, BMat, CMat, DMat;
    cv::Mat ADMat, BCMat;

    auto imageA = device.getOutputQueue("CAM_A", 1, false);
    auto imageB = device.getOutputQueue("CAM_B", 1, false);
    // auto imageC = device.getOutputQueue("CAM_C", 1, false);
    auto imageD = device.getOutputQueue("CAM_D", 1, false);

    while (true)
    {
        auto A = imageA->get<dai::ImgFrame>();
        auto B = imageB->get<dai::ImgFrame>();
        // auto C = imageC->get<dai::ImgFrame>();
        auto D = imageD->get<dai::ImgFrame>();

        AMat = A->getCvFrame();
        BMat = B->getCvFrame();
        // CMat = C->getCvFrame();
        DMat = D->getCvFrame();

        auto tsA = A->getTimestampDevice();
        auto tsB = B->getTimestampDevice();
        // auto tsC = C->getTimestampDevice();
        auto tsD = D->getTimestampDevice();

        auto timeDifference = tsA - tsB;
        auto timeDifferenceUs = std::chrono::duration_cast<std::chrono::microseconds>(timeDifference).count();
        std::cout << "Time difference between messages is: " << std::abs(timeDifferenceUs / 1000.0) << " ms" << std::endl;

        // timeDifference = tsC - tsB;
        // timeDifferenceUs = std::chrono::duration_cast<std::chrono::microseconds>(timeDifference).count();
        // std::cout << "Time difference between messages is: " << std::abs(timeDifferenceUs / 1000.0) << " ms" << std::endl;

        timeDifference = tsD - tsB;
        timeDifferenceUs = std::chrono::duration_cast<std::chrono::microseconds>(timeDifference).count();
        std::cout << "Time difference between messages is: " << std::abs(timeDifferenceUs / 1000.0) << " ms" << std::endl;

        currTime = std::chrono::duration_cast<std::chrono::nanoseconds>(std::chrono::system_clock::now().time_since_epoch()).count();
        auto Ts = currTime - lastTime;
        std::cout << "read 1 count;Ts = " << Ts / (1000.0f) << "us" << std::endl;
        lastTime = currTime;

        int key = cv::waitKey(1);
        if (key == 'q')
        {
            break;
        }
    }
    return 0;
}

The result is as follows

Time difference between messages is: 15.496 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 31927.5us
Time difference between messages is: 15.681 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33455.4us
Time difference between messages is: 15.866 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33237us
Time difference between messages is: 16.052 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33648.6us
Time difference between messages is: 16.237 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 32952.1us
Time difference between messages is: 16.422 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33447.5us
Time difference between messages is: 16.608 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33302.4us
Time difference between messages is: 16.793 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 24855.3us
Time difference between messages is: 16.978 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 41743.3us
Time difference between messages is: 17.163 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 24911.8us
Time difference between messages is: 17.349 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33330.5us
Time difference between messages is: 17.534 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 31817.8us
Time difference between messages is: 17.719 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33400.9us
Time difference between messages is: 17.905 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33321.5us
Time difference between messages is: 18.09 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33280.4us
Time difference between messages is: 18.275 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33403.9us
Time difference between messages is: 18.461 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33301.3us
Time difference between messages is: 18.646 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33319us
Time difference between messages is: 18.831 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33308.3us
Time difference between messages is: 19.017 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33329.3us
Time difference between messages is: 19.202 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33298.9us
Time difference between messages is: 19.387 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33434.2us
Time difference between messages is: 19.572 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33381.4us
Time difference between messages is: 19.758 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33206.2us
Time difference between messages is: 19.943 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33371.4us
Time difference between messages is: 20.128 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33273.5us
Time difference between messages is: 20.314 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33521.1us
Time difference between messages is: 20.499 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33146.4us
Time difference between messages is: 20.684 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33411.6us
Time difference between messages is: 20.87 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33324us
Time difference between messages is: 21.055 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33303.9us
Time difference between messages is: 21.24 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33418us
Time difference between messages is: 21.426 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33243.8us
Time difference between messages is: 21.611 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33410.5us
Time difference between messages is: 21.796 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33277.6us
Time difference between messages is: 21.982 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33358.2us
Time difference between messages is: 22.167 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33361us
Time difference between messages is: 22.352 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33288.1us
Time difference between messages is: 22.538 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33337.2us
Time difference between messages is: 22.723 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33390.5us
Time difference between messages is: 22.908 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33347.6us
Time difference between messages is: 23.093 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33334.9us
Time difference between messages is: 23.279 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33339.1us
Time difference between messages is: 23.464 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33334.2us
Time difference between messages is: 23.65 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33329.7us
Time difference between messages is: 23.835 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33344.9us
Time difference between messages is: 24.02 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33313.7us
Time difference between messages is: 24.205 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33297.9us
Time difference between messages is: 24.391 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33415.8us
Time difference between messages is: 24.576 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33307.8us
Time difference between messages is: 24.761 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33351.2us
Time difference between messages is: 24.947 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33301.3us
Time difference between messages is: 25.132 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33319.4us
Time difference between messages is: 25.317 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33356.4us
Time difference between messages is: 25.503 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33454us
Time difference between messages is: 25.688 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33218.8us
Time difference between messages is: 25.873 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33299.1us
Time difference between messages is: 26.058 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33346.6us
Time difference between messages is: 26.244 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33337.1us
Time difference between messages is: 26.429 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33527.1us
Time difference between messages is: 26.614 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33177us
Time difference between messages is: 26.8 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33389us
Time difference between messages is: 26.985 ms
Time difference between messages is: 22.579 ms
read 1 count;Ts = 33325.2us

erik · Mar 16, 2024

@aiyangsky could you share some photo of the wiring as well?

Aaiyangsky · Mar 18, 2024

erik

Of course, a schematic diagram of the line connection (orange cable from FSIN to XVS):

Below is the FSIN signal (blue to CAM_A, yellow is CAM_B

jakaskerl · Mar 18, 2024

Hi @aiyangsky
Try setting the OUTPUT mode to the CAM_B and the rest to INPUT.

Change gpio6 to gpio42 on R7.

dai::Device device(pipeline);
dai::Device::Config c = pipeline.getDeviceConfig();
c.board.gpio[42] = dai::BoardConfig::GPIO(dai::BoardConfig::GPIO::Direction::OUTPUT, dai::BoardConfig::GPIO::Level::HIGH);

Thanks,
Jaka

Aaiyangsky · Mar 19, 2024

jakaskerl Thanks for the suggestion. But I tried this way and he didn't improve.

erik · Mar 20, 2024

@aiyangsky the scoped signal doesn't look correct - it starts at 1V (should be 0v?) and goes to 1.8V. Is this a measurement error?

Aaiyangsky · Mar 21, 2024

erik I double-checked today that this is indeed not a measurement error.

Aaiyangsky · Mar 21, 2024

erik Is this reference voltage change caused by the FFC module? Or camera？What can I do to investigate this problem?