• DepthAIHardware

  • Synchronization issues between the OAK-FFC4P module and the host clock

I understand from the documentation that the depthai library automatically synchronizes the device's timestamp with the host's timestamp: https://docs.luxonis.com/software/depthai-components/device/

When I used the getTimestamp() function provided by the depthai-core library to get the timestamp of the image, I tested the difference between each frame's image time and the host's system time and found that the device clock was drifting.

In addition, we compared the radar timestamp with the host timestamp and calculated the difference for each frame. When the radar timestamp has clock drift, the host clock can correct it and maintain it around a value.

The clock synchronization problem is very important for me to do multi-sensor data fusion, and I look forward to the answer.

    Amovlab-matnav
    There is predictable drift because the images aren't taken at exactly the same time - there is slight deviation -eg. at 10FPS the drift is 160us drift, at 20FPS there is 80us..

    HW triggering can possibly solve this, or just account for the drift and you should be fine.

    Thanks,
    Jaka

        I understand what you're saying about predictable variation due to sampling.

        However, it can be seen from the figure that the time delay gradually increases as the sampling sequence advances. This increasing delay makes it difficult for other sensors that are strictly time synchronized with the host to incorporate the data provided by the OAK device, since the sample points of the OAK device are constantly drifting over time.

        It appears that the OAK device's internal clock may be out of sync with the host device's clock. If it is determined that two devices share the same base clock, can the OAK device compensate for this fixed bias? This will ensure that the sampling sequence is consistent with the host clock reference.

          jakaskerl

          I did a preliminary test with both of my devices and found a fixed offset of around 200us at 10Hz, so we can infer that the clock source is indeed synced to the host.

          I can compensate for this bias a bit in the software, but to get the best fusion, I need the OAK device to sample perfectly to the host clock.

          These are my test data:

          First equipment

          Second equipment:

          The phase difference is obtained by subtracting the timestamp of the OAK device's 10Hz sampling data from the timestamp output of the 10Hz timer on the host computer.

          thanks,

          sky

              aiyangsky

              aiyangsky I can compensate for this bias a bit in the software, but to get the best fusion, I need the OAK device to sample perfectly to the host clock.

              You should be able to get a perfect sync by using a HW trigger to capture frames. Perhaps you already have one for the radar, so it would be best to just connect the two together to achieve synchronization.
              https://docs.luxonis.com/hardware/platform/deploy/frame-sync/

              cc @Luxonis-Alex Would this work on IMX as well since it doesn't support snapshot mode or would the drift still occur?

              Thanks,
              Jaka

                    jakaskerl

                    Thank you for your suggestion. In fact, we try to do this.

                    However, the current radar uses PTPv2 to synchronize time with the host computer, and from the above results, he works very well.

                    In order to simplify the hardware design as much as possible, we would like to compensate for the fixed offset in the device.

                    At present, this offset value may be related to the hardware design? Does it perform the same on the same hardware? Maybe I could try to measure the offset first and then tell the device how to compensate.

                    Thanks，

                    Sky

                        aiyangsky

                        This also involves the synchronization problem of the IMU, which is known to be an integral system and very sensitive to time.

                        We insisted that it was best to keep the whole system working on the same clock base as much as possible.

                        Thanks，

                        Sky

                          jakaskerl

                          @jakaskerl Yeah, I see all your devices are hardware synced using the FSYNC Y-adapter. I want to confirm, are the sensors connected to the FSYNC Y-adapter via external triggering?

                          Thanks,

                          Nav

                              jakaskerl Using hardware synchronization does not resolve our issue, as the four cameras cannot achieve complete synchronization. For more details, please refer to the other post: https://discuss.luxonis.com/d/5879-oak-ffc4p-external-hardware-trigger-asynchronization-problem/2.

                              We hope you can address this bug by implementing a software-based synchronization solution, where the cameras periodically sync with the main system clock. Thank you for your attention to this matter!

                                jakaskerl

                                hello, Jaka. I have carefully checked the API in depthai-core and found the settings related to host synchronization, but it does not work and the clock still drifts.

                                  jakaskerl

                                  I noticed that cam_B and cam_C share the iic bus, so their sampling is always synchronized. Can I make sure that the 4 Sensers are using the same CMOS by sharing the iic bus?

                                  Thanks,

                                  Sky

                                      aiyangsky
                                      AFAIK sharing the I2C bus will only affect the camera controls (3A and more) to be the same across all 4 sensors. The pairs still have separate clocks and separate FSYNC signals. Not sure if it's even possible to link more than 2 of the same sensors on the same i2c line (depends on how FW handles slave addresses) cc @Luxonis-Alex