• DepthAI-v2

  • OAK-FFC-4P : Inconsistent IMU sample rate

Dear Luxonis Experts,

Currently, we received the report of a potential issue of the IMU on OAK-FFC-4P with the SpectacularAI VIO/SLAM as below :

Sometimes the frequency is 5-10Hz, restarting usually fixed this (also tinkered with the IMU settings, perhaps those help too)
I've noticed at least 80ms timeshift between the camera and IMU clocks, while our algorithm can cope with some shift, this much causes tracking issues, especially in the beginning before it has had a chance to estimate it. This is likely why trying to calibrate it's location with our calibration tools fail

While waiting for their support to debug, I would like to know if their is any efficient/independent way to verify the IMU sample rate, please ? The board we are using for the testing is R5.

Best Regards,
Khang

Hi @l4es ,
Sample rates are documented here: https://docs.luxonis.com/projects/api/en/latest/components/nodes/imu/#imu-sensor-frequencies
So for BNO085/6 that should be on the FFC-4P, you have:

Accelerometer: 15Hz, 31Hz, 62Hz, 125Hz, 250Hz 500Hz
Gyroscope: 25Hz, 33Hz, 50Hz, 100Hz, 200Hz, 400Hz
Magnetometer: 100Hz

80ms timeshift between the camera and IMU clocks

Which is sooner? Both IMU/camera frames are "sharing" the same transfer link, so if you are streaming eg. high-res frames as well then there will be additional latency.

  • l4es replied to this.

    Hi erik ,

    Which is sooner?

    Could you elaborate the idea of the question so that I can check with them, please ?

    Both IMU/camera frames are "sharing" the same transfer link, so if you are streaming eg. high-res frames as well then there will be additional latency.

    We are using 3 x AR0234 and default/supported resolution is 1280x720.

    Best Regards,
    Khang

    • erik replied to this.

        l4es ,
        You mentioned there's a timeshift between 2 msgs (imu/frame), which arrives to the host 80ms before the other?

        • l4es replied to this.
            5 days later

            Hi erik,

            which arrives to the host 80ms before the other?

            Following is the answer from our partner for your above question :

            IMU timestamps is older. For example, IMU timestamps at 5.9 seconds == camera at 6.0 seconds

            Best Regards,
            Khang

              Hi @l4es ,
              That would be expected, as IMU packets are smaller and don't need any processing - while images require ISP (takes a few/dozens ms, depends on the resolution. Frames are also larger, so take more time to get transferred, which means that approx. 100ms latency isn't unexpected. Thoughts?
              BR, Erik

              • l4es replied to this.

                Hi erik ,

                Thanks for your reply. Below is the comment of our collaborator based on your anser :

                The problem isn't latency. It's fine to send imu and images with up to 500ms latency. The problem is that the actual timestamps in the data are wrong. If I take IMU sample at time X and camera frame at the exact same time X, I would expect the IMU timestamp to be X and camera timestamp to be X. Instead the IMU timestamp is X-100ms and camera is X.

                It looks like that he expects that the timestamps of the IMU and the camera frame are as nearest as possible. What do you think?

                  @l4es so the messages themselves have incorrect timestamps attached to them? Do you have any simple example we could try out to repro/evaluate that ourselves?