IMU timestamp timings on Oak-d W PoE are very jitterish

youpi

We have 2x Oak-d W PoE with the latest bootloader (0.0.28) and depthai version 2.26.1
and depthai ubuntu package ros-noetic-depthai-ros-driver v 2.9.0 on ubuntu 20.04.

We analyze the timestamping of the imu and the cameras, and make sure that we have sufficient bandwidth on the PoE link (15FPS, stereo only, no rgb, downscaled and jpg compression enabled)

The analysis is by recording a rosbag of the oak topics and then checking jitter on the timestamps. For the camera images, the timestamp is always <1ms in deviation of the 15Hz or even at the 30Hz sampling rate. For the IMU however, both at 100Hz and 200Hz (gyro and accel are set at the same rate, no interpolation, we use COPY) the jitter is large, up to 7ms in some instances. The ros driver is running for at least a 30 seconds before we start capturing data.

We checked setting the imu_i_get_base_device_timestamp to true or false, with the same result. We can confirm that the devices is properly time synchronized with the PC time, as the ros base time updates are always <1000ns.

The above plot is an example of ~3ms jitter, but we can have worse. We have no samples dropped, the ros topic hz /oak/imu/data nicely shows 100Hz or 200Hz, same for the camera images, no frames are dropped.

How come the camera timestamps are so accurate while the imu timestamp is so jittery? Is there any setting we are overlooking? Is the imu internally sampled at 100Hz (or 200Hz) and could we just assume the timetamps are wrong, and restamp them ourselves with nice 10ms / 5ms intervals ?

jakaskerl

youpi

cc @Luxonis-Adam Is this a ros issue?

Thanks,
Jaka

youpi

Thanks for looking into it. I checked the ROS driver source code and it gets the timestamp directly from the SDK when no interpolation is done (COPY). So it is imo not a ROS driver issue, but a problem on the camera firmware.

Luxonis-Adam

Hi, indeed when COPY method is used, timestamps are taken straight from the camera so if there is an issue it is most likely on firmware level. Did you try setting different parameters for IMU such as frequency, batch size etc and if yes what were the results?

youpi

Hi Adam, yes we tried that, but more worryingly, we have it on 3 different cameras, all of the Poe Wide type.

When we saturate the PoE link (uncompressed all 3 cams), we get all data (including IMU) at ~6Hz and this debug reading from leonOS:

Cpu Usage - LeonOS 100.00%, LeonRT: 3.70%

When managing the link, enabling compression etc. we get as debug from leonOS:

Cpu Usage - LeonOS 54.18%, LeonRT: 9.19%
And camera images are nicely at 30Hz -without jitter- on all 3 channels, but IMU, although hitting 100Hz, jitters between 6ms and 17ms instead of 10ms without jitter.

So again - cams hit perfection while IMU is all over the place, while the leonOS is only 50% loaded.

youpi

To answer your question directly: occurs at 100Hz and 200Hz sampling rate (both acc&gyro), as well as with batch size = 1 and batch size threshold 3 / max 5

jakaskerl

youpi
I think it's the current limitation of the BNO (how it's integrated). Tested on USB and same thing happens.

cc @GergelySzabolcs

Thanks,
Jaka

youpi

Could this be fixed on the software level? And also, do you think that the timestamp itself is correct, or is it more likely that the sensor is sampled at a fixed frequency (by use of a hardware clock) and that the timestamping/arrival of the data is wrong?

Having a reliable IMU timestamping is very critical for visual slam applications.

jakaskerl

youpi
The jitter is due to the M0 cortex core from BNO, it raises the interrupt with delay. We don't know why, because we don't have access to the firmware. It can be considered that the data is sampled at even rate (according to discrete sampling rates, see datasheet), just the timestamp is skewed due to late interrupt.

Thanks,
Jaka