IMU data synchronization issues on the Oak D Wide

Doing some more investigation with the IMU measurements, the data rates are very odd to me. Below are the measurement rates which are stamped by the device.

The gyroscope and accelerometer rates vary very wildly on the OAK D WIDE. These rates are basically calculated via 1.0 / (current_time - previous_time).

With the OAK D LITE things are *better per se but still not ideal. I feel the "synchronization" strategy is not ideal...

One last thing, both experiments are configured with the IMU Acc at 500 and IMU Gyro at 400. While the OAK D WIDE does sort of forfil the rates albeit with very wide variances, the OAK D LITE is no where near the defined 500 / 400 Hz for the IMU Acc and Gyro respectively.

Here is the script I used to produce the above:

#!/usr/bin/env python3
import cv2
import numpy as np
import matplotlib.pylab as plt
import depthai as dai


def ts2sec(delta):
  """Timestamp to Seconds"""
  return delta.total_seconds()


if __name__ == "__main__":
  # Create pipeline
  pipeline = dai.Pipeline()

  # Configure IMU
  imu_xout = pipeline.create(dai.node.XLinkOut)
  imu_xout.setStreamName("imu")

  imu = pipeline.create(dai.node.IMU)
  imu.enableIMUSensor(dai.IMUSensor.ACCELEROMETER_RAW, 500)
  imu.enableIMUSensor(dai.IMUSensor.GYROSCOPE_RAW, 400)
  imu.setBatchReportThreshold(1)
  imu.setMaxBatchReports(1)
  imu.out.link(imu_xout.input)

  # Connect to device and start pipeline
  device = dai.Device(pipeline)

  # Loop
  imu_queue = device.getOutputQueue(name="imu", maxSize=50, blocking=True)

  num_measurements = 0
  imu_acc_time = []
  imu_acc_data = []
  imu_gyr_time = []
  imu_gyr_data = []

  loop = True
  while loop:
    imu_data = imu_queue.get()
    for imu_packet in imu_data.packets:
      imu_acc = imu_packet.acceleroMeter
      imu_gyr = imu_packet.gyroscope
      imu_acc_ts = ts2sec(imu_acc.getTimestampDevice())
      imu_gyr_ts = ts2sec(imu_gyr.getTimestampDevice())

      # Keep track of measurements
      imu_acc_time.append(imu_acc_ts)
      imu_gyr_time.append(imu_gyr_ts)
      imu_acc_data.append([imu_acc.x, imu_acc.y, imu_acc.z])
      imu_gyr_data.append([imu_gyr.x, imu_gyr.y, imu_gyr.z])

      # Break loop?
      num_measurements += 1
      if num_measurements > 100:
          loop = False
          break

  # Plot data
  imu_acc_time = np.array(imu_acc_time)
  imu_acc_data = np.array(imu_acc_data)
  imu_gyr_time = np.array(imu_gyr_time)
  imu_gyr_data = np.array(imu_gyr_data)
  print(f"imu rate: {1.0 / np.mean(np.diff(imu_acc_time))}")
  print(f"gyr rate: {1.0 / np.mean(np.diff(imu_gyr_time))}")

  # Boxplot
  plt.figure()
  plt.boxplot([1.0 / np.diff(imu_acc_time), 1.0 / np.diff(imu_gyr_time)])
  plt.xticks([1, 2], ["Accelerometer", "Gyroscope"])
  plt.ylabel("Rate [Hz]")

  # Time series plot
  plt.figure()
  plt.plot(1.0 / np.diff(imu_acc_time), 'r.', label="Acceleromenter")
  plt.plot(1.0 / np.diff(imu_gyr_time), 'b.', label="Gyroscope")
  plt.legend(loc=0)
  plt.xlim([0, num_measurements])
  plt.xlabel("i-th Measurement")
  plt.ylabel("Rate [Hz]")

  plt.show()

The accelerometer is synced with gyro, the combined sync output rate is the minimum between the two (400 hz in this case).

Yes, but this is not what I'm seeing with the Oak D Wide. The Oak D Wide accelerometer and gyroscope outputs are not synchronized (in terms of time), as I have shown in the screenshots above. This is important because state-of-the art SLAM / VIO algorithms assume at time-step k you have the IMU acceleration and angular velocity measurement at that time instance. While the OAK D Lite does satisfy this requirement the OAK D Wide does not.

Reference:

• On-Manifold Preintegration for Real-Time Visual-Inertial Odometry by Christian Forster, Luca Carlone, Frank Dellaert, Davide Scaramuzza
• A Multi-State Constraint Kalman Filter for Vision-aided Inertial Navigation by Anastasios I. Mourikis and Stergios I. Roumeliotis

Essentially every 5th accelerometer is discarded to get a synced output w/ gyro.

This is where I would disagree technically. You don't have to throw away anything if one does a simple linear interpolation. Is there a way for me to obtain the raw accelerometer and gyroscope measurement data independently? So that I can synchronize / interpolate them myself?

erik

Could you expand on this - what's currently missing from the API perspective?

From an API perspective I would prefer getting the Accelerometer / Gyroscope measurements independently. i.e. Accelerometer / Gyroscope at 500Hz and 400Hz respectively as I have configured them while enabling the IMU sensors via

imu = pipeline.create(dai.node.IMU)
imu.enableIMUSensor(dai.IMUSensor.ACCELEROMETER_RAW, 500)
imu.enableIMUSensor(dai.IMUSensor.GYROSCOPE_RAW, 400)

This would allow the developer to obtain the full capability of the sensor themselves (gyro / accel / or both) or interpolate the measurements to how they see fit. From my understanding there is no way to get them independently.

As an example, this is how intel realsense exposes their IMU measurements: link