IMU Calibration/Sensor Fusion for Dynamic Environment

Hi y'all!

We have used multiple different cameras and boards (FFC3/FFC4, Oak-1, …) mainly with the 9-axis IMU that is onboard.

Our use-case are cameras on small to medium-sized boats. We experience permanent unpredictable movement from waves hitting the boat, the boat accelerating promptly. For example, in the following video, we want to get the horizon based on the orientation vector and mapping it to pixel space, but it can't cope with the acceleration:

Hence, I was wondering if there is an option for changing the intrinsic sensor fusion module? Are there different modi/drivers/… that could be changed?

So far, doing stuff on the host-side was not too successful as we run into issues with sample rate and synchronization (we work in high-level Python). Generally, it's best to do this as low-level as possible.

Could you give me some advice on what to do in this case?

Thanks in advance!

In order to get the orientation vector, you have to fuse accelerometer and gyroscope (and potentially magnetometer) measurements (either through Kalman, Madgwick or similar). However, there are various hyperparameters one can set.

I wondered whether there is any way these are exposed via the Luxonis API so that we can adapt them to different use-cases. As shown in the video, the current settings are not really useful.

Thanks for the suggestion of the feature tracker. The problem in our domain is that the features are pretty bad most of the time (water moves, often time no land visible). Hence, we wanted to rely on an IMU only. We also did some tests with fusing the IMU and vision, but no success so far.

Maybe, it's an inherent weakness of the sensor, but I was just curious to hear if there's anything else that can be done.