We're struggling to understand Rotation Vector returned by the IMU. We currently have several different models of Oak cameras (Oak-D-POE, Oak-D-Pro-POE, Oak-SR-POE), which should all have the BNO086 IMU, albeit all mounted in different orientations. The depthai-ros URDFs do not seem to support all the current models (PoE support is especially patchy). After carefully checking the circuit board CAD files, we've created URDFs based on depthai-ros so our TF tree should correctly handle the IMU orientations of all our camera types.

After carefully reading the BNO08X datasheet, "The rotation vector provides an orientation output that is expressed as a quaternion referenced to magnetic north and gravity", when enabling the ROTATION_VECTOR sensor, we expected to get a quaternion that expressed the rotation from a NED frame to the IMU frame. However, in testing one of our Oak-D-PoEs we get an identity quaternion returned only when the X axis of the IMU points toward magnetic North, the Y axis points West (not East) and the Z axis of the IMU points UP (not down, as we expected). This would imply the IMU is referencing a North-West-Up (or Forward-Left-Up, FLU) frame. Has anyone had this same result?

Also, does Luxonis enable any dynamic calibrations of the IMU sensors? What data (if any) is written to the FRS records on the IMU? In particular, does Luxonis intentionally write any orientation into the IMUs "system orientation" field? Is there anyway as a user for us to trigger the camera to command the IMU to persist the dynamic calibration data (DCD) to the FRS record (as mentioned in section 3.4 of the Datasheet)?

    danl5
    cc @Luxonis-Adam Afaik this was fixed in depthaiV3 if I'm not mistaken.

    danl5 Also, does Luxonis enable any dynamic calibrations of the IMU sensors? What data (if any) is written to the FRS records on the IMU? In particular, does Luxonis intentionally write any orientation into the IMUs "system orientation" field? Is there anyway as a user for us to trigger the camera to command the IMU to persist the dynamic calibration data (DCD) to the FRS record (as mentioned in section 3.4 of the Datasheet)?

    The dynamic calibration (self-calibration) is enabled by default I think - depthai doesn't expose overriding that or doing any other low-level configurations for the IMU.

    Thanks,
    Jaka

    cc @Luxonis-Adam Afaik this was fixed in depthaiV3 if I'm not mistaken.

    Yes, in ROS2 Kilted release IMU outputs will be output in RDF format automatically and transforms will be published based on calibration data by default

    To be clear, we've written our own ROS2 driver nodes that wrap depthai-core and are not using Luxonis's ROS driver. We only extracted the URDFs from the description folder. So, what I need to know is, when I use depthai-core to process packets from the IMU queue, between what coordinate frames does the ROTATION_VECTOR quaternion express a rotation?

    I expected NED -> IMU but that doesn't appear to be correct.

    If this was fixed in depthaiV3, what was changed? We're not ready to adopt v3 yet and I need the definitions of the transforms to correct it in v2.30.0.0.

    11 days later

    @jakaskerl @Luxonis-Adam Thanks for the responses, but I'm still confused.

    In depthai v2.30.0.0, what frame is the IMUs ROTATION_VECTOR expressed with respect to and is it different for different models of Oak camera?

      danl5
      Gravity and magnetic north. All devices that have BNO085/6 will behave the same.

      Thanks,
      Jaka

      Can you be more specific? There are multiple ways to be "referenced" to gravity and magnetic north: NED (Noth-East-Down), ENU (East-North-Up), FLU (Forward-Left-Up / North-West-Up). We're finding that getting to the bottom of this extremely difficult as we don't seem to get consistent results from our cameras - sometimes the rotation vector returns an identity quaternion when the camera is not aligned to any Cardinal direction at all, sometimes it appears it might be FLU. It's almost as if the BNU's Tare function might be triggered at some point (but it's not clear what or when this might be happening). Since we don't have visibility into what commands are sent to the IMU chip, we're relying on Luxonis to provide the information.