Oak4 S and IMU

MatteoL

I can read neither the device acceleration, the rotation speed, nor the rotation vector from a connected Oak4 S, but I do receive the magnetometer measures. If I enable MAGNETOMETER_RAW alone or with some other quantity, I do receive messages, but only the magnetometer values are filled. Otherwise, if I enable only ACCELEROMETER or other measurements not related to the magnetometer, the output queue blocks indefinitely.

I updated both the Luxonis OS to 1.15.0 and the DepthAI SDK to 3.0.0rc3.

So my questions are:

I know you are still finalizing the software stack, but our application needs the IMU data to run. Would there be a way to receive at least the raw acceleration, just for testing purposes?
Is there some documentation about all quantities that are provided by the IMU on the Oak 4?

Minimal reproducible example:

import depthai as dai

with dai.Device() as device:
    print(f'Camera {device.getProductName()}, MXID {device.getDeviceId()}')
    print(f"IMU type: {device.getConnectedIMU()}, firmware version: {device.getIMUFirmwareVersion()}")

with dai.Pipeline() as pipeline:
    imu = pipeline.create(dai.node.IMU)
    imu.enableIMUSensor([dai.IMUSensor.ACCELEROMETER, dai.IMUSensor.MAGNETOMETER_RAW], 60)
    imu.setBatchReportThreshold(10)
    imu.setMaxBatchReports(10)
    imu_output_queue = imu.out.createOutputQueue(maxSize=128, blocking=False)

    pipeline.start()
    while pipeline.isRunning():
        imu_batch: dai.IMUData = imu_output_queue.get()

        if len(imu_batch.packets) <= 0:
            continue

        data = imu_batch.packets[0]

        print(f'{data.acceleroMeter.x:.3f} {data.acceleroMeter.y:.3f} {data.acceleroMeter.z:.3f} {data.acceleroMeter.accuracy}')
        print(f'{data.magneticField.x:.3f} {data.magneticField.y:.3f} {data.magneticField.z:.3f} {data.magneticField.accuracy}')

Output:

Camera OAK4-S-W, MXID 761681342
IMU type: accel, firmware version: 0.45.0
0.000 0.000 0.000 Accuracy.UNRELIABLE
14.400 -27.300 -29.250 Accuracy.HIGH
0.000 0.000 0.000 Accuracy.UNRELIABLE
12.150 -28.800 -31.500 Accuracy.HIGH

jakaskerl

MatteoL
This is what I get:

-0.000 -9.763 -0.191 Accuracy.HIGH
221.850 -9.450 139.650 Accuracy.HIGH
-0.005 -9.768 -0.191 Accuracy.HIGH
220.200 -9.150 136.800 Accuracy.HIGH
-0.005 -9.768 -0.196 Accuracy.HIGH
219.450 -9.600 138.000 Accuracy.HIGH

Can ssh into the device and paste the output of

ssc_drva_test -sensor=accel -duration=10 sample_rate=10
ssc_drva_test -sensor=gyro -duration=10 sample_rate=10
ssc_drva_test -sensor=mag -duration=10 sample_rate=10

Thanks,
Jaka

MatteoL

Sure! Here they are:

~ # ssc_drva_test -sensor=accel -duration=10 sample_rate=10
6 ssc_drva_test version 1.24k
6 ssc_drva_test -sensor=accel -duration=10 sample_rate=10
diag: Diag_LSM_Init: invoked for pid: 60836 with init_count: 0
diag:successfully connected to socket 5
diag: Diag_LSM_Init: done for pid: 60836 with init_count: 1
6 event_cb attribute event for da_test
6 event_cb attribute event for da_test
6 using da_test name=da_test, suid = [high addeaddeaddeadde, low addeaddeaddeadde
6 enter send_memory_log_req cookie: 6
6 exit send_memory_log_req
6 enter da_test runner. -rumifact=1
6 -time_to_first_event=16150604
6 -time_to_last_event=-5740092
6 -sample_ts=373491345415
6 -total_samples=13
6 -avg_delta=13085447
6 -recvd_phy_config_sample_rate=25
6 -random_seed_used=3932138129
6 -num_request_sent=2
6 -first_sample_timestamp=373315493961
6 received event: PASS
6 enter send_memory_log_req cookie: 6
6 exit send_memory_log_req
6 PASS

~ # ssc_drva_test -sensor=gyro -duration=10 sample_rate=10
7 ssc_drva_test version 1.24k
7 ssc_drva_test -sensor=gyro -duration=10 sample_rate=10
diag: Diag_LSM_Init: invoked for pid: 60957 with init_count: 0
diag:successfully connected to socket 5
diag: Diag_LSM_Init: done for pid: 60957 with init_count: 1
7 event_cb attribute event for da_test
7 event_cb attribute event for da_test
7 using da_test name=da_test, suid = [high addeaddeaddeadde, low addeaddeaddeadde
7 enter send_memory_log_req cookie: 7
7 exit send_memory_log_req
7 enter da_test runner. -rumifact=1
7 -time_to_first_event=16151221
7 -time_to_last_event=-5742692
7 -sample_ts=374165926549
7 -total_samples=13
7 -avg_delta=13085202
7 -recvd_phy_config_sample_rate=25
7 -random_seed_used=311751538
7 -num_request_sent=2
7 -first_sample_timestamp=373990075762
7 received event: PASS
7 enter send_memory_log_req cookie: 7
7 exit send_memory_log_req
7 PASS

~ # ssc_drva_test -sensor=mag -duration=10 sample_rate=10
8 ssc_drva_test version 1.24k
8 ssc_drva_test -sensor=mag -duration=10 sample_rate=10
diag: Diag_LSM_Init: invoked for pid: 61075 with init_count: 0
diag:successfully connected to socket 5
diag: Diag_LSM_Init: done for pid: 61075 with init_count: 1
8 event_cb attribute event for da_test
8 event_cb attribute event for da_test
8 using da_test name=da_test, suid = [high addeaddeaddeadde, low addeaddeaddeadde
8 enter send_memory_log_req cookie: 8
8 exit send_memory_log_req
8 enter da_test runner. -rumifact=1
8 -time_to_first_event=2288758
8 -time_to_last_event=-1556825
8 -sample_ts=374830355449
8 -total_samples=99
8 -avg_delta=1900567
8 -recvd_phy_config_sample_rate=10
8 -random_seed_used=976182346
8 -num_request_sent=2
8 -first_sample_timestamp=374640642028
8 received event: PASS
8 enter send_memory_log_req cookie: 8
8 exit send_memory_log_req
8 PASS

jakaskerl

Hi MatteoL
Does see_workhorse -sensor=accel -display_events=1 return measurements from thr accelerometer?

Thanks,
Jaka

MatteoL

Here the output of the command

~ # see_workhorse -sensor=accel -display_events=1
see_workhorse I  1847  1847 BufferAllocator.cpp:100] Using DMA-BUF heap named: system
see_workhorse version 1.63
00:02:44.101 see_workhorse -sensor=accel -display_events=1
begin usta_get_sensor_list
diag: Diag_LSM_Init: invoked for pid: 1847 with init_count: 0
diag:successfully connected to socket 13
diag: Diag_LSM_Init: done for pid: 1847 with init_count: 1
end   usta_get_sensor_list
parse error: invalid "attr_id" : name
parse error: invalid "attr_id" : name
dynamically assign attr_id for: SNS_STD_SENSOR_ATTRID_HLOS_INCOMPATIBLE
por file: /data/sensors/por_spec.txt not found.
00:02:44.297 psalt->begin() took 196 milliseconds
00:02:44.297 get_sensor_suid( diag_sensor)
00:02:44.297 + diag_sensor suid = [high 1292e618203987b5, low 94436fa912689085]
00:02:44.297 lookup: accel
00:02:44.297 suid = [high 61ab5376b4a5c9aa, low 58442ede47acd316]
00:02:44.297 -sensor=accel -rigid_body=display  found suid = 0x61ab5376b4a5c9aa58442ede47acd316
00:02:44.297 is NOT POR.
00:02:44.297 has max_rate: 500 hz
00:02:44.298 begin iteration: 1
00:02:44.298 stream_sensor( accel)
00:02:44.298 + sample_rate: 10.000000 hz
00:02:44.303 config_stream_sensor() complete rc 0
00:02:44.303 sleep(10) seconds

"sns_client_event_msg" : {
 "suid" : {
  "suid_low" : "0x58442ede47acd316",
  "suid_high" : "0x61ab5376b4a5c9aa"
 },
 "events" : [
  {
   "msg_id" : 768,
   "timestamp" : 3158762613,
   "payload" : {
    "sample_rate" : 25.000000,
    "water_mark" : 2,
    "range" : [
     -156.906403,
     156.906403
    ],
    "resolution" : 0.000299,
    "operation_mode" : "NORMAL",
    "active_current" : 240,
    "dri_enabled" : 1,
    "DAE_watermark" : 2
   }
  }
 ]
},
 "Event Counter" : 1,
"Time Elapsed" : 0
}

"sns_client_event_msg" : {
 "suid" : {
  "suid_low" : "0x58442ede47acd316",
  "suid_high" : "0x61ab5376b4a5c9aa"
 },
 "events" : [
  {
   "msg_id" : 1022,
   "timestamp" : 3158763713,
   "payload" : {
    "bias" : [
     0.000000,
     0.000000,
     0.000000
    ],
    "comp_matrix" : [
     1.000000,
     0.000000,
     0.000000,
     0.000000,
     1.000000,
     0.000000,
     0.000000,
     0.000000,
     1.000000
    ],
    "status" : "SNS_STD_SENSOR_SAMPLE_STATUS_ACCURACY_HIGH"
   }
  }
 ]
},
 "Event Counter" : 2,
"Time Elapsed" : 0
}

"sns_client_event_msg" : {
 "suid" : {
  "suid_low" : "0x58442ede47acd316",
  "suid_high" : "0x61ab5376b4a5c9aa"
 },
 "events" : [
  {
   "msg_id" : 130,
   "timestamp" : 3172919522,
   "payload" : {
    "unknown_fields" : 1,
    "field_1" : 6
   }
  }
 ]
},
 "Event Counter" : 3,
"Time Elapsed" : 1
}

MatteoL

@jakaskerl Could you please help me interpret the output?

jakaskerl

MatteoL
this log is strange
parse error: invalid "attr_id" : name dynamically assign attr_id for: SNS_STD_SENSOR_ATTRID_HLOS_INCOMPATIBLE
looks like some parameters are changed. Did you perhaps change the IMU config files manually? I would assume no, but just making sure.

Thanks,
Jaka

MatteoL

As of DepthAI v3.0.0rc4 the issue still persists. How do I check if the IMU configuration has changed? Honestly, I do not know how to edit it

MatteoL

After a factory reset the IMU seems working properly.

@jakaskerl Where can I find some documentation on the IMU features of the Oak 4? For instance, what are the supported quantities? On https://docs.luxonis.com/software-v3/depthai/depthai-components/nodes/imu/, I can only find information about the BNO086 and the BMI270 models.

jakaskerl

MatteoL
Huh, that is strange. Thanks for reporting back. Will notify the team that the issue has been resolved.

The IMU on OAK4 can output accelero and gyro + additionally the magnetometer reading (different chip). We are writing the docs atm - the IMU is still a bit of WIP - to see what works best.

Thanks,
Jaka