C++ Basalt VIO very slow and inaccurate compared to python in dephai-core

PaulanerAlex

We need to use the c++ implementation of the luxonis dephai-core basalt vio with our oak-d wide. I've tried the example with rerun from dephai 3.5.0 but it delivers only about one frame per second, which presumably leads to the very incorrect state estimation. The python example works great.

According to the note on this documentation page i've also tried using the newest develop branch at 2492f7b but there seems to be an issue with the camera communication. When i build the example (or my own code) it gave me the following in both the python and c++ example of basalt vio:

terminate called after throwing an instance of 'std::runtime_error'
  what():  Device 14442C109125B5D600 error: IMU(2) - IMU invalid settings!: IMU not detected. Your board doesn't have IMU or requires firmware update (Set imu.enableFirmwareUpdate(True) explicitly.). 0, 395
[1]    90096 IOT instruction (core dumped)  ./basalt_vio

Then i tried updating the imu firmware with in the develop branch using the script from dephai-python but this gave me:

IMU type: , firmware version: 0.0.0, embedded firmware version: 0.0.0
Warning! Flashing IMU firmware can potentially soft brick your device and should be done with caution.
Do not unplug your device while the IMU firmware is flashing.
Type 'y' and press enter to proceed, otherwise exits: 
y
IMU FW update status: 0.0%
[2026-03-23 12:24:27.100] [depthai] [error] Device with id 14442C109125B5D600 has crashed. Crash dump logs are stored in: /home/paul/akamav/resources/depthai/cora-main-src-cora_vision-cora_depthai_driver/src/vendor/depthai-core/examples/python/.cache/depthai/crashdumps/8197693a91687a76bac00e592e3682b32a552b7d/crash_dump.json - please report to developers.
Traceback (most recent call last):
  File "/home/paul/akamav/resources/depthai/cora-main-src-cora_vision-cora_depthai_driver/src/vendor/depthai-core/examples/python/update_imu.py", line 31, in <module>
    fwUpdateFinished, percentage = device.getIMUFirmwareUpdateStatus()
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Device already closed or disconnected: Input/output error

So i tried updating the imu firmware using the depthai-python repo, which didn't update because the firmware was already the newest. I suspect there is some other problem with the communication with the camera or imu in the develop version which is why the imu version is shown as 0.0.0 rather than 3.9.9.

OskarSonc

Hey PaulanerAlex - these look like two separate issues.

For the C++ example on v3.5.0, the ~1 FPS result is likely host-side performance, not IMU. Basalt VIO runs on the host, and the C++ sample also streams through the Rerun host node, so a Debug build or slow Rerun path can bottleneck the whole pipeline. I would try a Release build and test once without Rerun. One more thing which python example work great?

For the 2492f7b error, for now use the stable version(v3.5.0)

Also, depthai-python is the v2 repo. Basalt VIO requires DepthAI v3, so using the v2 IMU updater is not a valid check for the v3 VIO stack.

Thanks,
Oskar

PaulanerAlex

OskarSonc

Ok i tried building it for release which made only a minor change in performance but that doesn't fix the issue. The Python example does the same as the c++ example running about 30 times (or something like that) faster. My own C++ code which utilizes only the part of the c++ example without rerun is similary slow. Also, the timestamp of the Transform message is always zero which doesn't seem right.

OskarSonc

PaulanerAlex The Python example does the same as the c++ example running about 30 times (or something like that) faster

Just to make sure I understood correctly: you mean the Python VIO example runs about 30x faster than the equivalent C++ example on the same setup, correct?

If your C++ path is still very slow even without Rerun, then Rerun is probably not the bottleneck. Since the Python v3 and C++ examples use the same basic BasaltVIO backend, the large speed gap points more to the native C++ build/runtime path than to the device or pipeline itself.

Also, the TransformData timestamp always being 0 looks like a real issue on our side, will check with dev team, but this shouldn't be conected to the pipeline being slow ...

Thanks,
Oskar

PaulanerAlex

OskarSonc If your C++ path is still very slow even without Rerun, then Rerun is probably not the bottleneck. Since the Python v3 and C++ examples use the same basic BasaltVIO backend, the large speed gap points more to the native C++ build/runtime path than to the device or pipeline itself.

Ok i actually did build with the release option this time and it worked like a charm. I must have mixed up my build scripts which is why i thought it didn't work last time, sorry for that. Problem fixed, thanks for the help!

OskarSonc Also, the TransformData timestamp always being 0 looks like a real issue on our side, will check with dev team, but this shouldn't be conected to the pipeline being slow ...

Yeah, the time point is still 0 although i build it correctly this Time.

OskarSonc

PaulanerAlex - happy to hear 🙂
Yes, that is not yet implemented...