RGBD examples not working with OAK-D lite camera

flash

I tried to run these two examples from the docs:

I've installed depthai v3 (following these instructions https://docs.luxonis.com/software-v3/depthai/ ) and have an OAK-D lite camera connected via USB.

Running the code examples in python gives the following error:

[194430108128D32600] [1.1] [16.440] [Camera(2)] [error] Invalid configuration or input image - skipping frame

[194430108128D32600] [1.1] [16.472] [Camera(2)] [error] Not possible to create warp params. Error: WARP_SWCH_ERR_CACHE_TO_SMALL

Could you please help me to fix this?

OskarSonc

Hey flash ,

I tested the examples on a similar RVC2 device and they’re working fine.
Which version of depthai are you using? Could you try updating to the latest version and see if the issue persists?

Best,

Oskar

flash

I'm using:

depthai 3.0.0rce installed in a conda environment

Ubuntu 22.04

Camera is OAK-D Lite fixed focus

Local host shows device not detected even though it is plugged in:

OskarSonc

@flash,

Did you follow our USB deployment guide? On Linux there are a few extra setup steps (like installing udev rules) to get the camera detected properly:
https://docs.luxonis.com/hardware/platform/deploy/usb-deployment-guide/

Once you've gone through that, I recommend testing the camera with the OAK Viewer to confirm everything is working:
https://docs.luxonis.com/software-v3/depthai/tools/oak-viewer/

Thanks
Oskar

flash

Thanks, after installing the udev rules, uninstalling depthai-viewer, and installing the OAK-viewer .deb the original error is still there. Restarting the computer did not help either.

The camera is still not working for both RGBD examples.

Other examples, like capturing a still image (https://docs.luxonis.com/software-v3/depthai/examples/camera/camera_still_max_res/) work fine.

OAK-viewer works as shown below:

flash

I have two OAK-D Lite cameras. Why does the example work on one camera and not the other? Both cameras are new, kept in the box and not used much apart from running some examples a few months back.

Do I need to run some kind of firmware update on the faulty camera?

The error message seems to suggest memory issues on camera 2 (not sure what that means in Luxonis context).

Also, is there a way to check the sensors on both cameras, like if both have an internal IMU?

OskarSonc

Hey @flash,

Sorry for the wait, had to check with the team …
It’s likely the issue is with the calibration on the camera that’s failing.
Could you run this script on both cameras (one at a time) and send us the generated files?

import depthai as dai
from pathlib import Path
import json

def to_obj(x):
    """DepthAI may return a dict or a JSON string depending on version."""
    return x if isinstance(x, dict) else json.loads(x)

with dai.Device() as dev:
    devid = dev.getDeviceId()

    # Effective (user) calib = factory + any user overrides
    eff = to_obj(dev.readCalibration2().eepromToJson())
    # Factory-only calib (read-only from manufacturing)
    fac = to_obj(dev.readFactoryCalibration().eepromToJson())

    bundle = {
        "meta": {
            "deviceId": devid,
            "depthai_version": getattr(dai, "__version__", "unknown"),
        },
        "USER CALIBRATION": eff,
        "FACTORY CALIBRATION":   fac,
    }

    out = Path(f"{devid}_calibration.json")
    out.write_text(json.dumps(bundle, indent=2))
    print(f"Wrote {out.resolve()} "
          f"(effective size: {len(json.dumps(eff))} bytes, "
          f"factory size: {len(json.dumps(fac))} bytes)")

Thanks,
Oskar

flash

The output from the first camera (the one where the examples don't work, yielding the errors above) was generated. However, this forum does not allow uploading of .json files. You can download it from here.

The code failed to run on the second camera (the one where the RGBD examples worked). The error is:

File "/home/main.py", line 15, in <module>

fac = to_obj(dev.readFactoryCalibration().eepromToJson())

^^^^^^^^^^^^^^^^^^^^^^^^^^^^

depthai.EepromError: No or invalid EEPROM configuration flashed, error: EEPROM_INVALID_DATA

OskarSonc

Hey @flash,

Thanks for sending the calibration dump from the camera that’s failing.

Could you please run the updated calibration bundle script below on your working OAK-D Lite as well?

import depthai as dai, json
from pathlib import Path

def to_obj(x):  # dict vs str across versions
    return x if isinstance(x, dict) else json.loads(x)

with dai.Device() as dev:
    devid = dev.getDeviceId()
    eff = to_obj(dev.readCalibration2().eepromToJson())

    fac = None
    try:
        fac = to_obj(dev.readFactoryCalibration().eepromToJson())
    except dai.EepromError:
        fac = None  # factory partition not present/invalid is OK on some units

    bundle = {
        "meta": {"deviceId": devid, "depthai_version": getattr(dai, "__version__", "unknown")},
        "USER CALIBRATION": eff,
        "FACTORY CALIBRATION": fac,
    }
    out = Path(f"{devid}_calibration_bundle.json")
    out.write_text(json.dumps(bundle, indent=2))
    print(f"Wrote {out}  "
          f"(user bytes={len(json.dumps(eff))}, "
          f"factory bytes={len(json.dumps(fac)) if fac is not None else 0})")

Once we have the dump from the working camera too, we can compare the two and confirm the next steps.

Thanks!
Oskar

flash

Do some OAK-D Lite cameras not store calibration data in EEPROM? Where is it otherwise stored?

You can download the generated file for camera 2 here.

OskarSonc

Hey @flash,

Thanks for sending both dumps — they’re quite different, which points to a calibration issue on the camera that fails. This can cause the WARP_SWCH_ERR_CACHE_TO_SMALL errors in the RGB-D examples.

Also, to answer your earlier question: yes, some OAK-D Lite units don’t have a separate factory calibration partition in EEPROM. In those cases, the device only has a user/effective calibration area in EEPROM, which is what DepthAI uses at runtime.

You can try flashing the working camera’s calibration onto the failing one by following our guide here:
👉 https://docs.luxonis.com/software/depthai/examples/calibration_load/

Note: calibration flashing is currently only available with DepthAI v2, but once it’s done you can switch back and use the camera normally with DepthAI v3.

Let us know how it goes after the flash!
Thanks

flash

Looking at the calibration files and also running the IMU example I concluded that:

1. camera 1 (RGBD examples not working) has a built-in IMU

2. camera 2 (RGBD examples working) does not have an IMU since the IMU example code outputs:

RuntimeError: IMU(0) - IMU invalid settings!: IMU not detected. Your board doesn't have IMU or requires firmware update (Set imu.enableFirmwareUpdate(True) explicitly.). 0, 395

Is there a definitive test I should run to confirm the missing IMU for camera 1 (I didn’t buy the camera off Kickstarter)?

Could the IMU be a factor for the original error?

I'm concerned about flashing camera 2’s calibration parameters onto camera 1 since this will effectively erase the IMU calibration parameters on camera 1. I would like to be able to access and use the IMU data for future projects.

How should I proceed?

OskarSonc

Hey @flash,

Here’s a quick code snippet to check if your device has an IMU:

import depthai as dai
with dai.Device() as dev:
    t = dev.getConnectedIMU()
    print("IMU type:", t)  # None/IMUType.NONE => no IMU present

I don’t think the IMU is related to the RGB-D error.

From the calibration dump, the failing camera’s RGB <-> stereo extrinsics look invalid, which explains the warp error. To confirm safely (without touching IMU calibration), please try the temporary load approach from our v2 example:
https://docs.luxonis.com/software/depthai/examples/calibration_load/

flash

@OskarSonc Sorry for the late reply, I'm still interested in solving this camera issue.

I can confirm that only one camera has an IMU and that is the camera where the RGBD example is not working.

Below is the output for the IMU detection and a section from the associated calibration json file for each camera.

camera 1 (RGBD examples not working) : IMU type: BMI270

"imuExtrinsics": {
"rotationMatrix": [
[
1.0,
0.0,
0.0
],
[
0.0,
1.0,
0.0
],
[
0.0,
0.0,
1.0
]
],
"specTranslation": {
"x": -3.1500000953674316,
"y": 0.6000000238418579,
"z": 0.0
},

camera 2 (RGBD examples working): IMU type:

"imuExtrinsics": {
"rotationMatrix": [
[
0.0,
0.0,
0.0
],
[
0.0,
0.0,
0.0
],
[
0.0,
0.0,
0.0
]
],
"specTranslation": {
"x": 0.0,
"y": 0.0,
"z": 0.0
},

Next steps:

I have set up a depthai v2 workspace, as you suggested, to try out the temporary load approach. The calibration_load.py script provides as input a ../models/depthai_calib.json file. Is this the json file I should load onto the camera?

Or should I load the json file from the working camera, generated using the script you provided, linked above on 13 Aug?

Or is there a calibration json file you can provide for me to load (that includes IMU calibration params)?

After successfully running calibrationload.py, should I just try run the failing RGBD example script again on the camera without unplugging it? I assume the script only updates the params temporarily and not the EEPROM but I'm not sure if this persists once the script ends or if I'll need to incorporate the RGBD code into calibration_load.py?

OskarSonc

Hey flash - could you provide me with the device MXID, that way I can check the factory calibration

flash

I ran the following code:

import depthai as dai
with dai.Device() as dev:
t = dev.getDeviceId()
print("Device MXID:", t)

camera 1 (RGBD examples not working):

Device MXID: 194430108128D32600

camera 2 (RGBD examples working):

Device MXID: 18443010A1F70C1300

OskarSonc

So few things we will do:

update to the latest depthai v3 version (3.2.1)
plug in camera 2 and ran IMU example but add this line before pipeline.start():
imu.enableFirmwareUpdate(True), then try and run the updated IMU example
for camera 1, are any examples working? Try running depthai-docre/utilites/cam.test.py
with camera 1 ran this line in powershell: $env:DEPTHAI_LEVEL="debug" then ran rgbd.py example and send me the terminal.
also did you buy the cameras from our store and if you did when did you order?

flash

I have updated to depthai v 3.2.1 (pip list: depthai 3.2.1)
Running the IMU example on camera 2, after adding imu.enableFirmwareUpdate(True) produced the following output:

[18443010A1F70C1300] [1.1] [1.553] [IMU(0)] [warning] IMU 'enableFirmwareUpdate' is deprecated! Use 'device.startIMUFirmwareUpdate'

[18443010A1F70C1300] [1.1] [1.557] [IMU(0)] [warning] Performing IMU firmware update, do NOT disconnect the device.

[18443010A1F70C1300] [1.1] [1.680] [IMU(0)] [error] IMU firmware update failed. Your board likely doesn't have IMU!
Yes, other examples do work for camera 1 such as the feature tracker: https://docs.luxonis.com/software-v3/depthai/examples/feature_tracker/feature_tracker/

I tried to run the cam_test.py file found on Github https://github.com/luxonis/depthai-python/blob/main/utilities/cam_test.py but it is complaining about the stress_test module. I couldn’t locate this module in conda or pip, where can I find it?
In terminal, I set the environment variable: export DEPTHAI_LEVEL="debug" then ran the RGBD code https://docs.luxonis.com/software-v3/depthai/examples/rgbd/rgbd/ . The terminal output can be found here.
One camera was purchased from the store in March 2023, and the other camera I received at ROSCon in October 2023.