Change in device hardware/firmware

awetzel

We have been using the oak-d pro poe in production systems for a bit over a year now and recently we have been seeing some of the devices we've order drop offline and then quickly reconnect. We use the same pipeline and code to connect to these cameras with two other oak-d pro poe's in the same system and this seems to be only happening with this new batch. Was there any changes made to their firmware or hardware in latest batches that would require an update in that from our side?

erik

Hi @awetzel
There were some changes, also to the bootloader. Do cameras have the same bootloader / are you using the same depthai library version for the new cams as well as the old cams? You can check hardware revision with:

import depthai as dai
with dai.Device() as device:
    calib = device.readCalibration()
    print(calib.getEepromData().boardRev)

AFAIK our latest revision is R4M2E4, which was designed in December 2022 (looking at altium designs).
Thoughts?

awetzel

Looks like the R4M2E3 boards were fine, but the R4M2E4 is the one with the problem. They were all running the same bootloader and depthai version. We just updated the bootloader as well and that seemed to diminish the issue but didn't fix it.

Also, with some testing we've found that it resets much more often with larger models than smaller ones. With a nano detection model it was happening around once an hour and much more frequently with a medium one.

erik

I see, thank you for the report @awetzel , let me check with the team wrt HW changes E3 -> E4. From the context it seems it's a power issue - have you tried with different POE injectors/switches, to eliminate the possibility that injector can't provide enough power?

awetzel

erik We have not. I'll see if we can check that out today. Here's the actual output from our logs if this helps:

Aug 5 09:52:09 lpunit0006 Application.out[29929]: [184430104139E9F400] [10.10.1.8] [594.596] [system] [critical] Fatal error. Please report to developers. Log: 'stereoShave' '487'

Aug 5 09:53:41 lpunit0006 Application.out[29929]: [184430104139E9F400] [10.10.1.8] [1722866021.841] [host] [warning] Monitor thread (device: 184430104139E9F400 [10.10.1.8]) - ping was missed, closing the device connection
Aug 5 09:53:43 lpunit0006 Application.out[29929]: Exception (Communication exception - possible device error/misconfiguration. Original message 'Couldn't read data from stream: 'cam' (X_LINK_ERROR)') in EOA Camera thread. Trying again...

erik

@awetzel that's a bit of a different issue - which depthai version are you using? Do you get any crash log when running https://docs.luxonis.com/software/depthai/examples/crash_report/ ?

awetzel

Looks like there were no crash dumps saved unfortunately

erik

@awetzel are you able to provide a minimal repro example, so we could debug this issue (stereoShave 487)?

awetzel

Sorry for the delay. We are working on producing the mre in the next few days and will be sending it over. We consider our code proprietary so I will be sending it over email.