Persistent "No available devices error" on PoE

jsiic · Apr 12, 2024

jakaskerl I am running the exact same script, so the revision numbers are the same. Or are you referring to something regarding the Oak-D hardware?

jakaskerl · Apr 12, 2024

jsiic
I'm talking about the PCB revision. You can check the number on the PCB.

Thanks,
Jaka

jsiic · Apr 15, 2024

jakaskerl

The devices were tested with this script: luxonis/depthai-experimentsblob/master/gen2-record-replay/record.py

I couldn't tell which one was the PCB revision so I've uploaded the photo here. The top device is the one with the issues.

jakaskerl · Apr 15, 2024

Hi @jsiic
Hmm, only thing that comes to mind are the bootloader differences. Can you update both bootloaders to 0.0.26 please?
https://docs.luxonis.com/projects/api/en/latest/components/bootloader/#danger-zone

Thanks,
Jaka

jsiic · Apr 17, 2024

jakaskerl It's fixed now. Thanks!

YYosuke · May 14, 2024

Hello @jakaskerl ,

I'm facing same problem.
I have 3 sets of OAK-D Pro W PoE and only one of them having the same issue while the others work fine for many times.
Below is what I've done.

updated bootloader to 0.0.27
using latest depthai 2.25.1.0
tried both of static and dynamic ip address config

I found that board rev of the fail camera is "R4M2E3", and the others' is "R4M2E4".
Is there any chances that failure is caused by the difference of board rev?

I attach 3 eeprom json dumps for information.

Mxid:1844301011399D0F00 (fail camera)

  {
      "batchName": "",
      "batchTime": 1676710223,
      "boardConf": "IR-C00M00-00",
      "boardCustom": "",
      "boardName": "NG9097",
      "boardOptions": 3,
      "boardRev": "R4M2E3",
      "cameraData": [
          [
              2,
              {
                  "cameraType": 0,
                  "distortionCoeff": [
                      3.5048553943634033,
                      0.8226749897003174,
                      -0.0004995953058823943,
                      0.0004501539224293083,
                      0.005537083838135004,
                      3.8670060634613037,
                      1.8429312705993652,
                      0.08602307736873627,
                      0.0,
                      0.0,
                      0.0,
                      0.0,
                      0.0,
                      0.0
                  ],
                  "extrinsics": {
                      "rotationMatrix": [
                          [
                              0.9999178051948547,
                              -0.012165973894298077,
                              0.004051733762025833
                          ],
                          [
                              0.012164151296019554,
                              0.9999259114265442,
                              0.0004739780561067164
                          ],
                          [
                              -0.004057200159877539,
                              -0.0004246531752869487,
                              0.9999916553497314
                          ]
                      ],
                      "specTranslation": {
                          "x": 3.75,
                          "y": 0.0,
                          "z": 0.0
                      },
                      "toCameraSocket": 0,
                      "translation": {
                          "x": 3.7470760345458984,
                          "y": 0.002487661549821496,
                          "z": 0.0497138649225235
                      }
                  },
                  "height": 800,
                  "intrinsicMatrix": [
                      [
                          567.6001586914063,
                          0.0,
                          661.7894897460938
                      ],
                      [
                          0.0,
                          567.6001586914063,
                          393.4454345703125
                      ],
                      [
                          0.0,
                          0.0,
                          1.0
                      ]
                  ],
                  "lensPosition": 0,
                  "specHfovDeg": 127.0,
                  "width": 1280
              }
          ],
          [
              1,
              {
                  "cameraType": 0,
                  "distortionCoeff": [
                      0.3509557545185089,
                      -0.057527393102645874,
                      -0.0002248714881716296,
                      -0.0002267207601107657,
                      -0.002574219135567546,
                      0.6851815581321716,
                      -0.02803005278110504,
                      -0.01444168295711279,
                      0.0,
                      0.0,
                      0.0,
                      0.0,
                      0.0,
                      0.0
                  ],
                  "extrinsics": {
                      "rotationMatrix": [
                          [
                              0.9997594952583313,
                              0.021471060812473297,
                              -0.004467455670237541
                          ],
                          [
                              -0.02145538479089737,
                              0.9997636079788208,
                              0.0035275688860565424
                          ],
                          [
                              0.004542140290141106,
                              -0.0034308694303035736,
                              0.9999837875366211
                          ]
                      ],
                      "specTranslation": {
                          "x": -7.5,
                          "y": 0.0,
                          "z": 0.0
                      },
                      "toCameraSocket": 2,
                      "translation": {
                          "x": -7.5123090744018555,
                          "y": 0.08199021220207214,
                          "z": -0.025399429723620415
                      }
                  },
                  "height": 800,
                  "intrinsicMatrix": [
                      [
                          563.6264038085938,
                          0.0,
                          628.09521484375
                      ],
                      [
                          0.0,
                          563.6264038085938,
                          418.28759765625
                      ],
                      [
                          0.0,
                          0.0,
                          1.0
                      ]
                  ],
                  "lensPosition": 0,
                  "specHfovDeg": 127.0,
                  "width": 1280
              }
          ],
          [
              0,
              {
                  "cameraType": 0,
                  "distortionCoeff": [
                      -2.5844860076904297,
                      53.71508026123047,
                      0.0006256784545257688,
                      0.00015372849884442985,
                      8.364314079284668,
                      -2.445420503616333,
                      53.30659103393555,
                      20.663192749023438,
                      0.0,
                      0.0,
                      0.0,
                      0.0,
                      0.0,
                      0.0
                  ],
                  "extrinsics": {
                      "rotationMatrix": [
                          [
                              0.0,
                              0.0,
                              0.0
                          ],
                          [
                              0.0,
                              0.0,
                              0.0
                          ],
                          [
                              0.0,
                              0.0,
                              0.0
                          ]
                      ],
                      "specTranslation": {
                          "x": -0.0,
                          "y": -0.0,
                          "z": -0.0
                      },
                      "toCameraSocket": -1,
                      "translation": {
                          "x": 0.0,
                          "y": 0.0,
                          "z": 0.0
                      }
                  },
                  "height": 2160,
                  "intrinsicMatrix": [
                      [
                          2298.441650390625,
                          0.0,
                          1903.8328857421875
                      ],
                      [
                          0.0,
                          2298.441650390625,
                          1074.5648193359375
                      ],
                      [
                          0.0,
                          0.0,
                          1.0
                      ]
                  ],
                  "lensPosition": 0,
                  "specHfovDeg": 108.0,
                  "width": 3840
              }
          ]
      ],
      "deviceName": "",
      "hardwareConf": "F1-FV01-BC001",
      "housingExtrinsics": {
          "rotationMatrix": [],
          "specTranslation": {
              "x": 0.0,
              "y": 0.0,
              "z": 0.0
          },
          "toCameraSocket": -1,
          "translation": {
              "x": 0.0,
              "y": 0.0,
              "z": 0.0
          }
      },
      "imuExtrinsics": {
          "rotationMatrix": [
              [
                  0.0,
                  0.0,
                  0.0
              ],
              [
                  0.0,
                  0.0,
                  0.0
              ],
              [
                  0.0,
                  0.0,
                  0.0
              ]
          ],
          "specTranslation": {
              "x": 0.0,
              "y": 0.0,
              "z": 0.0
          },
          "toCameraSocket": -1,
          "translation": {
              "x": 0.0,
              "y": 0.0,
              "z": 0.0
          }
      },
      "miscellaneousData": [],
      "productName": "OAK-D-PRO-W-POE",
      "stereoEnableDistortionCorrection": false,
      "stereoRectificationData": {
          "leftCameraSocket": 1,
          "rectifiedRotationLeft": [
              [
                  0.999943733215332,
                  0.010547235608100891,
                  -0.0011248956434428692
              ],
              [
                  -0.010545255616307259,
                  0.9999428391456604,
                  0.001751492964103818
              ],
              [
                  0.0011433048639446497,
                  -0.0017395321046933532,
                  0.9999978542327881
              ]
          ],
          "rectifiedRotationRight": [
              [
                  0.999934732913971,
                  -0.010913403704762459,
                  0.0033808210864663124
              ],
              [
                  0.010919288732111454,
                  0.9999389052391052,
                  -0.0017271068645641208
              ],
              [
                  -0.0033617657609283924,
                  0.0017639102879911661,
                  0.9999927878379822
              ]
          ],
          "rightCameraSocket": 2
      },
      "stereoUseSpecTranslation": true,
      "version": 7,
      "verticalCameraSocket": -1
  }

Mxid:18443010E15E960F00 (working fine)

  {
      "batchName": "",
      "batchTime": 1681371289,
      "boardConf": "IR-C00M00-00",
      "boardCustom": "",
      "boardName": "NG9097",
      "boardOptions": 7,
      "boardRev": "R4M2E4",
      "cameraData": [
          [
              2,
              {
                  "cameraType": 0,
                  "distortionCoeff": [
                      2.2474868297576904,
                      0.6692854762077332,
                      0.00046299261157400906,
                      -9.837499965215102e-05,
                      0.018597930669784546,
                      2.591661214828491,
                      1.3092966079711914,
                      0.11096686869859695,
                      0.0,
                      0.0,
                      0.0,
                      0.0,
                      0.0,
                      0.0
                  ],
                  "extrinsics": {
                      "rotationMatrix": [
                          [
                              0.9999992847442627,
                              -0.0009960209717974067,
                              -0.0006217662594281137
                          ],
                          [
                              0.0010001721093431115,
                              0.9999769926071167,
                              0.006712025962769985
                          ],
                          [
                              0.0006150666158646345,
                              -0.006712642963975668,
                              0.9999772906303406
                          ]
                      ],
                      "specTranslation": {
                          "x": 3.75,
                          "y": 0.0,
                          "z": 0.0
                      },
                      "toCameraSocket": 0,
                      "translation": {
                          "x": 3.733102321624756,
                          "y": 0.0015009926864877343,
                          "z": -0.016050558537244797
                      }
                  },
                  "height": 800,
                  "intrinsicMatrix": [
                      [
                          574.37451171875,
                          0.0,
                          655.1437377929688
                      ],
                      [
                          0.0,
                          574.37451171875,
                          409.7769470214844
                      ],
                      [
                          0.0,
                          0.0,
                          1.0
                      ]
                  ],
                  "lensPosition": 0,
                  "specHfovDeg": 127.0,
                  "width": 1280
              }
          ],
          [
              1,
              {
                  "cameraType": 0,
                  "distortionCoeff": [
                      0.9543554186820984,
                      -0.15627902746200562,
                      -0.0005260875332169235,
                      9.12299074116163e-05,
                      -0.02378588356077671,
                      1.298899531364441,
                      0.048646025359630585,
                      -0.08654411882162094,
                      0.0,
                      0.0,
                      0.0,
                      0.0,
                      0.0,
                      0.0
                  ],
                  "extrinsics": {
                      "rotationMatrix": [
                          [
                              0.9999978542327881,
                              0.0017117680981755257,
                              -0.0011842198437079787
                          ],
                          [
                              -0.00170683185569942,
                              0.999989926815033,
                              0.004156853538006544
                          ],
                          [
                              0.0011913235066458583,
                              -0.004154823254793882,
                              0.9999906420707703
                          ]
                      ],
                      "specTranslation": {
                          "x": -7.5,
                          "y": 0.0,
                          "z": 0.0
                      },
                      "toCameraSocket": 2,
                      "translation": {
                          "x": -7.44867467880249,
                          "y": -0.04259622469544411,
                          "z": -0.17102035880088806
                      }
                  },
                  "height": 800,
                  "intrinsicMatrix": [
                      [
                          576.297119140625,
                          0.0,
                          624.130859375
                      ],
                      [
                          0.0,
                          576.297119140625,
                          407.7364196777344
                      ],
                      [
                          0.0,
                          0.0,
                          1.0
                      ]
                  ],
                  "lensPosition": 0,
                  "specHfovDeg": 127.0,
                  "width": 1280
              }
          ],
          [
              0,
              {
                  "cameraType": 0,
                  "distortionCoeff": [
                      -0.5994359254837036,
                      44.40129852294922,
                      0.00023757426242809743,
                      -0.0003660324146039784,
                      6.122718334197998,
                      -0.49412497878074646,
                      44.577911376953125,
                      15.948250770568848,
                      0.0,
                      0.0,
                      0.0,
                      0.0,
                      0.0,
                      0.0
                  ],
                  "extrinsics": {
                      "rotationMatrix": [
                          [
                              0.0,
                              0.0,
                              0.0
                          ],
                          [
                              0.0,
                              0.0,
                              0.0
                          ],
                          [
                              0.0,
                              0.0,
                              0.0
                          ]
                      ],
                      "specTranslation": {
                          "x": -0.0,
                          "y": -0.0,
                          "z": -0.0
                      },
                      "toCameraSocket": -1,
                      "translation": {
                          "x": 0.0,
                          "y": 0.0,
                          "z": 0.0
                      }
                  },
                  "height": 2160,
                  "intrinsicMatrix": [
                      [
                          2319.90283203125,
                          0.0,
                          1968.7259521484375
                      ],
                      [
                          0.0,
                          2319.90283203125,
                          1056.239013671875
                      ],
                      [
                          0.0,
                          0.0,
                          1.0
                      ]
                  ],
                  "lensPosition": 0,
                  "specHfovDeg": 108.0,
                  "width": 3840
              }
          ]
      ],
      "deviceName": "",
      "hardwareConf": "F1-FV01-BC001",
      "housingExtrinsics": {
          "rotationMatrix": [],
          "specTranslation": {
              "x": 0.0,
              "y": 0.0,
              "z": 0.0
          },
          "toCameraSocket": -1,
          "translation": {
              "x": 0.0,
              "y": 0.0,
              "z": 0.0
          }
      },
      "imuExtrinsics": {
          "rotationMatrix": [
              [
                  0.0,
                  0.0,
                  0.0
              ],
              [
                  0.0,
                  0.0,
                  0.0
              ],
              [
                  0.0,
                  0.0,
                  0.0
              ]
          ],
          "specTranslation": {
              "x": 0.0,
              "y": 0.0,
              "z": 0.0
          },
          "toCameraSocket": -1,
          "translation": {
              "x": 0.0,
              "y": 0.0,
              "z": 0.0
          }
      },
      "miscellaneousData": [],
      "productName": "OAK-D-PRO-W-POE",
      "stereoEnableDistortionCorrection": false,
      "stereoRectificationData": {
          "leftCameraSocket": 1,
          "rectifiedRotationLeft": [
              [
                  0.9997355937957764,
                  0.007332895416766405,
                  0.021793076768517494
              ],
              [
                  -0.007378414273262024,
                  0.9999707341194153,
                  0.0020090106409043074
              ],
              [
                  -0.02177770622074604,
                  -0.002169277984648943,
                  0.9997605085372925
              ]
          ],
          "rectifiedRotationRight": [
              [
                  0.9997202157974243,
                  0.00571703165769577,
                  0.022953413426876068
              ],
              [
                  -0.005669059231877327,
                  0.9999815821647644,
                  -0.0021544997580349445
              ],
              [
                  -0.022965308278799057,
                  0.0020237728022038937,
                  0.9997342228889465
              ]
          ],
          "rightCameraSocket": 2
      },
      "stereoUseSpecTranslation": true,
      "version": 7,
      "verticalCameraSocket": -1
  }

Mxid:18443010419A8E0E00 (working fine)

  {
      "batchName": "",
      "batchTime": 1681373872,
      "boardConf": "IR-C00M00-00",
      "boardCustom": "",
      "boardName": "NG9097",
      "boardOptions": 7,
      "boardRev": "R4M2E4",
      "cameraData": [
          [
              2,
              {
                  "cameraType": 0,
                  "distortionCoeff": [
                      1.1420116424560547,
                      -0.12159973382949829,
                      0.00016930163837969303,
                      1.48746103150188e-05,
                      -0.022195111960172653,
                      1.4924085140228271,
                      0.13922181725502014,
                      -0.08151122182607651,
                      0.0,
                      0.0,
                      0.0,
                      0.0,
                      0.0,
                      0.0
                  ],
                  "extrinsics": {
                      "rotationMatrix": [
                          [
                              0.9999328255653381,
                              0.002885618479922414,
                              0.011227047070860863
                          ],
                          [
                              -0.0028396674897521734,
                              0.9999875426292419,
                              -0.0041066864505410194
                          ],
                          [
                              -0.011238757520914078,
                              0.004074529279023409,
                              0.9999285340309143
                          ]
                      ],
                      "specTranslation": {
                          "x": 3.75,
                          "y": 0.0,
                          "z": 0.0
                      },
                      "toCameraSocket": 0,
                      "translation": {
                          "x": 3.7847630977630615,
                          "y": -0.0011661133030429482,
                          "z": -0.10201772302389145
                      }
                  },
                  "height": 800,
                  "intrinsicMatrix": [
                      [
                          570.9481201171875,
                          0.0,
                          626.4712524414063
                      ],
                      [
                          0.0,
                          570.9481201171875,
                          405.6054992675781
                      ],
                      [
                          0.0,
                          0.0,
                          1.0
                      ]
                  ],
                  "lensPosition": 0,
                  "specHfovDeg": 127.0,
                  "width": 1280
              }
          ],
          [
              1,
              {
                  "cameraType": 0,
                  "distortionCoeff": [
                      0.527733564376831,
                      -0.02382327988743782,
                      0.00020704128837678581,
                      0.00022398377768695354,
                      -0.0026558975223451853,
                      0.8656452298164368,
                      0.0608961321413517,
                      -0.01260523684322834,
                      0.0,
                      0.0,
                      0.0,
                      0.0,
                      0.0,
                      0.0
                  ],
                  "extrinsics": {
                      "rotationMatrix": [
                          [
                              0.9999582171440125,
                              -0.0007228651084005833,
                              -0.009114222601056099
                          ],
                          [
                              0.0007744028698652983,
                              0.9999837279319763,
                              0.005652391351759434
                          ],
                          [
                              0.009109987877309322,
                              -0.005659212823957205,
                              0.9999424815177917
                          ]
                      ],
                      "specTranslation": {
                          "x": -7.5,
                          "y": 0.0,
                          "z": 0.0
                      },
                      "toCameraSocket": 2,
                      "translation": {
                          "x": -7.4916276931762695,
                          "y": -0.048582371324300766,
                          "z": 0.004791563376784325
                      }
                  },
                  "height": 800,
                  "intrinsicMatrix": [
                      [
                          568.4283447265625,
                          0.0,
                          634.8511352539063
                      ],
                      [
                          0.0,
                          568.4283447265625,
                          419.8392639160156
                      ],
                      [
                          0.0,
                          0.0,
                          1.0
                      ]
                  ],
                  "lensPosition": 0,
                  "specHfovDeg": 127.0,
                  "width": 1280
              }
          ],
          [
              0,
              {
                  "cameraType": 0,
                  "distortionCoeff": [
                      -2.7425057888031006,
                      22.622543334960938,
                      0.00010579380614217371,
                      -0.0005987813929095864,
                      1.3354629278182983,
                      -2.6044418811798096,
                      22.41253089904785,
                      5.934658527374268,
                      0.0,
                      0.0,
                      0.0,
                      0.0,
                      0.0,
                      0.0
                  ],
                  "extrinsics": {
                      "rotationMatrix": [
                          [
                              0.0,
                              0.0,
                              0.0
                          ],
                          [
                              0.0,
                              0.0,
                              0.0
                          ],
                          [
                              0.0,
                              0.0,
                              0.0
                          ]
                      ],
                      "specTranslation": {
                          "x": -0.0,
                          "y": -0.0,
                          "z": -0.0
                      },
                      "toCameraSocket": -1,
                      "translation": {
                          "x": 0.0,
                          "y": 0.0,
                          "z": 0.0
                      }
                  },
                  "height": 2160,
                  "intrinsicMatrix": [
                      [
                          2288.113037109375,
                          0.0,
                          1938.6259765625
                      ],
                      [
                          0.0,
                          2288.113037109375,
                          1088.8843994140625
                      ],
                      [
                          0.0,
                          0.0,
                          1.0
                      ]
                  ],
                  "lensPosition": 0,
                  "specHfovDeg": 108.0,
                  "width": 3840
              }
          ]
      ],
      "deviceName": "",
      "hardwareConf": "F1-FV01-BC001",
      "housingExtrinsics": {
          "rotationMatrix": [],
          "specTranslation": {
              "x": 0.0,
              "y": 0.0,
              "z": 0.0
          },
          "toCameraSocket": -1,
          "translation": {
              "x": 0.0,
              "y": 0.0,
              "z": 0.0
          }
      },
      "imuExtrinsics": {
          "rotationMatrix": [
              [
                  0.0,
                  0.0,
                  0.0
              ],
              [
                  0.0,
                  0.0,
                  0.0
              ],
              [
                  0.0,
                  0.0,
                  0.0
              ]
          ],
          "specTranslation": {
              "x": 0.0,
              "y": 0.0,
              "z": 0.0
          },
          "toCameraSocket": -1,
          "translation": {
              "x": 0.0,
              "y": 0.0,
              "z": 0.0
          }
      },
      "miscellaneousData": [],
      "productName": "OAK-D-PRO-W-POE",
      "stereoEnableDistortionCorrection": false,
      "stereoRectificationData": {
          "leftCameraSocket": 1,
          "rectifiedRotationLeft": [
              [
                  0.9999361634254456,
                  0.005765415262430906,
                  -0.009716913104057312
              ],
              [
                  -0.005737767554819584,
                  0.9999794363975525,
                  0.0028707573655992746
              ],
              [
                  0.009733264334499836,
                  -0.002814820734784007,
                  0.9999486804008484
              ]
          ],
          "rectifiedRotationRight": [
              [
                  0.99997878074646,
                  0.006484751123934984,
                  -0.0006395755335688591
              ],
              [
                  -0.006486542988568544,
                  0.9999749064445496,
                  -0.0028407752979546785
              ],
              [
                  0.000621137791313231,
                  0.002844863571226597,
                  0.999995768070221
              ]
          ],
          "rightCameraSocket": 2
      },
      "stereoUseSpecTranslation": true,
      "version": 7,
      "verticalCameraSocket": -1
  }

Could you help me to find solution?

Thank you.

jakaskerl · May 14, 2024

Hi @Yosuke
How did you get the calibration if you can't connect to the device?

Could be that the calibration file is causing this issue.

Try loading the working config to the non-working camera. Script.
Have you tried reflashing the bootloader and selecting NETWORK mode?

Thanks,
Jaka

YYosuke · May 14, 2024

Hi @jakaskerl , I really appreciate your prompt reply.

My camera has the same symptons as jsiic's; the camera doesn't respond anymore after using it few times, and it continues until reconnecting LAN cable.

I coud get eeprom dumps before being hunged it up because it surely works after reconnection.

Thank you for proposal, I'll report if it works.

Best wishes,
Yosuke

YYosuke · May 20, 2024

~~I tried both of your proposals, and now it seems fixed after reflashing bootloader with selecting NETWORK mode.~~

~~Thank you so much!~~

it happens again after tens times of reconnection.
I have no idea anymore…

Regards,
Yosuke

jakaskerl · May 20, 2024

Hi @Yosuke
Can you run the script with DEPTHAI_DEBUG=1 python3 script.py and paste the green part.

once during normal operation (when the device boots up fine on the next try)
once during the last working try (so the device doesn't boot on the next try

Thanks,
Jaka

Iivan_sc · Jul 15, 2024

Hello @jakaskerl

I am having the same issue described above on one of my units. Device stops responding and need to unplug cable to get it back. Reboot of PC is not enough.

OAK-D W, powered by industrial PC POE. Bootloader version is 0.0.28. Had the same issues on 0.0.26.

I am developing some ROS Noetic code, and sometimes the device stops responding once I want to try my code again. I try to wait after stopping the script to get the device back into some idle state, as mentioned above, but I still face the issue eventually.

How can I debug this issue?

jakaskerl · Jul 16, 2024

ivan_sc
I'd suggest doing the same thing as I wrote one reply above. Run it with the ROS node. What is the faulty device revision (and the working ones)? - Can be seen on the PCB: something like R3M0E3 or similar.

Thanks,
Jaka