Hi,

For the OAK-1 POE I receive the following error usually after 30 min to 12 hours of script execution.

Traceback (most recent call last):
File "runoak.py", line 118, in inference
inRgb = qRgb.tryGet()
RuntimeError: Communication exception - possible device error/misconfiguration. Original message 'Couldn't read data from stream: 'rgb' (X_LINK_ERROR)'

Is there a recommended approach to dealing with this error, such as restarting the pipeline as shown below?

device = dai.Device(pipeline, device_info)
device.startPipeline()

The documentation https://docs.luxonis.com/en/latest/pages/tutorials/getting-started-with-poe/ mentions insufficient power supply as a possible culprit - is there a decent way to troubleshoot this?

The pipeline I am using is below:

pipeline = dai.Pipeline()

# Define sources and outputs
cam = pipeline.createColorCamera()
nn = pipeline.createMobileNetDetectionNetwork()

xoutFrame = pipeline.createXLinkOut()
xoutNN = pipeline.createXLinkOut()

xoutFrame.setStreamName("rgb")
xoutNN.setStreamName("nn")

# Properties
cam.setPreviewKeepAspectRatio(True)
cam.setPreviewSize(300, 300)
cam.setInterleaved(False)
cam.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1080_P)
cam.setImageOrientation(dai.CameraImageOrientation.ROTATE_180_DEG)

# Define a neural network
nn.setBlobPath(self.nnPath)
nn.setConfidenceThreshold(0.7)
nn.setNumInferenceThreads(2)
nn.input.setBlocking(False)

# Linking
cam.preview.link(xoutFrame.input)
cam.preview.link(nn.input)
nn.out.link(xoutNN.input)

The queues I have set are:

qRgb = device.getOutputQueue(name="rgb", maxSize=4, blocking=False)
qDet = device.getOutputQueue(name="nn", maxSize=4, blocking=False)
  • erik replied to this.

    Hello gbanuru,
    sorry for the inconvenience. We did some long-term testing (days) with some basic pipelines and AFAIK @jebba from discord had it running for few days as well (asking him again to double-check). Could you share with us what POE injector/switch are you using?
    Thanks, Erik

      I had one camera run in simple playback, non-recording mode for 10+ days without issue until I stopped it. The camera was plugged into a Netgear GS308P (inexpensive 8-port, 4 of which are PoE). This is the script I ran for that, taken from the Luxonis provided samples, fwiw:

      #!/usr/bin/env python3
      
      import cv2
      import depthai as dai
      
      # aicam6
      aicamip="192.168.1.6"
      
      found, device_info = dai.Device.getDeviceByMxId(aicamip)
      print(aicamip)
      
      print(device_info)
      
      if not found:
          raise RuntimeError("Device not found!")
      
      # Create pipeline
      pipeline = dai.Pipeline()
      
      # Define source and output
      camRgb = pipeline.createColorCamera()
      xoutVideo = pipeline.createXLinkOut()
      
      xoutVideo.setStreamName("video")
      
      # Properties
      camRgb.setBoardSocket(dai.CameraBoardSocket.RGB)
      camRgb.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1080_P)
      camRgb.setVideoSize(1920, 1080)
      
      xoutVideo.input.setBlocking(False)
      xoutVideo.input.setQueueSize(1)
      
      # Linking
      camRgb.video.link(xoutVideo.input)
      
      # Connect to device and start pipeline
      with dai.Device(pipeline) as device:
      
          video = device.getOutputQueue(name="video", maxSize=1, blocking=False)
      
          while True:
              videoIn = video.get()
      
              # Get BGR frame from NV12 encoded video frame to show with opencv
              # Visualizing the frame on slower hosts might have overhead
              cv2.imshow("video", videoIn.getCvFrame())
      
              if cv2.waitKey(1) == ord('q'):
                  break

      I do have some other cameras that I have been recording from, using this:

      
      #!/usr/bin/env python3
      
      import depthai as dai
      
      aicamip="192.168.1.6"
      
      found, device_info = dai.Device.getDeviceByMxId(aicamip)
      print(aicamip)
      
      if not found:
          raise RuntimeError("Device not found!")
      
      # Create pipeline
      pipeline = dai.Pipeline()
      
      # Define sources and output
      camRgb = pipeline.createColorCamera()
      videoEnc = pipeline.createVideoEncoder()
      xout = pipeline.createXLinkOut()
      
      xout.setStreamName('h265')
      
      # Properties
      camRgb.setBoardSocket(dai.CameraBoardSocket.RGB)
      camRgb.setResolution(dai.ColorCameraProperties.SensorResolution.THE_4_K)
      videoEnc.setDefaultProfilePreset(3840, 2160, 30, dai.VideoEncoderProperties.Profile.H265_MAIN)
      
      # Linking
      camRgb.video.link(videoEnc.input)
      videoEnc.bitstream.link(xout.input)
      
      # Connect to device and start pipeline
      with dai.Device(pipeline, device_info) as device:
      
          # Output queue will be used to get the encoded data from the output defined above
          q = device.getOutputQueue(name="h265", maxSize=30, blocking=True)
      
          # The .h265 file is a raw stream file (not playable yet)
          with open('ai-cam6-deer.h265', 'wb') as videoFile:
              print("Press Ctrl+C to stop encoding...")
              try:
                  while True:
                      h265Packet = q.get()  # Blocking call, will wait until a new data has arrived
                      h265Packet.getData().tofile(videoFile)  # Appends the packet data to the opened file
              except KeyboardInterrupt:
                  # Keyboard interrupt (Ctrl + C) detected
                  pass
      
          print("To view the encoded data, convert the stream file (.h265) into a video file (.mp4) using a command below:")
          print("ffmpeg -framerate 30 -i video.h265 -c copy video.mp4")

      This sometimes terminates for unknown reasons, but typically can run for hours/days. Causes for outages can include power or network. You could leave a ping running to the device and another device similarly situated on the network and see if there are issues. Also, have your PoE switch on UPS/battery.

      Happy hacking!

      Hi erik

      The POE Switch used:
      NETGEAR 8-Port Gigabit Ethernet Unmanaged PoE Switch (GS108PP) - with 8 x PoE+ @ 123W

      a year later

      I'm a little late but I wanted to know your response to the 'recommended approach to dealing with this error' question from the OP?

      I'm using a similar switch, GS108P (one 'P') powering a RasPi 4B with the PoE+ hat along with three OAK-1-POEs.

      For example I started my script at 09-16 15:02 but around 09-17 14:47 one of the camera's dropped out with:

      Cam3-Exception: RuntimeError
      Cam3-Exception message: Communication exception - possible device error/misconfiguration. Original message 'Couldn't read data from stream: 'preview' (X_LINK_ERROR)'

      The RasPi wasn't bothered and neither were the other two cameras. Sometimes it runs for days, sometimes not. Even the USB camera's could occasionally have this. Its not often but when this happens what is the best way of getting the camera back up? Initially I was putting a 1 second wait in and then going back into the loop which didn't work and after my 9th failed attempt I would just kill the thread for that specific camera. Do I just need to create a new device object like below or is there a better way of handling it? Should a wait, time.sleep(), be necessary?

      if needtoreinitialize:
        device = dai.Device(pipeline, device_info)
        preview = device.getOutputQueue("preview", 2, False)
        tracklets = device.getOutputQueue("tracklets", 2, False)
        needtoreinitialize = False

      The main thing is to get the camera back up and running.

      Thank you!

      • erik replied to this.

        Hi theDesertMoon ,
        Sorry about the issue, it's a known stability problem in the firmware that's much more prominent for OAK PoEs. I believe the latest 2.17.4 depthai (released today) added some xlink stability bugfixes, could you try that out?
        Thanks, Erik

          erik Will do! I had just updated to 2.17.3.1 this morning aaaaaand now I'm at 2.17.4.0!

          But to the original question - what is the most correct way to restart the stream?

          Thank you!

          • erik replied to this.

            Hi theDesertMoon I'd suggest waiting a couple of seconds and creating a new dai.Device object and passing the pipeline to it, that will reinitialize everything required.
            Thanks, Erik