• DepthAI-v2
  • [OAK-D Pro PoE] Fame drop for 1080p in 30 fps

I want to capture color images(1920*1080) at 30 fps without frame drop.
I've try following code, but there seems to be frame drop constantly.

prev_fc = 0
pipeline = dai.Pipeline()

cam_rgb = pipeline.createColorCamera()
cam_rgb.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1080_P)
cam_rgb.setFps(30)
cam_rgb.setInterleaved(False)
# If following downscale process is uncommented, frame drops doesn't happen
# cam_rgb.setIspScale(2, 3)

xout_rgb = pipeline.createXLinkOut()
xout_rgb.setStreamName("rgb")
cam_rgb.isp.link(xout_rgb.input)

with dai.Device(pipeline) as device:
    q_rgb = device.getOutputQueue("rgb")
    frame = None

    while True:
        in_rgb = q_rgb.get()
        frame = in_rgb.getCvFrame()
        curr_fc = in_rgb.getSequenceNum()
        diff = curr_fc - prev_fc

        if diff > 1:
            print(f"[Missing {diff-1}] Prev: {prev_fc}, Current: {curr_fc}")

        prev_fc = curr_fc

Even if I set "blocking" and "maxSize" for "device.getOutputQueue", frame drop counts seems not to be improve.
But I uncommented "cam_rgb.setIspScale(2, 3)", frame drops doesn't happen in my environment.

I'm using OAK-D Pro PoE and NETGEAR GS108PE. These are connected CAT.6 cable and speed seems to be recognized as 1Gb/s.
Is there better way to access all frames without frame drop at 1080p?

  • erik replied to this.

    Hi izumi ,
    I think you are above the limit of OAK GigE connection:

    1280 * 720 * 1.5 * 30fps * 8bits = 331 mbps
    1920 * 1080 * 1.5 * 30fps * 8bits = 747 mbps

    Have you tried encoding 1080P frames (eg MJPEG) before sending them?
    Thanks, Erik

      Hi erik ,
      Thank you for pointing out that limitation. I tried MJPEG encoding but I got following warning.
      Arrived frame type (2) is not either NV12 or YUV400p (8-bit Gray)
      So I tried ImageManip node to convert image to NV12. But I got following error.
      Output image is bigger (1382400B) than maximum frame size specified in properties (1048576B) - skipping frame.
      What would be the best way to encode 1080P images in MJPEG?
      Thank you.

      • erik replied to this.

        Hi izumi ,
        You should increase max frame size: manip.setMaxOutputFrameSize(1382400)
        I hope that will work๐Ÿ™‚
        Thanks, Erik

          Hi erik ,
          I assume from encoded data will numpy array from your comment. I tried q.get().getCvFrame() but I get following error.

          Traceback (most recent call last):
            File ".\sample.py", line 54, in <module>
              sn = frame.getSequenceNum()
          AttributeError: 'numpy.ndarray' object has no attribute 'getSequenceNum'

          So what is the best way to check frame drop encoded stream?
          Thank you.

          • erik replied to this.

            Hi izumi ,

            imgFrame = q.get() # Returns ImgFrame (depthai message)
            
            cvFrame = imgFrame.getCvFrame() # Gets np array (opencv Format)
            seq = imgFrame.getSequenceNum() # Gets seq num of ImgFrame

              Hi erik ,
              Sorry for bothering you but still I get similar error.

              App starting streaming MJPEG encoded frames into file video.mp4
              Traceback (most recent call last):
                File ".\sample.py", line 53, in <module>
                  sn = q.get().getCvFrame().getSequenceNum()
              AttributeError: 'numpy.ndarray' object has no attribute 'getSequenceNum'

              Do I miss something?

              • erik replied to this.

                Hi izumi ,
                Please refer to my comment above on correct API usage.
                Thanks, Erik