[OAK-D Pro PoE] Fame drop for 1080p in 30 fps

izumi

I want to capture color images(1920*1080) at 30 fps without frame drop.
I've try following code, but there seems to be frame drop constantly.

prev_fc = 0
pipeline = dai.Pipeline()

cam_rgb = pipeline.createColorCamera()
cam_rgb.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1080_P)
cam_rgb.setFps(30)
cam_rgb.setInterleaved(False)
# If following downscale process is uncommented, frame drops doesn't happen
# cam_rgb.setIspScale(2, 3)

xout_rgb = pipeline.createXLinkOut()
xout_rgb.setStreamName("rgb")
cam_rgb.isp.link(xout_rgb.input)

with dai.Device(pipeline) as device:
    q_rgb = device.getOutputQueue("rgb")
    frame = None

    while True:
        in_rgb = q_rgb.get()
        frame = in_rgb.getCvFrame()
        curr_fc = in_rgb.getSequenceNum()
        diff = curr_fc - prev_fc

        if diff > 1:
            print(f"[Missing {diff-1}] Prev: {prev_fc}, Current: {curr_fc}")

        prev_fc = curr_fc

Even if I set "blocking" and "maxSize" for "device.getOutputQueue", frame drop counts seems not to be improve.
But I uncommented "cam_rgb.setIspScale(2, 3)", frame drops doesn't happen in my environment.

I'm using OAK-D Pro PoE and NETGEAR GS108PE. These are connected CAT.6 cable and speed seems to be recognized as 1Gb/s.
Is there better way to access all frames without frame drop at 1080p?

erik

Hi izumi ,
I think you are above the limit of OAK GigE connection:

1280 * 720 * 1.5 * 30fps * 8bits = 331 mbps
1920 * 1080 * 1.5 * 30fps * 8bits = 747 mbps

Have you tried encoding 1080P frames (eg MJPEG) before sending them?
Thanks, Erik

izumi

Hi erik ,
Thank you for pointing out that limitation. I tried MJPEG encoding but I got following warning.
Arrived frame type (2) is not either NV12 or YUV400p (8-bit Gray)
So I tried ImageManip node to convert image to NV12. But I got following error.
Output image is bigger (1382400B) than maximum frame size specified in properties (1048576B) - skipping frame.
What would be the best way to encode 1080P images in MJPEG?
Thank you.

erik

Hi izumi ,
You should increase max frame size: manip.setMaxOutputFrameSize(1382400)
I hope that will work🙂
Thanks, Erik

izumi

Hi erik ,
I assume from encoded data will numpy array from your comment. I tried q.get().getCvFrame() but I get following error.

Traceback (most recent call last):
  File ".\sample.py", line 54, in <module>
    sn = frame.getSequenceNum()
AttributeError: 'numpy.ndarray' object has no attribute 'getSequenceNum'

So what is the best way to check frame drop encoded stream?
Thank you.

erik

Hi izumi ,

imgFrame = q.get() # Returns ImgFrame (depthai message)

cvFrame = imgFrame.getCvFrame() # Gets np array (opencv Format)
seq = imgFrame.getSequenceNum() # Gets seq num of ImgFrame

izumi

Hi erik ,
Sorry for bothering you but still I get similar error.

App starting streaming MJPEG encoded frames into file video.mp4
Traceback (most recent call last):
  File ".\sample.py", line 53, in <module>
    sn = q.get().getCvFrame().getSequenceNum()
AttributeError: 'numpy.ndarray' object has no attribute 'getSequenceNum'

Do I miss something?

erik

Hi izumi ,
Please refer to my comment above on correct API usage.
Thanks, Erik