I want to use the OAK-D POE to gather training data, but I am having difficulty with dropped frames.

I need to record either RGBD or RGB + mono left + mono right. I need to capture the RGB imagery at 4K. I've been working with the example code from depthai-experiments/gen2-seq-num-sync and depthai-experiments/gen2-record-replay.

My problem is that I have frequent dropped frames.

  • The resulting video from the gen2-record-replay example has noticable stutter.
  • The verbose information from gen2-seq-num-sync reports frequent frame drops if the RGB resolution is 1080p, and severe frame drops if the resolution is 4K, and the live video feed stutters.

Here is my hardware setup:

  • OAK-D POE
  • Cat-6 network cable (20 m)
  • Netgear GS108PE (gigabit POE switch)
  • Cat-5e network cable (3 m)
  • System76 Oryx5 Pro onboard gigabit ethernet

This is the POE model, so I do know that I need to tune my system to increase the TCP/IP buffers and allow for jumbo packets. I have followed these instructions, but my problem persists.

I've tried compressed, but the streams are not synchronized if I do that. Compressed (but with a decent bitrate) is fine for my application, so if there is a way to guarantee frame synchronization via that method, that would be a great alternative.

  • erik replied to this.

    Hello pbarsic , I would first suggest compression to reduce packet size. The gen2-record-replay actually uses compression already (by default MJPEG compression), you could also try python record.py -q LOW for .h265 compression which would significantly reduce packet size.
    Another thing to notice is that both of these examples are doing host-side sync, so all streams (RGB, left, right) are in sync - which might be the reason that it's dropping frames (as it doesn't find all stream frames for sync).
    And finally I would try setting different values with XLinkChunkSize(), eg. pipeline.XLinkChunkSize(0).
    Thanks, Erik

      Hi @erik ,
      Thanks for your prompt reply. The .h265 compression is working for me, and I have a lot of headroom to increase the bitrate.
      Best,
      Paul