I am writing a ROS2 node in C++ for streaming the oak-1 PoE 4k video at 20 FPS. I'm using the H.265 encoder settings for this. I see that we can set the bitrate, keyframe frequency, and numBframes. I am wondering how these settings will influence the P frames under the hood. Please provide as much detail as possible.

I am trying to optimize for latency, please also suggest the best settings for prioritizing latency with 4k 20 FPS video stream and H.265 encoding.

Here is how I'm currently creating the pipeline:

  dai::Pipeline pipeline;
  auto colorCam = pipeline.create<dai::node::ColorCamera>();
  colorCam->setBoardSocket(dai::CameraBoardSocket::CAM_A);
  colorCam->setResolution(dai::ColorCameraProperties::SensorResolution::THE_4_K); /// 3840 × 2160
  colorCam->setFps(frame_rate_);

  auto videoEnc = pipeline.create<dai::node::VideoEncoder>();
  videoEnc->setDefaultProfilePreset(frame_rate_, dai::VideoEncoderProperties::Profile::H265_MAIN);
  videoEnc->setBitrate(0);
  videoEnc->setKeyframeFrequency(1);
  videoEnc->setNumBFrames(0);
  videoEnc->setFrameRate(frame_rate_);
  videoEnc->setQuality(compression_quality_);
  colorCam->video.link(videoEnc->input);

  auto xout = pipeline.create<dai::node::XLinkOut>();
  xout->setStreamName("rgb");
  videoEnc->bitstream.link(xout->input);

    Igor
    Generally speaking,
    More keyframes -> less efficient the compression is.
    More B frames inserted -> better compression but higher latency as the frames are bi-directional
    Bitrate -> the lower the more efficient, but looks worse

    Thanks,
    Jaka

    • Igor replied to this.

        jakaskerl

        I am wondering what the specific breakdown of the packets is when I set KeyframeFrequnecy to 1 (key frames every frame) and B frames to 0. Does this result in a intra-frame-only stream? I am implementing this pipeline with NVIDIA NVDEC SDK so specifics are important.

          jakaskerl Currently it seems that if I set the stream to intra frame only by setting the keyframe frequency to 1 (once every frame). And then I set the NVDEC decoder to treat the stream and intraframe only it fails to decode.