Hi everyone! I'm attempting to execute an example provided by Luxonis/Examples on [video streaming via RTSP](https://github.com/luxonis/depthai-experiments/tree/d6a1dafb988e74e42f3250c71ca9905a6fe44b0e/gen2-rtsp-streaming), and I'm encountering some issues. I'm operating in an **Ubuntu** environment: Distributor ID: Ubuntu Description: Ubuntu 22.04.2 LTS Release: 22.04 Codename: jammy I've attempted to perform the retransmission in both H264 and H265, but in both cases, I'm encountering the following errors. Here's my **H264** codec configuration: ``` $ apt depends libx264-dev libx264-dev Depends: libx264-163 (= 2:0.163.3060+git5db6aa6-2build1) ``` When I run the script with an H264 configuration, upon opening playback, I receive the following warning: ``` [h264 @ 0x558109e892c0] Illegal short-term buffer state detected [h264 @ 0x558109e892c0] Missing reference picture; default is 65540 ``` The Gstreamer LOGS are as follows: [upl-image-preview url=https://discuss.luxonis.com/assets/files/2023-11-14/1699952348-543269-screenshot-from-2023-11-14-09-58-55.png] Upon triggering these warnings, the image appears choppy, though the streaming continues to be transmitted: [upl-image-preview url=https://discuss.luxonis.com/assets/files/2023-11-14/1699949813-221031-screenshot-from-2023-11-13-17-37-12.png] For **H265** here's my codec configuration: ``` $ apt depends libx265-dev libx265-dev Depends: libx265-199 (= 3.5-2) Suggests: libx265-doc ``` When I run the script with an H265 configuration, upon opening the streaming window, it works flawlessly until it closes with the following ERROR message: ``` [hevc @ 0x556682882080] Could not find reference with POC 46 [hevc @ 0x556682944900] Duplicate POC in a sequence: 48 ``` The Gstreamer LOGS are as follows, no error when window is closed this logs appear just when I start script with opencv to view the streaming. [upl-image-preview url=https://discuss.luxonis.com/assets/files/2023-11-14/1699951803-808513-screenshot-from-2023-11-14-09-49-41.png] To open a streaming window I use this script: ``` import cv2 IP = "localhost_IP" PORT = "PORT" IMG = "preview" cap = cv2.VideoCapture(f"rtsp://{IP}:{PORT}/{IMG}") while True: ret, frame = cap.read() if ret: cv2.imshow("RTSP View H264", frame) cv2.waitKey(1) else: print("unable to open camera") break cap.release() cv2.destroyAllWindows() ``` Has anyone here encountered this issue? I would appreciate any help or potential avenues of research to resolve this. Thank you very much in advance.

Issues Streaming RTSP with H264 and H265

Irena

Hi everyone!

I'm attempting to execute an example provided by Luxonis/Examples on video streaming via RTSP, and I'm encountering some issues.

I'm operating in an Ubuntu environment:

Distributor ID: Ubuntu
Description: Ubuntu 22.04.2 LTS
Release: 22.04
Codename: jammy

I've attempted to perform the retransmission in both H264 and H265, but in both cases, I'm encountering the following errors.

Here's my H264 codec configuration:

$ apt depends libx264-dev
libx264-dev
  Depends: libx264-163 (= 2:0.163.3060+git5db6aa6-2build1)

When I run the script with an H264 configuration, upon opening playback, I receive the following warning:

[h264 @ 0x558109e892c0] Illegal short-term buffer state detected
[h264 @ 0x558109e892c0] Missing reference picture; default is 65540

The Gstreamer LOGS are as follows:

Upon triggering these warnings, the image appears choppy, though the streaming continues to be transmitted:

For H265 here's my codec configuration:

$ apt depends libx265-dev
libx265-dev
  Depends: libx265-199 (= 3.5-2)
  Suggests: libx265-doc

When I run the script with an H265 configuration, upon opening the streaming window, it works flawlessly until it closes with the following ERROR message:

[hevc @ 0x556682882080] Could not find reference with POC 46
[hevc @ 0x556682944900] Duplicate POC in a sequence: 48

The Gstreamer LOGS are as follows, no error when window is closed this logs appear just when I start script with opencv to view the streaming.

To open a streaming window I use this script:

import cv2
IP = "localhost_IP"
PORT = "PORT"
IMG = "preview"

cap = cv2.VideoCapture(f"rtsp://{IP}:{PORT}/{IMG}")
while True:
    ret, frame = cap.read()
    if ret:
        cv2.imshow("RTSP View H264", frame)
        cv2.waitKey(1)
    else:
        print("unable to open camera")
        break
cap.release()
cv2.destroyAllWindows()

Has anyone here encountered this issue? I would appreciate any help or potential avenues of research to resolve this.

Thank you very much in advance.

jakaskerl

Hi Irena
I'm not too familiar with RTSP so best I can do is post a GPT answer:

Hope it solves the issue or at least helps you find it.

It seems you are facing common issues that can occur when streaming video over RTSP, particularly with the H264 and H265 codecs. Here's what the errors generally indicate and some potential solutions you can explore:

H264 Codec Issue

Error:

[h264 @ 0xaddress] Illegal short-term buffer state detected
[h264 @ 0xaddress] Missing reference picture; default is 65540

Explanation:

This error suggests that there's a problem with the frame buffering. It could be due to network issues, incorrect codec parameters, or a bug in the decoder.

Potential Solutions:

Check Network Stability: Ensure that your network connection is stable and has sufficient bandwidth for streaming.
Update Codecs: Make sure you have the latest versions of the required libraries and codecs.
Adjust GStreamer Pipeline: If you are using GStreamer in the background, tweak the pipeline parameters, especially related to buffering and sync.

H265 Codec Issue

Error:

[hevc @ 0xaddress] Could not find reference with POC 46
[hevc @ 0xaddress] Duplicate POC in a sequence: 48

Explanation:

These errors are related to Picture Order Count (POC) which is used in video compression to manage frame sequences. It could indicate a problem with how frames are being handled or an issue with the encoder settings.

Potential Solutions:

Frame Handling: Check the encoder settings to ensure frames are being handled in the correct order.
Upgrade Libraries: Similar to H264, make sure all related libraries and codecs are up to date.
GStreamer Pipeline: Adjust your GStreamer pipeline settings for HEVC streaming to ensure proper synchronization and ordering of frames.

General Tips

Debugging: Enable verbose logging in GStreamer to get more detailed logs that could help pinpoint the issue.
Software Update: Ensure that your Ubuntu system and all video-related packages are fully updated.
Hardware Support: Verify that your hardware supports the encoding and decoding of H264 and H265. Sometimes hardware acceleration can cause issues.

RTSP Streaming Script

Your RTSP streaming script seems standard. However, you may want to add error checking after cv2.VideoCapture to ensure the stream is properly initialized:

cap = cv2.VideoCapture(f"rtsp://{IP}:{PORT}/{IMG}")
if not cap.isOpened():
    print("Unable to connect to the camera")
    # Exit or handle the error as needed

Thanks,
Jaka

Irena

Hi jakaskerl !!

Thank you so much for your prompt response.

The truth is, I had previously tried using ChatGPT, but without much success 🙁

I'm still encountering the same issues, but I suspect it's related to codecs and how OpenCV handles frame reading. In other words, I have a feeling the problem lies not with the RTSP server or the data source (OAK-D).

In case anyone is interested, I am following this StackOverFlow thread, looking for more information on how to manage the reading of frames on the openCV side (using threads or multithreading). I will share the results with you and in the meantime if anyone has any ideas about the issue, it would be welcome.

Thanks a lot.
Irena