Black screen when streaming H.265 encoded frames on HTTP server

Hi @jakaskerl

I am using the VideoEncoder node to encode the frames to H.265 format for streaming it over a local HTTP server.

Here is the test code I am suing for doing so.

import depthai as dai
import time
import threading
from http.server import BaseHTTPRequestHandler, HTTPServer
from socketserver import ThreadingMixIn

HTTP_SERVER_PORT = 8000

# HTTPServer MJPEG
class VideoStreamHandler(BaseHTTPRequestHandler):
    def do_GET(self):
        self.send_response(200)
        self.send_header('Content-type', 'multipart/x-mixed-replace; boundary=frame')
        self.end_headers()
        while True:
            time.sleep(0.1)
            if hasattr(self.server, 'frametosend'):
                encoded_h265 = self.server.frametosend
                self.wfile.write(b'--frame\r\n')
                self.wfile.write(b'Content-Type: video/h265\r\n\r\n')
                self.send_header('Content-length', str(len(encoded_h265)))
                self.end_headers()
                self.wfile.write(encoded_h265)
                self.end_headers()



class ThreadedHTTPServer(ThreadingMixIn, HTTPServer):
    """Handle requests in a separate thread."""
    pass


# start MJPEG HTTP Server
server_HTTP = ThreadedHTTPServer(('localhost', HTTP_SERVER_PORT), VideoStreamHandler)
th2 = threading.Thread(target=server_HTTP.serve_forever)
th2.daemon = True
th2.start()


# Create pipeline
pipeline = dai.Pipeline()

# Define sources and output
camRgb = pipeline.create(dai.node.ColorCamera)
videoEnc = pipeline.create(dai.node.VideoEncoder)
xout = pipeline.create(dai.node.XLinkOut)

xout.setStreamName('h265')

# Properties
camRgb.setBoardSocket(dai.CameraBoardSocket.CAM_A)
camRgb.setResolution(dai.ColorCameraProperties.SensorResolution.THE_4_K)
videoEnc.setDefaultProfilePreset(30, dai.VideoEncoderProperties.Profile.H265_MAIN)

# Linking
camRgb.video.link(videoEnc.input)
videoEnc.bitstream.link(xout.input)


# Connect to device and start pipeline
with dai.Device(pipeline) as device:

    # Output queue will be used to get the encoded data from the output defined above
    q = device.getOutputQueue(name="h265", maxSize=30, blocking=True)

    # The .h265 file is a raw stream file (not playable yet)
    startTime = time.monotonic()
    counter = 0
    while True:
        h265Packet = q.get()  # Blocking call, will wait until a new data has arrived
        #h265Packet.getData().tofile(videoFile)  # Appends the packet data to the opened file
        frame = h265Packet.getData()
        print("frame: ", frame)
        
        server_HTTP.frametosend = frame
        
        if frame is not None:
            counter += 1
    
        pipeline_fps  = counter / (time.monotonic() - startTime)
        print("Pipeline FPS: ", pipeline_fps)

However, I am unable to see any frames on the local server at the Port mentioned in the code. All I see is a black screen.

I know there is an example where these encoded h265 frames are being written to the disk in its raw encoded format and then decoded using ffmpeg to convert it to a video file. However, i don't want to save it, rather i want to stream each frame over a local server. So how can I decode these encoded frames while they are in the memory on the host computer so that I can send them to a local server?

Can you look into it and see if you are having the same issue and how can we resolve it?

I am using a Oak-D S2 PoE camera. I look forward to hearing from you soon.

Thanks
Yishu