OAK-D ProW PoE bandwidth and latency issues

Hello!

I am currently developing an application with an OAK-D ProW PoE camera but I am experiencing bandwidth and latency issues. My end goal is to use 3 cameras at the same time.

I define my pipeline using

def create_luxonis_pipeline(
    depth_res: str = "800", 
    rgb_res: str = "800", 
    median_kernel: str = "5x5", 
    depth_alignment: bool = True,
    alpha: Optional[float] = None, 
) -> dai.Pipeline:
    """
    Create a Luxonis OAK-D pipeline for depth and RGB cameras.

    """
    # Validate depth_res
    if depth_res not in RES_MAP_DEPTH:
        raise ValueError(
            f"Invalid depth_res value: {depth_res}. Valid values are {', '.join(RES_MAP_DEPTH.keys())}"
        )

    # Validate rgb_res
    if rgb_res not in RES_MAP_RGB:
        raise ValueError(f"Invalid rgb_res value: {rgb_res}. Valid values are {', '.join(RES_MAP_RGB.keys())}")

    # Validate median_kernel
    if median_kernel not in MEDIAN_MAP:
        raise ValueError(
            f"Invalid median_kernel value: {median_kernel}. Valid values are {', '.join(MEDIAN_MAP.keys())}"
        )

    resolution_depth = RES_MAP_DEPTH[depth_res]
    resolution_rgb = RES_MAP_RGB[rgb_res]
    median = MEDIAN_MAP[median_kernel]

    pipeline = dai.Pipeline()
    
    camRgb = pipeline.create(dai.node.ColorCamera)
    camLeft = pipeline.create(dai.node.MonoCamera)
    camRight = pipeline.create(dai.node.MonoCamera)
    stereo = pipeline.create(dai.node.StereoDepth)
    
    camRgb.setFps(20)    
    camLeft.setFps(20)
    camRight.setFps(20)

    xoutRgb = pipeline.create(dai.node.XLinkOut)
    xoutRgb.input.setBlocking(False)
    xoutRgb.input.setQueueSize(1)

    xoutDepth = pipeline.create(dai.node.XLinkOut)
    xoutDepth.input.setBlocking(False)
    xoutDepth.input.setQueueSize(1)

    for monoCam in (camLeft, camRight):
        monoCam.setResolution(resolution_depth["res"])

    camRgb.setResolution(resolution_rgb["res"])
    camRgb.setPreviewSize(resolution_rgb["w"], resolution_rgb["h"])

    stereo.setDefaultProfilePreset(dai.node.StereoDepth.PresetMode.HIGH_DENSITY)
    stereo.initialConfig.setMedianFilter(median)

    stereo.setRectifyEdgeFillColor(0)
    stereo.setLeftRightCheck(False)
    stereo.setExtendedDisparity(True)
    stereo.setSubpixel(False)
    
    if depth_alignment:
        stereo.setLeftRightCheck(True)
        stereo.setDepthAlign(dai.CameraBoardSocket.CAM_A)

    if alpha is not None:
        stereo.setAlphaScaling(alpha)
        config = stereo.initialConfig.get()
        config.postProcessing.brightnessFilter.minBrightness = 0
        stereo.initialConfig.set(config)

    xoutRgb.setStreamName("rgb")
    xoutDepth.setStreamName("depth")

    camRgb.preview.link(xoutRgb.input)
    camLeft.out.link(stereo.left)
    camRight.out.link(stereo.right)
    stereo.depth.link(xoutDepth.input)
    # pipeline.setXLinkChunkSize(0)

    return pipeline

Setup 1

The camera connected to the switch (https://www.tp-link.com/baltic/business-networking/omada-sdn-switch/tl-sg105pe/) is returning ~2 or less FPS in depthai_demo.py the script. When executing poe_test_script.py all tests return OK.

I tried to Wireshark the connection, and interestingly the frame length was constantly increasing (look image) and the received output had a very low frame rate and was delayed by ~13 seconds.

I tried to manually set the length of the frame to any number by doing

    openvino_version = dai.OpenVINO.Version.VERSION_2021_4
    config = dai.Device.Config()
    config.board.network.mtu = 9000
    config.board.sysctl.append("net.inet.tcp.path_mtu_discovery=0")
    config.board.sysctl.append("net.inet.tcp.rfc1323=0")
    config.board.network.xlinkTcpNoDelay = False # Default True
    config.board.sysctl.append("net.inet.tcp.delayed_ack=1") # configure sysctl settings. 0 by default.
    config.version = openvino_version
    dev = dai.Device(config, dev_info)
    
    device: dai.Device = stack.enter_context(dev)

    device.startPipeline(create_luxonis_pipeline())
    dic["rgb-" + mxid] = device.getOutputQueue(name="rgb",maxSize=1, blocking=False)
    dic["depth-" + mxid] = device.getOutputQueue(name="depth",maxSize=1, blocking=False)

But I still received frames like those in the image above, not following value specified in the config.

Setup 2
Camera connected to Ubiquiti Gigabit PoE Injector, same way of creating pipeline as before without adjusting config.board. In Wireshark I can see the same length of every frame (image below), the camera streams with 20 FPS in the demo code, and my scripts smoothly with acceptable latency.

To be completely honest I don't have any further debugging ideas. Could you please indicate to me what I did incorrectly? Is my switch a limitation (according to documentation, switching speed speed should be more then sufficient)? Why values configured manually are not being observed?

Best,

Hubert

Dear jakaskerl ,

The network looks ok.

Indeed MTU settings are problematic, but this does not fix the problem. The camera does not work with values different than default.

I used also this switch https://store.ui.com/us/en/products/us-xg-6poe, but there is no difference.

Interestingly with the PoE injector, it works.

Best,

Hubert