Hi BradSteiner
Means there was an issue retrieving a frame from the RGB queue - mostly caused by queue filling up, crashing the device pipeline.

Thanks,
Jaka

What could cause that? Sometimes it seems to run fine, but I was seeing it a lot yesterday. It would only run for a few minutes before crashing due to that error. Is it the USB cable? Are the PoE version more reliable?

    BradSteiner
    Could be USB cable (lower bandwidth causing a block), POE (even lower bandwidth + ETH stack on CPU).. What does your pipeline look like? Is your host side loop fast enough to read the queues? If you set all queues to non-blocking, does the issue disappear.

    Thanks,
    Jaka

      2 months later

      Hi, we are still seeing issue even with the PoE version. It looks like this:

      [component_container-1] [ERROR] [1742415245.811915967] [oak]: No data on logger queue!
      [component_container-1] [ERROR] [1742415245.812111873] [oak]: Camera diagnostics error: Communication exception - possible device error/misconfiguration. Original message 'Couldn't read data from stream: 'sys_logger_queue' (X_LINK_ERROR)'

      We are launching with this command:
      ros2 launch depthai_ros_driver camera.launch.py params_file:=camera_config.yaml

      where camera_config.yaml looks like this:

      /oak:
        ros__parameters:
          camera:        
            i_pipeline_type: rgbd
            i_nn_type: none
            i_floodlight_brightness: 1000  # 1500 is max
            i_laser_dot_brightness: 1200  # 1200 is max
          rgb:
            i_fps: 10.0                                                                                                                                                                  
            i_resolution: '4K' #'720P' #'4K'       
            i_set_isp_scale: false
      
          stereo:
            i_width: 1280 #512 1280
            i_height: 720 #288 720
            # i_resolution: '720P'
            i_extended_disp: true
            i_subpixel: false
            i_depth_preset: HIGH_DENSITY
            i_enable_decimation_filter: true
            i_enable_temporal_filter: false
            i_enable_speckle_filter: true
            i_enable_spatial_filter: true
            i_lr_check: true
      
          left:
            i_publish_topic: true

        Hi, could you check camera CPU usage? You can get that either via subscribing to diagnostics topic or enabling DEBUG logs via running export DEPTHAI_DEBUG=1. You can also disable diagnostics queue just in case by setting

        pipeline_gen:
            i_enable_diagnostics: false

        Sorry for the late reply. Here is a snapshot of the camera usage.

        status:
        - level: "\0"
          name: 'oak: sys_logger'
          message: System Information
          hardware_id: /oak_18443010616C980F00_OAK-D-PRO-POE
          values:
          - key: Leon CSS CPU Usage
            value: '99.859'
          - key: Leon MSS CPU Usage
            value: '9.071'
          - key: Ddr Memory Usage
            value: '229.081'
          - key: Ddr Memory Total
            value: '333.263'
          - key: Cmx Memory Usage
            value: '2.10254'
          - key: Cmx Memory Total
            value: '2621440'
          - key: Leon CSS Memory Usage
            value: '68.5048'
          - key: Leon CSS Memory Total
            value: '81.6971'
          - key: Leon MSS Memory Usage
            value: '4.67026'
          - key: Leon MSS Memory Total
            value: '39.8734'
          - key: Average Chip Temperature
            value: '54.6903'
          - key: Leon CSS Chip Temperature
            value: '56.7999'
          - key: Leon MSS Chip Temperature
            value: '53.3338'
          - key: UPA Chip Temperature
            value: '54.8573'
          - key: DSS Chip Temperature
            value: '53.7703'
        ---

        Values are pretty consistent across messages

        jakaskerl This is using the ROS Humble driver. Any idea on how this gets dealt with there?

        Hi, it seems that CPU usage is too high (99%), this can cause connection issues as mentioned in the documentation. To improve it, you would need to adjust your pipeline accordingly (lower resolutions, reduce FPS, limit 3A algorithms cc @jakaskerl what else could might help here)
        More information:
        https://docs.luxonis.com/hardware/platform/deploy/poe-deployment-guide#PoE%20deployment%20guide-Runtime-Debugging
        https://docs.luxonis.com/software/depthai/debugging#cpu-usage

          Our framerate is already fairly low at 10 FPS. Is it expected to see such high CPU usage at this framerate?

          7 days later

          Luxonis-Adam Even with the following configuration

          /oak:
            ros__parameters:
              camera:        
                i_pipeline_type: rgbd
                i_nn_type: none
                i_floodlight_brightness: 800  # 800 is max
                i_laser_dot_brightness: 1200  # 1200 is max
              rgb:
                i_low_bandwidth: true
              left:
                i_low_bandwidth: true
              right:
                i_low_bandwidth: true
              stereo:
                i_low_bandwidth: true

          We do see an increased framerate (~20 fps) at the expense of poor depth images.

          However, the CPU utilization seems to be still quite high. Here is the /diagnostics output:

          status:
          - level: "\0"
            name: 'oak: sys_logger'
            message: System Information
            hardware_id: /oak_18443010516DE7F400_OAK-D-PRO-POE
            values:
            - key: Leon CSS CPU Usage
              value: '98.693'
            - key: Leon MSS CPU Usage
              value: '17.734'
            - key: Ddr Memory Usage
              value: '183.424'
            - key: Ddr Memory Total
              value: '333.263'
            - key: Cmx Memory Usage
              value: '2.37988'
            - key: Cmx Memory Total
              value: '2621440'
            - key: Leon CSS Memory Usage
              value: '67.9544'
            - key: Leon CSS Memory Total
              value: '81.6971'
            - key: Leon MSS Memory Usage
              value: '5.65635'
            - key: Leon MSS Memory Total
              value: '39.8734'
            - key: Average Chip Temperature
              value: '50.031'
            - key: Leon CSS Chip Temperature
              value: '50.9173'
            - key: Leon MSS Chip Temperature
              value: '49.3651'
            - key: UPA Chip Temperature
              value: '50.0317'
            - key: DSS Chip Temperature
              value: '49.8097'
          ---

          Is this expected behavior using the ROS node?

          9 days later

          Hi, sorry for the delay, I think with the default resolutions and fps it is to be expected since the camera has to run the network stack as well, you can try decreasing those parameters and/or ISP3A algorithms to see if it improves things much for you, you could maybe also try some other encoder options. cc @jakaskerl