Camera unreachable after network disconnect

henry0987

I submitted a related ticket a few days ago, but am creating this separate issue for clarity. Sometimes during operation (likely after some unknown network-disconnect event) my Oak-D Pro W PoE becomes permanently network-unreachable—I cannot ping it from the host computer.

While the host (ROS2 driver) correctly attempts to reconnect, it fails because no network communication with the camera is possible, and the camera never recovers on its own.

Currently, the only solution is to physically power cycle the camera (disconnect and reconnect), which isn't feasible long-term (there is no way for me to do this from software in our setup). This appears to be a firmware issue since the camera becomes completely unreachable and requires hardware intervention to recover.

Is there a known reason why the Oak firmware doesn't automatically recover network connectivity?

Is there anything I can configure while constructing the pipeline that would enable the camera to automatically reset or recover from network disconnects?

jakaskerl

Hi henry0987
The general recommendation is a SW controlled POE switch to turn off power remotely, but I'll check if there is something that we can fix.

Thanks,
Jaka

HenriqueCerqueira

gravacao-de-tela-de-08-15-2025-034128-pm.webm

9MB

i have a similar problem, on my case during initialization device can't be find but my script set the default device IP and in parallel i measure the IP ping that response normally about ~0.3ms. The video anexe it a test with stereo depth and the ping response is volatile, why? i know that data transfer is bigger but have a better way? bellow is my pipeline script:

try:

        device = dai.Device(dai.DeviceInfo("192.168.15.156"))

        print(f"INFO: Dispositivo {device.getMxId()} conectado com sucesso.")

        # PASSO 2: Criar o pipeline, informando qual é o seu dispositivo padrão.

        # O 'with' agora gerencia o ciclo de vida do pipeline.

        with dai.Pipeline(defaultDevice=device) as pipeline:

            

            device = pipeline.getDefaultDevice()                

            print('Connected cameras:',device.getConnectedCameras())

            # A definição dos nós permanece a mesma

            cam_rgb = pipeline.create(dai.node.Camera).build(

                boardSocket=dai.CameraBoardSocket.CAM_A,

                sensorResolution= [SENSOR_WIDTH,SENSOR_HEIGHT],

                sensorFps=FPS,

            )

            isp_out = cam_rgb.requestOutput((SENSOR_WIDTH, SENSOR_HEIGHT), dai.ImgFrame.Type.YUV420p, fps=FPS)

             # 1. Criamos o nó ImageManip. 

            manip = pipeline.create(dai.node.ImageManip)

            # 2. Lemos as coordenadas de corte iniciais do nosso estado centralizado.

            controls = self.camera_state.get_all_controls()

            is_depth_enabled = controls.get('depth_enabled', False)

            q_depth = None

            crop_x = controls.get('crop_x', 0)

            crop_y = controls.get('crop_y', 0)

            

            # 3. Configuramos a operação de corte. Usamos setCropRect para definir a região exata.

            # A configuração é feita através do atributo 'initialConfig'. [cite: 693]

            # Os parâmetros são (xmin, ymin, xmax, ymax) em coordenadas absolutas de pixel.

            print(f"INFO: Adicionando operação de corte em ({crop_x}, {crop_y}) com tamanho {CROP_WIDTH}x{CROP_HEIGHT}")

            manip.initialConfig.addCrop(crop_x, crop_y, CROP_WIDTH, CROP_HEIGHT)

            

            # 4. Garantimos que a saída do ImageManip tenha o formato esperado pelo VideoEncoder.

            manip.initialConfig.setFrameType(dai.ImgFrame.Type.NV12)

        

            video_enc = pipeline.create(dai.node.VideoEncoder)

            video_enc.setDefaultProfilePreset(fps=FPS, profile=dai.VideoEncoderProperties.Profile.H265_MAIN)

            if is_depth_enabled:

                print("INFO: Profundidade habilitada. Configurando pipeline estéreo...")

                mono_left = pipeline.create(dai.node.Camera).build(

                    boardSocket=dai.CameraBoardSocket.CAM_B, 

                    sensorFps=FPS

                )

                mono_right = pipeline.create(dai.node.Camera).build(

                    boardSocket=dai.CameraBoardSocket.CAM_C,

                    sensorFps=FPS,

                )

                stereo = pipeline.create(dai.node.StereoDepth)

                # Usamos o preset de alta precisão que é um bom equilíbrio

                stereo.setDefaultProfilePreset(dai.node.StereoDepth.PresetMode.DEFAULT)

                stereo.setDepthAlign(dai.CameraBoardSocket.CAM_A)

                

                # LeftRightCheck é MANTIDO ATIVO devido à exigência do alinhamento

                stereo.setLeftRightCheck(True)

                stereo.setSubpixel(False)

                mono_left_out = mono_left.requestOutput(size=(640, 400), type=dai.ImgFrame.Type.GRAY8, fps=FPS)

                mono_right_out = mono_right.requestOutput(size=(640, 400), type=dai.ImgFrame.Type.GRAY8, fps=FPS)

                mono_left_out.link(stereo.left)

                mono_right_out.link(stereo.right)

                # A fila só é criada se a profundidade estiver habilitada

                q_depth = stereo.depth.createOutputQueue(maxSize=4, blocking=False)

            else:

                print("INFO: Profundidade desabilitada. Pulando a criação dos nós estéreo.")

            

            isp_out.link(manip.inputImage) # Saída ISP vai para o ImageManip

            manip.setMaxOutputFrameSize(CROP_WIDTH \* CROP_HEIGHT \* 3 // 2) # NV12 tem 1.5 bytes por pixel

            manip.out.link(video_enc.input) # Saída de vídeo 1080p vai para o Manip

            print("INFO: Aplicando configurações iniciais da câmera...")

            self._apply_controls(cam_rgb.initialControl)

            q_h265_out = video_enc.out.createOutputQueue(maxSize=30, blocking=True)

            control_in = cam_rgb.inputControl.createInputQueue()

            

            sys_logger = pipeline.create(dai.node.SystemLogger)

            sys_logger.setRate(1) # Envia dados a cada 1 segundo

            

            q_sys_info = sys_logger.out.createOutputQueue(maxSize=4, blocking=False)

            

            pipeline.start()

            print("INFO: Pipeline V3 iniciado no dispositivo.")

            while self.is_running and pipeline.isRunning():

                self._process_commands(control_in)

                

                h265_packet = q_h265_out.get()

                sys_info = q_sys_info.tryGet()

                if q_depth:

                    depth_frame = q_depth.tryGet()

                    if depth_frame is not None:

                        depth_cv_frame = depth_frame.getFrame()

                        

                        # Obter a região de interesse (ROI) do estado central

                        controls = self.camera_state.get_all_controls()

                        roi_x = int(controls['selection_region_x'])

                        roi_y = int(controls['selection_region_y'])

                        roi_w = int(controls['selection_region_w'])

                        roi_h = int(controls['selection_region_h'])

                        # Calcular o centro da ROI

                        center_x = roi_x + roi_w // 2

                        center_y = roi_y + roi_h // 2

                        

                        # Garantir que as coordenadas estão dentro dos limites do frame

                        if 0 <= center_y < CROP_HEIGHT and 0 <= center_x < CROP_WIDTH:

                            # Obter o valor da distância (em mm) no pixel central

                            dist_mm = depth_cv_frame[center_y, center_x]

                            

                            # Atualizar o estado central com o novo valor

                            self.camera_state.update_depth_info({'center_depth_mm': dist_mm})

                if sys_info:

                    m = 1024 \* 1024 # MiB

                    temp = sys_info.chipTemperature.average # [cite: 3612, 4162]

                    css_cpu = sys_info.leonCssCpuUsage.average # [cite: 4170, 4202]

                    mss_cpu = sys_info.leonMssCpuUsage.average # [cite: 4170, 4202]

                    ddr_mem_info = {'used': sys_info.ddrMemoryUsage.used / m, 'total': sys_info.ddrMemoryUsage.total / m}

                    cmx_mem_info = {'used': sys_info.cmxMemoryUsage.used / m, 'total': sys_info.cmxMemoryUsage.total / m}

                    css_mem_info = {'used': sys_info.leonCssMemoryUsage.used / m, 'total': sys_info.leonCssMemoryUsage.total / m}

                    mss_mem_info = {'used': sys_info.leonMssMemoryUsage.used / m, 'total': sys_info.leonMssMemoryUsage.total / m}

                    self.camera_state.update_system_info({

                        'temp': sys_info.chipTemperature.average,

                        'css_cpu': sys_info.leonCssCpuUsage.average,

                        'mss_cpu': sys_info.leonMssCpuUsage.average,

                        'ddr_memory': ddr_mem_info,

                        'cmx_memory': cmx_mem_info,

                        'css_memory': css_mem_info,

                        'mss_memory': mss_mem_info

                    })

                    

                if h265_packet:

                    metadata = {

                        "lens_pos": h265_packet.getLensPosition(), # [cite: 4276]

                        "lens_pos_raw": h265_packet.getLensPositionRaw(), # [cite: 4277]

                        "exposure_us": h265_packet.getExposureTime().microseconds, # [cite: 4275]

                        "iso": h265_packet.getSensitivity(), # [cite: 4278]

                        "color_temp": h265_packet.getColorTemperature() # [cite: 4275]

                    }

                    with self.metadata_lock:

                        self.shared_metadata.update(metadata)

                    current_time = time.time()

                    if current_time - self.last_ui_update_time > 0.5: # Throttle a 2Hz

                        self.camera_state.update_from_camera_metadata(metadata)

                        self.last_ui_update_time = current_time

                    

                    buf = Gst.Buffer.new_wrapped(h265_packet.getData().tobytes())

                    if appsrc.emit('push-buffer', buf) != Gst.FlowReturn.OK:

                        print("AVISO: Appsrc (GStreamer) rejeitou o buffer. Parando.")

                        break

        

        # Quando o 'with pipeline' termina, o pipeline é parado.

        print("INFO: O pipeline foi parado.")

    except Exception as e:

        print(f"ERRO CRÍTICO na thread da câmera: {e}")

        traceback.print_exc()

    finally:

        self.is_running = False

        if appsrc:

            appsrc.emit('end-of-stream')

        print("INFO: Thread da câmera finalizada e EOS enviado.")

def stop(self):

    print("INFO: Sinal de parada recebido pela OakCameraManager.")

    self.is_running = False

henry0987

Thank you @jakaskerl Looking forward to hearing if there is a fix.

henry0987

Hi @jakaskerl just following up, were you able to find out if this is something you can fix?

jakaskerl

henry0987
I saw the same thing happening on plain depthaiV3 (without ROS). Not sure if related. It will take a little longer since it is a FW issue and we are swamped.. Thanks for your patience.