Error in retrieving images OAK 1-PoE

Stefano

Hello,
I have a problem with some OAK 1-PoE camera.
I have many cameras all running with the same code in standalone mode.
The cameras are working fine, but after many days, more then half had a problem in getting the images from the node.io queue.

here is a snippet of the pipeline and the retrieving functions:

    def get_pipeline(self) -> dai.Pipeline:
        pipeline = dai.Pipeline()
        cam_rgb = pipeline.create(dai.node.ColorCamera)
        cam_rgb.setPreviewKeepAspectRatio(False)
        cam_rgb.setPreviewSize(self.config['width'], self.config['height'])
        cam_rgb.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1080_P)
        cam_rgb.setInterleaved(False)
        cam_rgb.setColorOrder(dai.ColorCameraProperties.ColorOrder.RGB)
        cam_rgb.setFps(self.fps)

        if self.angle == '180':
            cam_rgb.setImageOrientation(dai.CameraImageOrientation.NORMAL)

        video_encoder = pipeline.create(dai.node.VideoEncoder)
        video_encoder.setDefaultProfilePreset(60, dai.VideoEncoderProperties.Profile.MJPEG)

        cam_rgb.video.link(video_encoder.input)

        nn = pipeline.create(dai.node.YoloDetectionNetwork)
        nn.setConfidenceThreshold(self.config['conf_threshold'])
        nn.setNumClasses(self.config['n_classes'])
        nn.setCoordinateSize(self.config['coordinates'])
        nn.setIouThreshold(self.config['iou_threshold'])
        nn.setBlobPath(Path(self.nn_weights_path))
        nn.setNumInferenceThreads(self.config['n_inference_threads'])
        nn.input.setBlocking(False)

        cam_rgb.preview.link(nn.input)

        script = pipeline.create(dai.node.Script)
        script.setProcessor(dai.ProcessorType.LEON_CSS)
        script = self.set_script(script)

        nn.out.link(script.inputs['detections'])
        video_encoder.bitstream.link(script.inputs['frame'])

        return pipeline`

`    def elaboration_loop(self) -> None:

        results = {}

        while True:
            try:
                data_im = None
                dets = None
                counter = 0

                if REQUESTS_RECEIVED.is_set():
                    while (data_im is None or not dets) and counter < 50:
                        dets = node.io['detections'].get()
                        pck = node.io['frame'].get()
                        if pck:
                            data_im = pck.getData()

                        if data_im is None or not dets:
                            counter += 1
                            time.sleep(0.2)

                    results['data_im'] = data_im if data_im else None
                    results['dets'] = dets if dets else []

                    threading.Thread(target=self.send_results, args=[results,]).start()
                    REQUESTS_RECEIVED.clear()

                else:
                    node.io['detections'].get()
                    node.io['frame'].get()

            except Exception as e:
                self.udp_client.send_message(f"Exception in the elaboration loop: {e} - {traceback.print_exc()}")`

function "elaboration_loop" is started as a thread from the main part of the code, and the actual image retrieving is called setting the REQUESTS_RECEIVED event.

The problem is that when the issue appears, data_im remains "None" and does not recover (meaning calling again the function does not change the result) until the code is flashed again.

Do you have any suggestions?
Is it possible to reboot the camera from within the camera itself without flashing the code?

Thank you

OskarSonc

Hey Stefano - this usually points to the on-device Script getting wedged, most often from a blocking queue read or from resource buildup if you spawn a new thread per request. Also be careful not to treat “zero detections” as “missing data”, since an empty detections packet is still valid.

Don’t use blocking get() in the Script loop. Make the Script inputs non-blocking + small queue and use tryGet() so you can’t deadlock if one stream stalls.

On DepthAI v2 in fully standalone mode there isn’t a supported way to reboot from inside the Script, the practical recovery path is PoE power-cycling or a small supervisor host that can detect the stall and reboot the device.

Thanks,
Oskar

Stefano

Hi @OskarSonc,
The thread is spawn just once at the beginning of my code (and the camera was working for days, I would aspect it to crush much earlier if that was the case), and of course the zero detection is not treated as an error but it is a valid result.

I do not understand exactly what difference would it make using the tryGet() in my specific case:
If I understand correctly, the difference between the two functions is that the simple get() will wait for the queue to have
at least an occurrence.
Therefore if my thread is blocked waiting for the image, it means that for some unknown reason I'm not receiving images anymore. What I mean is, even if I use tryGet(), I would still have the same issue of not receiving data until I reboot the camera.

Let me know if I'm missing something here,

Thanks,
Stefano

OskarSonc

Hi Stefano - tryGet() won’t magically bring frames back, but it prevents a common deadlock with blocking get(), if the Script blocks on one input it stops draining the other, upstream queues can fill, and that backpressure can stall the whole camera/NN path until reboot. Non-blocking inputs with a tiny queue + tryGet() keeps the loop alive and draining.

Also note you still start a send_results thread per request, if sending ever blocks, those can accumulate over days and starve the Script.