If you retrieve detections and passthrough from MobileNetDetection node queues at the same time are they guaranteed to be synchronized ? (i.e. detections were generated by image retrieved from the passthrough queue) I am saving passthrough images after drawing their detection boxes. I have notice when the target is moving I will occasionally get images saved that do not have any detectable objects in them, yet bounding boxes are drawn as if there were. This leads me to believe that the passthrough image is not the same one that the detections were generated with. I read from the detection queue and on the next line I read from the passthrough queue (as close to the same time as possible).

Hello @"TylerD"#p2978 , yes, if you get a new detection you will always get a new passthrough frame as well. Could you share the pipeline, so we can debug this? Thanks, Erik

This is a modified version of the license plate OCR example. I'm reading producer id "tattoos" placed on pigs that are processed at the facility where I work. I use one MobileNet to detect the tattoo and pass the roi to another MobileNet that locates the individual characters and sorts them left to right. Works great on still images, but moving object are where I experience the problem. `import argparse import numpy as np # numpy - manipulate the packet data returned by depthai import cv2 # opencv - display the video stream import depthai # depthai - access the camera and its data packets from pathlib import Path import os # os for file operations import time import threading import PyPigVision as ppv from PyPigVision import file_handling as fh from PyPigVision import tattoo_handling as th from pylogix import PLC parser = argparse.ArgumentParser() parser.add_argument('-cam', '--camera', action="store_true", help="Use DepthAI 4K RGB camera for inference (conflicts with -vid)") parser.add_argument('-vid', '--video', type=str, help="Path to video file to be used for inference (conflicts with -cam)") parser.add_argument('-rot', '--rotate', action="store_true", help="Rotates video 90° so camera can be mounted normally and accomodate a network that is trained with horizontal images.") parser.add_argument('-save', '--Save_Data', type=str, help="Write data to file") parser.add_argument('-plc', '--Use_PLC', action="store_true", help="Write data to PLC") parser.add_argument('-ocr_box', '--display_OCR_bboxes', action="store_true", help="Display OCR images with bboxes (creates images that cant be used for training)") args = parser.parse_args() if not args.camera and not args.video: raise RuntimeError( "No source selected. Use either \"-cam\" to run on RGB camera as a source or \"-vid \" to run on video" ) # Handle frames per second class FPSHandler: def __init__(self, cap=None): self.timestamp = time.time() self.start = time.time() self.framerate = cap.get(cv2.CAP_PROP_FPS) if cap is not None else None self.frame_cnt = 0 self.ticks = {} self.ticks_cnt = {} def next_iter(self): if not args.camera: frame_delay = 1.0 / self.framerate delay = (self.timestamp + frame_delay) - time.time() if delay > 0: time.sleep(delay) self.timestamp = time.time() self.frame_cnt += 1 def tick(self, name): if name in self.ticks: self.ticks_cnt[name] += 1 else: self.ticks[name] = time.time() self.ticks_cnt[name] = 0 def tick_fps(self, name): if name in self.ticks: if (time.time() - self.ticks[name]) > 0: return self.ticks_cnt[name] / (time.time() - self.ticks[name]) else: return 0 else: return 0 def fps(self): if (self.timestamp - self.start) > 0: return self.frame_cnt / (self.timestamp - self.start) else: return 0 def frameNorm(frame, bbox): normVals = np.full(len(bbox), frame.shape[0]) normVals[::2] = frame.shape[1] return (np.clip(np.array(bbox), 0, 1) * normVals).astype(int) def to_planar(arr: np.ndarray, shape: tuple) -> list: # Could possibly be how you have to load images into nn return cv2.resize(arr, shape).transpose(2, 0, 1).flatten() # Handles loading images into a frame sequence queue dictionary def get_frame(): global frame_det_seq if args.video: read_correctly, frame = cap.read() if read_correctly: frame_seq_map[frame_det_seq] = frame frame_det_seq += 1 return read_correctly, frame else: in_rgb = cam_out.get() frame = in_rgb.getCvFrame() frame_seq_map[in_rgb.getSequenceNum()] = frame return True, frame if args.camera: fps = FPSHandler() else: cap = cv2.VideoCapture(str(Path(args.video).resolve().absolute())) fps = FPSHandler(cap) # define image storage variables frame_seq_map = {} frame_det_seq = 0 tattoo_detections = [] rec_results = [] tat_last_seq = 0 tat_last_img = np.empty([300, 300, 3]) labels = {1: '0', 2: '1', 3: '2', 4: '3', 5: '4', 6: '6', 7: '7', 8: '8', 9: '9', 10: 'X'} decoded_text = "" running = True carc_dict = {} found = 0 read = 0 offset = 20 chars_thresh = 4 pipeline = depthai.Pipeline() # create blank pipeline if args.camera: cam_rgb = pipeline.create(depthai.node.ColorCamera) # create color camera object cam_rgb.setPreviewSize(576, 576) # set camera preview size cam_rgb.setInterleaved(False) cam_rgb.initialControl.setManualFocus(110) cam_rgb.initialControl.setAutoWhiteBalanceMode(depthai.CameraControl.AutoWhiteBalanceMode.AUTO) # --DETECTION-- det_nn = pipeline.createMobileNetDetectionNetwork() # create tattoo detection mobilenet network det_nn.setBlobPath("C:\\Luxonis\\DETECT_BLOBS\\Detect_2_17_2022.blob") # configure path to blob det_nn.setConfidenceThreshold(0.5) # set confidence threshold det_nn.input.setQueueSize(1) det_nn.input.setBlocking(False) if args.rotate: if args.camera: manipRgb = pipeline.createImageManip() rgbRr = depthai.RotatedRect() rgbRr.center.x, rgbRr.center.y = cam_rgb.getPreviewWidth() // 2, cam_rgb.getPreviewHeight() // 2 rgbRr.size.width, rgbRr.size.height = cam_rgb.getPreviewHeight(), cam_rgb.getPreviewWidth() rgbRr.angle = 90 manipRgb.initialConfig.setCropRotatedRect(rgbRr, False) cam_rgb.preview.link(manipRgb.inputImage) # Resize camera preview and map it to tattoo detection nn input manip = pipeline.createImageManip() manip.initialConfig.setResize(300, 300) manip.initialConfig.setFrameType(depthai.RawImgFrame.Type.RGB888p) manipRgb.out.link(manip.inputImage) manip.out.link(det_nn.input) # Create output que for rgb camera images cam_xout = pipeline.createXLinkOut() cam_xout.setStreamName("cam_out") manipRgb.out.link(cam_xout.input) else: det_xin = pipeline.createXLinkIn() det_xin.setStreamName("det_in") det_xin.out.link(det_nn.input) else: if args.camera: # Resize camera preview and map it to tattoo detection nn input manip = pipeline.createImageManip() manip.initialConfig.setResize(300, 300) manip.initialConfig.setFrameType(depthai.RawImgFrame.Type.RGB888p) cam_rgb.preview.link(manip.inputImage) manip.out.link(det_nn.input) # Create output que for rgb camera images cam_xout = pipeline.createXLinkOut() cam_xout.setStreamName("cam_out") cam_rgb.preview.link(cam_xout.input) else: det_xin = pipeline.createXLinkIn() det_xin.setStreamName("det_in") det_xin.out.link(det_nn.input) # --OCR-- rec_nn = pipeline.createMobileNetDetectionNetwork() # create tattoo ocr mobilenet network rec_nn.setBlobPath("C:\\Luxonis\\READ_BLOBS\\read_2_16_2022.blob") # configure path to blob rec_nn.setConfidenceThreshold(0.4) # set confidence threshold rec_nn.input.setQueueSize(1) rec_nn.input.setBlocking(False) rec_xin = pipeline.createXLinkIn() rec_xin.setStreamName("rec_in") rec_xin.out.link(rec_nn.input) # Create output queue for tattoo detection nn detections det_nn_xout = pipeline.createXLinkOut() det_nn_xout.setStreamName("det_nn") det_nn.out.link(det_nn_xout.input) # Create output queue for tattoo detection nn image passthrough det_pass = pipeline.createXLinkOut() det_pass.setStreamName("det_pass") det_nn.passthrough.link(det_pass.input) # Create output queue for tattoo ocr nn detections rec_xout = pipeline.createXLinkOut() rec_xout.setStreamName("rec_nn") rec_nn.out.link(rec_xout.input) # Create output queue for tattoo ocr nn image passthrough rec_pass = pipeline.createXLinkOut() rec_pass.setStreamName("rec_pass") rec_nn.passthrough.link(rec_pass.input) # Unloads tattoo detection nn output q and loads results into OCR nn input def detect_thread(det_queue, det_pass, rec_queue): global tattoo_detections, tat_last_seq, tat_last_img, found while running: try: in_det = det_queue.get().detections in_pass = det_pass.get() orig_frame = frame_seq_map.get(in_pass.getSequenceNum(), None) tat_last_img = orig_frame if orig_frame is None: continue tat_last_seq = in_pass.getSequenceNum() tattoo_detections = in_det for detection in tattoo_detections: bbox = frameNorm(orig_frame, (detection.xmin, detection.ymin, detection.xmax, detection.ymax)) cropped_frame = orig_frame[bbox[1] - offset:bbox[3] + offset, bbox[0] - offset:bbox[2] + offset] found += 1 shape = cropped_frame.shape if shape[0] > 0 and shape[1] > 0: tstamp = time.monotonic() img = depthai.ImgFrame() img.setTimestamp(tstamp) img.setType(depthai.RawImgFrame.Type.BGR888p) img.setData(to_planar(cropped_frame, (300, 300))) img.setWidth(300) img.setHeight(300) carc_dict[img.getSequenceNum()] = [bbox, orig_frame] rec_queue.send(img) fps.tick('detect') except RuntimeError: continue # Loads cropped tattoo images from queue loaded by the tattoo detection nn def rec_thread(q_rec, q_pass): global rec_results, decoded_text, read with PLC('10.166.137.120') as comm: box_color_rec = (205, 0, 0) while running: try: # Get detections from queue of cropped frames from tattoo detection nn rec_data = q_rec.get().detections rec_frame = q_pass.get().getCvFrame() seq = q_pass.get().getSequenceNum() char_detections = [detection for detection in rec_data] except RuntimeError: continue # Get top four characters if len(char_detections) >= chars_thresh: raw_results = char_detections[:4] # Declare storage variables Xmin_Char = [] ocr = [] decoded_text = '' frame_copy = rec_frame.copy() # Create list of detections xmin position and detection label for detection in raw_results: Xmin_Char.append([detection.xmin, labels[detection.label]]) bbox = frameNorm(frame_copy, (detection.xmin, detection.ymin, detection.xmax, detection.ymax)) ocr.append([bbox, labels[detection.label]]) if args.display_OCR_bboxes: cv2.rectangle(frame_copy, (bbox[0], bbox[1]), (bbox[2], bbox[3]), box_color_rec, 2) cv2.putText(frame_copy, '{} ({}%)'.format(labels[detection.label], int(detection.confidence * 100)), (bbox[0] - 10, bbox[1] - 20), cv2.FONT_HERSHEY_TRIPLEX, 0.4, box_color_rec) # Sort previously created list by xmin position to provide left to right decoded text for detection in sorted(Xmin_Char, reverse=False): decoded_text += detection[1] # Create result image to stack rec_results = [(cv2.resize(frame_copy, (300, 300)), decoded_text)] + rec_results[:9] # Extract image and annotation information to save current_ocr_bbox_label = ocr current_ocr_img = frame_copy # Retrieve detection image and bounding box corresponding to current ocr image current_detect_data = carc_dict.get(seq) # Delete dictionary entry containing detection image and bounding box once they have been extracted if current_detect_data is not None: current_detect_bbox = current_detect_data[0] current_detect_img = current_detect_data[1] current_detect_bbox_label = [[current_detect_data[0], 'tattoo']] del carc_dict[seq] # Save files try: if len(char_detections) >= chars_thresh: read += 1 if args.Save_Data is not None: if os.path.isdir(args.Save_Data): uid = fh.save_files(args.Save_Data, decoded_text, current_detect_bbox_label, current_ocr_bbox_label, current_detect_img, current_ocr_img) if args.Use_PLC: filtered_result = th.cond_tattoo(decoded_text) request = [("Program:P01_MainPeriodicProgram.DINT_HotTattoo", filtered_result), ("Program:P01_MainPeriodicProgram.Tattoo_UID", int(uid))] w_ret = comm.Write(request) except Exception as e: print(e) fps.tick('OCR') # intialize device with context manager using created pipeline with depthai.Device(pipeline) as device: if args.camera: cam_out = device.getOutputQueue("cam_out", 1, True) else: det_in = device.getInputQueue("det_in") rec_in = device.getInputQueue("rec_in") det_nn = device.getOutputQueue("det_nn", 1, False) det_pass = device.getOutputQueue("det_pass", 1, False) rec_nn = device.getOutputQueue("rec_nn", 1, False) rec_pass = device.getOutputQueue("rec_pass", 1, False) # Start tattoo detection thread det_t = threading.Thread(target=detect_thread, args=(det_nn, det_pass, rec_in)) det_t.start() # Start tattoo ocr thread rec_t = threading.Thread(target=rec_thread, args=(rec_nn, rec_pass)) rec_t.start() def should_run(): return cap.isOpened() if args.video else True # Main loop try: fps_rgb = 0 ticks = 0 start = time.time() # start main loop while should_run(): read_correctly, frame = get_frame() # Load frame from camera into queue if not read_correctly: # Could be used later if video is incorporated break for map_key in list(filter(lambda item: item 0: fps_rgb = round(ticks * (1 / (time.time() - start)), 1) ticks = 0 start = time.time() cv2.putText(debug_frame, f"RGB FPS: {round(fps.fps(), 1)}", (5, 15), cv2.FONT_HERSHEY_SIMPLEX, 0.5, text_color) # Display RGB fps on screen cv2.putText(debug_frame, f"DETECT FPS: {round(fps.tick_fps('detect'), 1)}", (5, 30), cv2.FONT_HERSHEY_SIMPLEX, 0.5, text_color) # Display Detection fps on screen cv2.putText(debug_frame, f"OCR FPS: {round(fps.tick_fps('OCR'), 1)}", (5, 45), cv2.FONT_HERSHEY_SIMPLEX, 0.5, text_color) # Display OCR fps on screen cv2.putText(debug_frame, f"FOUND: {found}", (5, 60), cv2.FONT_HERSHEY_SIMPLEX, 0.5, text_color) # Display tattoos found count cv2.putText(debug_frame, f"READ: {read}", (5, 75), cv2.FONT_HERSHEY_SIMPLEX, 0.5, text_color) # Display tattoos read count cv2.imshow("rgb", debug_frame) # Display main detection frame rec_stacked = None # Create variable to house ocr result que for rec_img, rec_text in rec_results: # Loop current tattoo ocr result queue rec_placeholder_img = np.zeros((300, 70, 3), np.uint8) cv2.putText(rec_placeholder_img, rec_text, (5, 25), cv2.FONT_HERSHEY_TRIPLEX, 0.5, (0, 255, 0)) rec_combined = np.hstack((rec_img, rec_placeholder_img)) if rec_stacked is None: rec_stacked = rec_combined else: rec_stacked = np.vstack((rec_stacked, rec_combined)) if rec_stacked is not None: # If result queue has contents, display them cv2.imshow("Recognized tattoos", rec_stacked) key = cv2.waitKey(1) if key == ord('q'): break except KeyboardInterrupt: pass running = False det_t.join() rec_t.join() print("FPS: {:.2f}".format(fps.fps())) `

MobileNetDetection Node synch question

erik

Hello TylerD , could you rather provide minimal reproducible code? As the code above is quite domain-specific, not purely issue related.
Thanks, Erik

TylerD

Sorry about that...here is the pipeline setup and threads used. The problem with detections and passthroughs not seeming to be synchronized happens with both MobileNets.

pipeline = depthai.Pipeline()

cam_rgb = pipeline.create(depthai.node.ColorCamera)  # create color camera object
cam_rgb.setPreviewSize(576, 576)  # set camera preview size
cam_rgb.setInterleaved(False)
cam_rgb.initialControl.setManualFocus(110)

det_nn = pipeline.createMobileNetDetectionNetwork()  # create tattoo detection mobilenet network
det_nn.setBlobPath("C:\\Luxonis\\DETECT_BLOBS\\Detect_2_17_2022.blob")  # configure path to blob
det_nn.setConfidenceThreshold(0.5)  # set confidence threshold
det_nn.input.setQueueSize(1)
det_nn.input.setBlocking(False)

manipRgb = pipeline.createImageManip()
rgbRr = depthai.RotatedRect()
rgbRr.center.x, rgbRr.center.y = cam_rgb.getPreviewWidth() // 2, cam_rgb.getPreviewHeight() // 2
rgbRr.size.width, rgbRr.size.height = cam_rgb.getPreviewHeight(), cam_rgb.getPreviewWidth()
rgbRr.angle = 90
manipRgb.initialConfig.setCropRotatedRect(rgbRr, False)
cam_rgb.preview.link(manipRgb.inputImage)

manip = pipeline.createImageManip()
manip.initialConfig.setResize(300, 300)
manip.initialConfig.setFrameType(depthai.RawImgFrame.Type.RGB888p)
manipRgb.out.link(manip.inputImage)
manip.out.link(det_nn.input)

cam_xout = pipeline.createXLinkOut()
cam_xout.setStreamName("cam_out")
manipRgb.out.link(cam_xout.input)

rec_nn = pipeline.createMobileNetDetectionNetwork()  # create tattoo ocr mobilenet network
rec_nn.setBlobPath("C:\\Luxonis\\READ_BLOBS\\read_2_16_2022.blob")  # configure path to blob
rec_nn.setConfidenceThreshold(0.4)  # set confidence threshold
rec_nn.input.setQueueSize(1)
rec_nn.input.setBlocking(False)

rec_xin = pipeline.createXLinkIn()
rec_xin.setStreamName("rec_in")
rec_xin.out.link(rec_nn.input)

det_nn_xout = pipeline.createXLinkOut()
det_nn_xout.setStreamName("det_nn")
det_nn.out.link(det_nn_xout.input)

det_pass = pipeline.createXLinkOut()
det_pass.setStreamName("det_pass")
det_nn.passthrough.link(det_pass.input)

rec_xout = pipeline.createXLinkOut()
rec_xout.setStreamName("rec_nn")
rec_nn.out.link(rec_xout.input)

rec_pass = pipeline.createXLinkOut()
rec_pass.setStreamName("rec_pass")
rec_nn.passthrough.link(rec_pass.input)


def detect_thread(det_queue, det_pass, rec_queue):
    global tattoo_detections, tat_last_seq, tat_last_img

    while running:
        try:

            in_det = det_queue.get().detections
            in_pass = det_pass.get()
            orig_frame = frame_seq_map.get(in_pass.getSequenceNum(), None)

            tat_last_img = orig_frame

            if orig_frame is None:
                continue

            tat_last_seq = in_pass.getSequenceNum()
            tattoo_detections = in_det

            for detection in tattoo_detections:
                bbox = frameNorm(orig_frame, (detection.xmin, detection.ymin, detection.xmax, detection.ymax))
                cropped_frame = orig_frame[bbox[1] - offset:bbox[3] + offset, bbox[0] - offset:bbox[2] + offset]
                shape = cropped_frame.shape

                if shape[0] > 0 and shape[1] > 0:
                    tstamp = time.monotonic()
                    img = depthai.ImgFrame()
                    img.setTimestamp(tstamp)
                    img.setType(depthai.RawImgFrame.Type.BGR888p)
                    img.setData(to_planar(cropped_frame, (300, 300)))
                    img.setWidth(300)
                    img.setHeight(300)
                    rec_queue.send(img)

            fps.tick('detect')

        except RuntimeError:
            continue

def rec_thread(q_rec, q_pass):
    global rec_results, decoded_text

    while running:

        try:
            # Get detections from queue of cropped frames from tattoo detection nn
            rec_data = q_rec.get().detections
            rec_frame = q_pass.get().getCvFrame()
            seq = q_pass.get().getSequenceNum()

            char_detections = [detection for detection in rec_data]

        except RuntimeError:
            continue

        decoded_text = ''

        # Create list of detections xmin position and detection label
        for detection in rec_data:

            bbox = frameNorm(rec_frame, (detection.xmin, detection.ymin, detection.xmax, detection.ymax))
            cv2.rectangle(rec_frame, (bbox[0], bbox[1]), (bbox[2], bbox[3]), (205, 0, 0), 2)
            cv2.putText(rec_frame, '{} ({}%)'.format(labels[detection.label], int(detection.confidence * 100)), (bbox[0] - 10, bbox[1] - 20), cv2.FONT_HERSHEY_TRIPLEX, 0.4, (205, 0, 0))

        # Create result image to stack
        rec_results = [(cv2.resize(rec_frame, (300, 300)), decoded_text)] + rec_results[:9]

    fps.tick('OCR')

with depthai.Device(pipeline) as device:

    cam_out = device.getOutputQueue("cam_out", 1, True)
    rec_in = device.getInputQueue("rec_in")
    det_nn = device.getOutputQueue("det_nn", 1, False)
    det_pass = device.getOutputQueue("det_pass", 1, False)
    rec_nn = device.getOutputQueue("rec_nn", 1, False)
    rec_pass = device.getOutputQueue("rec_pass", 1, False)


    det_t = threading.Thread(target=detect_thread, args=(det_nn, det_pass, rec_in))
    det_t.start()

    rec_t = threading.Thread(target=rec_thread, args=(rec_nn, rec_pass))
    rec_t.start()

erik

Hello TylerD , from my understanding of the code you are running first object detection (mobilenet), then crop the image on the host, send it back to the device and run another object detection model. Instead of sending imgs/NN results back to host for cropping, I would suggest using Script node instead (for ImageManipConfigs used for cropping), see demo here.
Thanks, Erik

TylerD

Understood, I'll give that a try.

TylerD

When using the script node it freezes. I used the existing warn diagnostic messages from the demo. The following is the output from the warn diagnostics and error message in the order they occured.

[Script(4)] [warning] Detection rect: 0.34814453125, 0.529296875, 0.45703125, 0.611328125
[Script(4)] [warning] 1 from nn_in: 0.34814453125, 0.529296875, 0.45703125, 0.611328125
[ImageManip(5)] [error] Processing failed, potentially unsupported config.

erik

Hello TylerD , could you share the current script? Do you send high-definition frame to the ImageManip node, together with the cropping config?

TylerD

cam_rgb = pipeline.create(depthai.node.ColorCamera)
cam_rgb.setPreviewSize(300, 300)
cam_rgb.setInterleaved(False)


det_nn = pipeline.createMobileNetDetectionNetwork()
det_nn.setBlobPath("C:\\Luxonis\\DETECT_BLOBS\\Detect_2_17_2022.blob")
det_nn.setConfidenceThreshold(0.5)
det_nn.input.setQueueSize(1)
det_nn.input.setBlocking(False)
cam_rgb.preview.link(det_nn.input)


image_manip_script = pipeline.create(depthai.node.Script)
det_nn.out.link(image_manip_script.inputs['nn_in'])
cam_rgb.preview.link(image_manip_script.inputs['frame'])
image_manip_script.setScript("""
import time
def limit_roi(det):
    if det.xmin <= 0: det.xmin = 0.001
    if det.ymin <= 0: det.ymin = 0.001
    if det.xmax >= 1: det.xmax = 0.999
    if det.ymax >= 1: det.ymax = 0.999
while True:
    frame = node.io['frame'].get()
    tat_dets = node.io['nn_in'].get().detections
    node.warn(f"Tats detected: {len(tat_dets)}")
    for det in tat_dets:
        limit_roi(det)
        node.warn(f"Detection rect: {det.xmin}, {det.ymin}, {det.xmax}, {det.ymax}")
        cfg = ImageManipConfig()
        cfg.setCropRect(det.xmin, det.ymin, det.xmax, det.ymax)
        cfg.setResize(300, 300)
        cfg.setKeepAspectRatio(False)
        node.io['manip_cfg'].send(cfg)
        node.io['manip_img'].send(frame)
        node.warn(f"1 from nn_in: {det.xmin}, {det.ymin}, {det.xmax}, {det.ymax}")
""")


manip_crop = pipeline.create(depthai.node.ImageManip)
image_manip_script.outputs['manip_img'].link(manip_crop.inputImage)
image_manip_script.outputs['manip_cfg'].link(manip_crop.inputConfig)
manip_crop.initialConfig.setResize(300, 300)
manip_crop.inputConfig.setWaitForMessage(True)

ocr_nn = pipeline.createMobileNetDetectionNetwork() 
ocr_nn.setBlobPath("C:\\Luxonis\\READ_BLOBS\\read_2_16_2022.blob") 
ocr_nn.setConfidenceThreshold(0.4)
ocr_nn.input.setQueueSize(1)
ocr_nn.input.setBlocking(False)
manip_crop.out.link(ocr_nn.input)

erik

Could you try setting both Script inputs to blocking=False and QueueSize=1?

image_manip_script.inputs['nn_in'].setBlocking(False)
image_manip_script.inputs['nn_in'].setQueueSize(1)
image_manip_script.inputs['frame'].setBlocking(False)
image_manip_script.inputs['frame'].setQueueSize(1)

Thanks, Erik

AdamPolak

erik

Erik why was this the recommendation here?

I am having an issue with my set up where a lot of frames are being dropped, I assume because some node is taking a while and then frames are being replaced. So when I try to grab synched messages on the host sometimes up to 4 seq numbers will skip.

I wanted to update to setQueueSize(10) on all my nodes and setBlocking(True) .

Is there some queue limit size I should be considering on the device?
it seems like NN's don't have setQueueSize they have setNumPoolFrames, is this equivalent
Does nn nodes have an equivalent for setBlocking?

Are all device nodes default setQueueSize(3) and setBlocking(False) ?

TylerD

Sorry for the delay...I made the recommended changes. I also found I had to replace the .get() and .get().detections with tryGets() and check that detections were not None before proceeding with the ImageManip. I am still get the warning "Processing failed, potentially unsupported config" error, but it doesn't cause a problem anymore. Setting the setKeepAspectRatio(True) stops this error from occurring, but has a negative effect on the second network's performance. The cropped image is being correctly sent to the second MobileNet and it is returning good results. I have a few more things to add, but will be doing more field testing soon. Thanks for the support (and the great product) !!!

erik

Hi AdamPolak ,

No, RAM is the limitation
No, pool is the output, queue is the input, see docs here: https://docs.luxonis.com/projects/api/en/latest/components/device/

Yes:

nn = pipeline.create(dai.node.MobileNetDetectionNetwork)
nn.input.setBlocking(False)

jakaskerl

Hi AdamPolak

AdamPolak Are all device nodes default setQueueSize(3) and setBlocking(False) ?

https://docs.luxonis.com/projects/api/en/latest/components/device/#blocking-behaviour

Though, I have found that on some nodes (object tracker) the default is non blocking.

Thanks,
Jaka