Hi, The pipeline I have includes two color cameras, encoding nodes and detection networks for both cameras. Both cameras should run at 15 FPS. When I run the code, sometimes the pipeline will launch correctly, all FPS are as expected. See the following pipeline graph [upl-image-preview url=https://discuss.luxonis.com/assets/files/2023-12-05/1701760162-911509-screenshot-from-2023-12-05-00-51-34.png] But sometimes one camera will be totally block. No data can go through in that path. And for the other camera, the FPS is not even correct (in the following pipeline graph, see 18 FPS instead of 15 FPS). [upl-image-preview url=https://discuss.luxonis.com/assets/files/2023-12-05/1701760275-872493-screenshot-from-2023-12-05-00-52-21.png] The chance for each case is almost 50 percent. The issue only happens during launching. Once it launched correctly, it won't have problem running for hours. Here is the system log when the pipeline is running correctly. All CMX is used. Will it cause this problem? [upl-image-preview url=https://discuss.luxonis.com/assets/files/2023-12-05/1701760521-951208-screenshot-from-2023-12-05-00-56-23.png] Thanks Lincoln

Hi @"lincolnxlw"#p14933 This is very strange, could you send the minimal reproducible example so I can try to debug it? Are you using any multiprocessing/threading that could cause a race condition or something similar? Perhaps there is a bug somewhere on our side. Thanks, Jaka

@"jakaskerl"#p14958 Thanks for the reply. I will prepare something for you. Lincoln

Hi @"jakaskerl"#p14958 The [link](https://public.3.basecamp.com/p/jGEVkEJP3PgjHBSSFSv6aLnz) should contain the MRE for this issue. Hardware * OAK FFC-3P with 2 IMX477 modules running in 1080P in CAM B and CAM C sockets The MRE is an minimum ROS package depends on [depthai-ros noetic](https://github.com/luxonis/depthai-ros/tree/noetic). `root@83d31607bd1d:/workspaces/depthai_catkin_ws/src/minimum_image_publisher# tree . -h` `.` `├── [1.2K] CMakeLists.txt` `├── [4.0K] config` `│ └── [1.4K] yolov6n.yaml` `├── [4.0K] include` `│ └── [4.0K] minimum_image_publisher` `├── [4.0K] models` `│ └── [8.9M] yolov6n_openvino_2022.1_6shave.blob` `├── [2.0K] package.xml` `└── [4.0K] src` ` └── [ 11K] image_publisher.cpp` `5 directories, 5 files` There are also instructions inside the link for how to reproduce the problem. Attached below are two screenshots, one when the pipeline is running correctly, and one when the left side camera path is totally blocked. [upl-image-preview url=https://discuss.luxonis.com/assets/files/2023-12-12/1702423547-364730-screenshot-from-2023-12-12-16-51-26.png] [upl-image-preview url=https://discuss.luxonis.com/assets/files/2023-12-12/1702423550-647636-screenshot-from-2023-12-12-16-49-45.png] Let me know what do you think. Thanks Lincoln

@"jakaskerl"#1112 Just to follow up, any questions regarding the MRE I provided. Also, I created the exact same pipeline in python running with [depthai-library docker image](https://hub.docker.com/layers/luxonis/depthai-library/latest/images/sha256-be59799f694ef7cc5c60519bd462ac2daa6bef27bd2462e60ed37791121d3c29?context=explore), and I will run 10 times without any problem. I attach the python script below. ``` #!/usr/bin/env python3 from pathlib import Path import cv2 import depthai as dai import numpy as np import argparse import time import errno import os import sys import json parser = argparse.ArgumentParser() parser.add_argument('-nn', '--nn_model', help='select model path for inference', default='/workspaces/yolov6n_openvino_2022.1_6shave.blob', type=str) parser.add_argument('-c', '--config', help='Provide config path for inference', default='/workspaces/yolov6n.json', type=str) parser.add_argument('-fps', '--fps', type=float, help='Frame rate of camera capturing', default=15) args = parser.parse_args() nn_path = Path(args.nn_model) config_path = Path(args.config) config_fps = args.fps if not nn_path.is_file(): sys.exit('NN not found!') if not config_path.is_file(): sys.exit('Config not found!') with config_path.open() as f: config = json.load(f) nn_config = config.get("nn_config", {}) if "input_size" in nn_config: nn_width, nn_height = tuple(map(int, nn_config.get("input_size").split('x'))) metadata = nn_config.get("NN_specific_metadata", {}) classes = metadata.get("classes", {}) coordinates = metadata.get("coordinates", {}) anchors = metadata.get("anchors", {}) anchor_masks = metadata.get("anchor_masks", {}) iou_threshold = metadata.get("iou_threshold", {}) confidence_threshold = metadata.get("confidence_threshold", {}) print(metadata) nn_mappings = config.get("mappings", {}) labels = nn_mappings.get("labels", {}) # pipeline pipeline = dai.Pipeline() camera_l = pipeline.create(dai.node.ColorCamera) detection_nn_l = pipeline.create(dai.node.YoloDetectionNetwork) xout_rgb_l = pipeline.create(dai.node.XLinkOut) xout_nn_l = pipeline.create(dai.node.XLinkOut) camera_r = pipeline.create(dai.node.ColorCamera) detection_nn_r = pipeline.create(dai.node.YoloDetectionNetwork) xout_rgb_r = pipeline.create(dai.node.XLinkOut) xout_nn_r = pipeline.create(dai.node.XLinkOut) xout_rgb_l.setStreamName("rgb_l") xout_nn_l.setStreamName("nn_l") xout_rgb_r.setStreamName("rgb_r") xout_nn_r.setStreamName("nn_r") camera_l.setPreviewSize(nn_width, nn_height) camera_l.setBoardSocket(dai.CameraBoardSocket.CAM_B) camera_l.setPreviewKeepAspectRatio(False) camera_l.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1080_P) camera_l.setInterleaved(False) camera_l.setColorOrder(dai.ColorCameraProperties.ColorOrder.BGR) camera_l.setFps(config_fps) camera_r.setPreviewSize(nn_width, nn_height) camera_r.setBoardSocket(dai.CameraBoardSocket.CAM_C) camera_r.setPreviewKeepAspectRatio(False) camera_r.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1080_P) camera_r.setInterleaved(False) camera_r.setColorOrder(dai.ColorCameraProperties.ColorOrder.BGR) camera_r.setFps(config_fps) detection_nn_l.setConfidenceThreshold(confidence_threshold) detection_nn_l.setNumClasses(classes) detection_nn_l.setCoordinateSize(coordinates) detection_nn_l.setAnchors(anchors) detection_nn_l.setAnchorMasks(anchor_masks) detection_nn_l.setIouThreshold(iou_threshold) detection_nn_l.setBlobPath(str(nn_path)) detection_nn_l.setNumInferenceThreads(1) detection_nn_l.setNumNCEPerInferenceThread(1) detection_nn_l.input.setBlocking(False) detection_nn_r.setConfidenceThreshold(confidence_threshold) detection_nn_r.setNumClasses(classes) detection_nn_r.setCoordinateSize(coordinates) detection_nn_r.setAnchors(anchors) detection_nn_r.setAnchorMasks(anchor_masks) detection_nn_r.setIouThreshold(iou_threshold) detection_nn_r.setBlobPath(str(nn_path)) detection_nn_r.setNumInferenceThreads(1) detection_nn_r.setNumNCEPerInferenceThread(1) detection_nn_r.input.setBlocking(False) # linking camera_l.preview.link(xout_rgb_l.input) camera_l.preview.link(detection_nn_l.input) detection_nn_l.out.link(xout_nn_l.input) camera_r.preview.link(xout_rgb_r.input) camera_r.preview.link(detection_nn_r.input) detection_nn_r.out.link(xout_nn_r.input) with dai.Device(pipeline) as device: print('Device name:', device.getDeviceName()) if device.getBootloaderVersion() is not None: print('Bootloader version:', device.getBootloaderVersion()) print('Usb speed:', device.getUsbSpeed().name) print('Connected cameras:', device.getConnectedCameraFeatures()) device_info = device.getDeviceInfo() print('Device mixid:', device_info.getMxId()) q_rgb_l = device.getOutputQueue(name="rgb_l", maxSize=4, blocking=False) q_nn_l = device.getOutputQueue(name="nn_l", maxSize=4, blocking=False) q_rgb_r = device.getOutputQueue(name="rgb_r", maxSize=4, blocking=False) q_nn_r = device.getOutputQueue(name="nn_r", maxSize=4, blocking=False) start_time = time.monotonic() counter_l = 0 detections_l = [] fps_l = 0 frame_l = None counter_r = 0 detections_r = [] fps_r = 0 frame_r = None text_color = (0, 0, 255) def to_planar(arr: np.ndarray, shape: tuple) -> np.ndarray: return cv2.resize(arr, shape).transpose(2, 0, 1).flatten() def frameNorm(frame, bbox): normVals = np.full(len(bbox), frame.shape[0]) normVals[::2] = frame.shape[1] return (np.clip(np.array(bbox), 0, 1) * normVals).astype(int) def displayFrame(name, frame, detections): color = (0, 0, 255) for detection in detections: bbox = frameNorm(frame, (detection.xmin, detection.ymin, detection.xmax, detection.ymax)) cv2.putText(frame, labels[detection.label], (bbox[0] + 10, bbox[1] + 20), cv2.FONT_HERSHEY_TRIPLEX, 0.5, color) cv2.putText(frame, f"{int(detection.confidence * 100)}%", (bbox[0] + 10, bbox[1] + 40), cv2.FONT_HERSHEY_TRIPLEX, 0.5, color) cv2.rectangle(frame, (bbox[0], bbox[1]), (bbox[2], bbox[3]), color, 2) cv2.imshow(name, frame) while True: in_nn_l = q_nn_l.tryGet() in_rgb_l = q_rgb_l.tryGet() in_nn_r = q_nn_r.tryGet() in_rgb_r = q_rgb_r.tryGet() if in_rgb_l is not None: frame_l = in_rgb_l.getCvFrame() fps_l = counter_l / (time.monotonic() - start_time) cv2.putText(frame_l, "NN fps: {:.2f}".format(fps_l), (2, frame_l.shape[0] - 4), cv2.FONT_HERSHEY_TRIPLEX, 0.5, text_color) if in_rgb_r is not None: frame_r = in_rgb_r.getCvFrame() fps_r = counter_r / (time.monotonic() - start_time) cv2.putText(frame_r, "NN fps: {:.2f}".format(fps_r), (2, frame_r.shape[0] - 4), cv2.FONT_HERSHEY_TRIPLEX, 0.5, text_color) if in_nn_l is not None: detections_l = in_nn_l.detections counter_l += 1 if in_nn_r is not None: detections_r = in_nn_r.detections counter_r += 1 if frame_l is not None: displayFrame("left_camera", frame_l, detections_l) if frame_r is not None: displayFrame("right_camera", frame_r, detections_r) if cv2.waitKey(1) == ord('q'): break ``` The pipeline graph for the python version is the same [upl-image-preview url=https://discuss.luxonis.com/assets/files/2023-12-14/1702595965-120644-screenshot-from-2023-12-14-17-12-19.png] This [link](https://public.3.basecamp.com/p/SXDBt446Pz14wkLaXapEkuSt) contains the code and model to run this python MRE Thanks Lincoln

Hi @"lincolnxlw"#1387 , Thank you for the MRE. Just to confirm, are you using latest depthai version (2.24)?

Hi @"erik"#p15213 For the python example, I am using this docker image that release yesterday https://hub.docker.com/layers/luxonis/depthai-library/latest/images/sha256-be59799f694ef7cc5c60519bd462ac2daa6bef27bd2462e60ed37791121d3c29?context=explore It looks like it is using "'2.23.0.0.dev0+dev'" For the ROS example, I rebuild it with this [docker file](https://github.com/luxonis/depthai-ros/blob/noetic/Dockerfile) yesterday and still having the issue. Let me know what do you think Lincoln

Hi @"lincolnxlw"#1387 I tried to reproduce with normal OAK-D (so 2x OV9282), but it worked as expected every time (out of 20x) running latest depthai. @"jakaskerl"#1112 , could you perhaps try to repro with FFC-3P and 2x IMX477 cam modules? I'd be surprised that would make a difference though. Perhaps the camera that fails doesn't have perfect connection, and it sometimes doesn't get enumerated, so it doesn't stream?

Hi @"lincolnxlw"#p15214 Tried with FFC-3P (r3) and two IMX477 connected to cam_L and cam_R. > @"lincolnxlw"#p15204 This [link](https://public.3.basecamp.com/p/SXDBt446Pz14wkLaXapEkuSt) contains the code and model to run this python MRE This link/code works without issues on my machine. Versions: depthai==2.24.0.0. Tried running 20 times, no errors/crashes. Couldn't setup the ROS version due to USB passthrough issues on M1 macs. @"Luxonis-Adam"#919 Can you check with ROS noetic please, so we can pinpoint the issue to a source? link: > @"lincolnxlw"#p15147 The [link](https://public.3.basecamp.com/p/jGEVkEJP3PgjHBSSFSv6aLnz) should contain the MRE for this issue. Thanks, Jaka

Hi @"jakaskerl"#1112 and @"erik"#175 Just want to make sure we are on the same page. I don't have issue running the python MRE, like I mentioned earlier > Also, I created the exact same pipeline in python running with **[depthai-library docker image](https://hub.docker.com/layers/luxonis/depthai-library/latest/images/sha256-be59799f694ef7cc5c60519bd462ac2daa6bef27bd2462e60ed37791121d3c29?context=explore)**, and I will run 10 times without any problem. I attach the python script below. The one has blocking issue is the ROS c++ MRE. It is weird because the pipeline is pretty much the same. Thanks Lincoln

Hi, regarding ROS Noetic on Mac, I don't it is officially supported, but you could use Docker images to get the latest version running. In general, if it's possible to connect to the camera with basic DepthAI libraries then it should also work for ROS. Could you share your ROS setup?

Loading...

Something went wrong while trying to load the full version of this site. Try hard-refreshing this page to fix the error.

Pipeline sometimes not being launched as expected

« Previous Page