Hello I have run both the nano and small version for the yolov8 on both camera's i get good detections on small but low frames on the other hand i get decent detections on the nano and good frames what my final goal is person detection even if the person is far away,good accuracy,real time do i have to put the load on the gpu of a pc rather than the camera to try out heavier models for higher accuracy Should i try other camera's? any comments is helpful

Hi @"krishnashravan"#p9591 The performance across all oak devices should be pretty similar. Since you need person detection, what about using a model that specializes in that aspect alone (stock yolo models have ~80 classes which hinders the speed of inference)? Also make sure you are running the NN in optimal mode - meaning correct amount of shaves that allow for 2 inference threads. ([docs](https://docs.luxonis.com/projects/api/en/latest/tutorials/low-latency/#reducing-latency-when-running-nn)) Instead of connecting the device to the PC, consider using something like the nvidia jetson series, which are designed for such tasks. Thoughts? Jaka

@"jakaskerl"#p9597 i have tried many person only detection models like pedestrian-detection-adas-0002,person-detection-0203, but it cant detect people far away very well yolo was one of thr better ones thats why i tried that any other models u can recommend i will check out the jetson series as well

Hi @"krishnashravan"#p9598 Perhaps something like https://docs.openvino.ai/2022.3/omz_models_model_person_detection_retail_0002.html would perform better at higher distances, but I feel it's going to be even slower. Thanks, Jaka

@"jakaskerl"#p9603 i have tried both the person detection retail 0002 and person detection retail 0013 as well even tho they give good results when people are close to the camera but when dealing with far away detections of people it struggles Do u think i should custom train any of these models i have custom trained only yolo v7 and v8 models any suggestions on which jetson models would be good enough for the heavier models? thinking of **Jetson Orin Nano 8GB Module** the custom trained v8 model (with people class only gives) doesnt give any detections I have attached the files [here](https://github.com/Krishna18698/yolov8-testing-pt-files/tree/main/yolo(people%20class)) this is trained with around 5000 images using the yolov8 custom training notebook. the quick fix that was suggested for the masks and anchor masks was solved for the 80 classes weights that is anchors: [10.0, 13.0, 16.0, 30.0, 33.0, 23.0, 30.0, 61.0, 62.0, 45.0, 59.0, 119.0, 116.0, 90.0, 156.0, 198.0, 373.0,326.0], anchor_masks: {"side52": [0, 1, 2], "side26": [3, 4, 5], "side13": [6, 7, 8]}, but the above values dont work when i used it on the custom model only the video shows up no detections and the value for the detections is also empty

Hi @"krishnashravan"#p9625 You could probably apply the transfer learning to just add detections for distant people detection on top of the already existing (and optimized) detection. > @"krishnashravan"#p9625 any suggestions on which jetson models would be good enough for the heavier models? I'm not sure I know enough to give a definitive answer to this. The orin nano features 40TOPS ai performance (3 TOPS on luxonis RVC3), so on paper it's much faster, I'm just not knowledgeable enough to know what this means for model performance. Perhaps anyone else reading this has better understanding. > @"krishnashravan"#p9625 the quick fix that was suggested for the masks and anchor masks was solved for the 80 classes weights cc @"Matija"#379; Afaik, the json obtained from tools.luxonis.com should have the anchors and anchor masks values. Thanks, Jaka

@"jakaskerl"#p9637 U mean add transfer learning on existing person detention models? The files that i posted were contents of the zip file u get after using the tools file The masks and anchor masks were empty i filled them with the values that were suggested it worked for normal nano and small weights but not for the custom one.

Do you have some example images about detection of people in the distance? Mainly to see what you're targeting. While heavier models could help, I believe the problem here is that resolution is not big enough, and so the distance people are just too small for the model to be able to detect them well. There are two ways that could solve that -- either a very high input resolution, which is slow. I am not sure how fast it would be on a Jetson -- my assumption is it should be faster since it has more tops, but since you increase the amount of operations by a lot it's hard to say what the FPS would be. Another option that you have, is that instead of using default [yolov8.yaml](https://github.com/ultralytics/ultralytics/blob/main/ultralytics/models/v8/yolov8.yaml) when finetuning/transfer learning, you use the [yolov8-p2.yaml](https://github.com/ultralytics/ultralytics/blob/main/ultralytics/models/v8/yolov8-p2.yaml). You can see in the config in L40 it says xsmall P2, and then that P2 is passed to the Detect head in the last line. This means that the model will be trained to detect much smaller objects, while the throughput should remain more or less the same. So you can try fine-tuning with that head. @"JanCuhel"#1367 , can we check why the converted model linked above is not detecting anything unless the mentioned fix is applied? And can we try exporting a model trained with yolov8-2.yaml config using tools.luxonis.com. @"krishnashravan"#1086 What DepthAI version are you using?

@"Matija"#p9647 depthai version is 2.21.2.0 [here](https://github.com/Krishna18698/yolov8-testing-pt-files/tree/main/yolo(people%20class)) i have added the pt file that i used to convert in tools i trained using the images from [here](https://universe.roboflow.com/shreks-swamp/coco-dataset-limited--person-only/dataset/1)

@"Matija"#p9647 [URL=https://i.imgur.com/dW7M1eu.jpg][IMG]https://i.imgur.com/dW7M1euh.jpg[/IMG][/URL] [URL=https://i.imgur.com/OK2kXio.jpg][IMG]https://i.imgur.com/OK2kXioh.jpg[/IMG][/URL] [URL=https://i.imgur.com/2vQgo1d.jpg][IMG]https://i.imgur.com/2vQgo1dh.jpg[/IMG][/URL] [URL=https://i.imgur.com/d8DaU1k.jpg][IMG]https://i.imgur.com/d8DaU1kh.jpg[/IMG][/URL]

@"Matija"#p9647 so are u saying that when training a custom model on the yolo v8 yolo task=detect mode=train model=yolov8n.pt imgsz=640 data=yolov8.yaml epochs=100 name=yolov8n my yaml file looks like this train: C:\\computervision\\yolov8_custom\\train val: C:\\computervision\\yolov8_custom\\val nc: 1 names: ["person"] i can add all the information about the layers that was in the p2 file here and then train or i have to do it some other way i'm new to fine tuning so any help would be great how would i train a custom model using the new p2 file i'm not sure how to do that.

running heavier models on oak-d, OAK-D CM4 PoE

JanCuhel

@krishnashravan we looked into the memory issue when exporting the YoloV8 with P2 head and found the issue is due to the softmax in the model. Unfortunately, there isn't anything we can with this. So, I'd suggest to you to try the options that @Matija has suggested to you with YoloV8 P2 head. Or alternatively, you can also try to use YoloV6 with higher resolution (but it has just 3 detection heads) or YoloV5 with P2 head. Here's the link to the config file. Be aware that this model is relatively big and so the inference will be very slow. To train this model you need to specify the config file using --cfg flag, e.g. to start the training/finetuning you can use this command python train.py --data coco128.yaml --weights yolov5n.pt --cfg yolov5-p2.yaml --device 0. More info can be found here.

krishnashravan

JanCuhel
the inference being in real time is the main focus
i'm doing inferencing on two videos besides live inference
made it work on the camera the far away video is at normal speed with blocking=False
if its true the video is slow.
with the close video (like a cctv video) video with blocking True the video is at normal pace with detections good 80-90% of the times.
with blocking False the video is fast
anyway to fix this to make it so that the video runs at a fixed fps any thing i can change ?
if video: q_rgb = device.getOutputQueue("manip") q_in = device.getInputQueue(name="inFrame", maxSize=4, blocking=True) videoPath = str((parentDir / Path('../../data/' + video_source)).resolve().absolute()) cap = cv2.VideoCapture(videoPath) inputFrameShape = (sizeX, sizeY)

krishnashravan

JanCuhel
both the yolo v6 and v5 will give lower frames?
in the oak d?

JanCuhel

@krishnashravan for more information about the blocking please look here.
What exactly do you mean by "lower frames"?

krishnashravan

JanCuhel
by lower frames i mean is it @ 30 frames or lower?

JanCuhel

@krishnashravan that depends on the input size that you use and also size (nano, small, medium, etc.) of the model. Generally speaking, out of the Yolo models, YoloV6 is the fastest. You can see the perfomances of several model, their sizes and inputs shapes in here or here.

« Previous Page