A new version of yolo has just been released, yolov10. Does luxonis plan to support this new version on their cameras?

Hi @"EwoudPool"#1266 We don't yet have a conversion for V9, so can't say for sure. If we get enough customer demand, we will implement it. Not sure how much of an improvement it would be though. Thanks Jaka

@"jakaskerl"#p22996 Yolo V10 is a significant upgrade, we'd like to see it supported asap. See attached [upl-image-preview url=https://discuss.luxonis.com/assets/files/2024-05-27/1716819893-740913-screenshot-2024-05-27-at-162356.png]

In addition to higher MAP, YOLOv10 has far fewer params to the point where the 's' size model may be practical to run on the Myriad (depending on how many operators spill over). Would you be able to open source the firmware so the community can make these changes itself rather than Luxonis having to do this internally each time a new model comes out? Previously Intel has been a blocker to open sourcing the parts relating to the MX, but is this still the case? What about with RVC4?

Hi @"rmaxwell"#1300 I doubt we can make it opensource (not sure on the contracts). I have passed the feature-request to our ML team. Thanks, Jaka

Hey, Thanks for the thread everyone! We are actively working on adding support for YoloV10 to tools.luxonis.com and device (FW). No ETA we can commit to, but should be in the next few weeks. Unfortunately it is not easy for us to open source the firmware, so I'll have to ask for some patience. We are aware of constraints on that, so we are looking to simplify the process with RVC4. While I doubt we could expose FW directly, we are working on concept of HostNodes. Those can then run on your host in case of RVC2 (this might be inefficient if you are sending data back to the device) or directly on the device in case of RVC4.

Hi all, Looking forward to the v10 release, will it be available from the conversion API with version: Literal["v10"] = "v10" and at the URL "https://tools.luxonis.com/yolov10"? Thanks, Chris

@"ChrisCoutureDelValle"#p23072 Likely yes for the version, can't yet guarantee for the URL.

Hi all, A small update on adding YOLOv10 support: 1. We are waiting for the YOLOv10 support in the Ultralytics library to be able to add it to tools.luxonis.com. Otherwise, it's ready in the development environment. 2. We are working on adding FW YOLOv10 post-processing support.

@"NikitaSokovnin"#p23421 Awesome thanks for the update.

Hello all, Just wondering on progress for v10 availability. Thanks, Chris

Support for V10

alexandrebenoit

Well, it would be nice if you could share :

the list of problematic yolov10 element-wise operations that are not supported by RVC2
the list of all available ops that the RVC2 can perform (a synthetic datasheet)

For example, if the Pixel-Adaptive Convolutions (PAC) operator is applicable for RVC2, then element-wise products could be replaced by PAC with some specific setup and so on (not as efficient as the base element-wise product on standard processors but this could compensate in some setups, maybe RVC2).

Alex

Matija

alexandrebenoit

Hey, we didn't do a deep dive into slow operations to point out where exactly the issue lies. Based on experience, I would say it's slow because of:

A lot of splitting, slicing, and concatenations.
SiLU - you can see there are a lot of "branch-outs" due to SiLU activation. Comparing this with YoloV6 which uses ReLU and reparametrization trick like in RepVGGs, you can see V8 or similar is slower.
MHSA module definitely doesn't help and is likely the "cherry on top".

You can see the ONNX file we use here for reference. Feel free to compile this to blob and benchmark it. If you want to dive into optimization a bit yourself you can use this as the baseline.

If you want per-op performance, OpenVINO provides also a benchmark app that can return per-layer latencies. Note that we are more focused on releasing this rather than optimizing, given that the gain for nano version is 1% mAP compared to V6.

Matija

To add, this seems to hold also on other HW (see relevant issue here). While paper optimizes for computational cost and parameter count, those do not always strongly correlate with the throughput and latency - typically, certain well-known operations might have more ops/params, but can execute faster.

alexandrebenoit

Yes, indeed, this is then mostly related to data exchange bottlenecks and some complex functions such as Silu.

Then, next steps could be Yolov10 engineering to adapt to hardware or on the research side to look for v11 ;o)

In this community, who would be interested in a given direction for collab ?

Alex

NikitaSokovnin

Hi all,

A new version of DepthAI tools with support for YOLOv10 export is deployed. Decoding on the device can be tested using this script.

alexandrebenoit

Hi,

thanks !

Well, regarding the provided online tools, is there any up to date standardized performance comparison table on a single or multiple Luxonis products ?

I saw some tables in the doc but it would be great if the update datetime and maybe model version could be provided.

Alex

jakaskerl

Hi alexandrebenoit
Which tables do you have in mind exactly?

Thanks,
Jaka

alexandrebenoit

Hi, this one:

https://docs.luxonis.com/hardware/platform/rvc/rvc2#rvc2-nn-performance

Maybe model version, hardware firmware version and timestamp could be added but i know this is boring to maintain while it allows the user to rapidly get a confident information to help choose the right model.

Alex

ChrisCoutureDelValle

Hi Team,

Just a quick follow up, what do I need to change here? Last used this for v7.

Code:
_URL = "https://tools.luxonis.com" #"http://tools.luxonis.com/upload" _OUTPUT_FILE_NAME = "output.zip" _FRACTIONS = { "none": 0, "read": 0.1, "initialized": 0.3, "onnx": 0.5, "openvino": 0.65, "blob": 0.8, "json": 0.9, "zip": 1 }

`def convert_yolo(file_path: str, shape: Union[int, Tuple[int, int]] = 416, version: Literal["v10"] = "v10"):
files = {'file': open(file_path, 'rb')}
values = {
'inputshape': shape if isinstance(shape, int) else " ".join(map(str, shape)),
'version': version,
'id': uuid4()
}
file_name = _OUTPUT_FILE_NAME
url = f"{_URL}/upload"
print(url)

# progress bar
proc = multiprocessing.Process(target=get_progress, args=(str(values["id"]),))
proc.start()

# upload files
session = requests.Session()
with session.post(url, files=files, data=values, stream=True) as r:
    r.raise_for_status()
    proc.terminate()
    print(f"Conversion complete. Downloading...")

    with open(file_name, 'wb') as f:
        for chunk in r.iter_content(chunk_size=8192):
            # If you have chunk encoded response uncomment if
            # and set chunk_size parameter to None.
            # if chunk:
            f.write(chunk)
return file_name`

Output:
https://tools.luxonis.com/upload

Progress

HTTP error occurred: 520 Server Error: UNKNOWN for url: https://tools.luxonis.com/upload

jakaskerl

cc @JanCuhel

JanCuhel

Hi @ChrisCoutureDelValle,

you can use this script:

import requests
import multiprocessing
from typing import Union, Tuple, Literal
from uuid import uuid4
import argparse


_URL = "https://tools.luxonis.com" #"http://tools.luxonis.com/upload" _OUTPUT_FILE_NAME = "output.zip" _FRACTIONS = { "none": 0, "read": 0.1, "initialized": 0.3, "onnx": 0.5, "openvino": 0.65, "blob": 0.8, "json": 0.9, "zip": 1 }
_OUTPUT_FILE_NAME = "output.zip"


def get_progress(id: str):
    while True:
        try:
            r = requests.get(f"{_URL}/progress/{id}")
            r.raise_for_status()
            data = r.json()
            print(f"Progress: {data['progress']}")
            if data["progress"] == 1:
                break
        except Exception as e:
            print(f"Error: {e}")
            break


def convert_yolo(file_path: str, shape: Union[int, Tuple[int, int]] = 416, version: Literal["v10"] = "v10"):
    files = {'file': open(file_path, 'rb')}
    values = {
        'inputshape': shape if isinstance(shape, int) else " ".join(map(str, shape)),
        'version': version,
        'id': uuid4()
    }
    file_name = _OUTPUT_FILE_NAME
    url = f"{_URL}/upload"
    print(url)

    # progress bar
    proc = multiprocessing.Process(target=get_progress, args=(str(values["id"]),))
    proc.start()

    # upload files
    session = requests.Session()
    with session.post(url, files=files, data=values, stream=True) as r:
        r.raise_for_status()
        proc.terminate()
        print(f"Conversion complete. Downloading...")

        with open(file_name, 'wb') as f:
            for chunk in r.iter_content(chunk_size=8192):
                # If you have chunk encoded response uncomment if
                # and set chunk_size parameter to None.
                # if chunk:
                f.write(chunk)
    return file_name


def main():
    parser = argparse.ArgumentParser(description="Convert YOLO models")
    parser.add_argument("path", type=str, help="Path to the model's weights")
    args = parser.parse_args()
    convert_yolo(args.path)


if __name__ == "__main__":
    main()

I tested it with yolov10 nano from Ultralytics and it worked. Btw, if you'd be looking for an inspiration how to write a call api to our tools, you can check out this script.

Best,
Jan

ChrisCoutureDelValle

JanCuhel Thanks but that is effectively what I'm using, here are the vars Im sending to the convert yolo function and still with the updated function error remains the same.

./runs/detect/yolov10n_640x640_model/weights/best.pt (640, 640) v10
https://tools.luxonis.com/upload
Progress: none
Progress: none
Progress: new
HTTP error occurred: 520 Server Error: UNKNOWN for url: https://tools.luxonis.com/upload

arnavaggs

JanCuhel

Hi Jan,

I used the script that you sent over to try and convert a custom yolov10n model that I trained and got the same error as Chris. Do you know why I could still be getting this error?

Thanks,

Arnav

JanCuhel

This seems like that your models are for some reason not compatible with the ones we are testing on. Could you please (both of you @ChrisCoutureDelValle and @arnavaggs) provide us with more information, for example, specify what is the source of the model you've used (e.g. did you downloaded from Ultralytics)? It would help me a lot if you'd share your models with me, so that I could take a closer look. You can either share it with me via Google Drive or you can send it to me via email. My email is jan.cuhel@luxonis.com.

Best,
Jan

ChrisCoutureDelValle

JanCuhel Following up its a standard custom trained model leveraging these weights. THU-MIG/yolov10releases/download/v1.1/yolov10n.pt

ChrisCoutureDelValle

JanCuhel I see that for the unit test, the Luxonis team is leveraging these weights. I have been using the weights directly from the yolov10 repo, should the weights from the v10 repo and ultralytics weights not be identical?

`'yolov10n': 'https://github.com/ultralytics/assets/releases/download/v8.2.0/yolov10n.pt',`

'yolov10s': 'https://github.com/ultralytics/assets/releases/download/v8.2.0/yolov10s.pt',

'yolov10m': 'https://github.com/ultralytics/assets/releases/download/v8.2.0/yolov10m.pt',

'yolov10b': 'https://github.com/ultralytics/assets/releases/download/v8.2.0/yolov10b.pt',

'yolov10l': 'https://github.com/ultralytics/assets/releases/download/v8.2.0/yolov10l.pt',

'yolov10x': 'https://github.com/ultralytics/assets/releases/download/v8.2.0/yolov10x.pt',

JanCuhel

@ChrisCoutureDelValle yes, you're right, we are using weights from Ultralytics. I'll check whether the weights from Ultralytics differ from the ones you are using and get back to you.

Best,

Jan

ChrisCoutureDelValle

JanCuhel Hi Jan, just as a follow up I attempted to upload a custom model trained on the ultralytics weights in the same fashion and ran into the same error.

ChrisCoutureDelValle

In addition when manually uploading the custom v10 weights from the ultralytics weights and the standard v10 weights and this is the error that occurs, I believe there may possibly be an error in the backend when detecting the custom v10 weights?
Automatic version detected: YoloV8 (detection only)

JanCuhel

ChrisCoutureDelValle

Have you tried to manually select YOLOv10 export option? And if not, could you please try that?

Best,
Jan

ChrisCoutureDelValle

JanCuhel When I try to manually select the YOLOv10 export option with the weights, I get this in response "Automatic version detected: YoloV8 (detection only)".

« Previous Page Next Page »