How to convert a YOLO instance segmentation model for C++ usage?

Goulart

I'm trying to convert my YOLO instance segmentation model from PyTorch to a format that is acceptable by "dai::node:: DetectionNetwork" to be used by a OAK 4D PRO in standalone mode.

The example found in the "ai-tutorials" repo (yolo11_instance_segmentation_training.ipynb) based on HubAI, doesn't seem to produce the format expected by "DetectionNetwork". I also couldn't generate the expected format with "tools" or "modelconverter".

I know it is possible because depthai-core repo has an example (detection_and_segmentation.cpp) that loads "luxonis/yolov8-instance-segmentation-large:coco-640x352" and it works.

The key difference seems to be some sort of a second stage model "mult_640x352.dlc" for postprocessing.

What are the steps to create a YOLO instance segmentation model (either YOLOv8, YOLO11, YOLO26) that can be used in C++?

Thanks for the help.

OskarSonc

cc @KlemenSkrlj

KlemenSkrlj

Hi Goulart,
We'll investigate the issues you are facing internally. Could you provide a bit more step by step description of what you've done so we can recrete the flow 1:1?

And in the meantime could you try using QuickConversion through Hub (selecting YOLO at the top and using .pt file as an input) to convert the model? This should generate the correct model NNArchive which you can then use in the downstream task.
Let me know how it goes.
Best,
Klemen

Goulart

Thanks for the help Klemen.

QuickConversion works. This tool also made me realise what I was doing wrong: incompatible snpe version.

Step-by-step I followed when attempting to convert a yolo11m-seg (followed model conversion):

onnx conversion

1.1: setup tools-cli
1.2: conversion command: tools yolo11m-seg.pt --imgsz "640 640" --no-use-rvc2
ModelConverter

2.1: pip install modelconv
2.2: created folders and copied onnx model to folder archives
2.3: conversion command: modelconverter convert rvc4 --path archives/yolo11m-seg.tar.xz --to nn_archive rvc4.quantization_mode FP16_STANDARD
-- I got the following error:
*Failed to pull the image, building it locally... docker_utils.py:434
WARNING SNPE archive not found at .build/rvc4/modelconverter-0.5.4-beta/docker/extra_packages/snpe-2.32.6.zip; attempting download from https://softwarecenter.qualcomm.com/api/download/software/sdks/Qualcomm_AI_Runtime_Community/All/2.32.6/v2.32.6.zip docker_utils.py:234

RuntimeError: Failed to download SNPE archive from https://softwarecenter.qualcomm.com/api/download/software/sdks/Qualcomm_AI_Runtime_Community/All/2.32.6/v2.32.6.zip: HTTP Error 404: Not Found. Download it manually from https://softwarecenter.qualcomm.com/catalog/item/Qualcomm_AI_Runtime_Community and save it as .build/rvc4/modelconverter-0.5.4-beta/docker/extra_packages/snpe-2.32.6.zip.*

2.4: Manually downloaded the latest snpe version from: https://softwarecenter.qualcomm.com/catalog/item/Qualcomm_AI_Runtime_Community. Currently, the defalult version is 2.46.
2.5: moved to the requested folder and executed the command again.
Attempted to load converted model in C++:

constexpr float FPS = 15.0f;
device_ = std::make_shared<dai::Device>();
pipeline_ = std::make_unique<dai::Pipeline>(device_);
auto camColour = pipeline_->create<dai::node::Camera>()->build(dai::CameraBoardSocket::CAM_A);
auto camLeft = pipeline_->create<dai::node::Camera>()->build(dai::CameraBoardSocket::CAM_B);
auto camRight = pipeline_->create<dai::node::Camera>()->build(dai::CameraBoardSocket::CAM_C);

auto objDetector = pipeline_->create<dai::node::DetectionNetwork>();
auto stereo = pipeline_->create<dai::node::NeuralDepth>();
auto pc = pipeline_->create<dai::node::PointCloud>();
auto imageAlign = pipeline_->create<dai::node::ImageAlign>();
auto sync = pipeline_->create<dai::node::Sync>();

dai::NNArchive modelArchive("/ws2/dnn_models/yolo11m-seg.rvc4.tar.xz");
auto imgsz = modelArchive.getInputSize().value();
objDetector->build(camColour, modelArchive, FPS, dai::ImgResizeMode::CROP);

auto leftOut = camLeft->requestFullResolutionOutput(std::nullopt, FPS);
auto rightOut = camRight->requestFullResolutionOutput(std::nullopt, FPS);
stereo->build(*leftOut, *rightOut, dai::DeviceModelZoo::NEURAL_DEPTH_864X540);

pc->initialConfig->setLengthUnit(dai::LengthUnit::METER);
pc->initialConfig->setOrganized(true);
pc->setTargetCoordinateSystem(dai::CameraBoardSocket::CAM_A);
pc->useCPUMT();

stereo->depth.link(imageAlign->input);
objDetector->passthrough.link(imageAlign->inputAlignTo);
imageAlign->outputAligned.link(pc->inputDepth);
//objDetector->passthrough.link(sync->inputs["colour"]);
objDetector->out.link(sync->inputs["dets"]);
pc->outputPointCloud.link(sync->inputs["cloud"]);

queue_ = sync->out.createOutputQueue(1, false);

pipeline_->start();
device_->setIrLaserDotProjectorIntensity(0.7f);
device_->setIrFloodLightIntensity(0.3f);
callback_id_ = queue_->addCallback(
[this](std::shared_ptr<dai::ADatatype> data) {
auto msg = std::dynamic_pointer_cast<dai::MessageGroup>(data);
if (msg) {
auto dets = msg->get<dai::ImgDetections>("dets");
auto cloud = msg->get<dai::PointCloudData>("cloud");
auto seg_msk = dets->getSegmentationMask();
if (!cloud || !dets || !seg_msk)
return;
const auto mask_data = seg_msk->getData();
…}});

Code compiles and runs ("oakctl app run ."); however, model always reported 0 detections.

After my experience with the QuickConversion, I replaced the snpe version I downloaded (2.46) with 2.32.6 and attempted to convert the model again with ModelConvert. This time, the converted model worked.

I would recommend an update to the ModelConverter page. I might be missing something, but I don't see anything stating that ModelConverter requires version 2.32.6. In my view, In the sentence "For RVC4, a full SNPE build version such as 2.32.6….", the term 'such as' indicates that other versions should work.

Some questions:

Since the resulting converted archive file contains only 1 dlc model (rather than 2 like "luxonis/yolov8-instance-segmentation-large"), am I correct to assume that mask post processing is done by the CPU instead of a neural network?
When converting a model using modelconvert, I added "--to nn_archive" to the command. How and when should I use the native version (which seems to give just a .dlc file)? Coulnd't find documentation about it.
(extra) Am I doing anything wrong/inefficiently in the code snippet I provided? For my use case, I want to extract data from the pointcloud based on the incoming instance segmentation masks.

Thanks for the support.

Regards,

Goulart

aljaz

Hi Goulart !
For your 3. point the code already looks pretty good! There is one change I would make at this stage:
Create a custom ThreadedHostNode (example) which would apply filtering of depth based on the segmentation mask. The implementation would be something like:

class SegmentationFilterNode : public dai::node::CustomThreadedNode<SegmentationFilterNode> {
   public:
    Input inputDetections = dai::Node::Input{*this, {}};
    Input inputDepth = dai::Node::Input{*this, {}};
    Output outputDepth = dai::Node::Output{*this, {}};

    void run() override {
        while(mainLoop()) {
            auto detections = inputDetections.get<dai::ImgDetections>();
            auto depth = inputDepth.get<dai::ImgFrame>();

            auto segmentationMask = detections->getSegmentationMask();
            // if you have opencv support, you can use getCvSegmentationMaskByClass funciton to get a filtered cv::Mat

            // filter the seg mask for only the class / instance you want
            // create a new depth frame with depth values only for the pixels that match the filtered seg mask and 0 for the rest
            // dont forget to copy the metadata from the original depth frame to the new filtered depth frame

            outputDepth.send(filteredDepth);
        }
    }
};

you can insert the node after ImageAling and link its output to the pointcloud node:

stereo->depth.link(imageAlign->input);
objDetector->passthrough.link(imageAlign->inputAlignTo);

auto segFilter = pipeline_->create<dai::node::SegmentationFilterNode>();
objDetector->out.link(segFilter->inputDetections);
stereo->depth.link(segFilter->inputDepth);

segFilter->outputDepth.link(pc->inputDepth);

You also need to pc->initialConfig->setOrganized(false); to only keep points with depth > 0.

After pipeline_->start(); the pointcloud output should already be filtered to only keep the points inside the seg mask.

If you're testing in peripheral mode (not "running oakctl app run .") use setRunOnHost(true) for PointCloud and Sync nodes because at that point Filter node already transfered data over to host and thus sending the output messages back to device for syncing would use a lot of unnecessarry bandwidth. Other than that I dont see any immediate issues with your code!

Hope this helps,

Aljaz

KlemenSkrlj

I would recommend an update to the ModelConverter page. I might be missing something, but I don't see anything stating that ModelConverter requires version 2.32.6. In my view, In the sentence "For RVC4, a full SNPE build version such as 2.32.6….", the term 'such as' indicates that other versions should work.

Yes this is a fair point, thanks for the feedback. The thing is that the SNPE version is tied to the OS and the DepthAI version you are using. The latest OS and DepthAI support models exporting with SNPE 2.41.0 (or older). And with some older DepthAI and OS versions you have to export with 2.32.6 (or older). This is generally visible from DepthAI and OS release notes (e.g. in DAI v.3.6.1 we released support for 2.41.0 mentioned here) but I 100% agree, we will make this information more visible and clear.

Since the resulting converted archive file contains only 1 dlc model (rather than 2 like "luxonis/yolov8-instance-segmentation-large"), am I correct to assume that mask post processing is done by the CPU instead of a neural network?

The second file in luxonis/yolov8-instance-segmentation-large is not really used anywhere, it was kind of a PoC for multi-stage model at that point. But it is not really used (yet) in native DepthAI unless you want to do it yourself. But for YOLO models they are already very heavily optimized with native DepthAI. The way we do it we run majority of the model on DSP and then run only the last postprocessing step (ie. NMS operation which is anyway dynamic so can't be part of the model) also directly on device (on CPU).

When converting a model using modelconvert, I added "--to nn_archive" to the command. How and when should I use the native version (which seems to give just a .dlc file)? Coulnd't find documentation about it.

You shouldn't really have a case where you would have .dlc instead of a packaged NNArchive if you are planning to run the model in DepthAI. Raw DLC file would be helpful if you plan to run some SNPE commands but even for that the .dlc is still present in the NNArchive, it is just packaged together with the metadata that is helpful for actually using the model.

(extra) Am I doing anything wrong/inefficiently in the code snippet I provided? For my use case, I want to extract data from the pointcloud based on the incoming instance segmentation masks.

I'll CC: @aljaz to the thread since this is the usecase that he is working on adding right now in a nicer way to DepthAI natively

Goulart

@KlemenSkrlj thanks for the clarifications. One more question: Does ModelConverter support YOLO26-seg quantization? YOLOv8-seg works but YOLO26-seg doesn't. FP16STANDARD works just fine on YOLO26 (and yolov8). The converted model load and the program runs but I get 0 detections (even after setting setConfidenceThreshold to 0.1).

Commands:
tools-cli: tools yolo26l-seg_custom.pt --imgsz "640 640" --no-use-rvc2
modelconverter: modelconverter convert rvc4 --tool-version 2.41.0 --path archives/yolo26l-seg_custom.tar.xz --to nn_archive rvc4.quantization_mode INT8_STANDARD calibration.path shared_with_container/calibration_data

Notes:

The PyTorch version of my YOLO26 (yolo26l-seg_custom.pt) works just fine;
I use the same calibration data to successfully convert my yolov8 model.

@aljaz Thanks for the help. I will start another thread later to ask some questions about your suggestion.

KlemenSkrlj

Does ModelConverter support YOLO26-seg quantization? YOLOv8-seg works but YOLO26-seg doesn't. FP16STANDARD works just fine on YOLO26 (and yolov8). The converted model load and the program runs but I get 0 detections (even after setting setConfidenceThreshold to 0.1).

On the framework level (ie. tools and modelconverter) there shouldn't be any difference between YOLOv8 and YOLO26 calibration. What is different though is the model architecture. We haven't really made a deepdive but most likely there is some operation in the 26 model which makes it so the model doesn't quantize well. One thing you can check out is the modelconverter analyze tool which will visualize the per-layer errors compared to the base ONNX. And you can then compare the v8 vs 26 visualizations and see which operation is causing issues.
I would also advise you to try out INT8_INT16_MIXED or any of the _ACCURACY_FOCUSED quantization modes. You might pay some price in a bit lower FPS but should get better accuracy.