userOfCamera

  • Apr 24, 2024
  • Joined Jan 9, 2024
  • 2 best answers
  • In How can I add Confidence Threshold on Custom YOLOv8 Classification Model

    @ThomasBr76 @jakaskerl I get this error:

    Error: 'depthai.node.NeuralNetwork' object has no attribute 'setConfidenceThreshold'

    To be clear, I uploaded a YOLOv8 image classification model as a NeuralNet node, it isn't an image detection model

  • In How can I add Confidence Threshold on Custom YOLOv8 Model

    jakaskerl I don't have an object detection yolo model - I'm using an Image Classification YOLO model. When I changed it from a NeuralNet node to a YoloObjectDetection node, the output detections were just a blank array

      • In How can I add Confidence Threshold on Custom YOLOv8 Classification Model

        Hello,

        I've finetuned a YOLOv8 nano classification model, and am using it with an OAK 1 PoE to classify an image into one of six categories.

        The classification works fine on those items, but the greatest issue I'm facing is that when something that isn't one of the 6 categories shows up, the model still gives it a percentage out of those six categories.

        I want to implement a confidence threshold, or be able to classify the items as 'None' (I'm assuming a confidence threshold is the best way to do this).

        There's info online on how to set a conf threshold when predicting with YOLO directly. How can I do this through DepthAI (my custom model is a Neural Network Node)? I want less false positives

        Edit: Just to be clear, this is a YOLOv8 classification model, not detection

      • In How can I add Confidence Threshold on Custom YOLOv8 Model

        Hello,

        I've finetuned a YOLOv8 nano classification model, and am using it with an OAK 1 PoE to classify an image into one of six categories.

        The classification works fine on those items, but the greatest issue I'm facing is that when something that isn't one of the 6 categories shows up, the model still gives it a percentage out of those six categories.

        I want to implement a confidence threshold, or be able to classify the items as 'None' (I'm assuming a confidence threshold is the best way to do this).

        There's info online on how to set a conf threshold when predicting with YOLO directly. How can I do this through DepthAI (my custom model is a Neural Network Node)? I want less false positives

      • In YoloV8 image classification deployed on OAK, wrong classification

        @jakaskerl I was able to figure out the flags which correctly converted my model (it is accurate now, tested on OAK with DepthAI and on my webcam normally)

        The flags I used to convert my YOLO model on the blob converter tool:

        --data_type=FP16 --scale_values=[255,255,255] --reverse_input_channels

      • In YoloV8 image classification deployed on OAK, wrong classification

        --data_type=FP16 --mean_values=[127.5,127.5,127.5] --scale_values=[255,255,255]--reverse_input_channels

        This made no difference for me. My Yolo classification model is still inaccurate @erik

      • In ImageManip queue becomes empty after about 3 minutes of running

        This issue only occurs when I XLink it out, not if it just exists without a xlink out

      • In ImageManip queue becomes empty after about 3 minutes of running

        I've narrowed this issue down a bit:

        It only occurs when:

        • I have an ImageManip thats XLinking out, as well as a still (which was originally passed into the ImageManip) XLinking out
      • In ImageManip queue becomes empty after about 3 minutes of running

        Follow up on this - its actually the still being captured which becomes None after a few minutes

      • In ImageManip queue becomes empty after about 3 minutes of running

        I'm using 2 ImageManips on a still - one to crop, another to resize

        After running for 3 minutes, taking a still a second, the when I tryget the imageManip from its queue it becomes empty (or None).

        I'm also capturing the still on a stream, and this isn't empty and continues to run fine. Does anyone know why this issue is happening?

      • In Still images to Yolo NN

        Solved this by setting frameType as RGB 🙂

        resize_rgb.initialConfig.setFrameType(depthai.ImgFrame.Type.RGB888p)

      • In Still images to Yolo NN

        jakaskerl Could you clarify how to do this? When I try, I get this error:

        [error] Input tensor 'images' (0) exceeds available data range. Data size (614400B), tensor offset (0), size (1228800B) - skipping inference

        • In How can I crop, then resize that cropped area (scale it bigger)
        • In How can I crop, then resize that cropped area (scale it bigger)

          Follow up to this message for anyone reading this - I removed cam_rgb.preview.link(resize_rgb.inputImage) and this removed the flicker issue.

          My updated code:

          import depthai
import cv2

pipeline = depthai.Pipeline()
cam_rgb = pipeline.createColorCamera()
cam_rgb.setPreviewSize(1920, 1080)  
cam_rgb.setInterleaved(False)
maxFrameSize = cam_rgb.getPreviewHeight() * cam_rgb.getPreviewWidth() * 3

crop_rgb = pipeline.createImageManip()
crop_rgb.initialConfig.setCropRect(325/1080, 367/1920, 1, 1)
crop_rgb.initialConfig.setResize(450, 450)
cam_rgb.preview.link(crop_rgb.inputImage)

resize_rgb = pipeline.createImageManip()
resize_rgb.initialConfig.setResize(640, 640)

xout_cropped = pipeline.createXLinkOut()
xout_cropped.setStreamName("cropped")
crop_rgb.out.link(xout_cropped.input)
crop_rgb.out.link(resize_rgb.inputImage)
crop_rgb.setMaxOutputFrameSize(maxFrameSize)

xout_resized = pipeline.createXLinkOut()
xout_resized.setStreamName("resized")
resize_rgb.out.link(xout_resized.input)
resize_rgb.setMaxOutputFrameSize(maxFrameSize)

device = depthai.Device(pipeline)
q_resize = device.getOutputQueue("resized")
q_cropped = device.getOutputQueue("cropped")

while True:
    in_cropped = q_cropped.tryGet()
    in_resize = q_resize.tryGet()

    if in_cropped is not None:
        stream = in_cropped.getCvFrame()
        if stream is not None:
            cv2.imshow("cropped", stream)
    
    if in_resize is not None:
        frame = in_resize.getCvFrame()
        if frame is not None:
            cv2.imshow("resized", frame)

    if cv2.waitKey(1) == ord('q'):
            break
        • In How can I crop, then resize that cropped area (scale it bigger)

          Follow up: I figured out how to do this but have a new problem

          My resized image keeps flickering between what I was previously seeing (a 640x640 of the original full size frame) and what I currently want (640x640 of the area I cropped and resized)

          Any ideas why? The code:

          from pathlib import Path
import depthai
import cv2
import numpy as np
import sys

pipeline = depthai.Pipeline()
cam_rgb = pipeline.createColorCamera()
cam_rgb.setPreviewSize(1920, 1080)  
cam_rgb.setInterleaved(False)
maxFrameSize = cam_rgb.getPreviewHeight() * cam_rgb.getPreviewWidth() * 3

crop_rgb = pipeline.createImageManip()
crop_rgb.initialConfig.setCropRect(325/1080, 367/1920, 1, 1)
crop_rgb.initialConfig.setResize(450, 450)
cam_rgb.preview.link(crop_rgb.inputImage)

resize_rgb = pipeline.createImageManip()
resize_rgb.initialConfig.setResize(640, 640)
cam_rgb.preview.link(resize_rgb.inputImage)

xout_cropped = pipeline.createXLinkOut()
xout_cropped.setStreamName("cropped")
crop_rgb.out.link(xout_cropped.input)
crop_rgb.out.link(resize_rgb.inputImage)
crop_rgb.setMaxOutputFrameSize(maxFrameSize)

xout_resized = pipeline.createXLinkOut()
xout_resized.setStreamName("resized")
resize_rgb.out.link(xout_resized.input)
resize_rgb.setMaxOutputFrameSize(maxFrameSize)

device = depthai.Device(pipeline)
q_resize = device.getOutputQueue("resized")
q_cropped = device.getOutputQueue("cropped") 

while True:
    in_cropped = q_cropped.tryGet()
    in_resize = q_resize.tryGet()

    if in_cropped is not None:
        stream = in_cropped.getCvFrame()
        if stream is not None:
            cv2.imshow("cropped", stream)
    
    if in_resize is not None:
        frame = in_resize.getCvFrame()
        if frame is not None:
            cv2.imshow("resized", frame)

    if cv2.waitKey(1) == ord('q'):
            break
        • In How can I crop, then resize that cropped area (scale it bigger)

          jakaskerl How can I resize the first one with the second manip?

          In my current code, the second imageManip (resize_rgb) is simply resizing the original image. Is there a way I can pass the output of the first image manip (crop_rgb) to resize_rgb?

          My code:

          from pathlib import Path
import depthai
import cv2
import numpy as np
import sys

pipeline = depthai.Pipeline()
cam_rgb = pipeline.createColorCamera()
cam_rgb.setPreviewSize(1920, 1080)  
cam_rgb.setInterleaved(False)
maxFrameSize = cam_rgb.getPreviewHeight() * cam_rgb.getPreviewWidth() * 3

crop_rgb = pipeline.createImageManip()
crop_rgb.initialConfig.setCropRect(325/1080, 367/1920, 1, 1)
crop_rgb.initialConfig.setResize(450, 450)
cam_rgb.preview.link(crop_rgb.inputImage)

xout_cropped = pipeline.createXLinkOut()
xout_cropped.setStreamName("cropped")
crop_rgb.out.link(xout_cropped.input)
crop_rgb.setMaxOutputFrameSize(maxFrameSize)

resize_rgb = pipeline.createImageManip()
resize_rgb.initialConfig.setResize(640, 640)
cam_rgb.preview.link(resize_rgb.inputImage)

xout_resized = pipeline.createXLinkOut()
xout_resized.setStreamName("resized")
resize_rgb.out.link(xout_resized.input)
resize_rgb.setMaxOutputFrameSize(maxFrameSize)

device = depthai.Device(pipeline)
q_resize = device.getOutputQueue("resized")
q_cropped = device.getOutputQueue("cropped") 

while True:
    in_cropped = q_cropped.tryGet()
    in_resize = q_resize.tryGet()

    if in_cropped is not None:
        stream = in_cropped.getCvFrame()
        if stream is not None:
            cv2.imshow("cropped", stream)
    
    if in_resize is not None:
        frame = in_resize.getCvFrame()
        if frame is not None:
            cv2.imshow("resized", frame)

    if cv2.waitKey(1) == ord('q'):
            break
          • In How can I crop, then resize that cropped area (scale it bigger)

            I'm using ImageManip's setCropRect() to move my images start to a specified area, then using setResize to ensure its 450x450

            I need to expand that 450x450 to 640x640 for my neural net. Anything I've tried just sets the image to 640x640 by increasing the crop (not by taking the existing 450x450 crop and expanding it to fill). I don't want to pick up things on the side that increasing the crop to 640x640 brings

          • In Cannot resize using config_nn

            Getting the following error:

            detection_nn.config_nn(resize_mode='stretch')

^^^^^^^^^^^^^^^^^^^^^^

            AttributeError: 'depthai.node.NeuralNetwork' object has no attribute 'config_nn'

            On this code:

            detection_nn = pipeline.createNeuralNetwork()
detection_nn.setBlobPath(nnPath)
detection_nn.config_nn(resize_mode='stretch')

            I was trying to follow this:

            https://docs.luxonis.com/projects/api/en/latest/tutorials/maximize_fov/#change-aspect-ratio

          • In Changing input_shape in blobconverter changes behavior; predicts same label

            When I change the input size of my model when converting to blob, I run into a weird issue.

            When I load and run the model with depthai it only outputs the same label, no matter what the camera sees. When I converted the model to blob without specifying an input shape, it worked fine. The reason I'm setting an input shape is because my images are 450x450 while the model takes 640x640 seemingly.

            Is there an issue with how I specified the input shape here? My model is the yolo model, I'm fairly confident it's nhwc in shape, but I force converted it from nchw to nhwc too and this didn't make a difference.

            My code:

            shape = [1, 3, 450, 450]

blob_path = blobconverter.from_onnx(
    model="./March8Model.onnx",
    data_type="FP16",
    shaves=5,
    optimizer_params=[
        f"--input_shape={shape}", #setting shape might be a mistake it keeps saying grate, to investigate
    ]
)
          • In Is it possible to use a Neural Network on stills (and not a RGB stream)?

            I'm currently capturing stills from the OAK-1-PoE on trigger. I want to use my neural net on these stills and not a full rgb stream, as there is too much delay and lag in the rgb stream.

            Why does my code (below) not print a prediction? I want q_nn to be filled with predictions every time the same trigger that takes the capture goes off, without using the rgb stream.

            from pathlib import Path
import depthai
import cv2
import numpy as np
import sys

CLASSES = ['SAMPLE1', 'SAMPLE2', 'SAMPLE3', 'SAMPLE4', 'SAMPLE5', 'SAMPLE6']

nnPath = str((Path(__file__).parent / Path('./best_captured_openvino_2022.1_5shave.blob')).resolve().absolute())
if len(sys.argv) > 1:
    nnPath = sys.argv[1]

if not Path(nnPath).exists():
    raise FileNotFoundError(f'Required file/s not found')

pipeline = depthai.Pipeline()
cam_rgb = pipeline.createColorCamera()

still_out = pipeline.createXLinkOut()
still_out.setStreamName("still")
cam_rgb.still.link(still_out.input)

control_in = pipeline.createXLinkIn()
control_in.setStreamName('control')
control_in.out.link(cam_rgb.inputControl)

detection_nn = pipeline.createNeuralNetwork()
detection_nn.setBlobPath(nnPath)

xout_nn = pipeline.createXLinkOut()
xout_nn.setStreamName("nn")
detection_nn.out.link(xout_nn.input)

device = depthai.Device(pipeline)
q_still = device.getOutputQueue(name="still", maxSize=1, blocking=False)
q_control = device.getInputQueue("control")
q_nn = device.getOutputQueue("nn")
ctrl = depthai.CameraControl()
ctrl.setCaptureStill(True)
q_control.send(ctrl)

while True:
    if cv2.waitKey(1) == ord('c'): 
        q_control.send(ctrl)
    in_still = q_still.tryGet()
    in_nn = q_nn.tryGet() 
    frame = None

    if in_still is not None:
        frame = in_still.getCvFrame()
    if in_nn is not None:
            data = in_nn.getFirstLayerFp16()
            result_pos = np.argmax(data)
            result = CLASSES[result_pos]
            print(result)

            frame = cv2.putText(frame, result, (50, 50), cv2.FONT_HERSHEY_SIMPLEX, 1, (255, 0, 0), 2)
    if frame is not None:       
        cv2.imshow("preview", frame)

    if cv2.waitKey(1) == ord('q'):
            break