C

chandrian

Hello,

I have trained my own model with great results in colab, but when I put the 416 blob into my code, I am not getting good results. My code works when using a blob from the zoo, but when I replace it with my own tested model, it is not working.

Ok thanks for the reply. I was not able to get the rotation working for the facial recognition. I am trying to train a model for rotated faces atm.

How does the "while True:" work with the script nodes? It seems like it would get stuck in there without a break..?

Thanks for the reply Erik. I am not sure what that is. The camera shows up as "S_VIDEO_CONTROL_1". Is that the MxID?

I actually got the rotated face detection working with the gen2_face detection examples so I'm trying to dissect that to make this work.

erik
Thanks for the reply. Probably low interest then. I wonder why UVC isn't more popular. Ok thanks

Hello,

Is it possible to rename the device so that it shows up on all computers (not just current registry)

Thanks,
Aaron

You probably dont have time to try it yourself but if you had any guidance on the approach that would help a lot. Hoping to get this done before production next week. Thanks again

I even tried it with a rotation of zero degrees to see if the facial recognition was working and it was not working.

Ok thanks Erik. Whats the average turn around time on something like this? (obviously not easy to predict)

I didnt quite follow. We rotated the camera so the faces coming in would be rotated 90 going into the NN so we are trying to rotate in software before the NN so that they are upright. Did I miss understand you?

what should the general approach be? This is the camera source flow through two nodes. The crop node works fine but I am not seeing the face recognition working when I add the rotation:

# Define source and output
camRgb = pipeline.create(dai.node.ColorCamera)
xoutVideo = pipeline.create(dai.node.XLinkOut)

xoutVideo.setStreamName("video")

# Properties
camRgb.setBoardSocket(dai.CameraBoardSocket.RGB)
camRgb.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1080_P)
camRgb.setVideoSize(1920, 1080)
camRgb.setPreviewSize(300, 300)

xoutVideo.input.setBlocking(False)
xoutVideo.input.setQueueSize(1)

# Create MobileNet detection network
mobilenet = pipeline.create(dai.node.MobileNetDetectionNetwork)
mobilenet.setBlobPath(
    blobconverter.from_zoo(name="face-detection-retail-0004", shaves=3)
)
mobilenet.setConfidenceThreshold(0.7)


crop_manip = pipeline.createImageManip()
rgbRr = dai.RotatedRect()
rgbRr.center.x, rgbRr.center.y = camRgb.getPreviewWidth() // 2, camRgb.getPreviewHeight() // 2
rgbRr.size.width, rgbRr.size.height = camRgb.getPreviewHeight(), camRgb.getPreviewWidth()
rgbRr.angle = 90
crop_manip.initialConfig.setCropRotatedRect(rgbRr, False)
camRgb.isp.link(crop_manip.inputImage)

crop_manip2 = pipeline.create(dai.node.ImageManip)
crop_manip2.initialConfig.setResize(300, 300)
crop_manip2.initialConfig.setFrameType(dai.ImgFrame.Type.BGR888p)
crop_manip.out.link(crop_manip2.inputImage)

crop_manip2.out.link(mobilenet.input)

https://github.com/luxonis/depthai-core/issues/670

Is this passable for an issue or should I change something?

Ah ok thanks. I tried to keep it minimal with the images above but maybe that was too much? Basically I fed the camera.isp output through two image crops to see if it would work and I got an error. Both image crops worked independently but it feeding through both nodes gave me an error.

erik
Hi Erik,

What do you mean when you say that? Do I post those things here in the forum post? Or do I submit something like a git issue?

Here's the code;

#!/usr/bin/env python3

import cv2
import depthai as dai
import blobconverter

# Create pipeline
pipeline = dai.Pipeline()

# Define source and output
camRgb = pipeline.create(dai.node.ColorCamera)
xoutVideo = pipeline.create(dai.node.XLinkOut)

xoutVideo.setStreamName("video")

# Properties
camRgb.setBoardSocket(dai.CameraBoardSocket.RGB)
camRgb.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1080_P)
camRgb.setVideoSize(1920, 1080)

xoutVideo.input.setBlocking(False)
xoutVideo.input.setQueueSize(1)

# Create MobileNet detection network
mobilenet = pipeline.create(dai.node.MobileNetDetectionNetwork)
mobilenet.setBlobPath(
    blobconverter.from_zoo(name="face-detection-retail-0004", shaves=3)
)
mobilenet.setConfidenceThreshold(0.7)

# manipRgb = pipeline.createImageManip()
# rgbRr = dai.RotatedRect()
# rgbRr.center.x, rgbRr.center.y = camRgb.getPreviewWidth() // 2, camRgb.getPreviewHeight() // 2
# rgbRr.size.width, rgbRr.size.height = camRgb.getPreviewHeight(), camRgb.getPreviewWidth()
# rgbRr.angle = 0
# manipRgb.initialConfig.setCropRotatedRect(rgbRr, False)
#
#
# camRgb.isp.link(manipRgb.inputImage)
# manipRgb.out.link(mobilenet.input)



crop_manip2 = pipeline.create(dai.node.ImageManip)
crop_manip2.initialConfig.setResize(300, 300)
crop_manip2.initialConfig.setFrameType(dai.ImgFrame.Type.BGR888p)
camRgb.isp.link(crop_manip2.inputImage)
#crop_manip2.out.link(mobilenet.input)


crop_manip = pipeline.create(dai.node.ImageManip)
crop_manip.initialConfig.setResize(300, 300)
crop_manip.initialConfig.setFrameType(dai.ImgFrame.Type.BGR888p)
crop_manip.out.link(crop_manip2.inputImage)

# camRgb.isp.link(crop_manip.inputImage)
# crop_manip2.out.link(crop_manip.inputImage)
crop_manip.out.link(mobilenet.input)




# Script node
script = pipeline.create(dai.node.Script)
mobilenet.out.link(script.inputs["dets"])
script.outputs["cam_cfg"].link(camRgb.inputConfig)
script.outputs["cam_ctrl"].link(camRgb.inputControl)
script.setScript(
    """
    ORIGINAL_SIZE = (5312, 6000) # 48MP with size constraints described on IMX582 luxonis page
    SCENE_SIZE = (1920, 1080) # 1080P
    x_arr = []
    y_arr = []
    AVG_MAX_NUM=7
    limits = [0, 0] # xmin and ymin limits
    limits.append((ORIGINAL_SIZE[0] - SCENE_SIZE[0]) / ORIGINAL_SIZE[0]) # xmax limit
    limits.append((ORIGINAL_SIZE[1] - SCENE_SIZE[1]) / ORIGINAL_SIZE[1]) # ymax limit
    cfg = ImageManipConfig()
    ctrl = CameraControl()
    def average_filter(x, y):
        x_arr.append(x)
        y_arr.append(y)
        if AVG_MAX_NUM < len(x_arr): x_arr.pop(0)
        if AVG_MAX_NUM < len(y_arr): y_arr.pop(0)
        x_avg = 0
        y_avg = 0
        for i in range(len(x_arr)):
            x_avg += x_arr[i]
            y_avg += y_arr[i]
        x_avg = x_avg / len(x_arr)
        y_avg = y_avg / len(y_arr)
        if x_avg < limits[0]: x_avg = limits[0]
        if y_avg < limits[1]: y_avg = limits[1]
        if limits[2] < x_avg: x_avg = limits[2]
        if limits[3] < y_avg: y_avg = limits[3]
        return x_avg, y_avg
    while True:
    

        dets = node.io['dets'].get().detections
        if len(dets) == 0: continue
        coords = dets[0] # take first
        width = (coords.xmax - coords.xmin) * ORIGINAL_SIZE[0]
        height = (coords.ymax - coords.ymin) * ORIGINAL_SIZE[1]
        x_pixel = int(max(0, coords.xmin * ORIGINAL_SIZE[0]))
        y_pixel = int(max(0, coords.ymin * ORIGINAL_SIZE[1]))
        # ctrl.setAutoFocusRegion(x_pixel, y_pixel, int(width), int(height))
        # ctrl.setAutoExposureRegion(x_pixel, y_pixel, int(width), int(height))
        # Get detection center
        x = (coords.xmin + coords.xmax) / 2
        y = (coords.ymin + coords.ymax) / 2
        x -= SCENE_SIZE[0] / ORIGINAL_SIZE[0] / 2
        y -= SCENE_SIZE[1] / ORIGINAL_SIZE[1] / 2
        # node.warn(f"{x=} {y=}")
        x_avg, y_avg = average_filter(x,y)
        # node.warn(f"{x_avg=} {y_avg=}")
        cfg.setCropRect(x_avg, y_avg, 0, 0)
        node.io['cam_cfg'].send(cfg)
        node.io['cam_ctrl'].send(ctrl)
    """
)

# Linking
camRgb.video.link(xoutVideo.input)

# Connect to device and start pipeline
with dai.Device(pipeline) as device:

    video = device.getOutputQueue(name="video", maxSize=1, blocking=False)

    while True:
        videoIn = video.get()
        print("Done in seconds")


        # Get BGR frame from NV12 encoded video frame to show with opencv
        # Visualizing the frame on slower hosts might have overhead
        cv2.imshow("video", videoIn.getCvFrame())

        if cv2.waitKey(1) == ord('q'):
            break

Thanks for the reply Eric,

I have been trying to get this to work with no luck. It seems when I add multiple nodes, it does not function well. I am just feeding isp output into a mobilenet.

I tried just duplicating the resize image ImageManip above to prove functionality since I got the single resize working and I cannot get it to pass to the mobilnet and function.

This is using the UVC demo.

Thanks!

I am not having much success with rotating the input to the facial recognition. Do you think this is possible? If not, do you have another suggestion?