Fakhrullo

  • Apr 4, 2024
  • Joined Jul 31, 2023
  • 0 best answers
  • Hi jakaskerl
    Sorry for the late response, I was busy with some other projects, Yeah I will see those examples, thank you so much man appreciate it.

    Thanks,
    Fakhrullo

  • Hi jakaskerl
    Yes kinda like that, instead of showing the video with CV2, I want to show it in a web browser, is it possible?

    Thanks,
    Fakhrullo

  • Hello everyone, I hope you are doing well.
    I am working on a project and I want to implement a feature where the admin can view real-time video streams from available cameras on the front-end side, specifically on the admin page. Can you please guide me on how to accomplish this? Thank you.

    • Hi @Fakhrullo
      Pretty sure that's how robothub works. If you are crafty you can also create your own version with web streaming.

      EDIT: Some examples to get you started luxonis/robothub-examples

      Thanks,
      Jaka

  • JanCuhel
    Thanks in advance bro, you helped me so much and took so many stress 😅 wish you best bro

    Best regards
    Fakhrullo

  • Hi JanCuhel
    I'm very new in this sphere, so I found some tutorials to train Yolov8 and I thought if I change the model it would just train the other models as well (in my case yolov6n) here's the code I used to train the model:


    # Load a model
    from ultralytics import YOLO
    
    model = YOLO('yolov6n.yaml')  # load a 
    pretrained model (recommended for training)
    
    # Train the model
    results = model.train(data='/content/Yolo/dataset/data.yaml', epochs=10, imgsz=640)
  • Please be aware that you need to download the .pt file, somehow google drive is opening .pt file like a zip file 😅

    Here's the image, after opening the drive link, just press the download button without opening best file

  • JanCuhel

    Hi JanCuhel
    Sorry for late response, I couldn't reach my laptop for a few days, well I also noticed that the file I shared seems to be broken, I will reupload the model I trained and I will make sure it's working properly before uploading it.

    Thanks,
    Fakhrullo

  • Hi JanCuhel
    Here's the image below you can see the configuration I used, and I also used all Yolo versions(related to yolov6) and none of them worked

    I hope it helps
    thanks,
    Fakhrullo

  • Hi jakaskerl
    Here's the link below where you can find the model I'm trying to convert to a blob file:

    I hope it will be fixed soon

    thanks,
    Fakhrullo

  • Hey guys, I'm trying to convert my trained yolov6n model (.pt file) on tools.luxonis.com to a blob file to use on the OAK camera, but I'm getting an error like this:

    I tried all three options and got the same error, how can I fix that issue?
    Thanks for the answers
    Fakhrullo

    • Ah, I see. Now, it makes sense. I saw in the ultralytics documentation that they support YoloV6 as well. So you can keep using the code you have and (for now) use the YoloV8 export option in the tools when converting the model. Another option would be to use YoloV6's training script. We have prepared a tutorial for that. You can also find more information in the YoloV6's repository. If you would choose this, then you could use the YoloV6 export option in the tools.

      Best,
      Jan

  • Is it possible to get a detected object's confidence level from tracklets? If yes, how can I do it? I couldn't find it. If not, how can I print the confidence level?

    • jakaskerl replied to this.
    • Hi Fakhrullo
      Afaik it's not possible. But you can use the passthroughDetections output to access confidence.

      Thanks,
      Jaka

    • Does anybody know or have YOLOv6 or above for head detection to use on OAK cameras?

      I trained YOLOv6s with more than 1.5K images but it's working slowly and detection is awful. Also when I converted it to blob it didn't have anchors and anchor_smasks does it affect performance?

      • jakaskerl
        After factory resetting to USB mode, can I connect to the OAK camera with MxID?

        Like this

        Or should I connect to the camera with a USB cable?

        • Hi jakaskerl
          Can it be still connected via MxID wirelessly after making a factory reset to USB mode?

          • Hi jakaskerl
            As you can see I can connect to the camera using IP in the device manager however I tried to connect using IP in other cases but had no luck, I couldn't connect to it

            • After a factory reset to the OAK-1 PoE camera, I cannot connect to the camera with MxID to run my project. How can I solve this problem?

              • I'm trying to get my project to work in standalone mode however there are some problems, the code below is for the camera and host, can you guys give me suggestions?
                The camera is flushing successfully with 100% and when I try to get in the host machine, after running the code, the terminal freezes and shows no result.
                Camera ⬇️⬇️⬇️

                from pathlib import Path
                import depthai as dai
                
                # tiny yolo v4 label texts
                labelMap = ["person",]
                
                nnPath = str((Path(__file__).parent / 
                Path('model/yolov6n_coco_640x640_openvino_2022.1_6shave.blob')).resolve().absolute())
                
                # Creating pipeline
                pipeline = dai.Pipeline()
                
                # Sources and outputs
                camRgb = pipeline.create(dai.node.ColorCamera)
                detectionNetwork = pipeline.create(dai.node.YoloDetectionNetwork)
                objectTracker = pipeline.create(dai.node.ObjectTracker)
                
                # Properties
                camRgb.setImageOrientation(dai.CameraImageOrientation.HORIZONTAL_MIRROR)
                camRgb.setPreviewSize(640, 640)
                camRgb.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1080_P)
                camRgb.setInterleaved(False)
                camRgb.setColorOrder(dai.ColorCameraProperties.ColorOrder.BGR)
                camRgb.setFps(40)
                
                #define a script node
                script = pipeline.create(dai.node.Script)
                script.setProcessor(dai.ProcessorType.LEON_CSS)
                
                # Network specific settings
                detectionNetwork.setConfidenceThreshold(0.65)
                detectionNetwork.setNumClasses(80)
                detectionNetwork.setCoordinateSize(4)
                # detectionNetwork.setAnchors([10, 14, 23, 27, 37, 58, 81, 82, 135, 169, 344, 319]) #for YOLOv4
                # detectionNetwork.setAnchorMasks({"side26": [1, 2, 3], "side13": [3, 4, 5]})
                detectionNetwork.setAnchors([10,13, 16,30, 33,23, 30,61, 62,45, 59,119, 116,90, 156,198, 373,326]) #for YOLOv5
                detectionNetwork.setAnchorMasks({"side52": [0,1,2], "side26": [3,4,5], "side13": [6,7,8]})
                detectionNetwork.setIouThreshold(0.5)
                detectionNetwork.setBlobPath(nnPath)
                detectionNetwork.setNumInferenceThreads(2)
                detectionNetwork.input.setBlocking(False)
                
                objectTracker.setDetectionLabelsToTrack([0])  # track only person
                # possible tracking types: ZERO_TERM_COLOR_HISTOGRAM, ZERO_TERM_IMAGELESS, SHORT_TERM_IMAGELESS, 
                SHORT_TERM_KCF
                objectTracker.setTrackerType(dai.TrackerType.ZERO_TERM_COLOR_HISTOGRAM)
                # take the smallest ID when new object is tracked, possible options: SMALLEST_ID, UNIQUE_ID
                objectTracker.setTrackerIdAssignmentPolicy(dai.TrackerIdAssignmentPolicy.UNIQUE_ID)
                
                #Define a video encoder
                videoEnc = pipeline.create(dai.node.VideoEncoder)
                videoEnc.setDefaultProfilePreset(30, dai.VideoEncoderProperties.Profile.MJPEG)
                
                #Linking
                camRgb.preview.link(detectionNetwork.input)
                camRgb.video.link(videoEnc.input)
                
                detectionNetwork.passthrough.link(objectTracker.inputTrackerFrame)
                
                detectionNetwork.passthrough.link(objectTracker.inputDetectionFrame)
                detectionNetwork.out.link(objectTracker.inputDetections)
                
                
                script.inputs['tracklets'].setBlocking(False)
                script.inputs['tracklets'].setQueueSize(1)
                objectTracker.out.link(script.inputs["tracklets"])
                
                script.inputs['frame'].setBlocking(False)
                script.inputs['frame'].setQueueSize(1)
                videoEnc.bitstream.link(script.inputs['frame'])
                
                script.setScript("""
                import socket
                import time
                server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
                server.bind(("0.0.0.0", 3005))
                server.listen()
                node.warn("Server up")
                             
                trackableObjects = {}
                startTime = time.monotonic()
                counter = 0
                fps = 0
                frame = None
                
                h_line = 380
                pos = {}
                going_in = 0
                going_out = 0
                obj_counter = [0, 0, 0, 0]  # left, right, up, down
                
                while(True):
                    conn, client = server.accept()
                    node.warn(f"Connected to client IP: {client}")
                    try:             
                        while True:
                             
                            pck = node.io["frame"].get()
                            tracklets = node.io["tracklets"].tryGet()
                            data = pck.getData()
                            ts = pck.getTimestamp()
                
                            counter+=1
                            current_time = time.monotonic()
                            if (current_time - startTime) > 1 :
                                fps = counter / (current_time - startTime)
                                counter = 0
                                startTime = current_time
                        
                            trackletsData = tracklets.tracklets
                            data_to_send = []
                            for t in trackletsData:
                                if t.status.name == "TRACKED":
                                    roi = t.roi.denormalize(frame.shape[1], frame.shape[0])
                                    x1 = int(roi.topLeft().x)
                                    y1 = int(roi.topLeft().y)
                                    x2 = int(roi.bottomRight().x)
                                    y2 = int(roi.bottomRight().y)
                
                                    # Calculate centroid
                                    centroid = (int((x2-x1)/2+x1), int((y2-y1)/2+y1))
                             
                                    coordinates = [x1, y1, x2, y2]
                
                                    # Calculate the buffer zone boundaries
                                    right_boundary = h_line + 15
                                    left_boundary = h_line - 15
                
                                    try:
                                        if not (left_boundary <= centroid[1] <= right_boundary):
                                            pos[t.id] = {
                                                'previous': pos[t.id]['current'],
                                                'current': centroid[1]      }
                                        if pos[t.id]['current'] > right_boundary and pos[t.id]['previous'] < right_boundary:
                
                                            obj_counter[1] += 1 #Right side
                                            going_out += 1
                                        if pos[t.id]['current'] < left_boundary and pos[t.id]['previous'] > left_boundary:
                                        
                                            obj_counter[0] += 1 #Left side
                                            going_in += 1
                                    except:
                                        pos[t.id] = {'current': centroid[1]}
                
                                    try:
                                        label = labelMap[t.label]
                                    except:
                                        label = t.label
                                if t.status != Tracklet.TrackingStatus.LOST and t.status != Tracklet.TrackingStatus.REMOVED:
                                    text = "ID {}".format(t.id)
                             
                                    # data_to_send.append([text, centroid[0], centroid[1], label, coordinates, obj_counter, fps])
                                    data_to_send.append([text, centroid[0], centroid[1]])
                        
                            # now to send data we need to encode it (whole header is 256 characters long)
                            header = f"ABCDE " + str(ts.total_seconds()).ljust(18) + str(len(data)).ljust(8) + str(counter).ljust(16) + str(data_to_send).ljust(208) 
                            conn.send(bytes(header, encoding='ascii'))
                            conn.send(data)
                             
                    except Exception as e:
                        node.warn(f"Error oak: {e}")
                        node.warn("Client disconnected")
                
                """)
                
                (f, bl) = dai.DeviceBootloader.getFirstAvailableDevice()
                bootloader = dai.DeviceBootloader(bl)
                progress = lambda p : print(f'Flashing progress: {p*100:.1f}%')
                bootloader.flash(progress, pipeline)

                Host ⬇️⬇️⬇️

                import socket
                import re
                import cv2
                import numpy as np
                
                # Enter your own IP!
                OAK_IP = "192.168.1.168"
                
                width = 640
                height = 640
                
                def get_frame(socket, size):
                    bytes = socket.recv(4096)
                    while True:
                        read = 4096
                        if size - len(bytes) < read:
                            read = size - len(bytes)
                        bytes += socket.recv(read)
                        if size == len(bytes):
                            return bytes
                
                sock = socket.socket()
                sock.connect((OAK_IP, 3005))
                
                frame = None  # Initialize frame outside the loop
                try:
                    while True:
                        header = str(sock.recv(256), encoding="ascii")
                        chunks = re.split(' +', header)
                        if chunks[0] == "ABCDE":
                        
                            ts = float(chunks[1])
                            imgSize = int(chunks[2])
                            img = get_frame(sock, imgSize)
                            buf = np.frombuffer(img, dtype=np.byte)
                            tracklet_data = eval(chunks[4])
                            counter = eval(chunks[3])
                        
                            frame = cv2.imdecode(buf, cv2.IMREAD_COLOR)
                            print(frame.shape)
                        
                            for tracklet in tracklet_data:
                                scale = 640/300
                                name = tracklet[0]
                                x = int(tracklet[1]*scale)
                                y = int(tracklet[2]*scale)
                                print(name, x, y)
                                cv2.putText(frame, name, (x - 10, y - 10),
                                        cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 255), 2)
                                cv2.circle(frame, (x, y), 4, (255, 255, 255), -1)
                
                        # Draw ROI line
                        cv2.line(frame, (int(0.5*width), 0),
                             (int(0.5*width), height), (0xFF, 0, 0), 5)
                
                        # display count and status
                        font = cv2.FONT_HERSHEY_SIMPLEX
                
                        cv2.putText(frame, f'Left: {counter[0]}; Right: {counter[1]}', (
                        10, 35), font, 0.8, (0, 0xFF, 0xFF), 2, cv2.FONT_HERSHEY_SIMPLEX)
                
                        cv2.imshow('cumulative_object_counting', frame)
                        if cv2.waitKey(25) & 0xFF == ord('q'):
                            break
                
                except Exception as e:
                    print("Error host:", e)
                
                sock.close()

                Can you please review the code and give me suggestions to get it done correctly? It is my first time trying to work with standalone mode.

                Thanks,
                Fakhrullo

              • Hey guys, how are you doing? I have this code, and how can I run it in standalone mode? I have both OAK-1 PoE and OAK-D PoE cameras, I want to run it in standalone mode to get the maximum performance as possible. Is there any step-by-step documentation or tutorial on how to convert into a standalone mode? I read the official website, But couldn't understand well exactly how to do it. I'll leave the code below, hope you will help me thanks 🙂

                from pathlib import Path
                import cv2
                import depthai as dai
                import time
                from environs import Env
                
                env = Env()
                env.read_env()
                
                MxID = env('MxID')
                # Set custom ROI coordinates (x, y, width, height)
                custom_roi = (350/640, 250/640, 640/640, 640/640)
                
                # Coordinates of the counting line
                line_start = (320, 0)
                line_end = (320, 640)
                
                # tiny yolo v4 label texts
                labelMap = ["person",]
                
                nnPath = str((Path(__file__).parent / Path('model/yolov6n_coco_640x640_openvino_2022.1_6shave.blob')).resolve().absolute())
                
                # Creating pipeline
                pipeline = dai.Pipeline()
                
                # Sources and outputs
                camRgb = pipeline.create(dai.node.ColorCamera)
                detectionNetwork = pipeline.create(dai.node.YoloDetectionNetwork)
                objectTracker = pipeline.create(dai.node.ObjectTracker)
                
                xlinkOut   = pipeline.create(dai.node.XLinkOut)
                trackerOut = pipeline.create(dai.node.XLinkOut)
                fullFrame  = pipeline.create(dai.node.XLinkOut) # creating new pipline for full_frame
                
                xlinkOut.setStreamName("preview")
                trackerOut.setStreamName("tracklets")
                fullFrame.setStreamName("full_frame") # To get Full Frame
                
                # Creating Manip node
                manip = pipeline.create(dai.node.ImageManip)
                # Setting CropRect for the Region of Interest
                # manip.initialConfig.setCropRect(*custom_roi)
                # Setting Resize for the neural network input size
                manip.initialConfig.setResize(640, 640)
                # Setting maximum output frame size based on the desired output dimensions
                max_output_width = 640
                max_output_height = 640
                max_output_frame_size = 3 * max_output_width * max_output_height # Assuming 3 channels for BGR image
                manip.setMaxOutputFrameSize(max_output_frame_size)
                
                # Properties
                if MxID == "14442C10C1AD3FD700":
                    camRgb.setImageOrientation(dai.CameraImageOrientation.HORIZONTAL_MIRROR)
                camRgb.setPreviewSize(640, 640)
                camRgb.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1080_P)
                camRgb.setInterleaved(False)
                camRgb.setColorOrder(dai.ColorCameraProperties.ColorOrder.BGR)
                camRgb.setFps(40)
                
                # Network specific settings
                detectionNetwork.setConfidenceThreshold(0.5)
                detectionNetwork.setNumClasses(80)
                detectionNetwork.setCoordinateSize(4)
                # detectionNetwork.setAnchors([10, 14, 23, 27, 37, 58, 81, 82, 135, 169, 344, 319]) #for YOLOv4
                # detectionNetwork.setAnchorMasks({"side26": [1, 2, 3], "side13": [3, 4, 5]})
                detectionNetwork.setAnchors([10,13, 16,30, 33,23, 30,61, 62,45, 59,119, 116,90, 156,198, 373,326]) #for YOLOv5
                detectionNetwork.setAnchorMasks({"side52": [0,1,2], "side26": [3,4,5], "side13": [6,7,8]})
                detectionNetwork.setIouThreshold(0.5)
                detectionNetwork.setBlobPath(nnPath)
                detectionNetwork.setNumInferenceThreads(2)
                detectionNetwork.input.setBlocking(False)
                
                objectTracker.setDetectionLabelsToTrack([0])  # track only person
                # possible tracking types: ZERO_TERM_COLOR_HISTOGRAM, ZERO_TERM_IMAGELESS, SHORT_TERM_IMAGELESS, SHORT_TERM_KCF
                objectTracker.setTrackerType(dai.TrackerType.ZERO_TERM_COLOR_HISTOGRAM)
                # take the smallest ID when new object is tracked, possible options: SMALLEST_ID, UNIQUE_ID
                objectTracker.setTrackerIdAssignmentPolicy(dai.TrackerIdAssignmentPolicy.UNIQUE_ID)
                
                #Linking
                # Connecting Manip node to ColorCamera
                camRgb.preview.link(manip.inputImage)
                camRgb.preview.link(fullFrame.input)
                # Connecting Manip node to YoloDetectionNetwork
                manip.out.link(detectionNetwork.input)
                # camRgb.preview.link(detectionNetwork.input)
                objectTracker.passthroughTrackerFrame.link(xlinkOut.input)
                
                detectionNetwork.passthrough.link(objectTracker.inputTrackerFrame)
                
                detectionNetwork.passthrough.link(objectTracker.inputDetectionFrame)
                detectionNetwork.out.link(objectTracker.inputDetections)
                objectTracker.out.link(trackerOut.input)
                
                device = dai.DeviceInfo(MxID)
                
                # Connecting to device and starting pipeline
                with dai.Device(pipeline, device) as device:
                
                    preview = device.getOutputQueue("preview", 4, False)
                    tracklets = device.getOutputQueue("tracklets", 4, False)
                    previewFull = device.getOutputQueue("full_frame", 4, False)
                
                    startTime = time.monotonic()
                    counter = 0
                    fps = 0
                    frame = None
                
                    h_line = 320
                    pos = {}
                    going_in = 0
                    going_out = 0
                    obj_counter = [0, 0, 0, 0]  # left, right, up, down
                
                    while(True):
                        imgFrame = preview.get()
                        track = tracklets.get()
                        imgFull = previewFull.get()
                
                        counter+=1
                        current_time = time.monotonic()
                        if (current_time - startTime) > 1 :
                            fps = counter / (current_time - startTime)
                            counter = 0
                            startTime = current_time
                
                        color = (255, 0, 0)
                        text_color = (0, 0, 255)
                        rectangle = (111, 147, 26)
                
                        frame = imgFrame.getCvFrame()
                        trackletsData = track.tracklets
                        # Draw the counting line on the frame
                        cv2.line(frame, line_start, line_end, (0, 255, 0), 2)
                
                        for t in trackletsData:
                            if t.status.name == "TRACKED":
                                roi = t.roi.denormalize(frame.shape[1], frame.shape[0])
                                x1 = int(roi.topLeft().x)
                                y1 = int(roi.topLeft().y)
                                x2 = int(roi.bottomRight().x)
                                y2 = int(roi.bottomRight().y)
                
                                # Calculate centroid
                                centroid = (int((x2-x1)/2+x1), int((y2-y1)/2+y1))
                
                                # Calculate the buffer zone boundaries
                                right_boundary = h_line + 15
                                left_boundary = h_line - 15
                
                                try:
                                    if not (left_boundary <= centroid[0] <= right_boundary):
                                        pos[t.id] = {
                                            'previous': pos[t.id]['current'],
                                            'current': centroid[0]      }
                                    if pos[t.id]['current'] > right_boundary and pos[t.id]['previous'] < right_boundary:
                
                                        obj_counter[1] += 1 #Right side
                                        going_in += 1
                                    if pos[t.id]['current'] < left_boundary and pos[t.id]['previous'] > left_boundary:
                                    
                                        obj_counter[0] += 1 #Left side
                                        going_out += 1
                                except:
                                    pos[t.id] = {'current': centroid[0]}
                
                                try:
                                    label = labelMap[t.label]
                                except:
                                    label = t.label
                
                                cv2.putText(frame, str(label), (x1 + 10, y1 + 20), cv2.FONT_HERSHEY_TRIPLEX, 0.5, text_color)
                                cv2.putText(frame, f"ID: {[t.id]}", (x1 + 10, y1 + 45), cv2.FONT_HERSHEY_TRIPLEX, 0.5, text_color)
                                cv2.putText(frame, t.status.name, (x1 + 10, y1 + 70), cv2.FONT_HERSHEY_TRIPLEX, 0.5, text_color)
                                cv2.rectangle(frame, (x1, y1), (x2, y2), rectangle, cv2.FONT_HERSHEY_SIMPLEX)
                                cv2.circle(frame, (centroid[0], centroid[1]), 4, (255, 255, 255), -1)
                    
                
                        cv2.putText(frame, f'Left: {obj_counter[0]}; Right: {obj_counter[1]}', (10, 35), cv2.FONT_HERSHEY_SIMPLEX, 0.8, (0, 0, 0xFF), 2, cv2.FONT_HERSHEY_SIMPLEX)
                        cv2.putText(frame, "FPS: {:.2f}".format(fps), (2, frame.shape[0] - 4), cv2.FONT_HERSHEY_TRIPLEX, 0.6, text_color)
                
                        # Displaying full frame
                        frameFull = imgFull.getCvFrame()
                        cv2.imshow("full_frame", frameFull)
                
                        # Displaying cropped frame with tracked objects
                        cv2.imshow("tracker", frame)
                
                        if cv2.waitKey(1) == ord('q'):
                            break`