DaniilPastukhov

  • Jun 13, 2024
  • Joined Jan 21, 2023
  • 0 best answers
  • We are thrilled to announce the release of DepthAI SDK 1.11.0, which brings a host of exciting new features and enhancements. In this blog post, we will highlight the main changes that have been introduced in this release!

    Auto IR Mode

    This new experimental feature in DepthAI SDK 1.11.0 automatically adjusts the power of the infrared (IR) projector and illumination based on the ambient lighting conditions. By dynamically adapting to the lighting environment, Auto IR Mode ensures optimal illumination for depth and spatial perception. This feature eliminates the need for manual adjustments, simplifying the setup process and making it easier for users to achieve accurate depth perception in varying lighting conditions.

    The following example illustrates how to activate this feature (view on GitHub):

    from depthai_sdk import OakCamera
    
    with OakCamera() as oak:
        left = oak.create_camera('left')
        right = oak.create_camera('right')
        stereo = oak.create_stereo(left=left, right=right)
    
        # Automatically estimate IR brightness and adjust it continuously
        stereo.set_auto_ir(auto_mode=True, continuous_mode=True)
    
        oak.visualize([stereo.out.disparity, left])
        oak.start(blocking=True)

    Point Cloud Support

    DepthAI SDK 1.11.0 introduces point cloud support, which is an exciting addition for generating immersive 3D visualizations. Point clouds provide a three-dimensional representation of the scene captured by the OAK device. By leveraging the depth information from the stereo cameras, you can now generate accurate and detailed 3D point cloud visualizations. This feature enables a more immersive and interactive experience, allowing you to explore the captured scene from different angles and perspectives.

    Make sure to install Rerun SDK using the following command: pip install rerun-sdk.

    The following example shows how to visualize point cloud using Rerun (view on GitHub):

    from depthai_sdk import OakCamera
    from depthai_sdk.classes.packets import PointcloudPacket
    import rerun as rr
    import subprocess
    import time
    
    subprocess.Popen(["rerun", "--memory-limit", "200MB"])
    time.sleep(1)  # Wait til rerun spins up
    rr.init("Rerun ", spawn=False)
    rr.connect()
    
    def callback(packet: PointcloudPacket):
        colors = packet.color_frame.getCvFrame()[..., ::-1] # BGR to RGB
        rr.log_image('Color Image', colors)
        points = packet.points.reshape(-1, 3)
        rr.log_points("Pointcloud", points, colors=colors.reshape(-1, 3))
    
    
    with OakCamera() as oak:
        pcl = oak.create_pointcloud()
        oak.callback(pcl, callback=callback)
        oak.start(blocking=True)

    To view the complete list of changes, please visit GitHub release page. We value your feedback and encourage you to share your experiences, suggestions, and ideas with our community. Together, we can continue to improve and evolve DepthAI SDK library to meet the ever-growing demands!

  • We are thrilled to announce the latest release of our DepthAl SDK, packed with a range of enhancements and new features. In this blog post, we will walk you through the noteworthy updates included in this release, enabling you to harness the power of our SDK more effectively.

    Trigger-Action Mechanism

    Trigger-Action API in the DepthAl SDK offers a simple yet powerful way to specify certain conditions and actions that should be executed when those conditions are met. With this feature, you can easily define a set of rules that determine what actions should be taken based on specific conditions. While the SDK provides a range of pre-defined conditions and actions, you also have the flexibility to create your own customized rules. This empowers you to automate processes and streamline workflows according to your unique requirements.

    The following example illustrates how to save a 15-second video when at least one person is detected:

    from depthai_sdk import OakCamera
    from depthai_sdk.trigger_action.actions.record_action import RecordAction
    from depthai_sdk.trigger_action.triggers.detection_trigger import DetectionTrigger
    
    with OakCamera() as oak:
        color = oak.create_camera('color', encode='jpeg')
        stereo = oak.create_stereo('400p')
    
        nn = oak.create_nn('mobilenet-ssd', color)
    
        trigger = DetectionTrigger(input=nn, min_detections={'person': 1}, cooldown=30)
        action = RecordAction(inputs=[color, stereo.out.disparity], dir_path='./recordings/',
                              duration_before_trigger=5, duration_after_trigger=10)
        oak.trigger_action(trigger=trigger, action=action)
    
        oak.visualize(nn)
        oak.start(blocking=True)

    You can find more examples on our GitHub.

    Infinite replay

    The new replay looping option has been added to our SDK. This feature enables you to effortlessly loop through recorded data, making the development process more convenient.

    The following example shows how to use this feature:

    from depthai_sdk import OakCamera
    
    with OakCamera(replay='cars-california-02') as oak:
        oak.replay.set_loop(True)  # Loop the video, otherwise app will exit when video ends
        color = oak.create_camera('color', '1080p')
        nn = oak.create_nn('vehicle-detection-adas-0002', input=color)
        oak.visualize(nn)
        oak.start(blocking=True)

    Updated documentation

    We have updated our documentation page to provide more comprehensive and accessible information. We aim to make it easier for you to understand and utilize the features and functionalities of DepthAI SDK.

    Visit our updated documentation page at DepthAl SDK docs and explore the wealth of information we have prepared for you!

    To view the complete list of changes, please visit GitHub release page. We value your feedback and encourage you to share your experiences. suggestions, and ideas with our community. Together, we can continue to improve and evolve DepthAI SDK library to meet the ever-growing demands!

  • Hi @NotGrant,

    Could you please try running the following script?

    import argparse
    import cv2
    from depthai_sdk import OakCamera, ArgsParser
    from depthai_sdk.classes import DetectionPacket
    from depthai_sdk.visualize.configs import TextPosition
    
    parser = argparse.ArgumentParser()
    parser.add_argument("-conf", "--config", help="Trained YOLO json config path", default='json/best.json', type=str)
    args = ArgsParser.parseArgs(parser)
    
    
    def callback(packet: DetectionPacket):
        visualizer = packet.visualizer
    
        num = len(packet.img_detections.detections)
        visualizer.add_text(f"Number of Objects: {num}", position=TextPosition.TOP_MID)
        visualizer.draw(packet.frame)
        cv2.imshow(f'frame {packet.name}', packet.frame)
    
    
    with OakCamera(args=args) as oak:
        color = oak.create_camera('color')
        nn = oak.create_nn(args['config'], color, nn_type='yolo', spatial=True)  # nn_type='yolo' decode_fn=decode
        oak.visualize(nn, callback=callback)
        oak.visualize(nn.out.passthrough, fps=True)
        oak.start(blocking=True)

    Also please specify the path to the blob in your json config. The following configuration worked in my case:

    {
        "model": {
            "blob": "../models/yolov5n_coco_416x416_openvino_2021.4_6shave.blob"
        },
        "nn_config":
        {
            "output_format" : "detection",
            "NN_family" : "YOLO",
            "input_size": "416x416",
            "NN_specific_metadata" :
            {
                "classes" : 80,
                "coordinates" : 4,
                "anchors" : [10,13, 16,30, 33,23, 30,61, 62,45, 59,119, 116,90, 156,198, 373,326],
                "anchor_masks" :
                {
                    "side52" : [0,1,2],
                    "side26" : [3,4,5],
                    "side13" : [6,7,8]
                },
                "iou_threshold" : 0.5,
                "confidence_threshold" : 0.5
            }
        },
        "mappings":
        {
            "labels":
            [
                "person",
                "bicycle",
                "car",
                "motorbike",
                "aeroplane",
                "bus",
                "train",
                "truck",
                "boat",
                "traffic light",
                "fire hydrant",
                "stop sign",
                "parking meter",
                "bench",
                "bird",
                "cat",
                "dog",
                "horse",
                "sheep",
                "cow",
                "elephant",
                "bear",
                "zebra",
                "giraffe",
                "backpack",
                "umbrella",
                "handbag",
                "tie",
                "suitcase",
                "frisbee",
                "skis",
                "snowboard",
                "sports ball",
                "kite",
                "baseball bat",
                "baseball glove",
                "skateboard",
                "surfboard",
                "tennis racket",
                "bottle",
                "wine glass",
                "cup",
                "fork",
                "knife",
                "spoon",
                "bowl",
                "banana",
                "apple",
                "sandwich",
                "orange",
                "broccoli",
                "carrot",
                "hot dog",
                "pizza",
                "donut",
                "cake",
                "chair",
                "sofa",
                "pottedplant",
                "bed",
                "diningtable",
                "toilet",
                "tvmonitor",
                "laptop",
                "mouse",
                "remote",
                "keyboard",
                "cell phone",
                "microwave",
                "oven",
                "toaster",
                "sink",
                "refrigerator",
                "book",
                "clock",
                "vase",
                "scissors",
                "teddy bear",
                "hair drier",
                "toothbrush"
            ]
        }
    }

    Thanks,
    Daniil

  • Hello @rpsavage1,

    Could you try installing newer Python version (e.g., 3.9) and give it a try? Also what is the output of pip show depthai-sdk?

    Thanks, Daniil