Hello,

I'd like to combine object tracking and spatial detection using SDK to output the spatial data for each of the tracked object. For example:

Frame 1:
ID 1: x = 5 y= 15 z= 13
ID 2: x= 8 y=20 z= 15

Frame 2:
ID 1: x = 5.1 y= 15.1 z= 13.1
ID 2: x= 8.2 y=20.2 z= 15.2

I've tried Object Tracking as explained here: https://docs.luxonis.com/projects/sdk/en/latest/samples/NNComponent/sdk_object_tracking/?highlight=tracker But I am not getting the ID number of the object in the box. I am only getting a line that is drawn tracking the path of the object. How could I get the object ID number? And then how could I get this to output with the spatial data?

Thanks a lot in advance,
Jae

import depthai as dai
from depthai_sdk import OakCamera
from depthai_sdk.classes import SpatialBbMappingPacket

# def packets for printing data
def cb(packet: SpatialBbMappingPacket):
    object_count = 0
    message = "object count:{} :" + "label: {}" + " X:{}" + " Y:{}" + " Z:{}"

    for detection in packet.spatials.detections:
        object_count += 1
        print(message.format(object_count, detection.label, int(detection.spatialCoordinates.x),
                             int(detection.spatialCoordinates.y), int(detection.spatialCoordinates.z)))

with OakCamera() as oak:
    left = oak.create_camera("left")
    right = oak.create_camera("right")

    stereo = oak.create_stereo(left=left, right=right, fps=30)
    stereo.set_auto_ir(auto_mode=False, continuous_mode=False)
    stereo.set_ir(dot_projector_brightness=200, flood_brightness=1000)

    nn = oak.create_nn('yolov8n_coco_640x352', left, spatial=stereo, tracker = True)

    nn.config_spatial(
        bb_scale_factor=0.5, # Scaling bounding box before averaging the depth in that ROI
        lower_threshold=300, # Discard depth points below 30cm
        upper_threshold=10000, # Discard depth pints above 10m
        #Average depth points before calculating X and Y spatial coordinates:
        calc_algo=dai.SpatialLocationCalculatorAlgorithm.AVERAGE
    )

    nn.config_tracker(
        tracker_type=dai.TrackerType.ZERO_TERM_COLOR_HISTOGRAM,
        track_labels=[0],  # Track only 1st object from the object map. If unspecified, track all object types
        # track_labels=['person'] # Track only people (for coco datasets, person is 1st object in the map)
        assignment_policy=dai.TrackerIdAssignmentPolicy.SMALLEST_ID,
        max_obj=10,  # Max objects to track, which can improve performance
        threshold=0.1  # Tracker threshold
    )

    #visualize on screen (stereo.out.depth
    oak.visualize([nn.out.tracker, stereo.out.depth], fps=True)
    #call back function to print data
    oak.callback(nn.out.spatials, callback=cb) #, enable_visualizer=True)

    #oak.show_graph()
    oak.start(blocking=True)

    Hi jsiic
    The script works for me on 1.12.0 depthai_sdk.

    • shows the ID of the person
    • shows spatial data for that ID

    It seems to be broken in the newest version. I'll create an issue on GH.

    Thanks,
    Jaka

      5 months later

      @jsiic did you ever get the ID of the tracked object? I'm also having a difficult time trying to get both xyz coords plus a tracklet ID using the SDK