how to get landmarks by label for 'human-pose-estimation-0001?'

Torsten

Hello…
I'm new to depthai and I'm facing problem that I can't solve by reading the docs.

I would like to know where a hand is in relation to the ellbow of all people in the frame. I don't need visual output, just the positions of the hands and wrists of each person.

I've seen the demo of 'human-pose-estimation-0001' and it seems to solve this, but I don't have any clue on how to implement it. Could someone point me in the right direction? Where is stuff like this documented?

from depthai_sdk import OakCamera
from depthai_sdk.classes import DetectionPacket
from depthai_sdk.classes.packets import ImgLandmarksPacket

def cb(packet: ImgLandmarksPacket):
     # what are packet.landmarks.landmarks? persons?
    for landmarks in packet.landmarks.landmarks:
        for landmark in landmarks:
            # what is a landmark? a wrist / hand / head ?
            pass;
            
with OakCamera() as oak:
    color = oak.create_camera('color')
    human_pose_nn = oak.create_nn('human-pose-estimation-0001', color)
    oak.visualize(human_pose_nn.out.main, callback=cb)
    oak.start(blocking=True)

SamiUddin

keypointsMapping = ['Nose', 'Neck', 'R-Sho', 'R-Elb', 'R-Wr', 'L-Sho', 'L-Elb', 'L-Wr', 'R-Hip', 'R-Knee', 'R-Ank',
'L-Hip', 'L-Knee', 'L-Ank', 'R-Eye', 'L-Eye', 'R-Ear', 'L-Ear']
POSE_PAIRS = [[1, 2], [1, 5], [2, 3], [3, 4], [5, 6], [6, 7], [1, 8], [8, 9], [9, 10], [1, 11], [11, 12], [12, 13],
[1, 0], [0, 14], [14, 16], [0, 15], [15, 17], [2, 17], [5, 16]]
mapIdx = [[31, 32], [39, 40], [33, 34], [35, 36], [41, 42], [43, 44], [19, 20], [21, 22], [23, 24], [25, 26], [27, 28],
[29, 30], [47, 48], [49, 50], [53, 54], [51, 52], [55, 56], [37, 38], [45, 46]]
colors = [[0, 100, 255], [0, 100, 255], [0, 255, 255], [0, 100, 255], [0, 255, 255], [0, 100, 255], [0, 255, 0],
[255, 200, 100], [255, 0, 255], [0, 255, 0], [255, 200, 100], [255, 0, 255], [0, 0, 255], [255, 0, 0],
[200, 200, 0], [255, 0, 0], [200, 200, 0], [0, 0, 0]]

SamiUddin

From API Example here: luxonis/depthai-experimentsblob/master/gen2-human-pose/pose.py

Kind Regards!

Torsten

thanks for your help!

But that mapping seemed wrong for the mode I'm using. I've reversed engineered the labels (by putting a big red circle and every keypoint one by one 😉) And came up with the following mapping that works great for me:

bodyParts = {

    "LShoulder": 0,

    "RShoulder": 1,

    "LElbow": 2,

    "LWrist": 3,

    "RElbow": 4,

    "RWrist": 5,

    "LHip": 6,

    "LKNee": 7,

    "LAnkle": 8,

    "RHip": 9,

    "RKnee": 10,

    "RAngle": 11,

    "Nose": 12,

    "LEye": 13,

    "LEar": 14,

    "REye": 15,

    "REear": 16,

}

Nevertheless, it's working now.

SamiUddin

That's Great. Good Luck.