Not able to get the 3D hand keypoint coordinates using detphai Hand Tracker

souhamseibi

Hello, I'm using https://github.com/geaxgx/depthai_hand_tracker and I'm wondering if someone could assist me in accessing the 3D key point coordinates of all 21 Hand keypoints . Any advice or guidance would be greatly appreciated.

jakaskerl

Hi souhamseibi

https://discuss.luxonis.com/d/1270-how-to-select-a-particular-hand-key-point-location for pixels, but should be very similar to:
https://discuss.luxonis.com/d/1229-how-can-we-get-coordinates-of-key-points-while-using-blazeposedepthai/2 for 3D, but this is for body tracking.

Hope this helps,
Jaka

souhamseibi

Hi jakaskerl ,
I've followed the suggestions provided in the forum previously, and I run the demo on edge mode and --use world_landmarks and I'm now able to extract the world_landmarks of 21 keypoints of the hand. However, I'm trying to compare world_landmark point 0 with the output of the script manager of the model of wrist coordinates to ensure that the world_landmark attribute of the hand has exactly what I need. Unfortunately, I'm getting different results and I'm wondering if there's some transformation that I should be doing to get the real-world landmarks

I've also tried to get more spatialCoordinates points by modifying the template_manager_script_duo script file
in the HandTrackerEdge : idk if I'm on the right path, and if these screenshots will help you to understand my needs.

jakaskerl

Hi souhamseibi,
it looks to me like the script manager uses a different base coordinate system. For the world_landmarks:

World landmarks: the latest versions of the landmark models have a 2nd output that yields estimated real-world 3D coordinates in meters with the origin at the hand’s approximate geometric center. Note that this brings very similar information to the "classic" output that yields also 3D coordinates but in a normalized form. To use the world landmarks, set the HandTracker's argument use_world_landmarks to True. It will fill the HandRegion's attribute world_landmarks

The coordinates will be relative to geometric center of the hand (roughly the key point 9). I'm not sure what the manager uses. You can pass the -xyz option when running the demo to also display the world base coordinates of the key point 0 and do transformations from there.

Hope this helps,
Jaka

souhamseibi

Thank you so much for your help, it was exactly what I needed.