Extracting Depth Map from Luxonis Camera for 3D Axis Creation

KoenvanWijlick

Hello Luxonis community,

I hope this post finds you well. I'm currently working on a project where I'm aiming to utilize the capabilities of the Luxonis camera, particularly in terms of its depth sensing. I have a couple of questions and would appreciate any insights or guidance you might be able to provide.

Extracting the Depth Map:
- I'm looking to extract the depth map data directly from the camera. I understand that the Luxonis camera has impressive depth-sensing capabilities, but I'm unsure about the best approach or methodology to retrieve this data. Are there specific APIs or SDK functions I should be focusing on? Any sample code or references would be extremely helpful.
Creating a 3D Axis from the Depth Map:
- Once I have the depth map data, my goal is to use it to generate a 3D axis system. The idea is to provide a spatial reference framework for the objects detected in the depth map. Are there any existing tools or libraries within the Luxonis ecosystem (or external ones) that can help me achieve this? If not, any general advice on the transformation of depth map data into a 3D axis would be appreciated.

I've been truly impressed by the Luxonis camera's capabilities so far and am excited about the potential applications of this depth information. Thank you in advance for any help or guidance you can offer. Looking forward to being a part of this innovative community and contributing in any way I can.

Best regards,

[Your Name]

jakaskerl

Hi KoenvanWijlick

KoenvanWijlick Extracting the Depth Map:

You have depth examples available here - API and here - SDK. Whichever one you want is up to you; the SDK is simpler to use, but does lose on some customization options that are available through API.

KoenvanWijlick Creating a 3D Axis from the Depth Map

The API and the SDK both have functionality for the SpatialLocationCalculator node which essentially computes the xyz of pixels on the image. The examples are available through here.

Hope this helps,
Jaka

KoenvanWijlick

Thank you for your response.

Your input has been immensely helpful in extracting the information I need. However, I'm aiming to achieve an output in the format {x, y, z}. For instance, for pixels at x: 0 and y: 0, the output would ideally be {0, 0, 980}. Is this feasible?

Alternatively, I'm considering an array where the position corresponds to the x and y values, and the value at that position represents the z coordinate.

I'd appreciate any insights or suggestions on this matter.

Warm regards, [Your Name]

KoenvanWijlick

For reference this is what i am trying to achieve!

jakaskerl

Hi KoenvanWijlick
Yes, a depth map would get you the results you are looking for.
The depth map looks like a standard mono image frame, but instead of gray values [0-255], you are getting the depth (in cm? i think). This way you could, for any pixel (x, y,) obtain its depth value (z).

These x and y are pixel indices, not coordinates so if you expect to have 3D spatial information, you would need to use the spatialcalculator or do the calculations by hand (like done here - https://github.com/luxonis/depthai-experiments/tree/master/gen2-calc-spatials-on-host).

Thanks,
Jaka

KoenvanWijlick

jakaskerl
Thank you for the detailed explanation. I had overlooked that aspect earlier, and I truly appreciate you pointing it out. Your insights are valuable. Thanks again!"