How to get XYZ coordinates in mm for specific pixels?

lcruz

I would like to obtain the XYZ coordinates in millimeters for specific pixels in an image. I have a color image aligned with the depth map, and I read that Spatial Location Calculator can be used when defining regions of interest (ROI). However, I am not interested in calculating the average coordinates of a region—I need the exact coordinates for specific pixels.

Is it possible to achieve this with Spatial Location Calculator, or is there another way to do it? Any guidance would be greatly appreciated!

jakaskerl

lcruz
I think you might need to do this manually, since by the looks of it, SLC only supports ROI based calculations.

luxonis/depthai-experimentstree/master/gen2-calc-spatials-on-host - here's the host side code that does the same thing as the SLC does on-device. You have depth already, so you only need the pixel i, j indices to get the x, y.

Thanks,
Jaka

lcruz

jakaskerl

Thank you so much!

I have another question. If I have the (x, y) pixel coordinates in the color image (instead of the depth frame), is it enough to use the following settings to properly align the depth map with the RGB image so that I can use the same (x, y) coordinates? Or do I need to do anything else?

stereo.setDepthAlign(dai.CameraBoardSocket.CAM_A)

stereo.setOutputSize(monoLeft.getResolutionWidth(), monoLeft.getResolutionHeight())

Thanks in advance!

jakaskerl

lcruz
Should be good. You should see if the overlap looks good to you. If it does, it's likely totally fine.

Thanks,
Jaka

lcruz

jakaskerl

Thank you so much!

My last question is: does the example from your first reply take into account the distortion of the RGB image or do I need to manually undistort it before calculating XYZ?

rgb-undistortion-depth-alignment.py - In this example I saw, a distortion mesh is used to undistort the image manually before processing. So now I'm not sure if that step is required or already handled by the device pipeline.

Thanks in advance!

jakaskerl

lcruz
It assumes your image is undistorted as it relies on the pinhole model of the camera.
You can use camera node for built-in undistortion:
https://docs.luxonis.com/software/perception/rgb-d/#RGB-D-Alignment%20using%20DepthAI-RGB-D%20on%20Wide%20FOV%20cameras-Using%20Camera%20node

Thanks,
Jaka