How to save a point cloud and access the individual points

flash

I would like to save the rgbd.pcl point cloud that is created using rgbd = p.create(dai.node.RGBD).build() following this example. How can I achieve this using open3d and depthai v3?

I would also like to access the individual points using getPoints and getPointsRGB methods from the documentation? Could you please provide an example of how to write this in Python?

I'm using an OAK-D Lite camera.

OskarSonc

Hi @flash,
Yes—you can do this with Open3D. Use:

o3d.geometry.PointCloud
o3d.utility.Vector3dVector (set points/colors)
o3d.io.write_point_cloud(...) (save PLY/PCD)

For accessing points in DepthAI v3, use getPoints() / getPointsRGB() on the rgbd.pcl message.

For exact usage and parameters, check the Open3D docs: https://www.open3d.org/docs/release/index.html

flash

From what I could see, rgbd.pcl does not have the getPoints() or getPointsRGB functions (see screenshot below). I tried creating an output queue but still struggling to figure out how to access these methods from DepthAI v3. From the documentation, these functions require a PointCloudData message but how do I get this from rgbd.pcl which is a RGBD message? The examples I came across online are for DepthAI v2. and use XLinkOut which I assume is now deprecated?

OskarSonc

@flash,
You’re looking at the port, not the message.
rgbd.pcl is an output port that emits dai.PointCloudData. The methods getPoints() / getPointsRGB() live on the message you read from that port — not on the port itself. (rgbd.rgbd is the port that emits dai.RGBDData frames.)

DepthAI v3:

import depthai as dai

with dai.Pipeline() as p:
    rgbd = p.create(dai.node.RGBD).build(autocreate=True, size=(640, 400))
    q = rgbd.pcl.createOutputQueue(maxSize=2, blocking=True)  # <-- host queue
    p.start()
    msg = q.get()  # <-- dai.PointCloudData

# Access points
xyz = msg.getPoints()                 # (N,3) float32
xyzc, rgba = msg.getPointsRGB()       # xyz:(N,3) float32, rgba:(N,4) uint8
print(type(msg))  # should be <class 'depthai.PointCloudData'>

flash

Thanks for sharing this important clarification! I was able to save the PCD files.
Looking at the node diagram in the docs, Nodes communicate via Messages which pass data from the pool to the queue between nodes.
To access the output of the RGBD node (i.e. the pool), you first createOutputQueue then get the message data stored in the pool.
To provide input to the RGBD node (i.e. the queue), you have to create links (color.out.link(rgbd.inColor) and depth.out.link(rgbd.inDepth)). Why can’t you createInputQueue instead?

Is there a technical paper I could read to better understand DepthAI linking mechanisms and the overall architecture?

OskarSonc

Right — createInputQueue() is only available on host-visible ports. For nodes like RGBD you feed data by linking from other device nodes (color, stereo, etc). If you need host input, you’d use a host-bridge node instead.

This links could be helpful:
https://docs.luxonis.com/software-v3/depthai/tutorials/v2-vs-v3/
https://docs.luxonis.com/software-v3/depthai/depthai-components/pipeline/