@jakaskerl ,

I am currently working with the depth test and spatial noise since there seems to be a way to save the point cloud forming. When I launched the main.py file from this I was able to see the Oak-D streaming, but I was unable to see the point cloud streaming, or be able to save it. Do you know why I am getting the errors below? And how I can fix it so I can save a point cloud.

luxonis/depthai-experimentstree/master/depth-test_z-acc_spatial-noise

(venv) C:\Users\sale4088\AppData\Local\Programs\DepthAI\venv\Scripts\depthai experiments\depthai-experiments\depth-test_z-acc_spatial-noise>python3 main.py

[Open3D WARNING] The number of points is 0 when creating axis-aligned bounding box.

[Open3D WARNING] The number of points is 0 when creating axis-aligned bounding box.

[Open3D WARNING] The number of points is 0 when creating axis-aligned bounding box.

[Open3D WARNING] The number of points is 0 when creating axis-aligned bounding box.

[Open3D WARNING] Write PLY failed: point cloud has 0 points.

Point clouds saved

Traceback (most recent call last):

File "C:\Users\sale4088\AppData\Local\Programs\DepthAI\venv\Scripts\depthai experiments\depthai-experiments\depth-test_z-acc_spatial-noise\main.py", line 167, in <module>

camera.update()

File "C:\Users\sale4088\AppData\Local\Programs\DepthAI\venv\Scripts\depthai experiments\depthai-experiments\depth-test_z-acc_spatial-noise\oak_camera.py", line 122, in update

self.rgbd_to_point_cloud()

File "C:\Users\sale4088\AppData\Local\Programs\DepthAI\venv\Scripts\depthai experiments\depthai-experiments\depth-test_z-acc_spatial-noise\camera.py", line 70, in rgbd_to_point_cloud

point_cloud = o3d.geometry.PointCloud.create_from_rgbd_image(

I just want to be able to generate a point cloud and save the information for later use. I really like the way that spectacular AI uses the Oak-D camera to generate a point cloud. Is there anyway to retain that point cloud as a .ply? Below I have posted the link to the page.

SpectacularAI/sdk-examplestree/main/python/oak

@jakaskerl

Do I need Iron, Humble, or Noetic to work with ROS?

Is there a easy way to retain a point cloud, I bought this camera so I could capture and retain information about the surrounding environment to apply to my research. I was under the impression that this camera was a plug-n-play camera, but I have had consistent error after error with this camera and am growing extremely frustrated with the product. I am currently struggling with the functionality with this camera, I have had this camera for over a month and haven't been able to record and replay any information, imagery or sensors. Is there a way to simply open the application, and save a point cloud generated with this camera as a .PLY, .XYZ, or .LAS. Please let me know if this is possible.

6 days later

Hi @michaelsalerno
We have branches for all three versions.

You can also record the pointcloud with below script (.ply), but this will create multiple .ply files which you can iterate through to create an illusion of replay.

import depthai as dai
import numpy as np
import cv2
import sys
import os

try:
    import open3d as o3d
except ImportError:
    sys.exit("Critical dependency missing: Open3D. Please install it using the command: '{} -m pip install open3d' and then rerun the script.".format(sys.executable))

def create_pipeline():
    pipeline = dai.Pipeline()
    camRgb = pipeline.create(dai.node.ColorCamera)
    monoLeft = pipeline.create(dai.node.MonoCamera)
    monoRight = pipeline.create(dai.node.MonoCamera)
    depth = pipeline.create(dai.node.StereoDepth)
    pointcloud = pipeline.create(dai.node.PointCloud)
    sync = pipeline.create(dai.node.Sync)
    xOut = pipeline.create(dai.node.XLinkOut)

    camRgb.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1080_P)
    camRgb.setBoardSocket(dai.CameraBoardSocket.RGB)
    camRgb.setIspScale(1, 3)
    camRgb.setFps(20)

    monoLeft.setResolution(dai.MonoCameraProperties.SensorResolution.THE_400_P)
    monoLeft.setBoardSocket(dai.CameraBoardSocket.LEFT)
    monoLeft.setFps(20)

    monoRight.setResolution(dai.MonoCameraProperties.SensorResolution.THE_400_P)
    monoRight.setBoardSocket(dai.CameraBoardSocket.RIGHT)
    monoRight.setFps(20)

    depth.setDefaultProfilePreset(dai.node.StereoDepth.PresetMode.HIGH_DENSITY)
    depth.initialConfig.setMedianFilter(dai.MedianFilter.KERNEL_7x7)
    depth.setLeftRightCheck(True)

    depth.setDepthAlign(dai.CameraBoardSocket.RGB)

    monoLeft.out.link(depth.left)
    monoRight.out.link(depth.right)
    depth.depth.link(pointcloud.inputDepth)
    camRgb.isp.link(sync.inputs['rgb'])
    pointcloud.outputPointCloud.link(sync.inputs['pcl'])
    sync.out.link(xOut.input)

    xOut.setStreamName("output")

    return pipeline

def main():
    pipeline = create_pipeline()

    with dai.Device(pipeline) as device:
        outQueue = device.getOutputQueue(name="output", maxSize=4, blocking=False)
        frame_count = 0
        output_dir = "recorded_pointclouds"
        os.makedirs(output_dir, exist_ok=True)

        while True:
            data = outQueue.get()
            if data is None:
                continue

            inColor = data["rgb"]
            inPointCloud = data["pcl"]
            cvColorFrame = inColor.getCvFrame()

            if inPointCloud:
                points = inPointCloud.getPoints().astype(np.float64)
                pcd = o3d.geometry.PointCloud()
                pcd.points = o3d.utility.Vector3dVector(points)
                colors = (cvColorFrame.reshape(-1, 3) / 255.0).astype(np.float64)
                pcd.colors = o3d.utility.Vector3dVector(colors)

                # Save the point cloud in PLY format
                ply_filename = f"{output_dir}/frame_{frame_count:04d}.ply"
                o3d.io.write_point_cloud(ply_filename, pcd)
                frame_count += 1

            cv2.imshow("RGB", cvColorFrame)
            if cv2.waitKey(1) == ord('q'):
                break

if __name__ == "__main__":
    main()

Thanks,
Jaka

    jakaskerl

    Thank you I really appreciate all of your consistent assistance. You and your team have been fantastic with responding and assisting me. Thank you again, I really appreciate it.