Sparse PCD

gdeanrexroth

The scattered points in the back ground seems to be parts of the objects. But as shown on the screenshots, it look like the points are trying to fully form but can not. I am using a oak-d-sr-poe. Once again, could the calibration and incorrect focal length be the reasoning for the sparseness? The configurations are set to the right properties so I am unsure what the issue is.

Any suggestions would be useful.

jakaskerl

gdeanrexroth
Which script are you using for pointcloud generation?

gdeanrexroth Once again, could the calibration and incorrect focal length be the reasoning for the sparseness?

Try tof-rgb align example, this should tell you if parameters are ok.

Thanks,
Jaka

gdeanrexroth

jakaskerl
I am running a script that i made. I did read the tof-rgb align to somewhat give me an idea on how to use both the color camera and tof nodes. It works as expected but everything so far is still sparse. All of the recommendations you have mentioned, has helped out a lot in different ways. But all the results are still sparse.

gdeanrexroth

jakaskerl
I can send you snippets of what I have. Or through email if needed.
More angles so you can see the points in the background. One screenshot is a back view of the pcd. As you can tell it seems that that it floating points are trying merge to the front. But they are not able to do so.

jakaskerl

Hi gdeanrexroth
Looks to me like the sparse points are actually part of the background. If you set a distance threshold, these points will not be included in the pointcloud.
It looks like a visualization error, not depth as far as I can see.

Thanks,
Jaka

gdeanrexroth

jakaskerl
Hey Jaka, is this a good source for me refer to for retrieving intrinsic data?
(https://docs.oakchina.cn/projects/api/samples/calibration/calibration_reader.html)
I ran it and got some results. I then added those results to my code and saw a difference with the generated pcd.
This is what I added
# Camera intrinsic parameters

fx = 472.8673095703125

fy = 472.693603515625

cx = 317.2562561035156

cy = 254.81143188476562

**intrinsic = o3d.camera.PinholeCameraIntrinsic(width=640, height=480, fx=fx, fy=fy, cx=cx, cy=cy)

The amount sparse points in the back disappeared. Which is a plus. Some are still trailing behind but more so at the bottom and they are closer. Yet you mentioned that visualization and a distance threshold. Is there any links you can refer me to? Or potentially point me in the right direction to fix this?
Here us how my pcd looks like with the new intrinsic parameter**

The objects are close to where they need to be. Yet the points seem to distributing at in pieces to form yet are not fully forming.

jakaskerl

gdeanrexroth
Generally yes, but the docs are a bit outdated; the source code should be the same. Make sure you select the correct resolution (same as used with the pcl), and to use useSpec=False argument whenever you can so actual calculated values (from calibration) are used.

gdeanrexroth Yet you mentioned that visualization and a distance threshold

https://docs.luxonis.com/hardware/platform/depth/configuring-stereo-depth/#Configuring Stereo Depth-2. Fixing noisy depth-Stereo postprocessing filters-Threshold filter
This is to remove the points from the background which you don't need and are usually noisy.

Thanks,
Jaka

gdeanrexroth

jakaskerl
When you mentioned visualization, I am a simply using the open3d library. What else is needed for visualization that is possibly causing an error to display the pcd? Here is an example script
"**import open3d as o3d
pcd*_file_path = r'color_pcd.pcd'
pcd = o3d.io.read_point_cloud(pcd_file_path)
o3d.visualzation.draw_geometries([pcd], window_name="PCD Viewer"

If visualization is an possible error then how can i fix this or what do i need to add or change?***

jakaskerl

gdeanrexroth
No need to change if you are using open3d. It's important the depth map looks good, and that color intrinsics and extrinsincs are correct. The pointcloud should then look good.

adding additional filtering to depth should improve the pointcloud (things like the above mentioned threshold filter and temporal filter, then enabling subpixel to avoid layering).

Thanks,
Jaka