Point cloud Generation with OAK-D Pro

MonalisaAchalla

Hello, I am trying to generate point clouds of objects using OAK-D pro. I am using the gen2-pointcloud code in the depthai-experiments repository. It somehow is very noisy and there are some areas in the point cloud that are empty and not being generated accurately. I need the point cloud for a project i am working on dimensioning objects and any missing details in the object point cloud would affect my accuracy.

Can anyone point me to how to accurately generate point clouds of objects with OAK-D pro camera. Thanks.

jakaskerl

Hi MonalisaAchalla
Could you please post an image of what the result looks like, so we can see what might be the cause?

Thanks,
Jaka

erik

Hi MonalisaAchalla , please see documentation here:
https://docs.luxonis.com/projects/api/en/latest/tutorials/configuring-stereo-depth/#fixing-noisy-pointcloud
Thanks, Erik

MonalisaAchalla

Hey jakaskerl , erik

Please look at the point cloud. So basically, the setup has a orange bowl on a table. Most of it is fine, but if you can see the points in the sides and bottom of the bowl are not clear. Also a part of the table is shown mistakenly above the plane of the table. I was wondering how to fix it. I need better point cloud of the bowl atleast, if not the table for dimensioning it properly.

jakaskerl

Hi MonalisaAchalla ,
Have you tried point cloud creation on something that has more contrast. Monotone white background and a relatively bright object probably aren't the best recipe as far as depth accuracy is concerned. Also, I would go step by step trough the docs Erik sent above to ensure best pointcloud generation for your usecase.

Thanks,
Jaka

MonalisaAchalla

Thankyou for your reply @jakaskerl .

In general what would be an ideal background setting be like for the best depth accuracy. Please let me know. Thanks.

erik

@MonalisaAchalla please also provide full recording, so we can cross-check locally. For background depth accuracy, see docs:
https://docs.luxonis.com/projects/api/en/latest/tutorials/configuring-stereo-depth/#fixing-noisy-depth
Thanks, Erik

MonalisaAchalla

Hey Eric,I tried uploading the recording of the setup. But I get an error saying request payload too large.

Also I am attaching pictures of setup and the pointcloud. As you can see there is a significant portion missing in the object point cloud. Please let me know what else to improve in the setup.

Thanks

erik

Hi MonalisaAchalla ,feel free to upload it to gdrive and share the link. From the screenshot it seems like there's a problem with the lack of texture. Thoughts?
Thanks, Erik

RuixiaoZhang

Hi, have you solved this problem? I'm facing the same issue as the generated point cloud has a lot of unclear/lost regions. I use OAK-D Pro. You can see that I have chosen a textured scene.

jakaskerl

Hi @RuixiaoZhang
From the looks of it, the perspective is different, that's why there are no pixels in the pointcloud for that specific region.
The camera view starts at coordinate origin and moves out in straight lines (along Z). That means if some part cannot be seen due to obstruction, no points will be generated for it, and it will seem like the pointcloud is missing some parts.

Thanks,
Jaka