Line scanning for 2D depth profile

Luke

Hello everyone,

I am working on a project using an oak-d pro and depthai-python. I am trying to output the 2D profile (cross-section) of a horizontal strip of a depth image.

Starting with the ‘spatial_location_calculator’ example, I configured the roi as a small narrow rectangle which moves incrementally across each frame from left to right, recording the X and Z values to a list. After each scan, I plot the values to get my cross-sectional profile.

This works, but is very slow for “higher resolution” scans where the step size is small.

Rather than shifting the roi over each new frame, I would like to pull all of the depth values (X and Z) across a horizontal line from a single frame.

If anyone has any suggestions on how to best accomplish this, I would greatly appreciate any guidance.

Thank you,
-Luke

erik

Hi Luke ,
I have created a simple multi-ROI spatial location calculator example, see PR here. I hope this helps!
Thanks, Erik
multi_roi_slc

Luke

erik

Luke

Hi Erik,

This is perfect, thank you!!

Have a great day,

-Luke

erik

This looks awesome, thanks for sharing Luke !

Luke

erik
Thanks for your help with it!

Luke

Hi Erik,

I am using line detection on the left and right mono frames, and would like to return depth coordinates for those lines (or points on the lines).

Can that be done? Would it help with disparity matching?

Thanks!
-Luke

erik

Hi Luke ,
Could you share some images of what that would look like? That would help with understanding🙂
Thanks, Erik

Luke

Hi erik ,

Here is an example of the type of profile I am wanting to map/track. In this picture it is viewed at an angle, but in my current setup the camera is directly overhead. Using Canny() and HoughLines() I add the red vertical lines marking the edges of the groove. In this photo there are only 2 lines, but typically there will be 4 vertical lines, marking the upper and lower edges.

My aim is to improve the accuracy and smooth out my 2D depth profiles by pinpointing the corners where a change in depth occurs.

Thanks!

erik

Hi Luke ,
Unfortunately, even if there was a way to improve disparity matching with these lines, the depthai stereo pipeline doesn't support it, as stereo blocks (silicon) on chip don't support it.
Thanks, Erik

Luke

Hi erik ,

Ok, thanks for the info!

-Luke

Luke

Hi erik ,

Sorry to keep dragging this question on. I have been looking through the depthai-experiments, and the gen2-triangulation example seems to do a similar type of thing I am trying to do.

Rather than detecting faces/facial landmarks in the mono frames, I would like to identify keypoints using line detection. Then, as in the example, I would like to return the depth/disparity of those keypoints.

Is this possible?

Thank you,

-Luke

erik

Hi Luke,
It is possible, but I am not sure how easy it would be to implement line detection with NNs (see docs here). We also plan to create such general CV functions available in firmware - feature request here - but no ETA yet (feel free to comment your thoughts).
Thanks, Erik