Using edges to produce lines

gregflurry

This is not strictly a DepthAI question, so please forgive me if it is inappropriate. Further I admit to being a novice and may be asking a dumb question.

I use the DepthAI EdgeDetection node to produce edges using the 'standard' Sobel 'operator'. The result shows lots of bright edges that to my eyes present a very accurate rendition of the scene. But, the edge image contains lots of "noise" -- faint blobs of pixels. When I run the edge image through OpenCV HoughLinesP function, I get on the order of 1800 lines. Drawing the lines on the color image from which the edges derive produces nothing but lines obscuring almost all pixels. I attribute this to noise. I've tried a couple of forms of noise reduction but have not seen much improvement.

I also tried taking the color image and using the OpenCV Sobel functions (see this). It produces an edge image quite similar to that of the EdgeDetector. However, running that result through HoughLinesP produces almost 2400 lines.

So, I found a lines example that used a Canny edge detector. I ran that example (actually had to modify it a bit to make it work) on the original color image. It produced a decent set of edges, if less intense. The HoughLinesP function produced 360 lines. Drawing the lines indicated relative fidelity to what I'd pick as edges. With one major exception -- a wall corner that appears clearly delineated in edge image appears as set of broken lines.

So, I am not sure if I am attempting something that can't be done, am simply doing something wrong, or just have not found the magic incantations for DepthAI and OpenCV. I am sure I'm out of additional ideas on what to try next.

Any suggestions appreciated.

Brandon

gregflurry Great questions.

I've wondered the same things actually. What is the end-use of the edge information (sorry if I missed it)?

I ask because we are seeing some pretty neat neural-network lane-following networks coming out. So if this is the end purpose, a NN might be a good solution. For example PINTO0309 just shared this in our Discord:

141_lanenet-lane-detection　Commited.
https://github.com/PINTO0309/PINTO_model_zoo/tree/main/141_lanenet-lane-detection
https://github.com/MaybeShewill-CV/lanenet-lane-detection

gregflurry

Thanks for the thoughtful response.

My end goal is related to lane detection, I guess. I am experimenting in indoor navigation. The space is of course full of lines formed by walls, furniture, and even the floor in some areas; many lines are nearly horizontal or nearly vertical. On occasion, sets of lines can resolve to objects, or, at when fused with additional information, location. While my long term plan includes using a neural-net (or maybe multiple), I have zero experience and am sort of working my way up to understanding how to choose and train a model. So, I guess I'd have to say that right now I'm exploring the art of the possible. The references you cite should help me make progress more quickly. Thanks!

luxonis-Sachin

gregflurry
Hey,
On your initial post I think some images from your experimenting would help in better analyzing what I should suggest.

But in general, If I am suggesting you need to use corrosion or erosion filters on top of edge filters it will remove some of the noise which helps in retaining only important big lines. this is normally what is done in contour detection also.

And I donna much about HoughLinesP but I think it is an estimation function. it sees a slightly curve and it tries to map the whole thing to a line and based on some threshold it breaks saying part of it is a line and I will search the next part and fit an another line there I think. And it goes on leading to so many lines. So you need to play with thresholds to figure that out if you are still going on that method.

If your goal is lane detection you should just use a Neural Network based lane detection.
And as of indoor navigation. What are you planning to do after detecting lines ? How do you classify which is from a wall and which is from furniture, etc?

Brandon

Got it. Thanks for the background. I'm trying to think who would be best to advise here. Actually @luxonis-Sachin I think would be. This would likely involve using ROS though, which is its own whole setup. But likely would be worth it as ROS provides a ton of useful things to help with Navigation. Will ask him offline.

gregflurry

Wow! Thanks for the thoughtful replies luxonis-Sachin and Brandon!

I should have provided test images and even code, but I had hoped there was something obvious I was doing wrong. I have played with tuning HoughLinesP (and other functions), but have not seen much, if any, improvement.

I am not familiar with corrosion or erosion filters, but will certainly do some investigation and experimentation. In the long run, ROS is on my list of topics to at least investigate, but it is such a different beast that I'd have to redo most of my current environment, and I'm not at a point where I'm willing to do that.

As far as my plans for indoor navigation, I really don't have firm plans. I was successful (mostly) in using a simple, but pretty accurate, LIDAR unit to facilitate localization and navigation, and I'm trying to find how best to use the OAK-D to replace it. So, sadly, I can't answer how I would classify items, because I simply don't yet have enough knowledge to do so.

I have a time-critical task in a very different area that will keep me busy for roughly a week. After that, I'll return to this topic and I hope to be a bit more specific with questions and answers.

Thanks again!

luxonis-Sachin

gregflurry
Hope this video helps in using some filters

gregflurry

luxonis-Sachin Wonderful video! I watched the entire video with fascination. I shall attempt to apply the techniques to my situation.