need tips on obstable avoidance and depth technologies

smoothVision

Hello,

i want to avoid obstacles in front of my minirobot and plan the path accordingly and i would like some suggestions on how to proceed. I have the OAK D lite and a raspberry pi and some ultrasonic sensors to avoid obstacles for now ( i can get rid of them if not needed ).

In the "code samples tab" i've found the "collision avoidance" project and some other drones examples but i don't know if this is the best approach for my needs.

Also , is it possible to use deep learning model to enhance depth map in the oak d Lite ? like MIdas or similars.

jakaskerl

smoothVision Also , is it possible to use deep learning model to enhance depth map in the oak d Lite ? like MIdas or similars.

What do you mean by enhance? Upscale the resolution? Make it denser? I think the depth map created should be good enough for collision avoidance systems.

smoothVision

i actually don't know … i've seen online that there's the possibility of combining depth data with deep learning. If i train a deep network with rgbd data ( taken with oak d lite ) do you think i can improve depth estimation ?

Any suggestion about my objective ( moving my mincar outdoor ) ?

jakaskerl

smoothVision improve depth estimation

I guess you could make the depth map less noisy, but I'm not sure if you would benefit a lot from it.

Are you trying to develop AI algorithm for driving or are you using a controller (like PID or similar)? I guess you could feed the NN depth data and target direction and have the model approximate best path while avoiding obstacles.

Thanks,
Jaka

smoothVision

jakaskerl

No, i just want to extrapolate informations/data from the depth camera to avoid obstacles and plan a clear path. i don't know how PID may be useful for me now maybe for steering and speed control. For now, If i know some objects are in front of the camera i would just implement the algorithm to turn left or right simple as that ; that's why i'm actually asking if there are better approaches.

Another approach i was thinking about is making a grid of the depth map and sum up each pixel's depth for each square box of the grid. Do you know how can i get those depth values ?

What do you mean AI algorithm for driving ? like path planning algorithms ?

Maybe feeding a NN with depth data is a bit advanced for now and also i should collect data , it is a bit time consuming.

jakaskerl

smoothVision

smoothVision Do you know how can i get those depth values ?

There is an example for it. https://discuss.luxonis.com/d/1301-detect-the-depth-of-a-roi-in-rgb-camera/12 Not sure if you have seen it. It's a nice way to get the average depth for each square and visualize it.

Thoughts?

smoothVision

thanks , i try with for now. Does this code run on the camera or locally ? How do know if the code i run is done on the camera ? i've read something on the documentation but maybe is only for neural networks

jakaskerl

Hi @smoothVision
Processing runs on camera. One way to know that is by checking the operations done inside the with Device()context manager. Anything done there is run on the host. Anything done inside the pipeline is run on the device.

Thanks,
Jaka