Hi... @Brandon @dhruvsheth-ai and @all I'm trying to run crash avoidance program on OAK-1 and have few queries. is there a way to simulate testing without actually installing the hardware setup in a vehicle? I checked if there is video as argument, it seems it's only camera... The camera video frame appears upside down, is there any specific reason for this or just by mistake... It would be great to test this on video of any given dash cam footage to be able to see how it performs. Does it rely on 3D processing? then video becomes irrelevant as it's in 2D. Can you share some video or link of this app working? Thanks & Best Regards, Ram

Hello ramkunchur , great application! Let me jump straight to the answers: Yes, you can use video and stream the frames to the device and perform inference on them. We have several example codes: stereo depth from host , mobilenet from video or running depthai_demo with python3.6 depthai_demo.py -cnn vehicle-detection-adas-0002 -vid https://www.youtube.com/watch?v=Y1jTEyb3wiI . If you were using OAK-D (or any other device that has stereo cameras on them), you could use depthai recordings that save mono frames and send mono+color frames back to the device to replay the recording. You can rotate it using camRgb.setImageOrientation(dai.CameraImageOrientation.AUTO) , besides AUTO you can also choose between: AUTO,NORMAL, HORIZONTAL_MIRROR, VERTICAL_FLIP, ROTATE_180_DEG . Here's an awesome collision detection demo that Noroc created: This demo was run on OAK-D (so it has spatial coordinates as well) and I believe that stereo cameras -> spatial locations would be beneficial for such an application - for example getting speed/acceleration from differences in spatial locations. I hope this helps! Thanks, Erik

Collision avoidance help

ramkunchur

Hi...@Brandon @dhruvsheth-ai and @all

I'm trying to run crash avoidance program on OAK-1 and have few queries.

is there a way to simulate testing without actually installing the hardware setup in a vehicle? I checked if there is video as argument, it seems it's only camera...
The camera video frame appears upside down, is there any specific reason for this or just by mistake...

It would be great to test this on video of any given dash cam footage to be able to see how it performs.
Does it rely on 3D processing? then video becomes irrelevant as it's in 2D.

Can you share some video or link of this app working?

Thanks & Best Regards,
Ram

erik

Hello ramkunchur, great application! Let me jump straight to the answers:

Yes, you can use video and stream the frames to the device and perform inference on them. We have several example codes: stereo depth from host, mobilenet from video or running depthai_demo with python3.6 depthai_demo.py -cnn vehicle-detection-adas-0002 -vid https://www.youtube.com/watch?v=Y1jTEyb3wiI. If you were using OAK-D (or any other device that has stereo cameras on them), you could use depthai recordings that save mono frames and send mono+color frames back to the device to replay the recording.
You can rotate it using camRgb.setImageOrientation(dai.CameraImageOrientation.AUTO), besides AUTO you can also choose between: AUTO,NORMAL, HORIZONTAL_MIRROR, VERTICAL_FLIP, ROTATE_180_DEG.

Here's an awesome collision detection demo that Noroc created:
This demo was run on OAK-D (so it has spatial coordinates as well) and I believe that stereo cameras -> spatial locations would be beneficial for such an application - for example getting speed/acceleration from differences in spatial locations. I hope this helps!
Thanks, Erik

ramkunchur

Hi erik
Thanks so much for detailed information.

I'll check it out!

Thanks & Best Regards,
Ram

ramkunchur

Hi erik

I tried to convert collision avoidance program to work with video

Following is section of code where I need some help

def run(self):
    try:
        log.info("Setup complete, parsing frames...")
        for frame, results in self.depthai.capture():
            self.parse_frame(frame, results)
    finally:
        del self.depthai

How do I pass video frame for this? this doesn't seem to be typical camera/video read method

For example, I can read video using following method.
cap = cv2.VideoCapture('demosd.mp4')

Not sure how to pass this to above function...I created a function do so as below:

def run_video(self):
    cap = cv2.VideoCapture('demosd.mp4')
    while cap.isOpened():
        results, self.frame = cap.read()
        if not results:
            break

        try:
            self.parse_frame(self.frame, results)
        except StopIteration:
            break
    cap.release()

I get following error, this is from following method

method:
def parse_frame(self, frame, results):
pts = [(item.depth_x, item.depth_z) for item in results]
tracker_objs = self.tracker.update(pts)

error:
pts = [(item.depth_x, item.depth_z) for item in results]
TypeError: 'bool' object is not iterable

Need some help here as it appears the method: "depthai.capture()" defaults to camera and I do not see any code to open camera and read frames where I could possibly replace with frames read from video file...

How do we pass video frame to above method?

Thanks in advance for your time and help...

Thanks & Best Regards,
Ram

erik

Hello ramkunchur ,
I would suggest starting from the mobilenet from video example code.

In the snipped below (from the mobilenet from video example), you read the video frame, create ImgFrame object, set the image information & data and send ImgFrame to the device via XLinkIn.

while cap.isOpened():
        read_correctly, frame = cap.read()
        if not read_correctly:
            break

        img = dai.ImgFrame()
        img.setData(to_planar(frame, (300, 300)))
        img.setTimestamp(monotonic())
        img.setWidth(300)
        img.setHeight(300)
        qIn.send(img)

But you could also use depthai_demo for this use-case and create custom callback.
Thanks, Erik

ramkunchur

Hi erik

Thanks a lot, I'll try to implement this.

Thanks & Best Regards,
Ram

constantijn77

Do you know if it is possible to use this (or other) script in realtime by using a raspberry pi combined with the camera? I hesitate because on my computer it is already real slow. My goal is to build a solution for a boy of 12 years old who has a limited view. When driving on his bicycle to school he sees large cars sometimes to late. This camera should make it possible to give him an alert when there is a large car left or right form him. Fast reaction is needed.

erik

Hello constantijn77 ,
I would say that the majority of the computation (if not all) is actually on the OAK camera itself when you are running live. If you are using prerecorded stream there's additional cpu usage as you have to read frames and send them to the camera.
Thoughts?
Thanks, Erik

constantijn77

This means that it should work? I am not sure that I can use it for the goal I wrote in my first message. The sound that there is a potential dangerous situation should sound before the car is near the boy of 12 year. Perhaps I should try to run the script with less FPS, but I don't know if it can recognize cars in the right way.

Thanks for your reaction.

erik

constantijn77 Yes, it should work. Spatial object detection and tracking can run at 30FPS (mobilenet, tiny yolo), so that should be a problem. The only thing on such small computers (like RPi) is that streaming&visualizing frames will be the bottleneck to the whole pipeline, so the FPS could drop to 5FPS. So I would suggest streaming only metadata (spatial coordinates, tracklets) or low-resolution frames (if needed).
Thanks, Erik

constantijn77

Do you know if I can connect this cameria with USB? In the documentation I see it is , but is it exposed? https://docs.luxonis.com/projects/hardware/en/latest/pages/SJ2088POE.html

erik

Hello constantijn77 ,
Yes, it's possible, see docs here. You would need to open up the enclosure and change the DIP switch so the OAK would boot through the USB.
Thanks, Erik