Hello ramkunchur, great application! Let me jump straight to the answers:

  1. Yes, you can use video and stream the frames to the device and perform inference on them. We have several example codes: stereo depth from host, mobilenet from video or running depthai_demo with python3.6 depthai_demo.py -cnn vehicle-detection-adas-0002 -vid https://www.youtube.com/watch?v=Y1jTEyb3wiI. If you were using OAK-D (or any other device that has stereo cameras on them), you could use depthai recordings that save mono frames and send mono+color frames back to the device to replay the recording.
  2. You can rotate it using camRgb.setImageOrientation(dai.CameraImageOrientation.AUTO), besides AUTO you can also choose between: AUTO,NORMAL, HORIZONTAL_MIRROR, VERTICAL_FLIP, ROTATE_180_DEG.

Here's an awesome collision detection demo that Noroc created:
This demo was run on OAK-D (so it has spatial coordinates as well) and I believe that stereo cameras -> spatial locations would be beneficial for such an application - for example getting speed/acceleration from differences in spatial locations. I hope this helps!
Thanks, Erik

      Hi erik
      Thanks so much for detailed information.

      I'll check it out!

      Thanks & Best Regards,
      Ram

        Hi erik

        I tried to convert collision avoidance program to work with video

        Following is section of code where I need some help

        def run(self):
    try:
        log.info("Setup complete, parsing frames...")
        for frame, results in self.depthai.capture():
            self.parse_frame(frame, results)
    finally:
        del self.depthai

        How do I pass video frame for this? this doesn't seem to be typical camera/video read method

        For example, I can read video using following method.
        cap = cv2.VideoCapture('demosd.mp4')

        Not sure how to pass this to above function...I created a function do so as below:

        def run_video(self):
    cap = cv2.VideoCapture('demosd.mp4')
    while cap.isOpened():
        results, self.frame = cap.read()
        if not results:
            break

        try:
            self.parse_frame(self.frame, results)
        except StopIteration:
            break
    cap.release()

        I get following error, this is from following method

        method:
        def parse_frame(self, frame, results):
        pts = [(item.depth_x, item.depth_z) for item in results]
        tracker_objs = self.tracker.update(pts)

        error:
        pts = [(item.depth_x, item.depth_z) for item in results]
        TypeError: 'bool' object is not iterable

        Need some help here as it appears the method: "depthai.capture()" defaults to camera and I do not see any code to open camera and read frames where I could possibly replace with frames read from video file...

        How do we pass video frame to above method?

        Thanks in advance for your time and help...

        Thanks & Best Regards,
        Ram

        • erik replied to this.

            Hello ramkunchur ,
            I would suggest starting from the mobilenet from video example code.

            In the snipped below (from the mobilenet from video example), you read the video frame, create ImgFrame object, set the image information & data and send ImgFrame to the device via XLinkIn.

            while cap.isOpened():
        read_correctly, frame = cap.read()
        if not read_correctly:
            break

        img = dai.ImgFrame()
        img.setData(to_planar(frame, (300, 300)))
        img.setTimestamp(monotonic())
        img.setWidth(300)
        img.setHeight(300)
        qIn.send(img)

            But you could also use depthai_demo for this use-case and create custom callback.
            Thanks, Erik

                8 months later

                Do you know if it is possible to use this (or other) script in realtime by using a raspberry pi combined with the camera? I hesitate because on my computer it is already real slow. My goal is to build a solution for a boy of 12 years old who has a limited view. When driving on his bicycle to school he sees large cars sometimes to late. This camera should make it possible to give him an alert when there is a large car left or right form him. Fast reaction is needed.

                • erik replied to this.

                  Hello constantijn77 ,
                  I would say that the majority of the computation (if not all) is actually on the OAK camera itself when you are running live. If you are using prerecorded stream there's additional cpu usage as you have to read frames and send them to the camera.
                  Thoughts?
                  Thanks, Erik

                    This means that it should work? I am not sure that I can use it for the goal I wrote in my first message. The sound that there is a potential dangerous situation should sound before the car is near the boy of 12 year. Perhaps I should try to run the script with less FPS, but I don't know if it can recognize cars in the right way.

                    Thanks for your reaction.

                    • erik replied to this.

                      constantijn77 Yes, it should work. Spatial object detection and tracking can run at 30FPS (mobilenet, tiny yolo), so that should be a problem. The only thing on such small computers (like RPi) is that streaming&visualizing frames will be the bottleneck to the whole pipeline, so the FPS could drop to 5FPS. So I would suggest streaming only metadata (spatial coordinates, tracklets) or low-resolution frames (if needed).
                      Thanks, Erik

                        6 days later

                        Hello constantijn77 ,
                        Yes, it's possible, see docs here. You would need to open up the enclosure and change the DIP switch so the OAK would boot through the USB.
                        Thanks, Erik