Record video output with annotations

barney2074

Hello

Is there a way to directly record video from DepthAI, with annotations ??
The -encout option in the demo records a particular camera stream (say color)- but without bounding boxes, labels, framerate etc

Or do I just use a screen capture utility ??

thanks

Andrew

erik

Hello barney2074 ,
We don't have an official solution for this yet, but we plan to make one where it would record video stream + detections (+ optionally IMU data) into ROS bag. It would be quite straighforward to create an app that would save all detections in eg. excel sheet, along with frame sequence number so detections would later be matched with the correct frame. Thoughts?
Thanks, Erik

LXNDR

erik Hi, Eric! I see that there is a new Record class that records videos in mp4. Have you managed to implement object detection trackers whilst recording say mp4 ?

barney2074

Hi Erik

thanks- I'll use a screen-cap utility for now

The export of the data sounds great- if you get a chance to look at this soon, that would be great

My personal preference would be rather than Excel something like a SQLite database as it is a simple, standalone file, but works well with concurrent access and large amounts of data- but I think an Excel or CSV version would be easily modified for this, or any other database.

Depending on the NN used- you sometimes get a number of brief false positives i.e something flashes up for 0.1 seconds, then the NN decides against it. I don't know whether it would be best to write out absolutely everything to Excel/DB and filter it there, or have some sort of threshold in DepthAI i.e only record detections >1 second ??

thanks,

Andrew

erik

Hello barney2074 ,
For more production environments I would agree with SQL (like sqlite). Regarding false positives, you could either filter them on host (python) or maybe increase threshold for detections (at the Detection Network node).
Thanks, Erik

barney2074

thanks Erik

I adapted one of the examples to write to an Influx (times series) database, without too much trouble.
And I'm building a real-time dashboard of object detections on top of that.
So far, it is working great

Andrew

erik

barney2074 Awesome that you got it working! Do you plan to opensource the project🙂?
Thanks, Erik

erik

Hi LXNDR ,
Yes, it does🙂

from depthai_sdk import OakCamera
with OakCamera() as oak:
    color = oak.create_camera('color')
    # List of models that are supported out-of-the-box by the SDK:
    # https://docs.luxonis.com/projects/sdk/en/latest/features/ai_models/#sdk-supported-models
    nn = oak.create_nn('yolov7tiny_coco_640x352', color)
    oak.visualize(nn.out.main, fps=True, record_path='recording.mp4') # Record frames that have obj. detections visualized
    oak.visualize(nn.out.passthrough)
    oak.start(blocking=True)

Terminal output:

> python .\nn_component.py
[2023-05-06 18:04:04] INFO [root.__exit__:292] Closing OAK camera
[2023-05-06 18:04:05] INFO [root.close:67] Video Recorder saved stream(s) to folder: recording.mp4

And recording.mp4:

recording.mp4

4MB

Hutch07

How can I view or decode an mp4 video? I have the camera recording OK but have not found anything that can open it.

from depthai_sdk import OakCamera, RecordType

from multiprocessing.sharedctypes import Value

import depthai as dai

import contextlib

import math

import time, datetime

from pathlib import Path

import signal

import threading

import csv

out_path = r"E:\Junk\vid"

OUTPUT_DIR = Path(r"F:\CAMERAS")

Output_csv = Path.joinpath(OUTPUT_DIR, "CSV")

print("Output", Output_csv)

def write_times(outime, big_array):

print("Writing Timestamp CSV.")

outname = outime.replace(":", "_") + ".csv"

outfile = Path.joinpath(Output_csv, outname)

header = ["Device", "Cam", "Timestamp"]

with open(outfile,"w", newline='') as csvfile:

spamwriter = csv.writer(csvfile, delimiter=',')

spamwriter.writerow(header)

for rowi in big_array:

spamwriter.writerow(rowi)

return True

with OakCamera() as oak:

left = oak.create_camera('left', resolution='800p', fps=20, encode='H265')

right = oak.create_camera('right', resolution='800p', fps=20, encode='H265')

# Synchronize & save all (encoded) streams

oak.record([ left.out.encoded, right.out.encoded], out_path, RecordType.VIDEO)

# Show left stream

oak.visualize([left.out.camera], scale=2/3, fps=True)

oak.start()

left.control.set_exposure_iso(1500, 600)

while oak.running():

key = oak.poll()

if key == ord('i'):

left.control.exposure_time_down()

elif key == ord('o'):

left.control.exposure_time_up()

elif key == ord('k'):

left.control.sensitivity_down()

elif key == ord('l'):

left.control.sensitivity_up()

elif key == ord('e'): # Switch to auto exposure

left.control.send_controls({'exposure': {'auto': True}}

[upl-image-preview url=https://discuss.luxonis.com/assets/files/2023-09-09/1694286258-701158-image.png]

jakaskerl

Hi Hutch07
You have the mp4 video file? What do you mean you cannot open it? Does it say the data is corrupted or are you getting some other error?

Thanks,
Jaka

Hutch07

jakaskerl I was not closing the cameras so the encoded files were missing information.

Thanks.