Get frames from Oak 1 Max

JaimilDalwadi · 2023-06-22T15:22:28+00:00

Just looking for some documentation or examples on how to save frames coming in from an oak at 60 fps (max) into a folder? I tried doing this with cv2.imwri...

JaimilDalwadi

If anyone can help,

I managed to save frames at 30 FPS but I think it's capping there because of this warning:

[184430108104DB0E00] [1.1] [0.801] [ColorCamera(0)] [warning] LCM48 supported resolutions: 4_K / 4000X3000 / 5312X6000. Defaulting to 4_K

I have it set to:

camRgb.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1080_P)

camRgb.setVideoSize(1920, 1080)

But I'm guessing it defaults to 4k. Can anyone help? This is on the Oak1 Max

jakaskerl

Hi JaimilDalwadi
The lowest sensor resolution for Max is indeed 4K. You can try downscaling it using ISP.
This capping you are referring to, does is come from camera being unable to process images fast enough or the ffmpeg?

Thanks,
Jaka

JaimilDalwadi

jakaskerl

This time I'm not using ffmpeg, I'm simply saving them to a dictionary and then saving them to files at the end like in the below script, so I'm not sure if it's being capped anywhere:

#!/usr/bin/env python3

# Script to save videos in .h265 format from oak camera

import depthai as dai
import time
import os
import cv2
# Create pipeline
pipeline = dai.Pipeline()

# Define sources and output
camRgb = pipeline.create(dai.node.ColorCamera)
ctrl = dai.CameraControl()
xout = pipeline.create(dai.node.XLinkOut)
xoutStill = pipeline.create(dai.node.XLinkOut)


xout.setStreamName('h265')
xoutStill.setStreamName('preview')

# Properties
camRgb.setBoardSocket(dai.CameraBoardSocket.RGB)
fps = 60
camRgb.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1080_P)
camRgb.setPreviewSize(360,360)
# camRgb.setIspScale(1,2)

# camRgb.setVideoSize(1920, 1080)
stillEnc.setDefaultProfilePreset(fps, dai.VideoEncoderProperties.Profile.H265_MAIN)




# Linking

camRgb.preview.link(xoutStill.input)

with dai.Device(pipeline) as device:
    device.setLogLevel(dai.LogLevel.DEBUG)
    device.setLogOutputLevel(dai.LogLevel.DEBUG)
    # Output queue will be used to get the encoded data from the output defined above
    qPreview = device.getOutputQueue(name="preview", maxSize=60,blocking=False)
    curr = int(round(time.time()))
    rawFile = rf'.\rawStreams\{curr}.h265'
    convertedFile = rf'.\convertedStreams\{curr}.mp4'
    temp = dict()
    fc = 0
    starttime = time.time()
    totaltime = 0
    with open(rawFile, 'wb') as videoFile:
        print("Press Ctrl+C to stop encoding...")
        
        try:
            while True:
                fc += 1
                frame = qPreview.get()
                frame_name = f"frame_{int(time.time()*1000)}.jpeg"
                temp[frame_name] = frame
                # cv2.imshow("rgb",frame)


        except KeyboardInterrupt:
            # Keyboard interrupt (Ctrl + C) detected
            totaltime = time.time() - starttime
            pass
    print(f"Total frame count: {fc}")
    print(f"Total time run: {totaltime}")
    print(f"FPS: {fc/totaltime}")
    output_folder = rf".\testPhotos"
    for filename, f in temp.items():
        pathout = f"../testPhotos/{filename}"
        f = f.getCvFrame()
        cv2.imwrite(pathout,f)

jakaskerl

Hi JaimilDalwadi
And the number of frames you are getting?
Since you have set the queue size to 60, when ending the stream with ctrl+c, I think there might still be some frames left in that queue that haven't yet been read, so you get lower number of frames than you should be getting.

Thoughts?
Jaka

JaimilDalwadi

jakaskerl

I was getting around 29 fps. I'm not sure if its the queue size, I tested with the ISP downscaling to (1,2) but it still only gives about the same. Do you think it could be a camera issue? Or something with the configuration such that it keeps using 4k resolution or limits it to 30 fps instead for some reason?

jakaskerl

Hi JaimilDalwadi
I don't know for sure. You can try running your script with DEPTHAI_LEVEL=TRACE python3 <file> to see how much time each frame acquisition took.

Thanks,
Jaka

JaimilDalwadi

jakaskerl

Thank you for all the help! I was able to save frames for x number of seconds on command.

I am facing issues though as it starts to not get any frames. I basically wait on the keyboard press for "c" and set some flags and run the following code:
`

if getImgs and time.time()-starttime <= 1:

frame = qPreview.tryGet()

if frame is not None:

fc += 1

frame_name = f"frame_{int(time.time()*1000)}.jpeg"

temp[frame_name] = frame

`
When I press c the first time, it works and gets around 40 fps, but when I press c the second time after everything has saved, it only gets around 10, and then anytime I press c again, it just doesn't get anything. Could it be anything to do with the video pipeline? Is it possible the queue gets empty for some reason or the camera doesn't get any more frames for some reason?

These are the pipelines:

xoutPrev = pipeline.create(dai.node.XLinkOut)

xoutPrev.setStreamName('preview')

camRgb.video.link(xoutPrev.input)
qPreview = device.getOutputQueue(name="preview", maxSize=10,blocking=False)

jakaskerl

Hi JaimilDalwadi
Can you add the code as well so I can test locally? And make it as minimal as you can.

Thanks,
Jaka

JaimilDalwadi

jakaskerl

I somehow got it to work with the MRE, maybe there were some issues in my original script. Are there any improvements I can make to this MRE code:

#!/usr/bin/env python3
import depthai as dai
import time
import os
import cv2
import keyboard
# Create pipeline
pipeline = dai.Pipeline()

# Define sources and output
camRgb = pipeline.create(dai.node.ColorCamera)

xoutPrev = pipeline.create(dai.node.XLinkOut)
xoutPrev.setStreamName('preview')

camRgb.setBoardSocket(dai.CameraBoardSocket.RGB)
fps = 30
camRgb.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1080_P)


camRgb.video.link(xoutPrev.input)

with dai.Device(pipeline) as device:

    qPreview = device.getOutputQueue(name="preview", maxSize=10,blocking=False)
    # The .h265 file is a raw stream file (not playable yet)
    temp = dict()
    fc = 0
    starttime = time.time()
    totaltime = 0
    getImgs = False
    try:
        while True:
            if getImgs and time.time()-starttime <= 1:
                frame = qPreview.tryGet()
                if frame is not None:
                    fc += 1
                    frame_name = f"{int(time.time()*1000)}.jpeg"
                    temp[frame_name] = frame
            elif getImgs:
                totaltime = time.time() - starttime
                print(f"Total frame count: {fc}")
                print(f"Total time run: {totaltime}")
                print(f"FPS: {fc/totaltime}")
                output_folder = rf".\testPhotos"
                for filename, f in temp.items():
                    pathout = f"../testPhotos/{filename}"
                    f = f.getCvFrame()
                    cv2.imwrite(pathout,f)
                getImgs = False
                temp.clear()
                fc = 0
                totaltime = 0
            
            if not getImgs:
                keyboard.wait("c")
                getImgs = True
                starttime = time.time()


    except KeyboardInterrupt:
        pass

jakaskerl

Hi JaimilDalwadi
Great! So I assume the code works as expected? The point of a MRE is to find the minimal amount of code you need to reproduce the issue you are facing. Very often the issue goes away on it's own when shrinking down the code. I am not sure what the cause for the above error is, but since the script is now working I feel there is no need to dig into what is possibly just a coding mistake.

Thanks,
Jaka

JaimilDalwadi

jakaskerl

Yessir, thank you so much for the help! It works as required for now, if anything breaks I know who to ask 🙂

« Previous Page