Hello,

I want to record encoded frames in a binary file. Below is the recording snippet. After recording the raw video, I transform the encoded file to .mp4 using ffmpeg.

When I set the camera's framerate at 10FPS all is ok but when I increase, I lose frames. A solution I thought about is to increase callback queues but I don't know how to do it with the SDK. Can you help me or do oy have any idea why I have problems?

Thank you,

François


# Initialize records file
self.videoFileRgb = open('raw_color.bin', 'wb')
self.videoFileLeft = open('raw_left.bin', 'wb')
self.videoFileRight = open('raw_right.bin', 'wb')
self.records_fd = {'rgb': self.videoFileRgb, 'left': self.videoFileLeft, 'right': self.videoFileRight}

# Intilialize, configure and start camera
self.oak = OakCamera()
color = self.oak.create_camera('color', resolution='800p', fps=10, encode=True)
left = self.oak.create_camera('left', resolution='800p', fps=10, encode=True)
right = self.oak.create_camera('right', resolution='800p', fps=10, encode=True)
self.oak.callback(color.out.encoded, callback=lambda x: self.cb_camera(x, 'rgb'))
self.oak.callback(left.out.encoded, callback=lambda x: self.cb_camera(x, 'left'))
self.oak.callback(right.out.encoded, callback=lambda x: self.cb_camera(x, 'right'))
self.oak.start()

# Define calback function (if recording activated, save frame in the coresponding file)
def cb_camera(self, packet: FramePacket, camera_name):
        if self.is_recording.value == 1:
            fd_camera = self.records_fd[camera_name]
            packet.frame.tofile(fd_camera)

# Main loop where "self.is_recording" can be set to record raw frames
while True:
  self.oak.poll()

    Hi fraymackers
    Could you minimize the code (and post a MRE) of the issue. Try maybe only using the rgb feed to see if it still causes problems.

    Thanks,
    Jaka

        Hi fraymackers ,
        Which OAK camera are you using - is it POE? If that's the case, throughput might not be high enough. I would also check whether you are actually receiving all the frames to the host, perhaps pipeline itself (on the cam) can't keep up and is dropping some frames. Also note that you can also record with the oak.record() - see docs here:
        https://docs.luxonis.com/projects/sdk/en/latest/features/recording/
        Thanks, Erik

            Hi jakaskerl and erik,

            Thank you both for your reply.

            I notice I didn't share with you all the elements of my problem: I'm using the Oak-D Pro W (OV9782 as RGB sensor) with a Raspberry Pi 4 (connected to USB3).

            I tried with only the RGB feed but the same problem: the framerate is respected (tested with filming a chronometer) but I have wrong number of frames (frames seem to be blocked on the camera or something like this). I tested on my computer, there is no problem but on Raspberry Pi I have the problem.

            About the recording on the SDK (oak.record()), I don't use it because I want to start/stop the record without rebuilding the pipeline and I want a personalized recording system.

            Thank you

            • erik replied to this.

                  Hi fraymackers ,

                  I tested on my computer, there is no problem but on Raspberry Pi I have the problem.

                  Wouldn't this mean it's a problem with the Pi - likely its writing speed isn't fast enough? I am not sure how this could be an issue with the OAK/pipeline.