how to delay (buffer) a frame in a pipeline

ynjiun

I would like to use a node to process frame(n) and frame(n-1) in the pipeline in real time.

How do I buffer a frame in the pipeline to sync up with next frame to achieve the above goal?

Thank you for your help.

erik

Hi ynjiun ,
You can use Script node to buffer frames, just note that number of buffered frames doesn't exceed pool size (docs here), otherwise the pipeline will freeze (as node won't be able to create new messages as pool will be "full".
Thanks, Erik

ynjiun

Hi @erik

Thank you for the information.

The following is the code I plan to test, please take a look if there is an obvious blunder.

    script = pipeline.create(dai.node.Script)
    script.setScript("""
        imgs = [ImgFrame(400*640),ImgFrame(400*640)]
        i=0
        while True:
            imgs[i]=node.io['in'].get()
            o = (i+1)%2
            node.io['out'].send(imgs[o])
            i = o
    """)

As you can see, I tried to create 2 img buffers (imgs) as a cyclic buffer. So the while loop just to get the current img and send the previous img.

The input of this script node may be from, says, a monoRight node as below:

`monoRight = pipeline.create(dai.node.MonoCamera)`

Do I need any clocking information to sync up the frame? or the node.io[].get() will automatically sync up the frame from the source?

In another words, would the following connection give me frame(n) and frame(n-1) to the sink node at the time n?

monoRight.out.link(sink.input1)
script.outputs['out'].link(sink.input2)

Please advise. Thanks.

erik

Hi ynjiun
Yes, I think this should work. If it doesn't work as expected, please share the shortened script (as short as possible) and I will take a look at it.
Thanks, Erik

ynjiun

Hi @erik

Thank you for your help.

Indeed the code is working!

However, I would like to shift the buffer by 48 pixels before sending it out as below:

        script = pipeline.create(dai.node.Script)
        script.setScript("""
            width,height=640,400
            imgs = [ImgFrame(height*width),ImgFrame(height*width)]
            for i in range(2):
                imgs[i].setWidth(width)
                imgs[i].setHeight(height)
            i=0
            while True:
                imgs[i]=node.io['in'].get()
                imgs[i][:,0:width-48] = imgs[i][:,48:width]
                o = (i+1)%2
                node.io['out'].send(imgs[o])
                i = o
        """)
        monoRight.out.link(script.inputs['in'])

Then I encounter the following error message:

[184430106183EB0F00] [1.11] [1.487] [Script(3)] [critical] TypeError: 'lpb.ImgFrame' object is not subscriptable

I did try use imgs.getFrame() but fail too.

Please advise how to get ImgFrame in 2D shape and do frame[:,0:width-48]=frame[:,48:width] shift and then save it back into imgs again.

Thank you very much for your help.

erik

Hi ynjiun .
Why would you want to do that? This is not numpy frame and you can't use it as such - for such cropping you should use ImageManip node. Thoughts?
Thanks, Erik

ynjiun

erik for such cropping you should use ImageManip node. Thoughts?

Nice! Thanks a lot for your help!

After tested it, I can crop it with manip but the resulting 592x400 imgFrame cannot be accepted by stereoDepth and generate the following error:

[184430106183EB0F00] [1.11] [2.314] [system] [critical] Fatal error. Please report to developers. Log: 'Fatal error on MSS CPU: trap: 2A, address: 8009F440' '0'

Any thoughts, Thanks.