I'm trying to control image transformations in my pipeline. A Camera node's output is linked to a Script node, and the Script node is linked to an ImageManip node. The Script node also has an input queue for "triggers". It listens for triggers and sends inputImage and inputConfig to the ImageManip node, from which I read the output frames. I implement a simple logic within the script node to decide what inputConfig should contain.
The pipeline is running fine until a trigger is received. Then, after handling the trigger, it's freezing. I'm not spamming triggers. The trigger frequency in my application will be very low (5 to 6 per minute).
I created a minimal example to explain the problem I'm facing. This example is built on Camera still image at max resolution example from the docs. In the example, I try to crop the frame when a trigger is received. I hard coded the ImageManip node's config for simplicity.
import cv2
import depthai as dai
import time
with dai.Pipeline() as pipeline:
cam = pipeline.create(dai.node.Camera).build(dai.CameraBoardSocket.CAM_A)
stream_highest_res = cam.requestOutput((5312, 6000), fps=10)
imgManip = pipeline.create(dai.node.ImageManip)
imgManip.setMaxOutputFrameSize(720*720*3)
imgManip.initialConfig.setOutputSize(720, 720)
script = pipeline.create(dai.node.Script)
script.setScript(
"""
import time
while True:
message = node.inputs["in"].get()
trigger = node.inputs["trigger"].tryGet()
if trigger is not None:
node.warn("Trigger received!")
node.outputs["highest_res"].send(message)
time.sleep(0.02)
""")
imgManipCrop = pipeline.create(dai.node.ImageManip)
imgManipCrop.setMaxOutputFrameSize(720*720*3)
imgManipCrop.initialConfig.addCrop(0, 0, 720, 720)
imgManipCrop.inputImage.setBlocking(False)
stream_highest_res.link(imgManip.inputImage)
stream_highest_res.link(script.inputs["in"])
script.outputs["highest_res"].link(imgManipCrop.inputImage)
q_trigger = script.inputs["trigger"].createInputQueue(blocking=False)
downscaled_res_q = imgManip.out.createOutputQueue(blocking=False)
highest_res_q = imgManipCrop.out.createOutputQueue(blocking=False)
pipeline.start()
print("To capture and crop high res image , press 'c'")
capture_count = 1
while pipeline.isRunning():
img_hd = downscaled_res_q.get()
frame = img_hd.getCvFrame()
cv2.imshow("Low Res", frame)
key = cv2.waitKey(1)
if key == ord("q"):
break
if key == ord('c'):
q_trigger.send(dai.Buffer())
time.sleep(0.01)
highres_img = highest_res_q.tryGet()
if highres_img is not None:
cropped_frame = highres_img.getCvFrame()
cv2.imshow("Cropped High Res", cropped_frame)
print("showing frame: ", capture_count)
capture_count += 1
time.sleep(0.01)
When I send a trigger, I see the cropped output the first time. Then, the pipeline freezes. Ideally, I expect the video stream from the Camera node to be active while the pipeline is running. The cropped frame should be displayed as and when a trigger is received.
Please help me with understanding the reason for the freeze and fixing it.
Thank you.
