Hello,
I'm using a device with the following specifications:
Available camera sensors: {<CameraBoardSocket.CAM_B: 1>: 'OV9282', <CameraBoardSocket.CAM_A: 0>: 'AR0234', <CameraBoardSocket.CAM_C: 2>: 'OV9282', <CameraBoardSocket.CAM_D: 3>: 'LCM48'}
Product name: OAK-FFC-4P
Board name: DD2090
Board revision: R7M1E7
I run a simple pipeline on this device to capture color image (using AR0234) and depth map of the scene (using the OV9282 pair and aligning to the AR0234 in CAM_A socket). In very rare cases the AR0234 sensor would not capture certain details, which I need for another application, in the scene. In such cases, I want to use the LCM48 sensor in CAM_D socket to capture a high resolution image of 32MP.
Basically, I'm trying to use the LCM48 sensor in CAM_D socket on demand. I wanted to use a Script node to achieve this, based on the example here. However, I couldn't create a pipeline due to insufficient memory. I'm getting the following error:
RuntimeError: Camera(1) - Out of memory while creating pool for source frames. Number of frames: 3 each with size: 39840000B
I only need a high resolution output a couple of times an hour. So, I don't have to reserve the memory for this node at all times.
Is there a way to run a separate pipeline on the device to capture high resolution image and come back to the regular pipeline once the image is captured? Or can I do any optimizations to hardware utilization? I'm sharing a minimal example of what I'm trying to do.
I appreciate any help or guidance.
Thank you.
import depthai as dai
import cv2
DEFAULT_FPS = 15
COLOR_WIDTH = 1920
COLOR_HEIGHT = 1200
COLOR_OUT_TYPE = dai.ImgFrame.Type.NV12
pipeline = dai.Pipeline()
device = pipeline.getDefaultDevice()
dai_config = device.Config()
dai_config.board.gpio[6] = dai.BoardConfig.GPIO(
dai.BoardConfig.GPIO.OUTPUT,
dai.BoardConfig.GPIO.Level.HIGH
)
colorNode = pipeline.create(dai.node.Camera).build(boardSocket=dai.CameraBoardSocket.CAM_A)
colorOut = colorNode.requestOutput((COLOR_WIDTH, COLOR_HEIGHT), fps=DEFAULT_FPS, type=COLOR_OUT_TYPE)
colorQueue = colorOut.createOutputQueue()
colorSlotD = pipeline.create(dai.node.Camera).build(boardSocket=dai.CameraBoardSocket.CAM_D)
# SlotDOut = colorSlotD.requestFullResolutionOutput(fps=DEFAULT_FPS, type=COLOR_OUT_TYPE, useHighestResolution=False)
SlotDOut = colorSlotD.requestOutput((5312, 6000), fps=2, type=COLOR_OUT_TYPE)
with pipeline:
script = pipeline.create(dai.node.Script)
script.setProcessor(dai.ProcessorType.LEON_CSS)
SlotDOut.link(script.inputs["in"])
script.setScript(
"""
while True:
message = node.inputs["in"].get()
trigger = node.inputs["trigger"].tryGet()
if trigger is not None:
node.io["highest_res"].send(message)
""")
highResQueue = script.outputs["highest_res"].createOutputQueue()
queueTrigger = script.inputs["trigger"].createInputQueue()
pipeline.start()
cv2.namedWindow("color", cv2.WINDOW_NORMAL)
cv2.resizeWindow("color", 640, 400)
cv2.namedWindow("color_cam_d", cv2.WINDOW_NORMAL)
cv2.resizeWindow("color_cam_d", 640, 400)
while pipeline.isRunning():
colorPayload = colorQueue.get()
assert isinstance(colorPayload, dai.ImgFrame)
cv2.imshow("color", colorPayload.getCvFrame())
key = cv2.waitKey(1)
if key == ord('q'):
pipeline.stop()
break
if key == ord('c'):
queueTrigger.send(dai.Buffer())
if highResQueue.has():
highres_img = highResQueue.get()
assert isinstance(highres_img, dai.ImgFrame)
highres_img_frame = highres_img.getCvFrame()
print(f"Read image from Cam_Slot_D with shape: [{highres_img_frame.shape}]")
cv2.imshow("color_cam_d", highres_img_frame)
