Yeah, "lag" maybe is not the proper term here.
So the idea of the code is that we use an ImageManip node to crop the incoming video frame and when 'p' is pressed, we change the crop parameters and start saving the frames. When I do this, the first 10-20 frames saved are with the initial image manip configuration and after that many frames I start to get crops with the new ImageManip configuration. Is this normal behaviour? Is there a way to get this as low as 1 frame latency?
- Windows 11
- OAK-D PRO W POE
- POE connection (1G cable and switch)
- Depthai 2.25
Tried at 15,30,60 fps same results, an offset of 10 (or more) frames before I get the new configuration cropped frames
import cv2
import depthai as dai
import random
import os
available_devices = dai.Device.getAllAvailableDevices()
p = dai.Pipeline()
p.setOpenVINOVersion(dai.OpenVINO.VERSION_2022_1)
camera = p.create(dai.node.ColorCamera)
camera.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1080_P)
camera.setPreviewSize(320, 180)
camera.setInterleaved(False)
camera.setColorOrder(dai.ColorCameraProperties.ColorOrder.BGR)
camera.setFps(60.0)
image_manip = p.create(dai.node.ImageManip)
image_manip.initialConfig.setResize(256,256)
image_manip.inputConfig.setWaitForMessage(False)
image_manip.inputConfig.setQueueSize(1)
image_manip.inputImage.setQueueSize(1)
image_manip.inputConfig.setBlocking(True)
video_encoder = p.create(dai.node.VideoEncoder)
video_encoder.setProfile(dai.node.VideoEncoder.Properties.Profile.MJPEG)
video_encoder_crop = p.create(dai.node.VideoEncoder)
video_encoder_crop.setProfile(dai.node.VideoEncoder.Properties.Profile.MJPEG)
camera_xout = p.create(dai.node.XLinkOut)
camera_xout.setStreamName("rgb_preview")
camera_xout_crop = p.create(dai.node.XLinkOut)
camera_xout_crop.setStreamName("rgb_crop")
camera.video.link(image_manip.inputImage)
image_manip.out.link(video_encoder_crop.input)
params = p.create(dai.node.XLinkIn)
params.setStreamName("params")
params.out.link(image_manip.inputConfig)
video_encoder_crop.bitstream.link(camera_xout_crop.input)
camera.preview.link(camera_xout.input)
device = dai.Device(p)
device.setLogLevel(dai.LogLevel.DEBUG)
device.setLogOutputLevel(dai.LogLevel.DEBUG)
sz ,blck = 10, False
camera_stream = device.getOutputQueue("rgb_preview", maxSize=sz, blocking=blck)
camera_stream_cropped = device.getOutputQueue("rgb_crop", maxSize=sz, blocking=blck)
params_q = device.getInputQueue("params", maxSize=sz, blocking=blck)
path = r"test"
os.makedirs(path, exist_ok=True)
counter = 0
press = False
while True:
if press:
frame_cropped = camera_stream_cropped.get().getRaw().data
image = cv2.imdecode(frame_cropped, cv2.IMREAD_COLOR)
cv2.imwrite(f"{path}\\{counter}.png", image)
counter += 1
if counter > 100:
break
frame = camera_stream.get().getCvFrame()
cv2.imshow(f"rgb", frame)
key = cv2.waitKey(1)
if key == ord('q'):
break
elif key == ord('p'):
press = True
config = dai.ImageManipConfig()
rr = dai.RotatedRect()
rr.center = dai.Point2f(random.randint(0, 1920), random.randint(0, 1080))
rr.size = dai.Size2f(800, 600)
rr.angle = (random.random() * 2 - 1) * 10
config.setCropRotatedRect(rr, False)
config.setResize(256,256)
config.setKeepAspectRatio(False)
params_q.send(config)
[19443010C1D17E1300] [192.168.1.27] [5.298] [VideoEncoder(3)] [debug] Bitstream frame size: 108544, num frames in pool: 4
[19443010C1D17E1300] [192.168.1.27] [6.065] [system] [info] Memory Usage - DDR: 73.51 / 333.39 MiB, CMX: 2.49 / 2.50 MiB, LeonOS Heap: 40.53 / 82.31 MiB, LeonRT Heap: 4.04 / 40.50 MiB / NOC ddr: 607 MB/s
[19443010C1D17E1300] [192.168.1.27] [6.066] [system] [info] Temperatures - Average: 50.08C, CSS: 52.02C, MSS 48.25C, UPA: 49.37C, DSS: 50.70C
[19443010C1D17E1300] [192.168.1.27] [6.066] [system] [info] Cpu Usage - LeonOS 88.88%, LeonRT: 17.02%
[19443010C1D17E1300] [192.168.1.27] [7.069] [system] [info] Memory Usage - DDR: 73.51 / 333.39 MiB, CMX: 2.49 / 2.50 MiB, LeonOS Heap: 40.78 / 82.31 MiB, LeonRT Heap: 4.04 / 40.50 MiB / NOC ddr: 2050 MB/s
[19443010C1D17E1300] [192.168.1.27] [7.070] [system] [info] Temperatures - Average: 50.47C, CSS: 52.02C, MSS 49.37C, UPA: 49.37C, DSS: 51.14C
[19443010C1D17E1300] [192.168.1.27] [7.070] [system] [info] Cpu Usage - LeonOS 99.48%, LeonRT: 13.62%
[19443010C1D17E1300] [192.168.1.27] [8.073] [system] [info] Memory Usage - DDR: 73.51 / 333.39 MiB, CMX: 2.49 / 2.50 MiB, LeonOS Heap: 40.78 / 82.31 MiB, LeonRT Heap: 4.04 / 40.50 MiB / NOC ddr: 2023 MB/s
[19443010C1D17E1300] [192.168.1.27] [8.074] [system] [info] Temperatures - Average: 50.36C, CSS: 52.24C, MSS 49.81C, UPA: 49.37C, DSS: 50.03C
[19443010C1D17E1300] [192.168.1.27] [8.074] [system] [info] Cpu Usage - LeonOS 99.89%, LeonRT: 13.15%
[19443010C1D17E1300] [192.168.1.27] [9.077] [system] [info] Memory Usage - DDR: 73.51 / 333.39 MiB, CMX: 2.49 / 2.50 MiB, LeonOS Heap: 40.78 / 82.31 MiB, LeonRT Heap: 4.04 / 40.50 MiB / NOC ddr: 2072 MB/s
[19443010C1D17E1300] [192.168.1.27] [9.078] [system] [info] Temperatures - Average: 50.53C, CSS: 52.46C, MSS 49.37C, UPA: 50.03C, DSS: 50.25C
[19443010C1D17E1300] [192.168.1.27] [9.078] [system] [info] Cpu Usage - LeonOS 99.49%, LeonRT: 14.60%
[19443010C1D17E1300] [192.168.1.27] [10.081] [system] [info] Memory Usage - DDR: 73.51 / 333.39 MiB, CMX: 2.49 / 2.50 MiB, LeonOS Heap: 40.78 / 82.31 MiB, LeonRT Heap: 4.04 / 40.50 MiB / NOC ddr: 2065 MB/s
[19443010C1D17E1300] [192.168.1.27] [10.081] [system] [info] Temperatures - Average: 50.80C, CSS: 52.90C, MSS 49.59C, UPA: 50.25C, DSS: 50.47C
[19443010C1D17E1300] [192.168.1.27] [10.081] [system] [info] Cpu Usage - LeonOS 99.45%, LeonRT: 13.62%
[19443010C1D17E1300] [192.168.1.27] [11.082] [system] [info] Memory Usage - DDR: 73.51 / 333.39 MiB, CMX: 2.49 / 2.50 MiB, LeonOS Heap: 40.78 / 82.31 MiB, LeonRT Heap: 4.04 / 40.50 MiB / NOC ddr: 2042 MB/s
[19443010C1D17E1300] [192.168.1.27] [11.082] [system] [info] Temperatures - Average: 50.58C, CSS: 52.24C, MSS 49.59C, UPA: 49.59C, DSS: 50.92C
[19443010C1D17E1300] [192.168.1.27] [11.082] [system] [info] Cpu Usage - LeonOS 99.47%, LeonRT: 15.38%
[19443010C1D17E1300] [192.168.1.27] [12.089] [system] [info] Memory Usage - DDR: 73.51 / 333.39 MiB, CMX: 2.49 / 2.50 MiB, LeonOS Heap: 40.78 / 82.31 MiB, LeonRT Heap: 4.04 / 40.50 MiB / NOC ddr: 2089 MB/s
[19443010C1D17E1300] [192.168.1.27] [12.089] [system] [info] Temperatures - Average: 51.30C, CSS: 52.90C, MSS 50.25C, UPA: 50.47C, DSS: 51.58C
[19443010C1D17E1300] [192.168.1.27] [12.089] [system] [info] Cpu Usage - LeonOS 99.26%, LeonRT: 14.87%
[19443010C1D17E1300] [192.168.1.27] [13.092] [system] [info] Memory Usage - DDR: 73.51 / 333.39 MiB, CMX: 2.49 / 2.50 MiB, LeonOS Heap: 40.78 / 82.31 MiB, LeonRT Heap: 4.04 / 40.50 MiB / NOC ddr: 2106 MB/s
[19443010C1D17E1300] [192.168.1.27] [13.093] [system] [info] Temperatures - Average: 51.08C, CSS: 52.68C, MSS 50.25C, UPA: 49.81C, DSS: 51.58C
[19443010C1D17E1300] [192.168.1.27] [13.093] [system] [info] Cpu Usage - LeonOS 99.61%, LeonRT: 13.96%
[19443010C1D17E1300] [192.168.1.27] [14.096] [system] [info] Memory Usage - DDR: 73.51 / 333.39 MiB, CMX: 2.49 / 2.50 MiB, LeonOS Heap: 40.78 / 82.31 MiB, LeonRT Heap: 4.04 / 40.50 MiB / NOC ddr: 2077 MB/s
[19443010C1D17E1300] [192.168.1.27] [14.097] [system] [info] Temperatures - Average: 51.25C, CSS: 52.68C, MSS 49.81C, UPA: 50.92C, DSS: 51.58C
[19443010C1D17E1300] [192.168.1.27] [14.097] [system] [info] Cpu Usage - LeonOS 99.69%, LeonRT: 14.18%
[19443010C1D17E1300] [192.168.1.27] [15.100] [system] [info] Memory Usage - DDR: 73.51 / 333.39 MiB, CMX: 2.49 / 2.50 MiB, LeonOS Heap: 40.78 / 82.31 MiB, LeonRT Heap: 4.04 / 40.50 MiB / NOC ddr: 2035 MB/s
[19443010C1D17E1300] [192.168.1.27] [15.101] [system] [info] Temperatures - Average: 51.08C, CSS: 52.90C, MSS 49.59C, UPA: 50.47C, DSS: 51.36C
[19443010C1D17E1300] [192.168.1.27] [15.101] [system] [info] Cpu Usage - LeonOS 99.38%, LeonRT: 13.94%
[19443010C1D17E1300] [192.168.1.27] [16.102] [system] [info] Memory Usage - DDR: 73.51 / 333.39 MiB, CMX: 2.49 / 2.50 MiB, LeonOS Heap: 40.78 / 82.31 MiB, LeonRT Heap: 4.04 / 40.50 MiB / NOC ddr: 2072 MB/s
[19443010C1D17E1300] [192.168.1.27] [16.103] [system] [info] Temperatures - Average: 51.19C, CSS: 52.68C, MSS 50.25C, UPA: 50.25C, DSS: 51.58C
[19443010C1D17E1300] [192.168.1.27] [16.103] [system] [info] Cpu Usage - LeonOS 99.74%, LeonRT: 15.25%
[19443010C1D17E1300] [192.168.1.27] [17.107] [system] [info] Memory Usage - DDR: 73.51 / 333.39 MiB, CMX: 2.49 / 2.50 MiB, LeonOS Heap: 40.78 / 82.31 MiB, LeonRT Heap: 4.04 / 40.50 MiB / NOC ddr: 1944 MB/s
[19443010C1D17E1300] [192.168.1.27] [17.107] [system] [info] Temperatures - Average: 51.25C, CSS: 52.46C, MSS 50.25C, UPA: 50.47C, DSS: 51.80C
[19443010C1D17E1300] [192.168.1.27] [17.107] [system] [info] Cpu Usage - LeonOS 99.81%, LeonRT: 16.81%
[19443010C1D17E1300] [192.168.1.27] [18.111] [system] [info] Memory Usage - DDR: 73.51 / 333.39 MiB, CMX: 2.49 / 2.50 MiB, LeonOS Heap: 40.78 / 82.31 MiB, LeonRT Heap: 4.04 / 40.50 MiB / NOC ddr: 1923 MB/s
[19443010C1D17E1300] [192.168.1.27] [18.111] [system] [info] Temperatures - Average: 51.36C, CSS: 52.68C, MSS 50.03C, UPA: 50.70C, DSS: 52.02C
[19443010C1D17E1300] [192.168.1.27] [18.111] [system] [info] Cpu Usage - LeonOS 99.54%, LeonRT: 16.52%