I would like to extend the rgb_camera_control.py functionality to change the level of lossless zoom at runtime. I don't see anything in ImageManipConfig that would allow me to do this. Is there something I am missing for runtime configuration, or can this be done by tearing down some of the pipline using "unlink" commands and then calling camRgb.setVideoSize() again?

Great question. From my quick testing it seems the best solution would be to wait for ImageManip rewrite (1/2 weeks) so you will be able to manipulate NV12 format frames (video output), and then use ImageManip to crop/resize the image in runtime.
Thanks, Erik

    Thanks erik , I will take a look into that once the ImageManip rewrite is complete (and great to hear that will be so soon!).

    I am trying to perform digital PTZ on the RGB 4k image as well as run MobileNetSpatialDetectionNetwork on the Mono camera. I have something mocked up currently, but for the RGB image, it uses the "preview" output from the color camera at 1080p resolution. That "works", but has a lot of latency on the color image (5-10 fps). Do you think performing ImageManip on the NV12 frames with the upcoming rewrite will improve that latency?

      Hello @atb4c , yes - 1080P preview will have quite a latency. If you stream 1080P camRgb.video instead, you will see that latency will be lower - as there are 1/2 bytes required to transfer same image. And using USB3 5gbps/10gbps will help as well, but your host has to have this kind of USB port.

      Great, thanks. And as I understand it, the OAK-D device will use USB3 5gbps/10gbps by default as long as it is plugged into a compatible port on the host, correct?

      Yes. It will use 5gbps by default if plugged into a 5gpbs-capable port. 5gbps USB3 seems pretty stable. A lot of computers struggle with 10gbps though, and very few cables are actually 10gbps. In fact a TON of USB3C cables, if you look closely say "USB3C charging cable" - which means they're actually 480mbps.

      So as a result, we disable 10gbps by default. But we have an option to enable it specifically in our develop branch. Which will likely be released this month.

      Thanks,
      Brandon

      13 days later

      Is there any update on the ImageManip rewrite? I didn't see any movement HERE, but also saw that there was a 2.11.0.0 library release recently without a lot of documentation. I assume that github ticket is the one I should be following, but let me know if there is a better way to track updates on this feature.

      Will I need to be on a development branch and building from source to test this out, or is this going to be in a "standard" release soon?

      7 days later

      Just checking up on this again. I see that this ticket does not have a 2021 tag associated with it. Has this been pushed to a release date in 2022?

      Martin is working on it now. Update from today:

      Continuing work on ImageManip refactor - hopefully having an equivalent ready soon to reduce the complexity and add support for more functionality.
      Afterwards will finish the libnop&shared refactor and continue on with XLink & BoardConfig

        5 months later

        Brandon Hello Brandon,
        I am having the same issue as @atb4c and would like to resize a 1080p NV12 Image into a 300 by 300 RGB Image.
        I want to capture STILL images at 1080p, and then resize them too 300 by 300 to feed them to a NN.
        My code is as follows:

        import cv2
import depthai as dai

xxx = cv2.imread("blue.png")

# Create the pipeline
pipeline = dai.Pipeline()

# Create Camera node and give its properties
camRGB = pipeline.create(dai.node.ColorCamera)
camRGB.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1080_P)
camRGB.setStillSize(1080, 1080)
camRGB.setPreviewSize(1080, 1080)
camRGB.setVideoSize(1080, 1080)
# camRGB.setInterleaved(False)
camRGB.setColorOrder(dai.ColorCameraProperties.ColorOrder.RGB)


# Create Image manipulation node, to resize image from 1080p to 300by300
manip = pipeline.create(dai.node.ImageManip)
manip.initialConfig.setResize(300,300)
manip.initialConfig.setFrameType(dai.ImgFrame.Type.RGB888p)

# Create output node for still images
outStillRGB = pipeline.create(dai.node.XLinkOut)
outStillRGB.setStreamName("rgbStill")

# Create input control node to acquire capture command
xinCaptureCommand = pipeline.create(dai.node.XLinkIn)
xinCaptureCommand.setStreamName("capture")

# Link output of encoder to input of xlinkout to send to deivce
camRGB.still.link(manip.inputImage)
manip.out.link(outStillRGB.input)

# Link output of xinCaptureCommand to camera input control
xinCaptureCommand.out.link(camRGB.inputControl)

# Connect to device and start the pipeline
with dai.Device(pipeline) as device:
    # Create output queue that will get RGB frame (Output from device, and input to host)
    stillQueue = device.getOutputQueue(name="rgbStill")


    # Create input queue to device, that receives capture command
    captureInputQueue = device.getInputQueue("capture")

    cv2.imshow("xxx",xxx)
    while True:
       
       
       
        stillFrame = stillQueue.tryGet()
        if stillFrame is not None:
            frame = stillFrame.getCvFrame()
            #frame = cv2.imdecode(stillFrame.getData(), cv2.IMREAD_UNCHANGED)
            cv2.imshow("frame", frame)

        # Send capture command from host to device
        key = cv2.waitKey(1)
        if key == ord("q"):
            break
           
        elif key == ord('c'):
            ctrl = dai.CameraControl()
            ctrl.setCaptureStill(True)
            captureInputQueue.send(ctrl)
            print("captured")

        The problems is that the output I get is a grayscale image.
        Any thoughts on how to resolve the issue?

          Hello hussain_allawati , I am not sure if ImageManip supports NV12 frames by default. Could you try moving to image_manip_refactor branch of depthai-python, running python3 examples/install_requirements.py (to get the latest version of depthai), and retrying the same script?
          Thanks, Erik