setPreviewSize() on zoomed or cropped image

@jakaskerl

Below is my current implementation. I feel like I'm missing some kind of "apply" API as I see no effect when the cropIn and cropOut calls are made. Do you see what I'm doing wrong?

This is my pipeline setup.

// 1. Create pipeline
dai::Pipeline corePipeline;

// 2. Define sources and outputs
colorCam = corePipeline.create<dai::node::ColorCamera>();
auto configIn = corePipeline.create<dai::node::XLinkIn>();
auto xLinkPreviewOut = corePipeline.create<dai::node::XLinkOut>();

// 3. Properties
colorCam->setPreviewSize(previewSize.width, previewSize.height);
colorCam->setResolution(dai::ColorCameraProperties::SensorResolution::THE_720_P);

configIn->setStreamName(STREAM_CONFIG_IN);

xLinkPreviewOut->setStreamName(STREAM_PREVIEW);

// 4. Linking
colorCam->preview.link(xLinkPreviewOut->input);
configIn->out.link(colorCam->inputConfig);

// 5. Connecting
device = std::shared_ptr<dai::Device>(new dai::Device(corePipeline, deviceInfo));

// 6. Queries
rgbQueue = device->getOutputQueue(STREAM_PREVIEW, 4, false);
configQueue = device->getInputQueue(STREAM_CONFIG_IN, 4, false);

The later calls to cropIn and cropOut are made, but I see no effect in the resulting preview image. I've confirmed the methods are executing and not erroring.

void CropIn()
{
    dai::ImageManipConfig cfg;
    cfg.setCenterCrop(.5);
    configQueue->send(cfg);
    std::cout << "crop in" << std::endl;
}

void CropOut()
{
    dai::ImageManipConfig cfg;
    cfg.setCenterCrop(1);
    configQueue->send(cfg);
    std::cout << "crop out" << std::endl;
}

jakaskerl Thank you. I was under the impression the ImageManipConfig accounted for the manipulation but now I see that's just part of the puzzle as it needs to be fed to the manip node.

I still need some clarity. Is the crop happening to the downsized preview image or is the crop happening to the original image before it's downsized to the preview image? Maybe it makes no difference from DepthAI's perspective (functionally), but from my API-user perspective it's unclear.

From what I can tell, it's the former (and this isn't what I want) as now the preview image is smaller than the desired colorCam->setPreviewSize(previewSize.width, previewSize.height); call. Maybe this is what you meant when you said "along with setResize()" in your initial comment? If true, the resize would now be stretching up and losing quality correct?

@jakaskerl My current understanding is:

1. The crop occurs to the preview image after it's been downsized from the original (which is set via setResolution)
2. A setResize call is required to get back to the original setPreviewSize
3. This results in a lossy image due to the upscale

Is this correct? If not, why?

If so, how do I achieve a crop of the original image (at the dimensions set via setResolution) before it's downsized to the preview resolution (set via setPreviewSize)?

@jakaskerl Regarding the above setResize followed by setCenterCrop calls, if I change setCenterCrop to setCropRect I get a crash. Through some digging, it looks like I need two ImageManipNodes (at least so far I don't get the crash with this approach).

1. When and when do you not need multiple ImageManipNodes? This isn't clear unless I'm missing something in the docs.

2. Assuming I do in fact need two ImageManipNodes, I'm able to get the initialConfig to work via the below, but then when I want to update the manip nodes based on user input it doesn't work as expected. Do you see anything that I'm missing or doing wrong?

      manipResize->initialConfig.setResize(resizeDimensionDefault, resizeDimensionDefault);
      manipResize->setMaxOutputFrameSize(colorCam->getResolutionHeight() * colorCam->getResolutionWidth() * 3);
      
      manipCrop->initialConfig.setCropRect(GetCropCoordinates());
      manipCrop->setMaxOutputFrameSize(colorCam->getResolutionHeight() * colorCam->getResolutionWidth() * 3);
   

   Here is the post initalization related code that executes on user input:

      dai::ImageManipConfig cfgResize;
      cfgResize.setResize(resizeDimension, resizeDimension);
      configQueue->send(cfgResize);
      
      dai::ImageManipConfig cfgCrop;
      cfgCrop.setCropRect(GetCropCoordinates());
      configQueue->send(cfgCrop);