Questions about ColorCamera scaling and cropping

gregflurry

I want to achieve the maximum possible VFOV for creating the preview frame sent to a MobileNetSpatialDetectionNetwork node. To do so I set the ColorCamera sensor resolution to 12_MP. That did not work out well due to an internal crop to a max of 3840x2160 in the preview processing (per Luxonis).

I then scaled the ISP down by 1/2. That produces an ISP frame size of 2028x1520, smaller than the preview max crop. I found it necessary to set the video size to 1520x1520. That worked; the preview showed the full VFOV. Nice.

Then I thought that there is no reason to be pushing so much unneeded data around in the pipeline, and without thinking, I set the ISP scaling to 1/3. I realized later the calculated ISP frame size of 1352x1013.33. I'd have expected something to fail. But everything worked ... mostly! The ISP frame when displayed with imshow is 1352x1014. After additional experimentation I concluded that the video dimensions appear to have to be a multiple of 16. Setting the video size to 1008x1008 produced a preview that was almost the full VFOV. With center cropping, the top 3 pixels and the bottom 3 pixels are cropped off. Not too bad.

I then tried an ISP scale of 1/4. I could get and display the ISP stream. I could find only one video size (300x300) that produced a preview stream. With any other size, the script produced no errors, but never displayed anything. I think it hung waiting on the preview queue.

So, on to the questions:

Is it helpful to reduce the size of the frames flowing thru the pipeline?
Why does ISP scaling of 1/3, that produces a calculated ISP frame size of 1352x1013.33 and results in an actual frame size of 1352x1014 work at all?
Is there any concern with the quality of the images with a scale factor of 1/3 compared to when the scale factor is 1/2?
What prevents the production of a preview stream with a scale factor of 1/4 for video sizes other than 300x300?

erik

Hello gregflurry ,

Yes, reducing the size of frames will reduce RAM/CPU usage, so use the lowest resolution/size as possible
It gets rounded up
Well they are both downscaled, so they lose quality. So depends on our application.
1/4 (1920x1080 => 480x270) is actually below 300x300, not sure why it would work

Thanks, Erik

gregflurry

@erik Thanks. One clarification, on my last question. I had the sensor resolution set to 12_MP and the ISP scale factor set to 1/4, which produces an ISP frame of 1014x760. I ran the test again and verified that only a video size of 300x300 produced a preview output; otherwise, the script hung waiting on something to get into the queue for preview frames.