I would like to crop the raw stream of a color camera to a square aspect ratio with a precise integer pixel offset from the left. The sensorCrop function only takes a single float (percentage of sensor) for the left, and does not offer a right side. How can I assure that I am employing the most low level region of interest crop so that the data coming off of the sensor is lessened for each frame. Hopefully this would give me a little more FPS as there is 1.33x less data.
Is this possible?
Hi Asa ,
WRT camRgb.setSensorCrop(x,y), it's used for ISP->video cropping (img below). By default, it will be center cropped (so setSensorCrop(0.25, 0.25)), but you could also configure it to get top-left quarter of the img (.setSensorCrop(0, 0)), or bottom-right quarter of the img (.setSensorCrop(0.5, 0.5)).
camRgb.setResolution(dai.ColorCameraProperties.SensorResolution.THE_4_K)
# camRgb,setSensorCrop(0.25, 0.25) # By default already, center crop
camRgb.setVideoSize(1920, 1080)
It doesn't actually do on-sensor crop (and MIPI stream only that region), but it does crop in postprocessing on RVC. I will update the documentation for this function as well.
Thoughts?
Thanks, Erik
I am looking for something that would be before isp. I am trying for the lowest latency, closest to the metal experience. Reducing the width of the readout ROI on the sensor should let you reduce the rolling shutter artifacting by a decent amount as well(less pixels to read out each line at the same clock rate = less overall time between rows). I would like to ideally set this up as my own SensorResolution (THE_12_MP_CENTER_CROP_1_TO_1?). Is this possible? I know the Sony sensors support this ROI type of setup. Being able to specify the correct integer column offset would be important so that we don't change the debayering order of course. I guess its up to you to setup support for something like this as you currently are pulling whole frames off all the time, even if you crop to 4k downstream?
Asa I don't think you can currently achieve that with OAK cameras, and we would need to implement this first. And since implementing such things takes very large efforts, I'm quite sure we wouldn't want to go down this rabbit hole unless there's a commitment for 1000s of devices.I hope you understand.
And you are right, it's 0..1 values, so 0.25 is a quarter, and if we have isp 4k and video 1080P, then it will do center cropping (from 0.25,0.25 to 0.75,0.75).
Thanks, Erik
Understood, albeit dissappointing. You would improve the performance of all of the cameras if you can support this as you are currently exposing and reading out a third more pixels than you need to. If you are center cropping 4k you are throwing away even more at around 4mp of data every frame. This has to limit the aggregate isp performance of all of your systems, increases the jello effect etc. As for this affecting a large commitment of devices, you may not be too far off. At any rate, I would love to track this issue. Perhaps this is an area that will open up in the SDK over time?
Hi Asa , I agree - in some cases, this would improve the system's performance. But the effort/benefit ratio would need to be good enough to work on this for RVC2. We will likely focus most of our energy on RVC3 / RVC4 (Q1 2024) in the future, as they provide a significant advantage (for the user) over RVC2. We will still maintain RVC2 in the future, but likely wouldn't add such large features to it.
Is the sensor crop pipeline different in RVC3? Are we able to have more of this kind of control there?
I am happy do do whatever is necessary as a manual task as long there is some kind of api exposed.Any hooks for that I can look at yet?
