• high performance sensor crop / region of interest

Hi Asa ,
WRT camRgb.setSensorCrop(x,y), it's used for ISP->video cropping (img below). By default, it will be center cropped (so setSensorCrop(0.25, 0.25)), but you could also configure it to get top-left quarter of the img (.setSensorCrop(0, 0)), or bottom-right quarter of the img (.setSensorCrop(0.5, 0.5)).

camRgb.setResolution(dai.ColorCameraProperties.SensorResolution.THE_4_K)
# camRgb,setSensorCrop(0.25, 0.25) # By default already, center crop
camRgb.setVideoSize(1920, 1080)

It doesn't actually do on-sensor crop (and MIPI stream only that region), but it does crop in postprocessing on RVC. I will update the documentation for this function as well.
Thoughts?
Thanks, Erik

    @erik Can you tell me what is the range of (X,y) coordinates we can set in camRgb,setSensorCrop(0.25, 0.25). Eg. what is the initial point and end point?

    • Asa replied to this.

      I am looking for something that would be before isp. I am trying for the lowest latency, closest to the metal experience. Reducing the width of the readout ROI on the sensor should let you reduce the rolling shutter artifacting by a decent amount as well(less pixels to read out each line at the same clock rate = less overall time between rows). I would like to ideally set this up as my own SensorResolution (THE_12_MP_CENTER_CROP_1_TO_1?). Is this possible? I know the Sony sensors support this ROI type of setup. Being able to specify the correct integer column offset would be important so that we don't change the debayering order of course. I guess its up to you to setup support for something like this as you currently are pulling whole frames off all the time, even if you crop to 4k downstream?

      Timmm these are "fractions of whole width and height" I believe.

      • erik replied to this.

          Asa I don't think you can currently achieve that with OAK cameras, and we would need to implement this first. And since implementing such things takes very large efforts, I'm quite sure we wouldn't want to go down this rabbit hole unless there's a commitment for 1000s of devices.I hope you understand.
          And you are right, it's 0..1 values, so 0.25 is a quarter, and if we have isp 4k and video 1080P, then it will do center cropping (from 0.25,0.25 to 0.75,0.75).
          Thanks, Erik

            Understood, albeit dissappointing. You would improve the performance of all of the cameras if you can support this as you are currently exposing and reading out a third more pixels than you need to. If you are center cropping 4k you are throwing away even more at around 4mp of data every frame. This has to limit the aggregate isp performance of all of your systems, increases the jello effect etc. As for this affecting a large commitment of devices, you may not be too far off. At any rate, I would love to track this issue. Perhaps this is an area that will open up in the SDK over time?

            • erik replied to this.

              Hi Asa , I agree - in some cases, this would improve the system's performance. But the effort/benefit ratio would need to be good enough to work on this for RVC2. We will likely focus most of our energy on RVC3 / RVC4 (Q1 2024) in the future, as they provide a significant advantage (for the user) over RVC2. We will still maintain RVC2 in the future, but likely wouldn't add such large features to it.

                Is the sensor crop pipeline different in RVC3? Are we able to have more of this kind of control there?

                • erik replied to this.

                  Asa very likely no by default. I am not very familiar with sensors & their configs, but my guess would be that you'd need custom camera config for each such "sensor cropping", which would likely be manual work.

                    I am happy do do whatever is necessary as a manual task as long there is some kind of api exposed.Any hooks for that I can look at yet?