Using ImageManip with UVC

adefranco

Hello,

We are interested in utilizing some ImageManip features and passing the results to a UVC node. For example, we would like to crop a 1920x1080 area from the 4K sensor and present that over UVC.

There seem to be some issues when applying ImageManip output to a UVC node, however. Primarily, the image usually shows up as green and cropping can sometimes tile the image. From reading the documentation, I believe the issue is that the UVC node requires NV12 input whereas the ImageManip node only supports outputting to RGB888p, BGR888p, or GRAY8.

Is this currently possible, or are there any plans to support this in the future? I have seen post #627 on the forums that mentions NV12 support is a WIP on the image_manip_refactor branch. I did try out this branch and seem to have the same results, but I was curious if this may solve our use case when completed?

Thank you in advance!

erik

Hello adefranco , I believe you are correct on the reason behind this issue - mainline ImageManip doesn't yet support NV12 manipulation, and there's no UVC support in mainline, so these features aren't "compatible" yet, until one of these braches gets merged and the other gets updated (rebased).
Thanks, Erik

adefranco

Hi erik , thank you for the confirmation! Is there any rough timeline planned for when ImageManip will support NV12 in main and/or when UVC will be merged into main? We're looking forward to trying out these features together once they're compatible and have been enjoying working with the Oak-1 in general. Thanks again!

erik

Hello adefranco , would XLink (normal mode, not UVC) not work for your application? We think that imagemanip will get refactored in a few weeks tops, then we would just need to update the UVC branch to latest main.
Thanks, Erik

adefranco

Hey erik , thank you for the information! That sounds great. Unfortunately, we do require UVC for our needs and normal XLink would not be sufficient. On the topic of UVC, are there any plans to expose UVC controls (pan/tilt/zoom) in the future? If you would rather I start a new thread for this question I'm more than happy to do so. Thanks again!

erik

Hi adefranco ,
Sorry about the delay. None of our cameras support pan/tilt/zoom, so I'm not sure how that would be useful as of now. Thoughts?
Thanks, Erik

adefranco

Hi erik ,

Thank you for your reply! We are interested in the possibility of using PTZ controls with the ImageManip node or similar if the camera is zoomed in. That is, for example, if we are looking at a 1920x1080 window inside of the full 4k view, we would like to be able to use tilt to adjust that zoomed window left and right, tilt to adjust the window up and down, and zoom to adjust the zoom of that window. We have seen this capability in a few other webcams in our testing. Would this be possible with any of your cameras/nodes? Thank you again!

erik

Hi adefranco ,
I believe we would first need to support this inside the firmware (via UVC - docs here). Since our cameras don't have zoom (only focus if AF color camera), this wouldn't be that useful, but could still control things like exposure/sensitivity/focus etc. I brought it up internally with the team to keep in mind after we mainline UVC support.
Thanks, Erik

adefranco

Hey erik ,

We appreciate you bringing this up with the rest of your team! Just to clarify a bit, we are only looking for digital zooming, panning, and tilting. For example, if the UVC output of the camera was zoomed out all the way (showing the maximum frame available to the camera), then PTZ controls would not change the output as the full frame is in view. However, if the user digitally zoomed in, PTZ controls would let the user pan and tilt the zoomed in frame to show other parts of the outer frame. We have included a picture to provide clarification on our use case. An example of this behavior is in the Anker PowerConf C300 that has no optical PTZ, but rather does the operations digitally. Several Logitech cameras have similar functionality.

In particular looking forward to larger image sensors, such as the IMX582, it would be quite handy to get more detail out of one area of an image over UVC.