Raw image from rgb camera that is demosaiced

telaydi

Hello!
I'm currently receiving a RAW10 video stream from my OAK-D color camera, and by using the provided example code, unpacking the stream to 16bits per channel. I then use cv2.cvtColor to demosiac the image to linear RGB. Normally these two steps are done on-device when receiving an isp stream. Is there a way to receive unpacked demosaiced images from the isp stream but without the YUV420 sub-sampling? I'm hoping to get a linear RGBF16F16F16p image directly from the OAK-D without subsampling or jpeg compression. I don't mind if the stream goes through the 3A algorithm or is de-noised. Any help is greatly appreciated!
(If it's a bandwidth limitation can the same be done with still images?)

Warmest regards,
Tarek

Luxonis-Alex

Hi Tarek,

At the moment we only have YUV420 8-bit planar output available from the ISP pipeline, but it's possible to tap before the chroma scaler and get YUV444, and potentially also RGB without subsampling. I can look at this by the end of this week and see about enabling it. FP16 output instead of U8 might also be possible, but could require more work.

Thanks,
Alex

telaydi

Thank you Alex! I have my setup working so I'm happy to wait. From an efficiency point of view though, having the spi do the heavy lifting makes more since. Looking forward to the new output type 😀

telaydi

Hi Alex!
Can you update us on whether you were able to add YUV444 (or RGB without subsampling) output streams.

Much appreciated!
Tarek

erik

Hello telaydi ,
you could also implement it with your own custom "NN" model, docs on that here. Kornia might help here, see rgb->yuv conversion here.
Thanks, Erik