RGB-Thermal align

MatBee

How do I run the RGB-Thermal align example if I'm using a non-Luxonis thermal camera?

https://docs.luxonis.com/software/depthai/examples/thermal_align/

MatBee

Figured it out, simply just using cv2.VideoCapture. I have to 3d print a proper mount to make sure theyre on the same plane but it seems to work.

MatBee

Actually no, how do I use a non-luxonis camera as a CameraNode?

jakaskerl

Hi @MatBee
You can't use it as a Camera node. You can however stream the frames via Xlink. The issue is that ImageAlign node uses extrinsics and intrinsics of the camera which can not be set to ImgFrame directly.

Your best bet would be to do this on host. At least until the depthai v3. I think the plan was to have ImgFrame carry the camera parameters.

Thanks,
Jaka

MatBee

I'm having issues with an ImageNode solution whereas the Thermal camera is 256x192 but the the RGB camera is much larger. How do I warp the thermal image to the RGB image using the Depthai library? It doesnt seem like it can work with different resolutions, but the hosts opencv functions can.

jakaskerl

MatBee
The image inside ImageAlign node will be transformed to whatever the inputAlignTo frame is. The it will warp the image according to the intrinsics and extrinsics.

Thanks,
Jaka

JackMead

jakaskerl

The commons are helpful here. Can you provide reference, say in OpenCV, for how one would do the alignment on host for these custom scenarios? I'm familiar enough with camera calibration that if you share the base algorithmic implementation in plain english, I can figure it out. I am a similar scenario, but I am using the Oak-T and don't want to utilize ImageAlign as it is giving me issues.

GergelySzabolcs

JackMead

You can find reference python implementation here: luxonis/depthai-pythontree/646e3fdffa358e2455b794a6d50f74fbb3baef13/examples/Reprojection