IR can't be aligned with RGB but Depth can.

FengkaiTian

Right now my depth image is perfectly aligned with RGB, however, there is no example code for IR to do alignment with RGB. I checked the document and used Claude Agent but still not figure out.

IR is originally aligned with depth so it's possible to do alignment with RGB right ? if RGB can !!

SimonvE

[I don't work at Luxonis so don't quote me]

Without the presence of the depth image it is fundamentally very hard to accurately align RGB and IR. This can only be done for a set distance, but not something that works on all distances. I have seen this is done in some examples with OAK thermal image alignment (maybe you can find that).

However, the presence of depth makes it theoretically possible! Although I don't know if the OAK tools support it., let me know if it does.

You could use the IR image to "colourise" the unaligned depth image. Next, align that colourised depth image and success. / Or somehow just apply the depth align transform on the IR image.

I don't think OAK exposes the right functions for this, so you might have to build it yourself together with Claude.

jakaskerl

SimonvE
Has been done in the past - luxonis/depthai-pythonblob/main/examples/ImageAlign/image_align.py but as @SimonvE said, you need depth info to align them correctly since you need to perform reprojection.

Thanks,
Jaka

1eqweq

jakaskerl Hello, I have checked the code I think this is for depthAI2, do you have code for depthAI3?

Also, after I run it, I can only get the depth image aligned with RGB, what about the IR image? I also need IR image aligned.

Thanks for helping.