Our method to try to align the depth, projected on (rectified) left image, to rgb camera (outside of pipeline) is not working, and we are trying to figure out why. Why would they stay the same? are there then any other computations going on to do this alignment internally?