Color Space / Channel Misalignment After Enabling `enableUndistortion=True`

MulongDeng

It appears that enabling enableUndistortion=True is causing color space or channel misalignment in the output image.

During testing, I observed that after undistortion, the color pixel values remain positioned as if no distortion correction had been applied. This results in visible color fringing or chromatic misregistration — essentially, the red, green, and blue channels seem misaligned relative to the geometrically corrected image.

Below is the test code I used:

#!/usr/bin/env python3
from datetime import datetime

import cv2
import depthai as dai

# Create pipeline
with dai.Pipeline() as pipeline:
    # Define source and output
    cam = pipeline.create(dai.node.Camera).build(dai.CameraBoardSocket.CAM_A)
    distortedQueue = cam.requestOutput((1920,1080), resizeMode=dai.ImgResizeMode.STRETCH, enableUndistortion=False).createOutputQueue()
    undistortedQueue = cam.requestOutput((1920,1080), resizeMode=dai.ImgResizeMode.STRETCH, enableUndistortion=True).createOutputQueue()


    # Connect to device and start pipeline
    pipeline.start()
    while pipeline.isRunning():
        distortedIn = distortedQueue.get()
        assert isinstance(distortedIn, dai.ImgFrame)
        distorted_frame = distortedIn.getCvFrame()
        cv2.imshow("distorted", distorted_frame)
    
        undistortedIn = undistortedQueue.get()
        assert isinstance(undistortedIn, dai.ImgFrame)
        undistorted_frame = undistortedIn.getCvFrame()
        cv2.imshow("undistorted", undistorted_frame)
        
        key = cv2.waitKey(1) & 0xFF
        if key == ord("q"):
            break
        if key == ord("s"):
            filename = f"cam_a_{datetime.now():%Y%m%d_%H%M%S}"
            cv2.imwrite(f"{filename}_distorted.png", distorted_frame, [cv2.IMWRITE_PNG_COMPRESSION, 0])
            cv2.imwrite(f"{filename}_undistorted.png", undistorted_frame, [cv2.IMWRITE_PNG_COMPRESSION, 0])

For reference, here are the before/after comparison images:

Original (without undistortion):

After undistortion (`enableUndistortion=True`):

Has anyone else encountered this issue? Any suggestions for debugging or workarounds would be greatly appreciated.

Device: OAK4-D R7
DepthAI: v3.0.0
Luxonis OS: RVC4 1.18.3 (debug)

OskarSonc

Hey @MulongDeng

I ran your script on the OAK4-D and couldn’t reproduce the colour fringing. One thing that stands out in your screenshots is the much heavier crop on the undistorted view, which usually points to a different undistortion map (calibration/ISP mode), not the API flag itself.

Could you change ImgResizeMode.LETTERBOX in your code and post output?

Also could you run the calibration dump and post the output:
https://docs.luxonis.com/software-v3/depthai/examples/calibration/calibration_dump/

Thanks,
Oskar

MulongDeng

OskarSonc

I tried using ImgResizeMode.LETTERBOX, but it doesn’t seem to behave any differently from ImgResizeMode.STRETCH.

Here’s the output:

And here’s the calibration:

Is EEPROM available: True
User calibration: {
  "batchName": "",
  "batchTime": 1748325751,
  "boardConf": "nIR-C56M52-03",
  "boardCustom": "",
  "boardName": "NG9498-ASM",
  "boardOptions": 0,
  "boardRev": "P7D1",
  "cameraData": [
    [
      0,
      {
        "cameraType": 0,
        "distortionCoeff": [
          4.154960632324219,
          11.986598014831543,
          -0.0001723415043670684,
          -0.0005735334707424045,
          -0.9605101346969604,
          4.229392051696777,
          13.612556457519531,
          0.5860109925270081,
          0.0,
          0.0,
          0.0,
          0.0,
          0.0010749325156211853,
          -0.005254020448774099
        ],
        "extrinsics": {
          "rotationMatrix": [
            [
              0.0,
              0.0,
              0.0
            ],
            [
              0.0,
              0.0,
              0.0
            ],
            [
              0.0,
              0.0,
              0.0
            ]
          ],
          "specTranslation": {
            "x": -0.0,
            "y": -0.0,
            "z": -0.0
          },
          "toCameraSocket": -1,
          "translation": {
            "x": 0.0,
            "y": 0.0,
            "z": 0.0
          }
        },
        "height": 3000,
        "intrinsicMatrix": [
          [
            2233.322509765625,
            0.0,
            2008.655029296875
          ],
          [
            0.0,
            2232.114501953125,
            1540.9024658203125
          ],
          [
            0.0,
            0.0,
            1.0
          ]
        ],
        "lensPosition": 0,
        "specHfovDeg": 108.0,
        "width": 4000
      }
    ],
    [
      1,
      {
        "cameraType": 0,
        "distortionCoeff": [
          0.33490827679634094,
          -0.04399221017956734,
          6.405573367374018e-05,
          8.02077556727454e-05,
          -0.0016113914316520095,
          0.667831301689148,
          -0.017263764515519142,
          -0.01005784422159195,
          0.0,
          0.0,
          0.0,
          0.0,
          -0.0008745337836444378,
          0.0015693728346377611
        ],
        "extrinsics": {
          "rotationMatrix": [
            [
              0.9999640583992004,
              -0.006885173264890909,
              -0.00494249165058136
            ],
            [
              0.006908210460096598,
              0.9999653100967407,
              0.00465930113568902
            ],
            [
              0.004910239949822426,
              -0.004693277645856142,
              0.9999769330024719
            ]
          ],
          "specTranslation": {
            "x": -7.5,
            "y": 0.0,
            "z": 0.0
          },
          "toCameraSocket": 2,
          "translation": {
            "x": -7.480626583099365,
            "y": -0.03341342881321907,
            "z": -0.04739514738321304
          }
        },
        "height": 800,
        "intrinsicMatrix": [
          [
            566.3438110351562,
            0.0,
            647.1285400390625
          ],
          [
            0.0,
            566.203125,
            415.7493591308594
          ],
          [
            0.0,
            0.0,
            1.0
          ]
        ],
        "lensPosition": 0,
        "specHfovDeg": 127.0,
        "width": 1280
      }
    ],
    [
      2,
      {
        "cameraType": 0,
        "distortionCoeff": [
          0.37178856134414673,
          -0.028114957734942436,
          6.70732551952824e-05,
          -6.304917769739404e-05,
          -0.0012017368571832776,
          0.7060509324073792,
          0.012137592770159245,
          -0.007517450489103794,
          0.0,
          0.0,
          0.0,
          0.0,
          1.1011356946255546e-05,
          -0.002646640408784151
        ],
        "extrinsics": {
          "rotationMatrix": [
            [
              0.9999170303344727,
              0.011888023465871811,
              -0.0049632820300757885
            ],
            [
              -0.011914782226085663,
              0.9999144673347473,
              -0.005397136323153973
            ],
            [
              0.004898696206510067,
              0.005455825012177229,
              0.9999731183052063
            ]
          ],
          "specTranslation": {
            "x": 3.75,
            "y": 0.0,
            "z": 0.0
          },
          "toCameraSocket": 0,
          "translation": {
            "x": 3.736459970474243,
            "y": -0.05492363125085831,
            "z": 0.08217036724090576
          }
        },
        "height": 800,
        "intrinsicMatrix": [
          [
            573.3753051757812,
            0.0,
            631.2189331054688
          ],
          [
            0.0,
            573.014892578125,
            434.4741516113281
          ],
          [
            0.0,
            0.0,
            1.0
          ]
        ],
        "lensPosition": 0,
        "specHfovDeg": 127.0,
        "width": 1280
      }
    ]
  ],
  "deviceName": "",
  "hardwareConf": "F1-FV01-BC000",
  "housingExtrinsics": {
    "rotationMatrix": [],
    "specTranslation": {
      "x": 0.0,
      "y": 0.0,
      "z": 0.0
    },
    "toCameraSocket": -1,
    "translation": {
      "x": 0.0,
      "y": 0.0,
      "z": 0.0
    }
  },
  "imuExtrinsics": {
    "rotationMatrix": [
      [
        0.0,
        0.0,
        0.0
      ],
      [
        0.0,
        0.0,
        0.0
      ],
      [
        0.0,
        0.0,
        0.0
      ]
    ],
    "specTranslation": {
      "x": 0.0,
      "y": 0.0,
      "z": 0.0
    },
    "toCameraSocket": -1,
    "translation": {
      "x": 0.0,
      "y": 0.0,
      "z": 0.0
    }
  },
  "miscellaneousData": [],
  "productName": "OAK4-D-W",
  "stereoEnableDistortionCorrection": false,
  "stereoRectificationData": {
    "leftCameraSocket": 1,
    "rectifiedRotationLeft": [
      [
        0.9999960064888,
        -0.0024483283050358295,
        0.0014138496480882168
      ],
      [
        0.0024450235068798065,
        0.9999942779541016,
        0.0023344550281763077
      ],
      [
        -0.001419557025656104,
        -0.002330988645553589,
        0.9999963045120239
      ]
    ],
    "rectifiedRotationRight": [
      [
        0.9999699592590332,
        0.00446652714163065,
        0.00633552810177207
      ],
      [
        -0.0044517358765006065,
        0.9999873638153076,
        -0.00234684394672513
      ],
      [
        -0.00634593004360795,
        0.002318569226190448,
        0.999977171421051
      ]
    ],
    "rightCameraSocket": 2
  },
  "stereoUseSpecTranslation": true,
  "version": 7,
  "verticalCameraSocket": -1
}
Factory calibration: {
  "batchName": "",
  "batchTime": 1748325751,
  "boardConf": "nIR-C56M52-03",
  "boardCustom": "",
  "boardName": "NG9498-ASM",
  "boardOptions": 0,
  "boardRev": "P7D1",
  "cameraData": [
    [
      0,
      {
        "cameraType": 0,
        "distortionCoeff": [
          4.154960632324219,
          11.986598014831543,
          -0.0001723415043670684,
          -0.0005735334707424045,
          -0.9605101346969604,
          4.229392051696777,
          13.612556457519531,
          0.5860109925270081,
          0.0,
          0.0,
          0.0,
          0.0,
          0.0010749325156211853,
          -0.005254020448774099
        ],
        "extrinsics": {
          "rotationMatrix": [
            [
              0.0,
              0.0,
              0.0
            ],
            [
              0.0,
              0.0,
              0.0
            ],
            [
              0.0,
              0.0,
              0.0
            ]
          ],
          "specTranslation": {
            "x": -0.0,
            "y": -0.0,
            "z": -0.0
          },
          "toCameraSocket": -1,
          "translation": {
            "x": 0.0,
            "y": 0.0,
            "z": 0.0
          }
        },
        "height": 3000,
        "intrinsicMatrix": [
          [
            2233.322509765625,
            0.0,
            2008.655029296875
          ],
          [
            0.0,
            2232.114501953125,
            1540.9024658203125
          ],
          [
            0.0,
            0.0,
            1.0
          ]
        ],
        "lensPosition": 0,
        "specHfovDeg": 108.0,
        "width": 4000
      }
    ],
    [
      1,
      {
        "cameraType": 0,
        "distortionCoeff": [
          0.33490827679634094,
          -0.04399221017956734,
          6.405573367374018e-05,
          8.02077556727454e-05,
          -0.0016113914316520095,
          0.667831301689148,
          -0.017263764515519142,
          -0.01005784422159195,
          0.0,
          0.0,
          0.0,
          0.0,
          -0.0008745337836444378,
          0.0015693728346377611
        ],
        "extrinsics": {
          "rotationMatrix": [
            [
              0.9999640583992004,
              -0.006885173264890909,
              -0.00494249165058136
            ],
            [
              0.006908210460096598,
              0.9999653100967407,
              0.00465930113568902
            ],
            [
              0.004910239949822426,
              -0.004693277645856142,
              0.9999769330024719
            ]
          ],
          "specTranslation": {
            "x": -7.5,
            "y": 0.0,
            "z": 0.0
          },
          "toCameraSocket": 2,
          "translation": {
            "x": -7.480626583099365,
            "y": -0.03341342881321907,
            "z": -0.04739514738321304
          }
        },
        "height": 800,
        "intrinsicMatrix": [
          [
            566.3438110351562,
            0.0,
            647.1285400390625
          ],
          [
            0.0,
            566.203125,
            415.7493591308594
          ],
          [
            0.0,
            0.0,
            1.0
          ]
        ],
        "lensPosition": 0,
        "specHfovDeg": 127.0,
        "width": 1280
      }
    ],
    [
      2,
      {
        "cameraType": 0,
        "distortionCoeff": [
          0.37178856134414673,
          -0.028114957734942436,
          6.70732551952824e-05,
          -6.304917769739404e-05,
          -0.0012017368571832776,
          0.7060509324073792,
          0.012137592770159245,
          -0.007517450489103794,
          0.0,
          0.0,
          0.0,
          0.0,
          1.1011356946255546e-05,
          -0.002646640408784151
        ],
        "extrinsics": {
          "rotationMatrix": [
            [
              0.9999170303344727,
              0.011888023465871811,
              -0.0049632820300757885
            ],
            [
              -0.011914782226085663,
              0.9999144673347473,
              -0.005397136323153973
            ],
            [
              0.004898696206510067,
              0.005455825012177229,
              0.9999731183052063
            ]
          ],
          "specTranslation": {
            "x": 3.75,
            "y": 0.0,
            "z": 0.0
          },
          "toCameraSocket": 0,
          "translation": {
            "x": 3.736459970474243,
            "y": -0.05492363125085831,
            "z": 0.08217036724090576
          }
        },
        "height": 800,
        "intrinsicMatrix": [
          [
            573.3753051757812,
            0.0,
            631.2189331054688
          ],
          [
            0.0,
            573.014892578125,
            434.4741516113281
          ],
          [
            0.0,
            0.0,
            1.0
          ]
        ],
        "lensPosition": 0,
        "specHfovDeg": 127.0,
        "width": 1280
      }
    ]
  ],
  "deviceName": "",
  "hardwareConf": "F1-FV01-BC000",
  "housingExtrinsics": {
    "rotationMatrix": [],
    "specTranslation": {
      "x": 0.0,
      "y": 0.0,
      "z": 0.0
    },
    "toCameraSocket": -1,
    "translation": {
      "x": 0.0,
      "y": 0.0,
      "z": 0.0
    }
  },
  "imuExtrinsics": {
    "rotationMatrix": [
      [
        0.0,
        0.0,
        0.0
      ],
      [
        0.0,
        0.0,
        0.0
      ],
      [
        0.0,
        0.0,
        0.0
      ]
    ],
    "specTranslation": {
      "x": 0.0,
      "y": 0.0,
      "z": 0.0
    },
    "toCameraSocket": -1,
    "translation": {
      "x": 0.0,
      "y": 0.0,
      "z": 0.0
    }
  },
  "miscellaneousData": [],
  "productName": "OAK4-D-W",
  "stereoEnableDistortionCorrection": false,
  "stereoRectificationData": {
    "leftCameraSocket": 1,
    "rectifiedRotationLeft": [
      [
        0.9999960064888,
        -0.0024483283050358295,
        0.0014138496480882168
      ],
      [
        0.0024450235068798065,
        0.9999942779541016,
        0.0023344550281763077
      ],
      [
        -0.001419557025656104,
        -0.002330988645553589,
        0.9999963045120239
      ]
    ],
    "rectifiedRotationRight": [
      [
        0.9999699592590332,
        0.00446652714163065,
        0.00633552810177207
      ],
      [
        -0.0044517358765006065,
        0.9999873638153076,
        -0.00234684394672513
      ],
      [
        -0.00634593004360795,
        0.002318569226190448,
        0.999977171421051
      ]
    ],
    "rightCameraSocket": 2
  },
  "stereoUseSpecTranslation": true,
  "version": 7,
  "verticalCameraSocket": -1
}

MulongDeng

Hi，@OskarSonc，I was wondering if there have been any updates to the session？Any insights would be greatly appreciated！