Noisy image from OAK-D Pro W

Rrbn23 · Aug 28, 2024

When I record left and right rectified images, my right image is very noisy. Here is an example:

Left

Right

I followed the samples for camera settings:

controlIn.out.link(monoLeft.inputControl)
controlIn.out.link(monoRight.inputControl)
controlQueue = device.getInputQueue(controlIn.getStreamName())
ctrl = dai.CameraControl()
ctrl.setAutoExposureEnable()
ctrl.setAutoExposureLimit(10000)
controlQueue.send(ctrl)

Did I miss something?

jakaskerl · Aug 30, 2024

Hi @rbn23
How are you recording the left and right streams? Are images for left and right mono cameras not noisy?

Thanks,
Jaka

Rrbn23 · Sep 4, 2024

I take each frame as opencv image from the queue and save it with cv::imwrite()

I have a second OAK-D-Pro W camera and with this one (same code) the images are not noisy. To verify if unrectified image is not noisy I need to take a test with strong sunlight, since the problem with the noisy image only happes in very bright scenes.

jakaskerl · Sep 4, 2024

rbn23
Does this happen when only using one camera (left or right) at one time? Seems strange to me that sensor exposure would cause this on one device but not on the other.

Generally the left camera drives the right so setting both to different settings would cause issues. Perhaps only set the master camera's control.

Thanks,
Jaka

Rrbn23 · Sep 5, 2024

jakaskerl

I am using dai.node.StereoDepth node. The quality of the depth image was very bad what made me wonder. When I looked at the rectified images I saw that the right one was noisy.

I followed the depthai examples that set coltrols for the mono cameras the following way:

controlIn.out.link(monoLeft.inputControl)
controlIn.out.link(monoRight.inputControl)

Is this wrong for unsing stereo node? When Left camera sets the right settings, why send the control also to the right camera?

This still wouldn't explain why it only happens with one OAK module. I have 3 OAK-D Pro W and only one has this problem. The others look fine even in full sunlight, all running with the same code. Maybe the image sensor is somehow broken?

jakaskerl · Sep 6, 2024

rbn23
The connection board-sensor might be a bit worse which could cause issues when setting very low exposure time. Is the unrectified image noisy as well?
I'm trying to figure out if this is a sensor fault or perhaps calibration or some other component.

Thanks,
Jaka

Rrbn23 · Sep 6, 2024

Right now it's cloudy outside, so I cannot test and check the unrectified images. I will test when I get full sunlight again

jakaskerl · Sep 6, 2024

rbn23 Can you shine a bright light perhaps?

Rrbn23 · Sep 11, 2024

I set the camera right underneath the ceiling lights and got these results with manual exposure of 100us and iso 100:

left

right

right image is noisy again

jakaskerl · Sep 11, 2024

rbn23
Ok, good, thanks.

cc @Luxonis-Alex

Rrbn23 · Oct 1, 2024

Is there any update from your side?

LLuxonis-Alex · Oct 2, 2024

@rbn23 I can't seem to replicate this noise on just one of the sensors. A couple things to check:

is a recent depthai release in use (e.g. 2.28.0.0), at some point we improved the sensor configs
might be temperature related, could you check with the device cooled down, does this issue appear right away after power on, or after running for some time. Try also a simple pipeline (no NN, no VideoEncoder, ...)
the hardware revision of the device, so we can recheck with the same version you have. Best if you could get the output of depthai-python/examples/calibration/calibration_dump.py

Rrbn23 · Oct 7, 2024

I use python package 2.26.0, that was the latest I could install using pip
The problem happens right after camera boot. The camera is not warm at all. There is no NN or anything else running on the camera except grabbing the two images.
I have 4 cameras of the same type. I cannot reproduce the problem with any of the other 3 cameras. When running the same script.
Here is the calibration from the camera:

User calibration: {

"batchName": "",

"batchTime": 1708949643,

"boardConf": "IR-C00M00-00",

"boardCustom": "",

"boardName": "NG9097",

"boardOptions": 7,

"boardRev": "R4M2E4",

"cameraData": [

[

  1,

  {

    "cameraType": 0,

    "distortionCoeff": [     

      3\.5571253299713135,    

      1\.0675731897354126,    

      0\.00011813067976618186,

      -4.179932602710323e-06,

      0\.019934868440032005,

      3\.9141414165496826,

      2\.124300003051758,

      0\.15198968350887299,

      0\.0,

      0\.0,

      0\.0,

      0\.0,

      0\.0,

      0\.0

    ],

    "extrinsics": {

      "rotationMatrix": [

        [

          0\.9998213648796082,

          0\.008929289877414703,

          -0.01665874384343624

        ],

        [

          -0.009426653385162354,

          0\.9995048642158508,

          -0.03002036362886429

        ],

        [

          0\.016382433474063873,

          0\.030172036960721016,

          0\.9994104504585266

        ]

      ],

      "specTranslation": {

        "x": -7.5,

        "y": 0.0,

        "z": 0.0

      },

      "toCameraSocket": 2,

      "translation": {

        "x": -7.511865139007568,

        "y": 0.006338753271847963,

        "z": 0.008916680701076984

      }

    },

    "height": 800,

    "intrinsicMatrix": [

      [

        563\.3818359375,

        0\.0,

        620\.244384765625

      ],

      [

        0\.0,

        563\.4485473632812,

        401\.442138671875

      ],

      [

        0\.0,

        0\.0,

        1\.0

      ]

    ],

    "lensPosition": 0,

    "specHfovDeg": 127.0,

    "width": 1280

  }

],

[

  0,

  {

    "cameraType": 0,

    "distortionCoeff": [

      12\.214872360229492,

      11\.166379928588867,

      0\.0009387647733092308,

      -0.0005655140266753733,

      -1.9312169551849365,

      12\.175280570983887,

      14\.985671043395996,

      -1.223604440689087,

      0\.0,

      0\.0,

      0\.0,

      0\.0,

      0\.0,

      0\.0

    ],

    "extrinsics": {

      "rotationMatrix": [

        [

          0\.0,

          0\.0,

          0\.0

        ],

        [

          0\.0,

          0\.0,

          0\.0

        ],

        [

          0\.0,

          0\.0,

          0\.0

        ]

      ],

      "specTranslation": {

        "x": -0.0,

        "y": -0.0,

        "z": -0.0

      },

      "toCameraSocket": -1,

      "translation": {

        "x": 0.0,

        "y": 0.0,

        "z": 0.0

      }

    },

    "height": 2160,

    "intrinsicMatrix": [

      [

        2311\.1796875,

        0\.0,

        1912\.238525390625

      ],

      [

        0\.0,

        2309\.768798828125,

        1092\.1376953125

      ],

      [

        0\.0,

        0\.0,

        1\.0

      ]

    ],

    "lensPosition": 0,

    "specHfovDeg": 108.0,

    "width": 3840

  }

],

[

  2,

  {

    "cameraType": 0,

    "distortionCoeff": [

      5\.346960544586182,

      1\.4969804286956787,

      -0.0008067575399763882,

      6\.0376351029844955e-05,

      0\.028692806139588356,

      5\.729836940765381,

      3\.11305832862854,

      0\.20841671526432037,

      0\.0,

      0\.0,

      0\.0,

      0\.0,

      0\.0,

      0\.0

    ],

    "extrinsics": {

      "rotationMatrix": [

        [

          0\.9998905658721924,

          -0.013739221729338169,

          0\.005482451058924198

        ],

        [

          0\.013782133348286152,

          0\.9998740553855896,

          -0.007867513224482536

        ],

        [

          -0.005373666994273663,

          0\.00794221181422472,

          0\.9999540448188782

        ]

      ],

      "specTranslation": {

        "x": 3.75,

        "y": 0.0,

        "z": 0.0

      },

      "toCameraSocket": 0,

      "translation": {

        "x": 3.768968343734741,

        "y": 0.059201680123806,

        "z": -0.07226227968931198

      }

    },

    "height": 800,

    "intrinsicMatrix": [

      [

        574\.7843627929688,

        0\.0,

        623\.1576538085938

      ],

      [

        0\.0,

        574\.4819946289062,

        429\.1021423339844

      ],

      [

        0\.0,

        0\.0,

        1\.0

      ]

    ],

    "lensPosition": 0,

    "specHfovDeg": 127.0,

    "width": 1280

  }

]

],

"deviceName": "",

"hardwareConf": "F1-FV01-BC001",

"housingExtrinsics": {

"rotationMatrix": [],

"specTranslation": {

  "x": 0.0,

  "y": 0.0,

  "z": 0.0

},

"toCameraSocket": -1,

"translation": {

  "x": 0.0,

  "y": 0.0,

  "z": 0.0

}

},

"imuExtrinsics": {

"rotationMatrix": [

  [

    0\.0,

    0\.0,

    0\.0

  ],

  [

    0\.0,

    0\.0,

    0\.0

  ],

  [

    0\.0,

    0\.0,

    0\.0

  ]

],

"specTranslation": {

  "x": 0.0,

  "y": 0.0,

  "z": 0.0

},

"toCameraSocket": -1,

"translation": {

  "x": 0.0,

  "y": 0.0,

  "z": 0.0

}

},

"miscellaneousData": [],

"productName": "OAK-D-PRO-W-POE",

"stereoEnableDistortionCorrection": true,

"stereoRectificationData": {

"leftCameraSocket": 1,

"rectifiedRotationLeft": [

  [

    0\.9998087882995605,

    0\.00805005244910717,

    -0.017819706350564957

  ],

  [

    -0.008317464031279087,

    0\.9998531341552734,

    -0.0149836465716362

  ],

  [

    0\.017696470022201538,

    0\.015128996223211288,

    0\.9997289180755615

  ]

],

"rectifiedRotationRight": [

  [

    0\.999998927116394,

    -0.0008438312215730548,

    -0.0011870115995407104

  ],

  [

    0\.0008616084815002978,

    0\.9998862743377686,

    0\.015056533738970757

  ],

  [

    0\.001174171338789165,

    -0.015057540498673916,

    0\.9998859167098999

  ]

],

"rightCameraSocket": 2

},

"stereoUseSpecTranslation": false,

"version": 7,

"verticalCameraSocket": -1

}

Factory calibration: {

"batchName": "",

"batchTime": 1708949643,

"boardConf": "IR-C00M00-00",

"boardCustom": "",

"boardName": "NG9097",

"boardOptions": 7,

"boardRev": "R4M2E4",

"cameraData": [

[

  1,

  {

    "cameraType": 0,

    "distortionCoeff": [

      3\.5571253299713135,

      1\.0675731897354126,

      0\.00011813067976618186,

      -4.179932602710323e-06,

      0\.019934868440032005,

      3\.9141414165496826,

      2\.124300003051758,

      0\.15198968350887299,

      0\.0,

      0\.0,

      0\.0,

      0\.0,

      0\.0,

      0\.0

    ],

    "extrinsics": {

      "rotationMatrix": [

        [

          0\.9998213648796082,

          0\.008929289877414703,

          -0.01665874384343624

        ],

        [

          -0.009426653385162354,

          0\.9995048642158508,

          -0.03002036362886429

        ],

        [

          0\.016382433474063873,

          0\.030172036960721016,

          0\.9994104504585266

        ]

      ],

      "specTranslation": {

        "x": -7.5,

        "y": 0.0,

        "z": 0.0

      },

      "toCameraSocket": 2,

      "translation": {

        "x": -7.511865139007568,

        "y": 0.006338753271847963,

        "z": 0.008916680701076984

      }

    },

    "height": 800,

    "intrinsicMatrix": [

      [

        563\.3818359375,

        0\.0,

        620\.244384765625

      ],

      [

        0\.0,

        563\.4485473632812,

        401\.442138671875

      ],

      [

        0\.0,

        0\.0,

        1\.0

      ]

    ],

    "lensPosition": 0,

    "specHfovDeg": 127.0,

    "width": 1280

  }

],

[

  0,

  {

    "cameraType": 0,

    "distortionCoeff": [

      12\.214872360229492,

      11\.166379928588867,

      0\.0009387647733092308,

      -0.0005655140266753733,

      -1.9312169551849365,

      12\.175280570983887,

      14\.985671043395996,

      -1.223604440689087,

      0\.0,

      0\.0,

      0\.0,

      0\.0,

      0\.0,

      0\.0

    ],

    "extrinsics": {

      "rotationMatrix": [

        [

          0\.0,

          0\.0,

          0\.0

        ],

        [

          0\.0,

          0\.0,

          0\.0

        ],

        [

          0\.0,

          0\.0,

          0\.0

        ]

      ],

      "specTranslation": {

        "x": -0.0,

        "y": -0.0,

        "z": -0.0

      },

      "toCameraSocket": -1,

      "translation": {

        "x": 0.0,

        "y": 0.0,

        "z": 0.0

      }

    },

    "height": 2160,

    "intrinsicMatrix": [

      [

        2311\.1796875,

        0\.0,

        1912\.238525390625

      ],

      [

        0\.0,

        2309\.768798828125,

        1092\.1376953125

      ],

      [

        0\.0,

        0\.0,

        1\.0

      ]

    ],

    "lensPosition": 0,

    "specHfovDeg": 108.0,

    "width": 3840

  }

],

[

  2,

  {

    "cameraType": 0,

    "distortionCoeff": [

      5\.346960544586182,

      1\.4969804286956787,

      -0.0008067575399763882,

      6\.0376351029844955e-05,

      0\.028692806139588356,

      5\.729836940765381,

      3\.11305832862854,

      0\.20841671526432037,

      0\.0,

      0\.0,

      0\.0,

      0\.0,

      0\.0,

      0\.0

    ],

    "extrinsics": {

      "rotationMatrix": [

        [

          0\.9998905658721924,

          -0.013739221729338169,

          0\.005482451058924198

        ],

        [

          0\.013782133348286152,

          0\.9998740553855896,

          -0.007867513224482536

        ],

        [

          -0.005373666994273663,

          0\.00794221181422472,

          0\.9999540448188782

        ]

      ],

      "specTranslation": {

        "x": 3.75,

        "y": 0.0,

        "z": 0.0

      },

      "toCameraSocket": 0,

      "translation": {

        "x": 3.768968343734741,

        "y": 0.059201680123806,

        "z": -0.07226227968931198

      }

    },

    "height": 800,

    "intrinsicMatrix": [

      [

        574\.7843627929688,

        0\.0,

        623\.1576538085938

      ],

      [

        0\.0,

        574\.4819946289062,

        429\.1021423339844

      ],

      [

        0\.0,

        0\.0,

        1\.0

      ]

    ],

    "lensPosition": 0,

    "specHfovDeg": 127.0,

    "width": 1280

  }

]

],

"deviceName": "",

"hardwareConf": "F1-FV01-BC001",

"housingExtrinsics": {

"rotationMatrix": [],

"specTranslation": {

  "x": 0.0,

  "y": 0.0,

  "z": 0.0

},

"toCameraSocket": -1,

"translation": {

  "x": 0.0,

  "y": 0.0,

  "z": 0.0

}

},

"imuExtrinsics": {

"rotationMatrix": [

  [

    0\.0,

    0\.0,

    0\.0

  ],

  [

    0\.0,

    0\.0,

    0\.0

  ],

  [

    0\.0,

    0\.0,

    0\.0

  ]

],

"specTranslation": {

  "x": 0.0,

  "y": 0.0,

  "z": 0.0

},

"toCameraSocket": -1,

"translation": {

  "x": 0.0,

  "y": 0.0,

  "z": 0.0

}

},

"miscellaneousData": [],

"productName": "OAK-D-PRO-W-POE",

"stereoEnableDistortionCorrection": true,

"stereoRectificationData": {

"leftCameraSocket": 1,

"rectifiedRotationLeft": [

  [

    0\.9998087882995605,

    0\.00805005244910717,

    -0.017819706350564957

  ],

  [

    -0.008317464031279087,

    0\.9998531341552734,

    -0.0149836465716362

  ],

  [

    0\.017696470022201538,

    0\.015128996223211288,

    0\.9997289180755615

  ]

],

"rectifiedRotationRight": [

  [

    0\.999998927116394,

    -0.0008438312215730548,

    -0.0011870115995407104

  ],

  [

    0\.0008616084815002978,

    0\.9998862743377686,

    0\.015056533738970757

  ],

  [

    0\.001174171338789165,

    -0.015057540498673916,

    0\.9998859167098999

  ]

],

"rightCameraSocket": 2

},

"stereoUseSpecTranslation": false,

"version": 7,

"verticalCameraSocket": -1

}

Rrbn23 · Oct 18, 2024

Do you have customer service? I'll need that camera for a project and it is not working right. From you there is no answer for days, so how to continue with that problem?

Suggestion from my side: I'll send you the camera and you can find the problems yourself. You will send me a new one.

jakaskerl · Oct 18, 2024

rbn23
Apologies for that, @Luxonis-Alex was not pinged

rbn23 I'll need that camera for a project and it is not working right

If you need it urgently, we can send you a replacement, and you send the problematic device to us.

Thanks,
Jaka

LLuxonis-Alex · Oct 21, 2024

Hi @rbn23 , sorry for the delay, I wasn't notified on the new replies on the topic, it appears it is not followed automatically after a reply ( @jakaskerl maybe there is a way we can enable this on the forum?)

The device in the question above is a PoE variant, but doesn't seem relevant for the issue. I was able to replicate it, and it seems it happens at low exposure (strong light, outdoors/sunlight) on some sensors, but it doesn't happen at the very minimum exposure of 1 row (that is 9.05 us). Could you try that as well to confirm the issue is no longer present on your side. To compensate for the very low exposure set, increase ISO. So for example:
.setManualExposure(10, 100)
.setManualExposure(10, 200)
.setManualExposure(10, 400)
.setManualExposure(10, 500) etc

There's a possibility it might be a driver issue as I don't see the problem present at the lowest exposure, I'm debugging more

Rrbn23 · Oct 24, 2024

I tested with manual exposure and your settings. here an example of (10, 300). There is no noise. When I enable autoExposure I get the same noisy results as before.

jakaskerl · Oct 25, 2024

rbn23
@Luxonis-Alex ping

LLuxonis-Alex · Oct 25, 2024

@rbn23 Thanks for confirming, yes we're seeing the same, a few OV9282 camera modules show a static pattern noise when the exposure is low, but not at the lowest level, and not when the exposure is higher (it's no longer noticeable in my tests for an exposure of 500us+).
We're doing more investigation on this issue.

Rrbn23 · Nov 7, 2024

any progress so far?