1) released depthai library does not support HDR (waiting for an answer on another thread)
2) color is brownish compared to reality. IMX378 is more realistic. See attached image (IMX582 on the right). I'm sure this is not ideal for the NN model
3) autofocus/autoexposure/autowhite cause issues every second in certain lighting conditions. See attached movie

We need to better understand if the above can be fixed, any help?

screen-recording-2023-12-29-at-160102.mov
7MB

I've been doing considerable research on this over the past week, myself. And unfortunately, I was reminded of something I learned in Physics - Optics class about a phenomenon called "Diffraction Limit" and "Airy Disks".

Generally speaking, the diffraction limit is reached when the Airy disc exceeds 2 times the pixel pitch of the sensor. But it's also variable by the wavelength of the light, so most calculations I see seems to use a 520nm green light for examples. I wonder if we use use a lens assembly with a larger aperture, if it would help with some of our image issues. On that topic, I think that the more greenish tint in the IMX582 images has more to with this diffusion-limitation exacerbated by the RGGB pattern from the Bayer filter...with twice as many green pixels in the pattern, and the diffusion limit hits first for green to red spectrum. Cool light filtering to lean more towards blue light should also help.

But please do your homework on what I'm remembering. There's some good articles available.

Hi @Thor ,
Have you already tried manually setting the white-balance setting of the sensor?
Thanks, Erik

No I have not, as I'm not sure if manually setting things is going to mess up the auto parameters. If I'm not mistaken you have autofocus, Auto white balance Auto exposure
Bottom line, keep in mind the OAK is unsupervised and will have to work 24/7 in all outdoor lighting conditions. Can you please provide guidance about the following:

  • if we manually provide white balance as a baseline (??), is it going to auto change it depending on external view/light? We don't want to have to change it for every OAK we deploy
  • we don't want to lose the auto exposure, as the oak has to adjust day/night automatically with the best exposure possible. Maximum exposure should be compatible with 20 frames/sec, is there a max limit that can be set for auto? What does lock auto exposure do?
  • what is the ISO setting for auto exposure? if we provide a certain ISO setting, is the autoexposure going to work and how is it possible to know the ranges it is going to use?
  • I still need an explanation about the fluctuations shown on the video that I attached to the post, and info about how to avoid the occurrence of such a thing

I've been experimenting with the gen2-color-isp-raw program
What the below parameters are for, how do they work (there is no information on the dephtai API docs)?
ae compensation
ae lock
awb lock
(
ae_comp = 0 # Valid: -9 .. +9
ae_lock = False
awb_lock = False
)

Some manual settings seem to improve on the auto settings at night, however again there is the question about how to specify eg a baseline ISO (eg 800) while having the exposure to auto change up to 50ms, or vice versa setting the autoexposure to 50ms and have the software to auto select the best ISO

What do the below parameters do?

I look forward to a reply. Thanks

    Hi Thor
    Regarding the first post in the discussion:

    For the colors, unfortunately we don't have a proper tuning, and reuse IMX378 tuning. We would need to manually create a proper tuning.
    For that slow variation issue with AE, it's unexpected, can check further if possible to reproduce, need camera settings and approximate exposure time for the scene

    Thor What do the below parameters do?

    Check the reference for descriptions: https://docs.luxonis.com/projects/api/en/latest/components/messages/camera_control/#reference

    Thanks,
    Jaka

        OK. The reference descriptions are sometimes not that useful.
        Let me ask:
        setAutoExposureLock(self: depthai.CameraControl, lock: bool) → depthai.CameraControl
        Set a command to specify lock auto exposure

        Lock to what? A region, a value, a something?

          Hi Thor
          The lock is set using a boolean, which essentially locks the exposure to the value at which it currently is (value perhaps set using autoexposure). This way, we don't need to know the value of the exposure. Same thing for AWB.

          Thanks,
          Jaka