Details in RGB cam of Oak-D Pro blurry

blank

We're currently trying to do protogrammetry with an Oak-D Pro (FF) mounted to a drone. We fly slowly above the area to be mapped, take a picture every few metres and use these pictures to generate a height model. For this algorithm to work, the pictures need to be very clear and detailed.

Unfortunately, the pictures taken by the Oak-D's color camera tend to get quite blurry in the finer details. The following picture was taken with a stationary camera out of my window to exclude any possible vibrations through gimbal or drone.

Notice how the sharpness degrades along the length of the building and that the finer tree structures in the back tend to get very blurry? I may seem to exaggerate here, but this is unfortunately the quality the photogrammetry algorithm demands…

I turned sharpness all the way up and disabled denoising completely, as the algorithm deals better with noise than with blur. This is the test script I used:

import depthai as dai
import time
import os

if __name__ == '__main__':
    
    # set image directory here
    workDir = "/home/foo/img"
    
    pipeline = dai.Pipeline()
    
    camRgb = pipeline.create(dai.node.ColorCamera)
    camRgb.initialControl.setSharpness(4)     # range: 0..4, default: 1
    camRgb.initialControl.setLumaDenoise(0)   # range: 0..4, default: 1
    camRgb.initialControl.setChromaDenoise(0) # range: 0..4, default: 1
    camRgb.setBoardSocket(dai.CameraBoardSocket.CAM_A)
    camRgb.setResolution(dai.ColorCameraProperties.SensorResolution.THE_12_MP)
    
    xin = pipeline.create(dai.node.XLinkIn)
    xin.setStreamName("control")
    xin.out.link(camRgb.inputControl)
    
    # Properties
    videoEnc = pipeline.create(dai.node.VideoEncoder)
    videoEnc.setDefaultProfilePreset(1, dai.VideoEncoderProperties.Profile.MJPEG)
    camRgb.still.link(videoEnc.input)
    
    # Linking
    xoutStill = pipeline.create(dai.node.XLinkOut)
    xoutStill.setStreamName("still")
    videoEnc.bitstream.link(xoutStill.input)
    with dai.Device(pipeline) as device:
        qStill = device.getOutputQueue(name="still", maxSize=30, blocking=True)
        qControl = device.getInputQueue(name="control") 
        while True:
            ctrl = dai.CameraControl()
            ctrl.setCaptureStill(True)
            qControl.send(ctrl)

            while True:
                if qStill.has():
                    file_name = f"{int(time.time() * 1000)}.jpeg"
                    print("Taking picture...")
                    with open(os.path.join(workDir, file_name), "wb") as f:
                        f.write(qStill.get().getData())
                        print('Image saved to', file_name)
                    break

Is there anything I can do to improve my images?

Also let me say: I realize that, nowadays, we tend to take stunning image quality for granted as every cheap smartphone is equipped with professional grade sensors, so maybe I'm asking too much and this is simply the best this sensor can deliver. If so, could we gain anything from using an Oak-1 Max instead?

Thanks in advance

jakaskerl

Hi blank
Are you sure you are using a FF camera? Could you try the mono cameras (which should be FF) to see if the effect is the same? I think there is nothing you can do software-wise since this is an optics issue.
test:

import depthai as dai
with dai.Device() as device:
    print(device.getConnectedCameraFeatures())

Thanks,
Jaka

blank

jakaskerl I'm a little confused right now.

Running your code, I get the following output:

{
   "socket":"RGB/CENTER/CAM_A",
   "sensorName":IMX378,
   "width":4056,
   "height":3040,
   "orientation":"AUTO",
   "supportedTypes":[
      "COLOR"
   ],
   "hasAutofocus":1,
   "name":"color"
},
{
   "socket":"LEFT/CAM_B",
   "sensorName":OV9282,
   "width":1280,
   "height":800,
   "orientation":"AUTO",
   "supportedTypes":[
      "MONO"
   ],
   "hasAutofocus":0,
   "name":"left"
},
{
   "socket":"RIGHT/CAM_C",
   "sensorName":OV9282,
   "width":1280,
   "height":800,
   "orientation":"AUTO",
   "supportedTypes":[
      "MONO"
   ],
   "hasAutofocus":0,
   "name":"right"
}

If I interpret this correctly, it claims that the RGB camera indeed has autofocus ("hasAutofocus":1). However,

I bought a FF camera,
the reciept and original packaging of the camera say "Fixed Focus"
the camera was successfully operated on a drone (although on a gimbal with vibration dampeners)
other cameras of the same type, which also say "FF", show the same behaviour and yield the same output to your code.

Could it really be that I accidentially got several autofocus cams instead of FF ones?

Also, this rises a side question: Are using autofocus cameras on drones really such a big issue? I always had this video in mind, but on hindsight, this guy uses neither a gimbal nor any kind of vibration dampeners.

jakaskerl

Hi blank
That would mean your camera is AF, yes. Looks like there was a mixup.
I'd advise you to recheck with https://docs.luxonis.com/projects/api/en/latest/samples/ColorCamera/rgb_camera_control/ to see if you can enable autofocus (F key).

The vibration is bad for AF cameras, yes. On the other hand, you can enable the autofocus which should improve the image you have sent above.

blank the camera was successfully operated on a drone (although on a gimbal with vibration dampeners)

Have you noticed any issues? If there are no/minimal vibrations, the AF camera should have no issues.

Thanks,
Jaka

blank

jakaskerl Have you noticed any issues? If there are no/minimal vibrations, the AF camera should have no issues.

No, I haven't. The aerial pictures look the same as the one I sent you, which was shot with a completely stationary camera.

jakaskerl I'd advise you to recheck with https://docs.luxonis.com/projects/api/en/latest/samples/ColorCamera/rgb_camera_control/ to see if you can enable autofocus (F key).

The commands all complete without exceptions. However, no command (t, f or even setting the AutoFocusMode to OFF) seems to have any effect on the picture. There is no visible focus adjustment.

Also, I noticed that the right part of the picture is much more blurry than the left. Looks like this one additionally suffers from a misaligned lens?

erik

Hi blank ,
As blurriness is barely visible, I suspect the lens itself is fixed (as it should be), but as the whole camera is moving a tiny bit (due to drone vibrations or drone movement), you are essentially having motion blur. If it's due to:

vibrations (camera moving into every direction), you could consider using anti-vibration mount or something similar
drone movement, the only option would be to either have drone "perfectly static" during the exposure, or lower exposure time to reduce motion blur.

Thoughts?
Thanks, Erik

blank

erik The camera that did the example shot was static, I glued it to the window with double sided tape. Vibrations also were my first suspects. Tests with a smaller expore

Again: Maybe I'm expecting too much from a 12MP machine vision camera and that's simply the best it can do. It would still be a good picture for most applications, but I'd have to look for another camera if I want to do photogrammetry.

erik

Hi blank ,
You can also configure ISP settings (sharpness, chroma/luma denoise), which could provide better IQ, I'd suggest checking the thread here:
https://discuss.luxonis.com/d/554-image-pipelines-compression-and-filtering/8