OAK-1 image pixels are displaced

I am using depthai-core v2.20.2 and I realized that the images have some displaced pixels (as seen in the image)

I have tested with three different cameras and I have a similar effect, just that the displaced pixels regions are not always the same. Do you have any clue of how to solve this issue or what is the cause? I am using depthai in a C++ application.

The distance at which the image was taken is around 45 cm and the displaced pixels zones are normally found on the right side of the image frame.

erik Yes Erik, it is the standard OAK-1 autofocus. The resolution is 4k, and the FPS tested are 20 fps when debugging and 30 when using the release version of the application. I have checked by saving the image in the release version and with a breakpoint when debugging, inspecting directly with image watch from Visual Studio 2019, to verify it was not the saving method the reason.

erik Sure:

To recover the frame I use:

`if (auto result = camera->TryGetFrame()) {

        cv::Mat frame = result.value();\`

And to save it I use:

`bool CaptureCalibImage(cv::Mat frame, int f_value) {

assert(!frame.empty());

if (count_calib_captures == 0) {

    CreateSaveDirectory(dataPath::calibration);

}

char str[200];

sprintf(str, "calib_%04d_Focus_%04d_ISO_%04d_SS_%04d_YofYUV.png", count_calib_captures, lens_position,

        iso, shutter_speed);

bool isSaved = cv::imwrite(save_calibration_path + str, frame);

if (isSaved) {

    count_calib_captures++;

    capture_calib = false;

}

return isSaved;\`

erik I've tried the same code but it was not working properly since on my side it only showed the first frame and then the image feed stopped. However, I tried with the depthai-python examples/ColorCamera/rgb_camera_control.py and the captures did seem to avoid this effect. The thing is that I am using an application in C++ and I would like to verify if the images obtained from the feed are not having this artifact. So, I will modify the saving method using the still frame but for c++ to see if that resolves the issue for the image capture, but there is still the issue of this artifact appearing even without saving, reading the image directly in the pipeline generate the mentioned effect.

to create my camera pipeline I use this code:

<script src=">

and this is the code used to get the image frame:

<script src=">

My application uses the images from the camera on real time, to estimate the pose of aruco markers, and even if a solve the saving issue I still have the artifact while runningn on real time the detection and pose estimation algorithms, and since the position of my markers corners are displaced then my pose will be displaced, increasing the pose estimation error.

I have been using this camera since january 2022 and this artifact was noticed last month, and the only thing that I have changed since last month is an update from depthai version 2.17.4 to 2.20.2. Do you think this is related to the hardware of the camear itself ?

Hello erik,

It took a while but I've managed to replicate the artifact with the following code.

<script src=">

You should look for the two saved images in line 171 and in line 189. This time I was using depthai version 2.20.2. I use the ImageManip to avoid loading the whole 3 image channels into the buffer due to frame rate. This step is key since I use the OAK-1 camera connected to an Android Device and if I don't do this the frame rate goes from 20 (taking only one channel from imgmanip node) to 10(taking the 3 channels image from isp).

I was not able to use the version 2.17.4 in python. But I've done the same experiment with the 2.17.4 in C++ and the artifact is less evident.

This is the complete image in case the scale is not clear in the first image. The artifact appears normally in the right side of the image as you can see.

I hope that you can help me with this issue. Thank you for your attention.

Hi erik, here is the same code without the unnecessary controls, but the one to save the image when the 'C' key is pressed. This is the minimum I can do since reducing something else from the code will not be representative or enough to reproduce the issue. This time you should look for the saved images in line 92 and 108 to verify the presence of the artifact when using the ImageManip node. The given description in my last message is still the same.

<script src=">

I hope that you can help me.

erik the one from line 108, which came from the ImageManip node, called 'gframe'. The image from line 92 is the one that come directly from the still encoder.

erik Yes, the artifact it is only produced when using the ImageManip node. About the ImageManip limitations I do not surpass the width of the image (3840 pixels which is also a multiple of 16). I've tried also with the GRAY8 image type and the artifact also appears with this configuration.

I've tested with the following versions of depthai-core:

• 2.17.4
• 2.18.0
• 2.19.1
• 2.20.2
• 2.21.2

The artifact appears in all the versions but for the 2.17.4. In this version the artifact is almost imperceptible, but it does appear when looking carefully (more evident in an image with checkerboard).

To give a little of background, I was working with the 2.17.4 since at that time it was the available release version, so thats it is why I have not notice this artifact. I was intending to update the depthai-core version for a new one to be able to recover the metadata from each image (lensPosition, ISO, exposureTime), and it may be related that from version 2.18.0 to 2.21.2 it is possible to obtain the imaga metada, but the artififact is also more evident for those versions.

Hello @erik ,

I would like to know if there has been an attempt to solve this issue, do you think it would be possible to correct it ?

Thank you, Alexander

erik You are right, I should have done that. I was waiting for an answer from your side and I have just thouhgt it was not possible.

robotaiguy As explained before, it is an OAK-1 AF