HI alexv, so which output (of the two) produces artifacts on your side?

      erik the one from line 108, which came from the ImageManip node, called 'gframe'. The image from line 92 is the one that come directly from the still encoder.

      • erik replied to this.

          Hi @alexv , I simplified this to 55 LOC, and can still confirm an issue. I have sent it over to FW engineers.

          #!/usr/bin/env python3

import depthai as dai
import cv2
import numpy as np

# Create pipeline
pipeline = dai.Pipeline()

# Define sources and outputs
camRgb = pipeline.create(dai.node.ColorCamera)
camRgb.setResolution(dai.ColorCameraProperties.SensorResolution.THE_4_K)
camRgb.setInterleaved(False)
#camRgb.setIspScale(3,3) # 1080P -> 720P
camRgb.setColorOrder(dai.ColorCameraProperties.ColorOrder.RGB)
camRgb.setImageOrientation(dai.CameraImageOrientation.NORMAL)

manipVideo = pipeline.create(dai.node.ImageManip)
manipVideo.setMaxOutputFrameSize(3840*2160)
manipVideo.initialConfig.setFrameType(dai.ImgFrame.Type.YUV400p)


ispOut = pipeline.create(dai.node.XLinkOut)
outVideo = pipeline.create(dai.node.XLinkOut) #########

outVideo.setStreamName('gray')  #########
ispOut.setStreamName('isp')

camRgb.isp.link(ispOut.input)
camRgb.isp.link(manipVideo.inputImage)   #########
manipVideo.out.link(outVideo.input)      #########

# Connect to device and start pipeline
with dai.Device(pipeline) as device:

    ispQueue = device.getOutputQueue('isp')
    grayQueue = device.getOutputQueue('gray',3,False) #########

    while True:
        ispFrames = ispQueue.tryGetAll()
        for ispFrame in ispFrames:
            f = ispFrame.getCvFrame()
            cv2.imshow('isp', cv2.pyrDown(cv2.pyrDown(f)))

        grayFrames = grayQueue.tryGetAll()
        for grayFrame in grayFrames:
            gframe = np.array(grayFrame.getData(),np.uint8).reshape(2160,3840)

        # Update screen (1ms pooling rate)
        key = cv2.waitKey(1)
        if key == ord('q'):
            break
        elif key == ord('c'):
            cv2.imwrite("2174_img_manip.png",gframe)

          alexv Seems like only imagemanip to YUV400 produces these artufacts, not the still frame from encoder. Could you confirm? Note also ImageManip's limitations - yuv400 isn't officially supported by ImageManip. Could you try to repro with the supported type as well?

              erik Yes, the artifact it is only produced when using the ImageManip node. About the ImageManip limitations I do not surpass the width of the image (3840 pixels which is also a multiple of 16). I've tried also with the GRAY8 image type and the artifact also appears with this configuration.

              I've tested with the following versions of depthai-core:

              • 2.17.4
              • 2.18.0
              • 2.19.1
              • 2.20.2
              • 2.21.2

              The artifact appears in all the versions but for the 2.17.4. In this version the artifact is almost imperceptible, but it does appear when looking carefully (more evident in an image with checkerboard).

              To give a little of background, I was working with the 2.17.4 since at that time it was the available release version, so thats it is why I have not notice this artifact. I was intending to update the depthai-core version for a new one to be able to recover the metadata from each image (lensPosition, ISO, exposureTime), and it may be related that from version 2.18.0 to 2.21.2 it is possible to obtain the imaga metada, but the artififact is also more evident for those versions.

                2 months later

                Hello @erik ,

                I would like to know if there has been an attempt to solve this issue, do you think it would be possible to correct it ?

                Thank you, Alexander

                • erik replied to this.

                  Hi alexv , to get updated on bug fixes I'd suggest posting an issue for it on depthai-core repo on github.

                      erik You are right, I should have done that. I was waiting for an answer from your side and I have just thouhgt it was not possible.

                        4 months later

                        I can confirm that I can still produce these artifacts in the current version of depthai. it is easiest to see this if you setup a "frame forwarding" application that uses script node to oscillate between using an imagemanip node for yuv400 and an imagemanip node for yuv420, keeping as much as possible the same. to mitigate any smoothing effect that my eyes/brain might do (because my old eyes often play tricks on me), I reduced FPS to 1 and used an even/odd modulo operator to determine which imagemanip to use.
                        If you do this, you will see that the offset shift does not appear to be linearly distributed throughout the image. Some objects in view are offset more, and some hardly seem to be offset at all. My colleague and I were looking at this a couple weeks ago, and I can't recall if we determined that it was dark-edged objects that appeared to shift, or if it was something else. At my next opportunity, I'll setup a MRE with frame-forwarding.

                          robotaiguy Hello, in my particular use case it is not possible to combine both ImageManip nodes since the hardware configuration I use do not support YUV420 at least at 20 fps, that is the reason I use YUV400 or GRAY8.

                          In my use case I identify the corners of an object in the image to perform high precision pose estimation (or at least as high as the image allows it) so a shift of the pixels positions (no matter if it is not systematic in the same place) creates an error that should not exist, degrading the quality of the system in general when it is known that an image without such artifact can achieve an expected precision when estimating the pose of an object.

                              6 months later