• DepthAI
  • OAK-D SR color weird while UVC mode

Hi... I tried out the instruction (link as https://docs.luxonis.com/projects/api/en/latest/samples/UVC/uvc_rgb/#) by using my OAK-D SR camera with slightly modification (CameraBoardSocket.CAM_A changed to CAM_B and resolution changed to THE_800_P or (1280, 800) where my hardware configuration is raspberry pi 5 + ubuntu Desktop 24.04 LTS. Then used test code to display frames output from UVC camera and it can be shown correctly. But the color is weird (seems one color missed). Already tried to set up color order as BGR (comply with openCV format) but it still kept weird. Any insights or assistance on this would be greatly appreciated.

Thanks in advance,

Robert Huang

Hi @RobertH
The frame type has to be NV12, otherwise it doesn't work.
Could you test the image using cam_test.py or rgb_preview.py? Maybe it's a sensor issue.

Thanks,
Jaka

Hi @jakaskerl
Yes, the sample code used NV12 frame type (listed complete code as below where the modifications are ones starting with Robert modification). Following your instruction by running cam_test.py, i snapshot these different outputs compared to real environment (iPhone 12 Pro Max rear camera) as comparison snapshot link. Could you please help give more clues? Thanks in advance.

parser = argparse.ArgumentParser()
parser.add_argument('-fb', '--flash-bootloader', default=False, action="store_true")
parser.add_argument('-f',  '--flash-app',        default=False, action="store_true")
parser.add_argument('-l',  '--load-and-exit',    default=False, action="store_true")
args = parser.parse_args()

if args.load_and_exit:
    import os
    # Disabling device watchdog, so it doesn't need the host to ping periodically.
    # Note: this is done before importing `depthai`
    os.environ["DEPTHAI_WATCHDOG"] = "0"

import depthai as dai

def getPipeline():
    enable_4k = False  # Will downscale 4K -> 1080p

    pipeline = dai.Pipeline()

    # Define a source - color camera
    cam_rgb = pipeline.createColorCamera()
    cam_rgb.setBoardSocket(dai.CameraBoardSocket.CAM_B) # (Robert modification) changed to CAM_B
    cam_rgb.setInterleaved(False)
    #cam_rgb.initialControl.setManualFocus(130)

    if enable_4k:
        cam_rgb.setResolution(dai.ColorCameraProperties.SensorResolution.THE_4_K)
        cam_rgb.setIspScale(1, 2)
    else:
        cam_rgb.setResolution(dai.ColorCameraProperties.SensorResolution.THE_800_P) # (Robert modification) THE_1080_P changed to THE_800_P

    # Create an UVC (USB Video Class) output node
    uvc = pipeline.createUVC()
    cam_rgb.video.link(uvc.input)

    # Note: if the pipeline is sent later to device (using startPipeline()),
    # it is important to pass the device config separately when creating the device
    config = dai.Device.Config()
    # config.board.uvc = dai.BoardConfig.UVC()  # enable default 1920x1080 NV12
    config.board.uvc = dai.BoardConfig.UVC(1280, 800) # (Robert modification) (1920, 1080) changed to (1280, 800)
    config.board.uvc.frameType = dai.ImgFrame.Type.NV12
    # config.board.uvc.cameraName = "My Custom Cam"
    pipeline.setBoardConfig(config.board)

    return pipeline

# Will flash the bootloader if no pipeline is provided as argument
def flash(pipeline=None):
    (f, bl) = dai.DeviceBootloader.getFirstAvailableDevice()
    bootloader = dai.DeviceBootloader(bl, True)

    # Create a progress callback lambda
    progress = lambda p : print(f'Flashing progress: {p*100:.1f}%')

    startTime = time.monotonic()
    if pipeline is None:
        print("Flashing bootloader...")
        bootloader.flashBootloader(progress)
    else:
        print("Flashing application pipeline...")
        bootloader.flash(progress, pipeline)

    elapsedTime = round(time.monotonic() - startTime, 2)
    print("Done in", elapsedTime, "seconds")

if args.flash_bootloader or args.flash_app:
    if args.flash_bootloader: flash()
    if args.flash_app: flash(getPipeline())
    print("Flashing successful. Please power-cycle the device")
    quit()

if args.load_and_exit:
    device = dai.Device(getPipeline())
    print("\nDevice started. Attempting to force-terminate this process...")
    print("Open an UVC viewer to check the camera stream.")
    print("To reconnect with depthai, a device power-cycle may be required in some cases")
    # We do not want the device to be closed, so terminate the process uncleanly.
    # (TODO add depthai API to be able to cleanly exit without closing device)
    import signal
    os.kill(os.getpid(), signal.SIGTERM)

# Standard UVC load with depthai
with dai.Device(getPipeline()) as device:
    print("\nDevice started, please keep this process running")
    print("and open an UVC viewer to check the camera stream.")
    print("\nTo close: Ctrl+C")

    # Doing nothing here, just keeping the host feeding the watchdog
    while True:
        try:
            time.sleep(0.1)
        except KeyboardInterrupt:
            break

Hi @RobertH
And could you screenshot the UVC image you are getting? I tested your code (removed flashing since it shouldn't be done) and it works as expected.

Thanks,
Jaka

Hi @RobertH
Sorry, didn't see the second image was UVC. It kinda looks like the wrong color order.

You can do frame = cv2.cvtColor(frame, cv2.COLOR_RGB2BGR) to fix this in the video cap script.

Thanks,
Jaka

Hi @jakaskerl

After applying cv2.cvtColor() the color is correct. If we'd like to convert color in OAK VPU internally (supposed to get a higher frame rate on host side, right?), is there any alternate for this target? Any sample code for sharing? Thanks in advance.

BR, Robert Huang

Hi @RobertH,
Yes, in general, you would set the color order for preview output using cam_rgb.setColorOrder(dai.ColorCameraProperties.ColorOrder.RGB or BGR).

Let me know if that works.

Thanks,
Jaka