Depth of modular OAK-D camera before calibration?

l4es

Dear Community,

I would like to know if it is possible to retrieve the default depth image from an OAK FFC 4P + a pair of AR0234 on CAMB, CAMC (and 3rd AR0234 on CAMD + IMX477 on CAMA) without the calibration, please ?

Best Regards,
Khang

erik

Hi l4es ,
If there is no calibration, depth would be more or less just noise. Either way, you could load empty calibration to the calibrated device to "disable calibration", perhaps check Load Calibration example, and just load a new CalibrationHandler. Thoughts?

l4es

Hi erik

When trying to visualize the depth, we got :
[14442C1081F4D5D600] [9.1] [18.519] [StereoDepth(8)] [error] There is no available extrinsic calibration between camera ID: '1' and '2'

Does it mean that calibration file is missing?

Other developer got different error with the custom script :

$ python3 ex1_c2.py

Reading calib data

...done

Check that everything is nice and cool

Setup done. Some checks first

Traceback (most recent call last):

File "/home/tmatila/tommi/DepthAI/ex1_c2.py", line 80, in <module>

with dai.Device(pipeline) as device:

RuntimeError: StereoDepth(4) - StereoDepth | Invalid resolution: Width 1920, Height 1200

The script :

import depthai as dai

import cv2

import numpy as np

def getFrame(queue):

frame = queue.get()

return frame.getCvFrame()

def getMonoCamera(pipeline,isLeft):

mono = pipeline.createMonoCamera()

mono.setResolution(dai.MonoCameraProperties.SensorResolution.THE_1200_P)

if isLeft:

mono.setBoardSocket(dai.CameraBoardSocket.LEFT)

else:

mono.setBoardSocket(dai.CameraBoardSocket.RIGHT)

return mono

if __name__ == '__main__':

# THE_720_P = 1280×720

# THE_800_P = 1280×800

# THE_400_P = 640×400

# THE_480_P = 640×480

# THE_1200_P = 1920×1200

pipeline = dai.Pipeline()

monoL = getMonoCamera(pipeline,isLeft= True )

monoR = getMonoCamera(pipeline,isLeft= False)

print("Reading calib data")

with dai.Device(pipeline) as device:

calibData = device.readCalibration()

print("...done")

# ======================================================

manipResize_L = pipeline.create(dai.node.ImageManip)

manipResize_L.initialConfig.setResize(640, 400)

manipResize_L.initialConfig.setKeepAspectRatio(True)

manipResize_R = pipeline.create(dai.node.ImageManip)

manipResize_R.initialConfig.setResize(640, 400)

manipResize_R.initialConfig.setKeepAspectRatio(True)

# ======================================================

monoR.out.link(manipResize_L.inputImage)

monoL.out.link(manipResize_R.inputImage)

# ======================================================

#print("--- Set up depth and disparity ---")

stereo = pipeline.createStereoDepth()

print("Check that everything is nice and cool")

# Check occluded pixels (performance hit)

stereo.setLeftRightCheck(True)

#stereo.setLeftRightCheck(False)

manipResize_L.out.link(stereo.left)

manipResize_R.out.link(stereo.right)

xoutDepth = pipeline.createXLinkOut()

xoutDepth.setStreamName("depth")

xoutDisp = pipeline.createXLinkOut()

xoutDisp.setStreamName("disparity")

#print("--- Set up rectified images links ---")

xoutRectLeft = pipeline.createXLinkOut()

xoutRectLeft.setStreamName("rectifiedLeft")

xoutRectRight = pipeline.createXLinkOut()

xoutRectRight.setStreamName("rectifiedRight")

#print("--- Set stereo config ---")

stereo.disparity.link(xoutDisp.input)

stereo.rectifiedLeft.link(xoutRectLeft.input)

stereo.rectifiedRight.link(xoutRectRight.input)

print("Setup done. Some checks first")

with dai.Device(pipeline) as device:

# Print MxID, USB speed, and available cameras on the device

print('MxId:',device.getDeviceInfo().getMxId())

print('USB speed:',device.getUsbSpeed())

print('Connected cameras:',device.getConnectedCameras())

print("Opening image queues")

with dai.Device(pipeline) as device:

print("Update debug log level")

# Set debugging level

device.setLogLevel(dai.LogLevel.DEBUG)

device.setLogOutputLevel(dai.LogLevel.DEBUG)

print("Set queues")

qDisp = device.getOutputQueue(name="disparity" , maxSize=1, blocking =False)

qRectL = device.getOutputQueue(name="rectifiedLeft" , maxSize=1, blocking =False)

qRectR = device.getOutputQueue(name="rectifiedRight" , maxSize=1, blocking =False)

scale = 255 / stereo.getMaxDisparity()

cv2.namedWindow("Stereo Pair")

cv2.setMouseCallback("Stereo Pair",mouseCallback)

sideBySide = False

print("Start loop")

while(True):

disparity = getFrame(qDisp)

disparity = (disparity*scale).astype(np.uint8)

#disparity = cv2.applyColorMap(disparity, cv2.COLORMAP_JET)

disparity = cv2.applyColorMap(disparity, cv2.COLORMAP_TURBO)

frameL = getFrame(qRectL)

frameR = getFrame(qRectR)

if sideBySide:

imOut = np.hstack((frameL,frameR))

else:

imOut = np.uint8(frameL/2 + frameR/2)

imOut = cv2.cvtColor(imOut,cv2.COLOR_GRAY2RGB)

imOut = cv2.line (imOut, (mouseX,mouseY), (1280,mouseY), (0,0,255), 2)

imOut = cv2.circle(imOut, (mouseX,mouseY), 2, (255,255,128), 2)

cv2.imshow("Stereo Pair",imOut)

cv2.imshow("Disparity",disparity)

k = cv2.waitKey(1)

if k==ord('q'):

break

elif k == ord('t'):

sideBySide = not sideBySide

Could you instruct more, please ?

Thanks and Best Regards,
Khang

erik

Hi l4es ,
So 2 problems here:

To actually calculate depth, you need to know extrinsics (baseline distance)
On RVC2, max input stereo frames must be below 1280 pixels in width - so you would need to crop/downscale your input stereo frames.

l4es

Hi erik ,
Currently, we set the resolution of each left and right as below :
mono.setResolution(dai.MonoCameraProperties.SensorResolution.THE_1200_P)

If we set a different resolutions, there's following complains :

[18443010C1BE5B1200] [1.2] [2.715] [MonoCamera(1)] [error] AR0234 only supports THE_1200_P resolution, defaulting to it
[18443010C1BE5B1200] [1.2] [2.716] [MonoCamera(0)] [error] AR0234 only supports THE_1200_P resolution, defaulting to it

The program threw exception at following line :

with dai.Device(pipeline) as device:
# Print MxID, USB speed, and available cameras on the device

In the example of oak-d lr, there's setIspScale() :

left.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1200_P)
left.setCamera("left")
left.setIspScale(2, 3)

But setIspScale() seems not available in my version. So I would like to know how to configure the correct scaled input stereo frames ?

Best regards,
Khang

erik

Hi l4es ,
Instead of

left = pipeline.create(dai.node.MonoCamera)
# Do:
left = pipeline.create(dai.node.ColorCamera)

And then you can use the ISP scale option. Other option would be to use ImageManip to downscale by 2/3, or to use Camera node (which supports both mono and color). Thoughts?