Multi-cam calibration to calculate extrinsics but how to calculate intrinsics?

Nearpoint

We have a unstandard setup so we are not using the standard luixonis calibration tool that is designed for color camera in between two stereo cameras.

So we need to calculate the cameras extrinsics and intrinsics manually for each camera.

We are utilizing this tool to calculate the extrinsics - luxonis/depthai-experimentstree/master/gen2-multiple-devices/multi-cam-calibration

But if we use that tool, then what do we do to calculate the intrinsics for each camera?

Nearpoint

I believe I can set the intrinsics using the calibration_handler like so:

calibration_handler.setDistortionCoefficients(stringToCam[camera], cam_info['dist_coeff'])
calibration_handler.setCameraIntrinsics(stringToCam[camera], cam_info['intrinsics'],  cam_info['size'][0], cam_info['size'][1])
calibration_handler.setFov(stringToCam[camera], cam_info['hfov'])

And to get the intrinsic parameters to set I can use the following function from calibrate utils

def calibrate_intrinsics(self, image_files, hfov):
        image_files = glob.glob(image_files + "/*")
        image_files.sort()
        assert len(
            image_files) != 0, "ERROR: Images not read correctly, check directory"

        allCorners, allIds, _, _, imsize, _ = self.analyze_charuco(image_files)

        coverageImage = np.ones(imsize[::-1], np.uint8) * 255
        coverageImage = cv2.cvtColor(coverageImage, cv2.COLOR_GRAY2BGR)
        coverageImage = self.draw_corners(allCorners, coverageImage)

        if self.cameraModel == 'perspective':
            ret, camera_matrix, distortion_coefficients, rotation_vectors, translation_vectors = self.calibrate_camera_charuco(
                allCorners, allIds, imsize, hfov)
            # (Height, width)
            if self.traceLevel == 4 or self.traceLevel == 5 or self.traceLevel == 10:
                self.undistort_visualization(
                    image_files, camera_matrix, distortion_coefficients, imsize)

            return ret, camera_matrix, distortion_coefficients, rotation_vectors, translation_vectors, imsize, coverageImage
        else:
            print('Fisheye--------------------------------------------------')
            ret, camera_matrix, distortion_coefficients, rotation_vectors, translation_vectors = self.calibrate_fisheye(
                allCorners, allIds, imsize, hfov)
            if self.traceLevel == 4 or self.traceLevel == 5 or self.traceLevel == 10:
                self.undistort_visualization(
                    image_files, camera_matrix, distortion_coefficients, imsize)
            print('Fisheye rotation vector', rotation_vectors[0])
            print('Fisheye translation vector', translation_vectors[0])

            # (Height, width)
            return ret, camera_matrix, distortion_coefficients, rotation_vectors, translation_vectors, imsize, coverageImage

Is this correct?

Then after I set the intrinsics I would then use the multi cam calibration to calculate the pose of each camera and generate the extrinsics?

luxonis/depthai-experimentstree/master/gen2-multiple-devices/multi-cam-calibration

jakaskerl

Hi Nearpoint
You can use CV2 approach to calculating the intrinsic parameters of the camera, or you can use the function you posted. Should work either way I think.

Nearpoint Then after I set the intrinsics I would then use the multi cam calibration to calculate the pose of each camera and generate the extrinsics?

Correct. Though if you use compact cameras (OAK-D, OAK-1..) then intrinsic calibration should not be needed.

Thanks,
Jaka