I have a pair of AR0234 M12 modules I'm using for the left and right cameras on my robot, I would like to use wide angle lenses on these and I already have some from other projects I can use. The problem is I've no HFOV details for them, and when I've tried to calculate this in the past I've not been able to calibrate them. The other issue is that they don't fill the frame so I've parts of the sensor that aren't being used. An example is below, you can see the black banding around the edge of the preview along with the disparity image. [upl-image-preview url=https://discuss.luxonis.com/assets/files/2023-09-06/1694001843-35034-image.png] Are there recommended lenses for this sensor? If not, is there a way to calibrate without a known HFOV? I know there's a process on ROS to calibrate without this info but no idea how to port it to this camera. Could I just publish image messages from the uncalibrated camera and use the results of it to put the calibration data manually on the FFC-3P? If anyone can give any advice here I'd appreciate it. I'm still new to computer vision and a lot of the documentation and advice is aimed at more experienced users, I'm struggling to understand a lot of it if I'm honest but slowly learning more with help!

Thank you for the feedback @"kneave"#p11755 , it's greatly appreciated! Let us know if something doesn't work as expected:)

Hi @"kneave"#p11796 , Perhaps relevant PR for you: https://github.com/luxonis/depthai/pull/1111 Best to continue conversation about issues/challenges there, as I am not very familiar with the calib script:)

M12 Lenses Recommendation/Unknown HFOV

kneave

I have a pair of AR0234 M12 modules I'm using for the left and right cameras on my robot, I would like to use wide angle lenses on these and I already have some from other projects I can use. The problem is I've no HFOV details for them, and when I've tried to calculate this in the past I've not been able to calibrate them. The other issue is that they don't fill the frame so I've parts of the sensor that aren't being used.

An example is below, you can see the black banding around the edge of the preview along with the disparity image.

Are there recommended lenses for this sensor?
If not, is there a way to calibrate without a known HFOV? I know there's a process on ROS to calibrate without this info but no idea how to port it to this camera.
Could I just publish image messages from the uncalibrated camera and use the results of it to put the calibration data manually on the FFC-3P?

If anyone can give any advice here I'd appreciate it. I'm still new to computer vision and a lot of the documentation and advice is aimed at more experienced users, I'm struggling to understand a lot of it if I'm honest but slowly learning more with help!

jakaskerl

Hi kneave
I believe you would need to use a specific calibration designed for fish-eye lenses. I don't think we have it implemented in the API as of now. You could try with the standard calibration but the results could be unsatisfactory.
AFAIK the FOV parameter is ignored when performing calibration. The script itself should calculate the FOV, again not sure how well that would perform on the fisheyes.

Thanks,
Jaka

kneave

Thanks Jaka, I'm reading the updated calibration scripts now and there's a bunch of references to fisheye lenses but still getting my head around it all. I'm not even sure if mine count as fisheye, I'm not sure where wide-angle and fish-eye overlap.

kneave

For anyone else heading down this rabbit hole, the difference between a fisheye and wide angle lens is that the wide angle is literally just that. Wide angle gives a much wider view along the horizontal, a fish eye gives a wider view of all angles. I suspect all "wide-angle" M12 lenses count as fish-eye in that case as you have to screw them in, there's no horizontal for them to orient across.

For calibration, it looks as though there is a fisheye option that's been added in the last few days that does indeed ignore the HFOV value! I'm going to try it with the new noInitCalibration option and hope for the best.

kneave

I've spent some time with the fisheye calibration mode this evening and I have to say I'm impressed with the improvements to the calibration script if nothing else!

Also, having lights around the eyes is proving handy for illuminating the charuco board! 🤣

erik

Thank you for the feedback kneave , it's greatly appreciated! Let us know if something doesn't work as expected🙂

kneave

erik No worries! One thing that doesn't seem to work for fisheye mode is minDetectedMarkersPercent, I was having trouble getting all the markers seen at once due to the fisheye warping things (as expected). I set mdmp to 0.75 expecting it to need 33 markers but it still required all 44 from my board.

Another thing I had to disable was cv2.fisheye.CALIB_CHECK_COND in calibration_utils.py:567. I think it checks for chessboards that are over the edge of the images, with fisheye that's an easy one to run foul of as above, but I couldn't seem to get good enough images with it set without losing coverage near the edges of the lenses curve.

I haven't finished calibrating yet as it got late and the script was taking a while to finish. I'll have another go this eve though.

erik

Hi kneave , Perhaps relevant PR for you: luxonis/depthai1111
Best to continue conversation about issues/challenges there, as I am not very familiar with the calib script🙂

kneave

It seems I circumvented that issue and hit a bunch more 😅
I'll create an issue once I understand what I'm asking 🙂