Hi, I've recently purchased an OAK-FFP-3P along with three Arducam AR0234 camera modules and I'm having difficulty with pretty much all the examples, I'm entirely expecting this to be user error and figure there's something fundamental I've missed. If I write my own simple scripts using bits from various examples I can get a list of the three camera modules and they are recognised fine, I can even run one of the RGB examples that shows the source and cropped image, but I can't get IMU data or seem to get images from the left or right cameras. Is there an idiots guide to getting started with the 3P board please? I'm a software engineer with a lot of experience but still relatively new to computer vision, if anyone could give me pointers to get started I'd appreciate it. Photo for attention, this is the bot I'll be putting the 3P in to once I've got it working :) [URL=https://i.imgur.com/rMM2Oqg.jpg][IMG]https://i.imgur.com/rMM2Oqgh.jpg[/IMG][/URL]

Hi @"kneave"#p5209 , I think most demos don't work because you have 3x ColorCamera, and many demos use MonoCamera depthai nodes. After [OAK FFC calibration ](https://docs.luxonis.com/projects/hardware/en/latest/pages/guides/calibration.html#modular-cameras-calibration)you can also do stereodepth, [demo here](https://raw.githubusercontent.com/alex-luxonis/depthai-python/e1c02e070d8ad3073c2dfbda7c8be45814d49999/examples/StereoDepth/rgb_depth_aligned.py), which also shows how to use multuple ColorCamera nodes (instead of MonoCamera nodes). Thoughts? And that robot looks really cool!! Would love to see picture with the OAK FFC on there:) Thanks, Erik

Ah! I thought that mono would just return a mono image, I didn't realise that it would also require or expect a mono camera. This all looks ideal, I'll give it a go today. Thanks for the tips! The plan is for wide angle colour cameras for the stereo pair and a narrower lens on the centre camera. I figure that'll give a good combination for SLAM and a more zoomed in camera for distance viewing. I'm hoping to find a small camera for the centre so that I can hide it behind the panel between his eyes and not spoil the aesthetic :)

I've switched to and built the develop branch of depthai_python, I saw that the multi_cam_support branch was merge in a few weeks ago, and I've gone through calibration.py from the depthai library and modified it for three colour cameras. I'm really struggling to more than a few calibration images accepted by the calibration script, none at all unless I disable the RGB sensor, could this be because I'm using fisheye lenses? The left and right camera have 160 degree lenses and the RGB has a 90 degree lens, do I have to do anything different to enable a fisheye mode? I know that's something I've needed to do with ROS before. Here is the calibration file I created in case it helps. `{ "board_config": { "name": "NE3P", "revision": "V1.0", "swap_left_and_right_cameras": false, "left_fov_deg": 160, "rgb_fov_deg": 90, "left_to_right_distance_cm": 8.0, "left_to_rgb_distance_cm": 4.0 } }` Any tips would be welcome, I'm happy I'm at least getting images now but it's still a little frustrating.

Hi @"kneave"#898, Try to calibrate using this branch (https://github.com/luxonis/depthai/tree/ar0234_x3), you have to add a custom board config to work, try adding something like this in depthai/resoruces/boards/FFC-3P.json, and then be sure to select the board while doing the calibration for example python3 calibrate.py -brd FFC-3P. Be sure that you use the HFOV, not DFOV or something else, and check if the left/right/rgb cameras are in the right place (change the CAM_A/B/C ports as needed) ``` { "board_config": { "name": "NE3P", "revision": "R1M0E1", "cameras":{ "CAM_B": { "name": "right", "hfov": 160, "type": "color", "extrinsics": { "to_cam": "CAM_C", "specTranslation": { "x": 8, "y": 0, "z": 0 }, "rotation":{ "r": 0, "p": 0, "y": 0 } } }, "CAM_C": { "name": "left", "hfov": 160, "type": "color", "extrinsics": { "to_cam": "CAM_A", "specTranslation": { "x": 4, "y": 0, "z": 0 }, "rotation":{ "r": 0, "p": 0, "y": 0 } } }, "CAM_A": { "name": "rgb", "hfov": 90, "type": "color" } }, "stereo_config":{ "left_cam": "CAM_B", "right_cam": "CAM_C" } } } ```

Thanks, I'll have a look at this today hopefully

I've gone through this a few times today and gotten much further so thanks for the tips! Pretty sure one of the issues I was facing was due to a poor quality calibration image, the printer put out something more grey than black so I reprinted on photo paper and it's much happier. First few times I got XLink errors, pretty sure I knocked a cable, and then when I was almost done the battery in my laptop died as it was taking the last image! I just got through them all for the first time after a recharge, it collected them all then hit an error in OpenCV. I've not had a chance to debug it yet, I'll have another look tomorrow. `OpenCV(4.5.1) C:\Users\appveyor\AppData\Local\Temp\1\pip-req-build-5rb_9df3\opencv_contrib\modules\aruco\src\charuco.cpp:448: error: (-215:Assertion failed) _markerCorners.total() == _markerIds.getMat().total() && _markerIds.getMat().total() > 0 in function 'cv::aruco::_interpolateCornersCharucoLocalHom'` Though it now looks fairly obvious, "appveyor" isn't my username so I suspect that there is a hard coded path somewhere in the calibration script. I'll have a look tomorrow, hopefully it's an easy win. :)

Hmm, no, AppVeyor isn't mentioned in the repo at all that I can find. How very odd! Edit:just realised AppVevor is a CI tool so that's likely the path for the file on the build server and I've the resulting assembly on my machine. Will dig in to the code and figure out what it's doing, or not, as the case may be.

Just for those who hit a similar issue, with the branch above I needed to downgrade OpenCV to the following version manually: opencv-contrib-python==4.5.5.64 I'm now hitting an issue where it seems to get through everything but complain there's too big a difference between stereo cameras and RGB, I'm assuming as stereo are 160deg and the RGB is 90deg. I'm going to try with the -drgb flag to see if it can calibrate the stereo cameras at least.

Hi @"kneave"#p5297 , We apologize for the inconvenience - could you copy the full error that you get? Thanks, Erik

Hi Erik, will do. I'll go through again and grab it shortly. Are the images saved to the disk too by the way? It'd be handy if I could run calibration again without having to recapture the images each time. It'd speed up debugging no end!

OAK-FFP-3P User Guide For Dummies?

DanielCalaver

kneave

sorry @kneave not sure how it got deleted (should stay for 2 weeks). Anyways, here is the wetransfer link

kneave

Brilliant, I've got them. Thanks!

kneave

Finally got my new modern art delivered, I can finally test this again!

erik

That's quite large kneave ! When you are calibrating it, just make sure the board is actually flat (eg on table, or on the ground), as currently, it looks like it's bending a bit, which would cause worse calibration.
Thanks, Erik

kneave

Thanks Erik, I had it made to the same specifications @DanielCalaver mentioned that you use for your wide-angle products. I used it on the ground and managed to avoid the cats "helping" but sadly I'm still getting the epipolar errors.

I'll try again tomorrow with natural light rather than the indoor lights we have and more images too.

DanielCalaver

kneave thanks, yeah, if the board reflects too much light from ahead it can be disruptive, hope it'll work this time!

kneave

Sadly it hasn't, though I'm down to only one epipolar error so that's progress I guess:

Reprojection error threshold -> 100
rgb Reprojection Error: 0.688267
1200
Reprojection error threshold -> 166.66666666666666
left Reprojection Error: 0.614719
1200
Reprojection error threshold -> 166.66666666666666
right Reprojection Error: 0.577443
['high epipolar error between right and rgb']
py: DONE.

I've uploaded the calibration images and board definition here: https://we.tl/t-5uGyFtsNkM
The command I ran was python calibrate.py -s 4 -ms 3.1 -brd FFC-3Pw -nx 22 -ny 16 -c 3 and this is from the ar0234x3 branch of DepthAI.

One thing I'm still unsure of which I suspect could have an impact is the HFOV value I'm using for the stereo cameras. They are the same as the centre with a 160deg lens instead of the 90 that the 0234 come with. I couldn't find a definition for the HFOV for these lenses but calculated it at 122deg, I'm not sure of my maths however as optics isn't my forte to say the least.

I'm out of ideas and kinda annoyed I can't get this working to be honest. If you've any advice, even suggestions for other lenses to try, I'd appreciate it. Heck, if I can just calibrate with the stereo cameras and ignore the centre could that be an option? I'm planning on using the colour stereo for SLAM and point cloud generation and the centre with narrower field for object detection. If I don't actually need the centre can we get rid of that as it's the RGB to right cam that's throwing the error?

kneave

I just ran the following to disable RGB and got the same error 🙁

python calibrate.py -s 4 -ms 3.1 -brd FFC-3Pw -nx 22 -ny 16 -c 3 -drgb -m process

Displaying Stereo Pair for visual inspection. Press the [ESC] key to exit.
1200
Reprojection error threshold -> 100
rgb Reprojection Error: 0.688267
1200
Reprojection error threshold -> 166.66666666666666
left Reprojection Error: 0.614719
1200
Reprojection error threshold -> 166.66666666666666
right Reprojection Error: 0.577443
['high epipolar error between right and rgb']
py: DONE.

kneave

I saw someone downloaded the dataset file, hopefully that means someone is looking at this please?
@erik, @DanielCalaver?

erik

(@DanielCalaver would be the best person to check this - I have no exp with calibration.)

DanielCalaver

Yes, I downloaded the file but didn't managed to take a close look, will get back on this soon, sorry for delay @kneave

kneave

DanielCalaver cool, thanks Daniel. I appreciate the help

DanielCalaver

Hi @kneave sorry for delay, can you check if the lenses for both LEFT and RIGHT are the same? They look a bit different for me and this might affect calibration, could you try to get some better lighting? It was a bit dark in the pictures, maybe try to put the calib board on a wall and the room to be well light? As this way, the lights from above won't be reflected.

kneave

Hi Daniel,

They both came off the same camera model so should be the same. I finally found the specs for hfov and they're 120 rather than 122. Not sure if that 2deg difference is enough to cause a problem.

Will check lighting when home too, thanks for the tips!

Keegan

kneave

Hi Folks, I had to take care of some other projects and I'm just picking this back up. Could I ask if the 3xar0234 branch is still the best one to follow for multi-cam calibration please? I see there has been a new release since I looked at this last and wanted to check there isn't some kind of new Easy Mode you've added! 😅

erik

Hi kneave ,
For calibration, I believe the modular-calibration branch (PR here) would be the best to follow, as it will also be used for calibrating OAK-D-LR (which have 3x ar0234). Thoughts?
Thanks, Erik

kneave

I'll have a look, thanks. Just to complicate matters I've an oak-ffc-imx378 I'm hoping to use for the centre camera now. I've not been able to get it to be detected by the 3P yet which is annoying but it's been that long since I've looked at this I've likely forgotten a trick needed. That or the one USB cable I have that works reliably has gone AWOL 😅

erik

kneave is it the case that only FFC-IMX378 isn't being recognized, while other cameras are? If so, then it's not an USB issue. I would also suggest using latest depthai version.
Thanks, Erik

kneave

erik It's a bit random at the minute, it's sometimes saying no cameras detected but as it's getting that response it at least means it's connected to the 3P and able to get a response back. I'm currently printing a new enclosure, once I've got everything in place I'll do some more tests.

Quick render of the new enclosure attached, it'll be 3d printed so not quite as shiny! You can see the centre camera hidden in the black separator between the eyes, I'm keen to make it look like he only has two as much as possible.

erik

Hi kneave ,
Could you copy the full terminal log of cam_test script, please? If it doesn't work as expected we will send a replacement unit. The render looks dope!! Please let us know how it will turn out🙂
Thanks, Erik

« Previous Page Next Page »