OAK-FFP-3P User Guide For Dummies?

erik It's a bit random at the minute, it's sometimes saying no cameras detected but as it's getting that response it at least means it's connected to the 3P and able to get a response back. I'm currently printing a new enclosure, once I've got everything in place I'll do some more tests.

Quick render of the new enclosure attached, it'll be 3d printed so not quite as shiny! You can see the centre camera hidden in the black separator between the eyes, I'm keen to make it look like he only has two as much as possible.

Sure thing, does it matter which branch I run cam-test from or can I install the production libraries and run against them?

Ok, I've done a lot of digging and should really be in bed but I may have stumbled upon something in this issues;
https://github.com/luxonis/depthai-hardware/issues/292

Current configuration is:
Left: AR0234
Centre: IMX378
Right: AR0234

I have a simple camera preview script which I've successfully used to focus the cameras in the past, it has this line that lists all the cameras:

with dai.Device(pipeline) as device:
    print('Connected cameras: ', device.getConnectedCameras())

First off, I ran this against the develop branch of depthai, making sure to run ./install_requirements.py:

I then switch to the ar0234_x3 branch which is the only one I've been able to get the cameras working on in the past:

This shows that the two AR0234 are detected, if I replace the center with the other AR0234 I have it is detected too and I'm able to preview from all three.

This made me wonder about the IMX378 which led me to the issue I linked above. Taking a macro photo of the board it looks like it does indeed have the K28 power regulator on board and it also has all the pins connected, I'm starting to think I have one of the broken ones?

Could I cut the problem pin and see if it works or having plugged it in would it have damaged it? I don't want to do any mods that may prevent a warranty claim but if all I need to so is cut that pin then it could be an easy win?

Success! Kinda...

If I run on the Main branch I can get cam_test to run and show the IMX378 camera but the two AR0234 cameras aren't detected. If I run on the ar0234x3 branch I can see all three cameras in my preview.py script however cam_test.py won't run as it doesn't recognise THE_1200_P as a valid resolution.

I also tried the modular-calibration branch however I couldn't run install_requirements.py as it failed with the following error:
ERROR: Could not find a version that satisfies the requirement depthai==2.19.1.0.dev+1eb01248fb7e994b51ceec91984bff46e0a5d5f2

I suspect my combination of cameras is throwing things off slighting and that the IMX378 is supported on one branch and the AR0234 on another but not both together maybe? Either that or depthai_python just doesn't have an updated version of cam_test?

I'm once again a little lost to be honest, if you know of a branch that has success with this camera combo could you point me towards it please?

Hi Eric,

To confirm, I'm now on the following branches for:
depthai: modular-calibration (Your commit ₃₀ minutes ago)
depthai-python: main (for cam-test.py in utilities, latest commit)

I ran `install-requirements.py' from the root of depthai and the utilities folder of depthai-python.

When I run cam_test.py I only see the IMX378, it comes up in the preview window quite happily and the console reports this:

Enabled cameras:
    rgb : color
   left : color
  right : color
DepthAI version: 2.20.2.0
DepthAI path: C:\Users\keegan\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\depthai.cp39-win_amd64.pyd
[184430105182410E00] [1.5] [0.865] [ColorCamera(6)] [error] Camera not detected on socket: 2
[184430105182410E00] [1.5] [0.866] [ColorCamera(4)] [error] Camera not detected on socket: 1
Connected cameras:
 -socket RGB   : IMX378 4056 x 3040 focus:auto  - COLOR
USB speed: SUPER
IR drivers: []
Cam:      rgb           left          right    [host | capture timestamp]
FPS:  30.00| 30.00   0.00|  0.00   0.00|  0.00

If I switch to the ar0234x3 branch the cam_test.py script doesn't run as it looks as though there was a bit refactor that deprecated or changed a lot of the methods in depthai between then and 2.20.

It looks as though if I try and run with any version of depthai other than depthai==2.17.2.0.dev+211dc1c0a135d5615dba75ebf2098ba8cdc05cd0 it isn't able to see the AR0234 cameras. If I look at the depthai-python branch it looks as though ar0234x3 was never merged in to any other branch either.

I may be misreading the commit graph, I've been digging between branches for a while now and I've lost track, but is the AR0234 supported on either main or develop of depthai-python? To me it looks like it isn't and that it's only that old 2.17 branch that it's available in.

I'm once again trying to calibrate against the AR0234x3 branch as it seems to be the only one that supports all my cameras, this time all new errors. Before I start digging even further and spending more time on this can I get a yes/no answer to the following please?

Has the combination of 2x AR0234 and single IMX378 been tested by yourselves or anyone else that you know of?

Right now I've no idea if this newest issue will be the last one and I can get passed it and do the actual work I want to with these cameras, or if this is the tip of an iceberg and there will be yet more work to do afterwards.

I understand that what I'm doing isn't simple and that the majority of your customers just use the standard integrated and tested cameras. I also understand that these things can take time, but right now I have an awful lot of unknowns and other projects I need to work on. If I know that this has worked for others I can continue, if I'm on my own with a new and untested combination I may have to shelve this project until the 3P has better support for multi-cameras.

If you could let me know if the light at the end of the tunnel is sunlight or an oncoming train, I would greatly appreciate it.

It's looking like the issue was with the image preview. The AR0234 output at 1200p and the IMX378 wants to output at 1080p/4k.

I'm thinking if ignoring the centre camera for now and just trying to get the depth camera working. I think that the difference between HFOV of the centre and side cameras is too great for calibrating together. At least with the side cameras being rgb I should still be able to get a coloured point cloud. Hopefully I'll be able to use the centre camera independently, I'll have to have a play around with it at some point.

Thanks Erik, will do

Done: Issue 763

I included the preview.py script I've been using to list and preview the cameras, if nothing else it's really handy for manually checking the focus of the cameras.

Apparently this combo of cameras should work but the config file on the 3P may now be in a bad state prevent calibration from working. I've tried running the factory_reset script and it just says the factory calibration isn't present, I've no idea where to find an empty one either. If you could point me towards one I'd greatly appreciate it, I'm worried that if it's this difficult to get the camera calibrated that it'll be impossible to do anything else with it.

Getting closer, now stuck on RuntimeError: No PROTECTED permissions to override protected EEPROM fields so something else on the EEPROM isn't quite right, or is locked in some way?

Thanks Erik, I was able to use that to update the calibration but getting other errors permissions errors now
Also, it doesn't have a factory calibration apparently. Is this supposed to be the case with the 3P? It doesn't have cameras included so I can see why it wouldn't, but also it seems like not having a calibration at all may be the cause of a bunch of other issues.

Thanks for all your help so far Erik, and colleagues, I think I'm going to have to spend some time properly getting my head around calibration and how to get it to work with my setup. I was previously using a StereoPi with two fisheye lenses and using the ROS calibration method without issue so I think I got complacent. With the extra power of the 3P it seems it needs more attention to get working. That said, it's tempting just to publish the images from each cameras to ROS and see if I can use the same method to generate the calibration. A bit hacky but I'm willing to try anything at this point!

For Windows users seeing the previous post regarding the environmental variable, the equivalent is:
in command prompt:
set DEPTHAI_ALLOW_FACTORY_FLASHING="235539980"

in powershell:
$env:DEPTHAI_ALLOW_FACTORY_FLASHING="235539980"

Then run the python script. This will persist until the cmd/powershell session is closed.