Hi jakaskerl
So I went through the following processes but still no luck.
First, I modified the board config and added the extrinsics
{
"board_config":
{
"name": "Tennibot OAK FFC 3P",
"revision": "R0M0E0",
"cameras":{
"CAM_B": {
"name": "station_left_cam",
"hfov": 140,
"type": "color",
"extrinsics": {
"to_cam": "CAM_C",
"specTranslation": {
"x": 3.5,
"y": 0,
"z": 0
},
"rotation":{
"r": 0,
"p": 0,
"y": 0
}
}
},
"CAM_C": {
"name": "station_right_cam",
"hfov": 140,
"type": "color",
"extrinsics": {
"to_cam": "CAM_B",
"specTranslation": {
"x": -3.5,
"y": 0,
"z": 0
},
"rotation":{
"r": 0,
"p": 0,
"y": 0
}
}
}
}
}
}
Because I still have my dataset saved, I just ran the process
python calibrate.py -ih -s 2.5 -db -brd <camera-definition-file> -m process
But because now we have extrinsics in the config, according to this line, the calibration is looking for stereo_config, which I am not using it because I only uses the board for color cameras. But I added the following anyway just hope the calibration will success
"stereo_config":{
"left_cam": "CAM_B",
"right_cam": "CAM_C"
}
And it did, the following is the result json file
{
"batchName": "",
"batchTime": 0,
"boardConf": "",
"boardCustom": "",
"boardName": "",
"boardOptions": 0,
"boardRev": "",
"cameraData": [
[
2,
{
"cameraType": 0,
"distortionCoeff": [
-0.5117843151092529,
-0.06536504626274109,
-5.844790211995132e-05,
-0.0005799136124551296,
0.025351274758577347,
-0.16227206587791443,
-0.3164222836494446,
0.05816520377993584,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0
],
"extrinsics": {
"rotationMatrix": [
[
0.9998506307601929,
0.01509548258036375,
0.008416909724473953
],
[
-0.01534199807792902,
0.9994310736656189,
0.03003622405230999
],
[
-0.007958710193634033,
-0.030160870403051376,
0.9995133876800537
]
],
"specTranslation": {
"x": -3.5,
"y": 0.0,
"z": 0.0
},
"toCameraSocket": 1,
"translation": {
"x": 3.646730661392212,
"y": 0.18376091122627258,
"z": 0.02605484239757061
}
},
"height": 1080,
"intrinsicMatrix": [
[
872.6644897460938,
0.0,
981.9554443359375
],
[
0.0,
869.8309326171875,
528.7936401367188
],
[
0.0,
0.0,
1.0
]
],
"lensPosition": 0,
"specHfovDeg": 140.0,
"width": 1920
}
],
[
1,
{
"cameraType": 0,
"distortionCoeff": [
5.590981960296631,
-0.8370659947395325,
-0.00031772308284416795,
-0.0005036363145336509,
-0.3884274959564209,
5.948943138122559,
1.0271347761154175,
-1.038749098777771,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0
],
"extrinsics": {
"rotationMatrix": [
[
0.9998506307601929,
-0.01534199807792902,
-0.007958710193634033
],
[
0.01509548258036375,
0.9994310736656189,
-0.030160870403051376
],
[
0.008416909724473953,
0.03003622405230999,
0.9995133876800537
]
],
"specTranslation": {
"x": 3.5,
"y": 0.0,
"z": 0.0
},
"toCameraSocket": 2,
"translation": {
"x": -3.6431593894958496,
"y": -0.23791968822479248,
"z": -0.0622558519244194
}
},
"height": 1080,
"intrinsicMatrix": [
[
870.5143432617188,
0.0,
982.17822265625
],
[
0.0,
867.4022827148438,
517.3909912109375
],
[
0.0,
0.0,
1.0
]
],
"lensPosition": 0,
"specHfovDeg": 140.0,
"width": 1920
}
]
],
"hardwareConf": "",
"imuExtrinsics": {
"rotationMatrix": [],
"specTranslation": {
"x": 0.0,
"y": 0.0,
"z": 0.0
},
"toCameraSocket": -1,
"translation": {
"x": 0.0,
"y": 0.0,
"z": 0.0
}
},
"miscellaneousData": [],
"productName": "",
"stereoRectificationData": {
"leftCameraSocket": 1,
"rectifiedRotationLeft": [
[
0.9987073540687561,
0.05032545328140259,
0.007135476917028427
],
[
-0.05021293833851814,
0.9986235499382019,
-0.015156709589064121
],
[
-0.007888423278927803,
0.014778824523091316,
0.9998596906661987
]
],
"rectifiedRotationRight": [
[
0.9977293014526367,
0.06515757739543915,
0.017049621790647507
],
[
-0.06540568917989731,
0.9977545142173767,
0.014422908425331116
],
[
-0.01607157476246357,
-0.015505300834774971,
0.9997506141662598
]
],
"rightCameraSocket": 2
},
"version": 7
}
The result has all the translation and rotation parameters. Then I flashed it to the device and confirmed the flashing. But getConnectedCameraFeatures still can't find any camera.
Then I thought maybe the board config has to have all sockets defined. So I added CAM-A to the config
{
"board_config":
{
"name": "Tennibot OAK FFC 3P",
"revision": "R0M0E0",
"cameras":{
"CAM_A": {
"name": "station_middle_cam",
"hfov": 140,
"type": "color"
},
"CAM_B": {
"name": "station_left_cam",
"hfov": 140,
"type": "color",
"extrinsics": {
"to_cam": "CAM_A",
"specTranslation": {
"x": 3.5,
"y": 0,
"z": 0
},
"rotation":{
"r": 0,
"p": 0,
"y": 0
}
}
},
"CAM_C": {
"name": "station_right_cam",
"hfov": 140,
"type": "color",
"extrinsics": {
"to_cam": "CAM_A",
"specTranslation": {
"x": -3.5,
"y": 0,
"z": 0
},
"rotation":{
"r": 0,
"p": 0,
"y": 0
}
}
}
},
"stereo_config":{
"left_cam": "CAM_B",
"right_cam": "CAM_C"
}
}
}
When I retried the calibration, it gave me error saying missing key for the camera properties like "AutoFocus", "SensorType". It turns out the calibration code is using getConnectedCameraFeatures() to get those information, which I can't use now.
Even though I can manually provide information like "SensorType" in the board config, but eventually the calibration will fail because it couldn't find the cameras from CAM-B and CAM-C sockets to when it run the pipeline.
[{socket: RGB/CENTER/CAM_A, sensorName: IMX577, width: 4056, height: 3040, orientation: AUTO, supportedTypes: [COLOR], hasAutofocus: 0, name: color}]
Cam: station_middle_cam and focus: False
[18443010D1F3F40800] [3.6] [0.985] [ColorCamera(2)] [error] Camera not detected on socket: 1
[18443010D1F3F40800] [3.6] [0.985] [ColorCamera(4)] [error] Camera not detected on socket: 2
What's your suggestion for my next step? Thanks
