ShivamSharma
Can I see the full calibration file with the protected fields please?
Thanks,
Jaka
ShivamSharma
Can I see the full calibration file with the protected fields please?
Thanks,
Jaka
The following is the calibration file with the -nic option :
{
"batchName": "",
"batchTime": 0,
"boardConf": "",
"boardCustom": "",
"boardName": "Custom FFC",
"boardOptions": 0,
"boardRev": "R1M0E3",
"cameraData": [
[
1,
{
"cameraType": 0,
"distortionCoeff": [
-1.8108996152877808,
0.5892674922943115,
0.00020837810006923974,
0.0008147237240336835,
-0.4694308340549469,
-1.8531672954559326,
0.7104603052139282,
-0.5547073483467102,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0
],
"extrinsics": {
"rotationMatrix": [],
"specTranslation": {
"x": 0.0,
"y": 0.0,
"z": 0.0
},
"toCameraSocket": -1,
"translation": {
"x": 0.0,
"y": 0.0,
"z": 0.0
}
},
"height": 800,
"intrinsicMatrix": [
[
906.6533813476563,
0.0,
625.2329711914063
],
[
0.0,
906.753662109375,
399.9358825683594
],
[
0.0,
0.0,
1.0
]
],
"lensPosition": 0,
"specHfovDeg": 75.0,
"width": 1280
}
],
[
2,
{
"cameraType": 0,
"distortionCoeff": [
-1.1144567728042603,
0.4457603096961975,
-0.00030109440558589995,
0.000643925741314888,
1.2683645486831665,
-1.1590780019760132,
0.5456246733665466,
1.2029682397842407,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0
],
"extrinsics": {
"rotationMatrix": [
[
0.9946930408477783,
0.10237327218055725,
-0.010269088670611382
],
[
-0.1015079990029335,
0.9927471876144409,
0.06441415846347809
],
[
0.016788896173238754,
-0.06302992254495621,
0.9978703856468201
]
],
"specTranslation": {
"x": 9.0,
"y": 0.0,
"z": 0.0
},
"toCameraSocket": 1,
"translation": {
"x": -8.774820327758789,
"y": 0.4384150207042694,
"z": -0.33474013209342957
}
},
"height": 800,
"intrinsicMatrix": [
[
910.39697265625,
0.0,
646.8530883789063
],
[
0.0,
910.701904296875,
386.07025146484375
],
[
0.0,
0.0,
1.0
]
],
"lensPosition": 0,
"specHfovDeg": 75.0,
"width": 1280
}
]
],
"deviceName": "",
"hardwareConf": "",
"housingExtrinsics": {
"rotationMatrix": [],
"specTranslation": {
"x": 0.0,
"y": 0.0,
"z": 0.0
},
"toCameraSocket": -1,
"translation": {
"x": 0.0,
"y": 0.0,
"z": 0.0
}
},
"imuExtrinsics": {
"rotationMatrix": [],
"specTranslation": {
"x": 0.0,
"y": 0.0,
"z": 0.0
},
"toCameraSocket": -1,
"translation": {
"x": 0.0,
"y": 0.0,
"z": 0.0
}
},
"miscellaneousData": [],
"productName": "",
"stereoEnableDistortionCorrection": false,
"stereoRectificationData": {
"leftCameraSocket": 1,
"rectifiedRotationLeft": [
[
0.9980300664901733,
-0.049864426255226135,
0.03807265684008598
],
[
0.051083534955978394,
0.9981896281242371,
-0.03174852952361107
],
[
-0.036420609802007675,
0.03363087400794029,
0.9987704753875732
]
],
"rectifiedRotationRight": [
[
0.9984344244003296,
0.05026911944150925,
0.024530746042728424
],
[
-0.05104481801390648,
0.998180627822876,
0.03209204599261284
],
[
-0.02287287451326847,
-0.03329396992921829,
0.9991838335990906
]
],
"rightCameraSocket": 2
},
"stereoUseSpecTranslation": true,
"version": 7,
"verticalCameraSocket": -1
}
Hi ShivamSharma
I tried this .json file with calibration_load.py and experienced no issues (albeit the depth was bad, which was expected since the device is different). Do other examples work as expected (correct Opencv version, correct depthai version,..). I think the issue might be somewhere else.
Thanks,
Jaka
Okay, I tried the load calibration again today, and I saw the depth window open with the same file I pasted above. The depth was bad, even though the calibration is correct as far as I know. The following is the depth:
What should I do now?
Edit1:
The depth is still bad after switching left and right in the calibration load file. I don't think I changed the camera setup. Should I calibrate again just to be sure? A few days ago, switching left and right in the depth_preview file gave the correct depth, but that is not the case now.
Edit2:
I calibrated the camera again, and now depth_preview is better, but I keep seeing blue-colored shadows of the outline of objects. Why is this happening? Is this how it is supposed to work around the discontinuity in depth?
Edit3:
After switching left and right in the calibration_load.py and using the -nic calibration data, I got good depth. It looks like the following:
Do you still want me to switch numerical calibration values instead to make it work with ROS 2?
Edit 4:
I did what you asked. I switched the numeric calibration values in the -nic calibration file but got bad depth with and without switching the cameras in calibration_load.py. I have pasted the original and the modified below:
Original:
{
"batchName": "",
"batchTime": 0,
"boardConf": "",
"boardCustom": "",
"boardName": "Custom FFC",
"boardOptions": 0,
"boardRev": "R1M0E3",
"cameraData": [
[
1,
{
"cameraType": 0,
"distortionCoeff": [
-0.7255496382713318,
5.1313700675964355,
-0.0002011057222262025,
0.0008841821108944714,
-6.754741668701172,
-0.7626256942749023,
5.142012119293213,
-6.697120189666748,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0
],
"extrinsics": {
"rotationMatrix": [],
"specTranslation": {
"x": 0.0,
"y": 0.0,
"z": 0.0
},
"toCameraSocket": -1,
"translation": {
"x": 0.0,
"y": 0.0,
"z": 0.0
}
},
"height": 800,
"intrinsicMatrix": [
[
907.2047119140625,
0.0,
625.1134033203125
],
[
0.0,
907.6209716796875,
397.6976013183594
],
[
0.0,
0.0,
1.0
]
],
"lensPosition": 0,
"specHfovDeg": 75.0,
"width": 1280
}
],
[
2,
{
"cameraType": 0,
"distortionCoeff": [
-0.11288045346736908,
0.9440451264381409,
-0.0003182076325174421,
0.0005265712388791144,
-2.1388356685638428,
-0.15885838866233826,
1.0026153326034546,
-2.1278512477874756,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0
],
"extrinsics": {
"rotationMatrix": [
[
0.9948962926864624,
0.10089320689439774,
-0.0013819887535646558
],
[
-0.10070427507162094,
0.9937053918838501,
0.04907328262925148
],
[
0.006324450485408306,
-0.04868365451693535,
0.9987941980361938
]
],
"specTranslation": {
"x": 9.0,
"y": 0.0,
"z": 0.0
},
"toCameraSocket": 1,
"translation": {
"x": -8.728154182434082,
"y": 0.45325255393981934,
"z": -0.024646950885653496
}
},
"height": 800,
"intrinsicMatrix": [
[
907.8340454101563,
0.0,
646.8483276367188
],
[
0.0,
908.2755737304688,
384.77850341796875
],
[
0.0,
0.0,
1.0
]
],
"lensPosition": 0,
"specHfovDeg": 75.0,
"width": 1280
}
]
],
"deviceName": "",
"hardwareConf": "",
"housingExtrinsics": {
"rotationMatrix": [],
"specTranslation": {
"x": 0.0,
"y": 0.0,
"z": 0.0
},
"toCameraSocket": -1,
"translation": {
"x": 0.0,
"y": 0.0,
"z": 0.0
}
},
"imuExtrinsics": {
"rotationMatrix": [],
"specTranslation": {
"x": 0.0,
"y": 0.0,
"z": 0.0
},
"toCameraSocket": -1,
"translation": {
"x": 0.0,
"y": 0.0,
"z": 0.0
}
},
"miscellaneousData": [],
"productName": "",
"stereoEnableDistortionCorrection": false,
"stereoRectificationData": {
"leftCameraSocket": 1,
"rectifiedRotationLeft": [
[
0.9986503720283508,
-0.05185985937714577,
0.0028200335800647736
],
[
0.05191338434815407,
0.9983528256416321,
-0.024425296112895012
],
[
-0.0015486960764974356,
0.024538729339838028,
0.9996976852416992
]
],
"rectifiedRotationRight": [
[
0.99879390001297,
0.049086324870586395,
-0.0011084240395575762
],
[
-0.04904444143176079,
0.9984954595565796,
0.024524861946702003
],
[
0.0023105917498469353,
-0.02444092184305191,
0.9996985793113708
]
],
"rightCameraSocket": 2
},
"stereoUseSpecTranslation": true,
"version": 7,
"verticalCameraSocket": -1
}
Modified:
{
"batchName": "",
"batchTime": 0,
"boardConf": "",
"boardCustom": "",
"boardName": "Custom FFC",
"boardOptions": 0,
"boardRev": "R1M0E3",
"cameraData": [
[
1,
{
"cameraType": 0,
"distortionCoeff": [
-0.11288045346736908,
0.9440451264381409,
-0.0003182076325174421,
0.0005265712388791144,
-2.1388356685638428,
-0.15885838866233826,
1.0026153326034546,
-2.1278512477874756,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0
],
"extrinsics": {
"rotationMatrix": [
[
0.9948962926864624,
0.10089320689439774,
-0.0013819887535646558
],
[
-0.10070427507162094,
0.9937053918838501,
0.04907328262925148
],
[
0.006324450485408306,
-0.04868365451693535,
0.9987941980361938
]
],
"specTranslation": {
"x": 9.0,
"y": 0.0,
"z": 0.0
},
"toCameraSocket": 2,
"translation": {
"x": -8.728154182434082,
"y": 0.45325255393981934,
"z": -0.024646950885653496
}
},
"height": 800,
"intrinsicMatrix": [
[
907.8340454101563,
0.0,
646.8483276367188
],
[
0.0,
908.2755737304688,
384.77850341796875
],
[
0.0,
0.0,
1.0
]
],
"lensPosition": 0,
"specHfovDeg": 75.0,
"width": 1280
}
],
[
2,
{
"cameraType": 0,
"distortionCoeff": [
-0.7255496382713318,
5.1313700675964355,
-0.0002011057222262025,
0.0008841821108944714,
-6.754741668701172,
-0.7626256942749023,
5.142012119293213,
-6.697120189666748,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0
],
"extrinsics": {
"rotationMatrix": [],
"specTranslation": {
"x": 0.0,
"y": 0.0,
"z": 0.0
},
"toCameraSocket": -1,
"translation": {
"x": 0.0,
"y": 0.0,
"z": 0.0
}
},
"height": 800,
"intrinsicMatrix": [
[
907.2047119140625,
0.0,
625.1134033203125
],
[
0.0,
907.6209716796875,
397.6976013183594
],
[
0.0,
0.0,
1.0
]
],
"lensPosition": 0,
"specHfovDeg": 75.0,
"width": 1280
}
]
],
"deviceName": "",
"hardwareConf": "",
"housingExtrinsics": {
"rotationMatrix": [],
"specTranslation": {
"x": 0.0,
"y": 0.0,
"z": 0.0
},
"toCameraSocket": -1,
"translation": {
"x": 0.0,
"y": 0.0,
"z": 0.0
}
},
"imuExtrinsics": {
"rotationMatrix": [],
"specTranslation": {
"x": 0.0,
"y": 0.0,
"z": 0.0
},
"toCameraSocket": -1,
"translation": {
"x": 0.0,
"y": 0.0,
"z": 0.0
}
},
"miscellaneousData": [],
"productName": "",
"stereoEnableDistortionCorrection": false,
"stereoRectificationData": {
"leftCameraSocket": 1,
"rectifiedRotationLeft": [
[
0.99879390001297,
0.049086324870586395,
-0.0011084240395575762
],
[
-0.04904444143176079,
0.9984954595565796,
0.024524861946702003
],
[
0.0023105917498469353,
-0.02444092184305191,
0.9996985793113708
]
],
"rectifiedRotationRight": [
[
0.9986503720283508,
-0.05185985937714577,
0.0028200335800647736
],
[
0.05191338434815407,
0.9983528256416321,
-0.024425296112895012
],
[
-0.0015486960764974356,
0.024538729339838028,
0.9996976852416992
]
],
"rightCameraSocket": 2
},
"stereoUseSpecTranslation": true,
"version": 7,
"verticalCameraSocket": -1
}
Edit :
@jakaskerl I tried the ROS 1 noetic container, and the depth was good. But the fps was around 10, and it was unsynchronized even after setting synchronization to true. So the bug is in the ROS 2 driver, and the hardware is faulty?
I can't use the ROS 1 topic with RTabmap because the camera information is giving a negative baseline, even though the depth in rviz is correct. Rtabmap works with positive baseline. Rtabmap assumes the left camera is the reference, but I see in the rostopic info that it is switched.
Also, the image in ROS 1 topic has switched left and right. I move my right arm and see left side move in the image not like a mirror.
ShivamSharma ut I keep seeing blue-colored shadows of the outline of objects. Why is this happening?
This is normal and is a result of binocular occlusion, where one camera sees something the other does not (because the object is blocking it). This means the stereo depth can't be calculated there since stereo matching fails.
ShivamSharma Do you still want me to switch numerical calibration values instead to make it work with ROS 2?
No, if depth in the preview looks good without changing the code in any way, then leave it as it is. It is a problem of ROS.
cc @Luxonis-Adam
Thanks,
Jaka
Hi, DepthAI assumes different stereo cam order than OpenCV, you can toggle this behavior using a ROS parameter, as described here. Regarding socket order, it has some default parameters which can be overriden as well using parameters (cam_name_i_board_socket_id
). Stereo node also has predefined socket numbers which can be overriden using parameters.
This is very confusing. Are you saying that I change all of the following parameters? Set which to what?:
stereo_i_left_socket_id: 1
stereo_i_right_socket_id: 2
right_i_board_socket_id: 2
left_i_board_socket_id: 1
stereo_i_reverse_stereo_socket_order: false
Edit1:
When I set the following to true I get an error in the noetic container:
stereo_i_reverse_stereo_socket_order: true
The error:
river_sensor_nodes.so: undefined symbol: _ZN3dai3ros14ImageConverter24reverseStereoSocketOrderEv
[FATAL] [1719946738.205158589]: Failed to load nodelet '/oak` of type `depthai_ros_driver/Camera` to manager `oak_nodelet_manager'
[oak_nodelet_manager-3] process has died [pid 70, exit code 127, cmd /opt/ros/noetic/lib/nodelet/nodelet manager /nodelet_manager/load_nodelet:=oak/nodelet_manager/load_nodelet /nodelet_manager/unload_nodelet:=oak/nodelet_manager/unload_nodelet /nodelet_manager/list:=oak/nodelet_manager/list __name:=oak_nodelet_manager __log:=/root/.ros/log/1f8989e0-38a5-11ef-a2b1-1cce51f6c4df/oak_nodelet_manager-3.log].
log file: /root/.ros/log/1f8989e0-38a5-11ef-a2b1-1cce51f6c4df/oak_nodelet_manager-3*.log
================================================================================REQUIRED process [oak-4] has died!
process has died [pid 71, exit code 255, cmd /opt/ros/noetic/lib/nodelet/nodelet load depthai_ros_driver/Camera oak_nodelet_manager __name:=oak __log:=/root/.ros/log/1f8989e0-38a5-11ef-a2b1-1cce51f6c4df/oak-4.log].
log file: /root/.ros/log/1f8989e0-38a5-11ef-a2b1-1cce51f6c4df/oak-4*.log
Initiating shutdown!
@Luxonis-Adam (ping)
There is also one more parameter:
stereo_i_board_socket_id: 2
I have moved on from the Depthai ROS 2 driver, and I have created my own ROS 2 driver and calibration script for the oak cameras I have. I get good depth with my own driver in ROS 2. But the issue is synchronization and varying time difference between consecutive frames with my ROS 2 driver. I will create a separate post for it.
Will the depthai ROS 2 driver or the calibration script be fixed in the near future?
I can't get my ROS 2 driver to synchronize the images. So I also want to pursue the other option of using DepthAI- ros driver. @Luxonis-Adam
Is there an active issue with the ROS 2 driver or I am using an old branch. I cloned the main branch and installed the driver, but it gave me numerous issues. I also posted this issue on GitHub but haven't heard back from you.