I believe to mistaken calibrating my camera with another process. I have the oak-d-sr-poe camera, I have clone luxnonis/depthai/calibrate.py repo. I now I am trying to use the calibrate the camera correctly based off this link: [Calibration (luxonis.com)](https://docs.luxonis.com/hardware/platform/depth/calibration/) So far I have ran the script and have came upon this: PS S:\\DEPT\\SVM4\\Shared\\Crossfunctional_Work\\Projects\\DepthCameras\\LuxonisDepthAI\\test_run\\depthai> python calibrate.py -s 3.0 -ms 3.5 -brd OAK-D-SR-POE -m process Cam: rgb and focus: False Cam: left and focus: False Cam: right and focus: False Traceback (most recent call last): File "S:\\DEPT\\SVM4\\Shared\\Crossfunctional_Work\\Projects\\DepthCameras\\LuxonisDepthAI\\test_run\\depthai\\calibrate.py", line 1183, in Main().run() ^^^^^^ File "S:\\DEPT\\SVM4\\Shared\\Crossfunctional_Work\\Projects\\DepthCameras\\LuxonisDepthAI\\test_run\\depthai\\calibrate.py", line 453, in __init__ self.charuco_board = cv2.aruco.CharucoBoard_create( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ cv2.error: OpenCV(4.5.5) D:\\a\\opencv-python\\opencv-python\\opencv_contrib\\modules\\aruco\\src\\charuco.cpp:126: error: (-215:Assertion failed) squaresX > 1 && squaresY > 1 && markerLength > 0 && squareLength > markerLength in function 'cv::aruco::CharucoBoard::create' The camera does lit up green but then it crashes and displays the message above. Any suggestions? I also believe that this could be the issue for some of my point cloud scripts being sparse. Due to an uncalibrated camera. I thought I did it correctly the first time however I mistaken it for another process.

@"gdeanrexroth"#p25563 Update- I came across this link:https://luxonis-depthai-hardware.readthedocs-hosted.com/en/latest/pages/guides/calibration/#tof-calibration I followed the git command line"git checkout new_tof_calib", the terminal then outputed this "warning: unable to rmdir 'depthai_calibration': Directory not empty Updating files: 100% (316/316), done. M resources/depthai_boards branch 'new_tof_calib' set up to track 'origin/new_tof_calib'." i ran the calibrate.py file and got a new output within terminal. Here is the result: "Starting image processing Starting image processingxxccx 11 squareX is 11 Device closed in exception.. ERROR: Images not read correctly, check directory Traceback (most recent call last): File "S:\\DEPT\\SVM4\\Shared\\Crossfunctional_Work\\Projects\\DepthCameras\\LuxonisDepthAI\\test_run\\depthai\\calibrate.py", line 968, in calibrate status, result_config = stereo_calib.calibrate( ^^^^^^^^^^^^^^^^^^^^^^^ File "S:\\DEPT\\SVM4\\Shared\\Crossfunctional_Work\\Projects\\DepthCameras\\LuxonisDepthAI\\test_run\\depthai\\depthai_helpers\\calibration_utils.py", line 121, in calibrate ret, intrinsics, dist_coeff, _, _, size = self.calibrate_intrinsics( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "S:\\DEPT\\SVM4\\Shared\\Crossfunctional_Work\\Projects\\DepthCameras\\LuxonisDepthAI\\test_run\\depthai\\depthai_helpers\\calibration_utils.py", line 258, in calibrate_intrinsics assert len( ^^^^ AssertionError: ERROR: Images not read correctly, check directory" Beside switching to the new branch there is nothing that has changed within the code.

Hi @"gdeanrexroth"#2732 ``` git checkout new_tof_calib git pull --recurse-submodules ``` LMK if this is enough to solve the issues. Thanks, Jaka

@"Brandon"#p25 After running the commands that you mentioned, The traceback changed: **python calibrate.py -db -nx 13 -ny 7 -c 1 -cd 0 -s 4 -ms 3 -brd OAK-D-SR-POE(command i entered in terminal)** **[{socket: CAM_A, sensorName: S5K33D, width: 640, height: 480, orientation: ROTATE_180_DEG, supportedTypes: [TOF], hasAutofocus: 0, hasAutofocusIC: 0, name: tof}, {socket: CAM_B, sensorName: OV9782, width: 1280, height: 800, orientation: ROTATE_180_DEG, supportedTypes: [COLOR], hasAutofocus: 0, hasAutofocusIC: 0, name: left}, {socket: CAM_C, sensorName: OV9782, width: 1280, height: 800, orientation: ROTATE_180_DEG, supportedTypes: [COLOR], hasAutofocus: 0, hasAutofocusIC: 0, name: right}]** **Cam: rgb and focus: False** **Cam: left and focus: False** **Cam: right and focus: False** **Camera type ToF** **Traceback (most recent call last):** **File "S:\\DEPT\\SVM4\\Shared\\Crossfunctional_Work\\Projects\\DepthCameras\\LuxonisDepthAI\\test_run\\depthai\\calibrate.py", line 1207, in ** **File "S:\\DEPT\\SVM4\\Shared\\Crossfunctional_Work\\Projects\\DepthCameras\\LuxonisDepthAI\\test_run\\depthai\\calibrate.py", line 333, in __init__** **pipeline = self.create_pipeline()** **^^^^^^^^^^^^^^^^^^^^^^** **File "S:\\DEPT\\SVM4\\Shared\\Crossfunctional_Work\\Projects\\DepthCameras\\LuxonisDepthAI\\test_run\\depthai\\calibrate.py", line 445, in create_pipeline** **tof_config.depthParams.freqModUsed = dai.RawToFConfig.DepthParams.TypeFMod.MIN** **^^^^^^^^^^^^^^^^^^^^^^^^^^^^** **AttributeError: type object 'depthai.RawToFConfig' has no attribute 'DepthParams'**

@"gdeanrexroth"#p25857 However when I commented out line 445, and ran the same command line. The interface to capture the images pops up on the screen. But the results are different: **Cam: rgb and focus: False** **Cam: left and focus: False** **Cam: right and focus: False** **Camera type ToF** **Starting image capture. Press the [ESC] key to abort.** **Will take 13 total images, 1 per each polygon.** **left 0:01:25.766316** **new minimum: {'ts': 0.0, 'indicies': {'left': 0}} min required: 0.2** **synced frames: None** **right 0:01:25.766330** **new minimum: {'ts': 1.3999999993075107e-05, 'indicies': {'left': 0, 'right': 0}} min required: 0.2** **synced frames: None** **rgb 0:01:25.667397** **new minimum: {'ts': 0.09893300000000238, 'indicies': {'left': 0, 'right': 0, 'rgb': 0}} min required: 0.2** **synced frames: None** **left 0:01:25.866318** **new minimum: {'ts': 0.09893300000000238, 'indicies': {'left': 0, 'right': 0, 'rgb': 0}} min required: 0.2** **synced frames: None** **right 0:01:25.866340** **rgb 0:01:25.767397** **new minimum: {'ts': 0.09893300000000238, 'indicies': {'left': 0, 'right': 0, 'rgb': 0}} min required: 0.2** **new minimum: {'ts': 0.0010949999999922966, 'indicies': {'left': 0, 'right': 0, 'rgb': 1}} min required: 0.2** **synced frames: None** **left 0:01:25.966316** **new minimum: {'ts': 0.09893300000000238, 'indicies': {'left': 0, 'right': 0, 'rgb': 0}} min required: 0.2** **new minimum: {'ts': 0.0010949999999922966, 'indicies': {'left': 0, 'right': 0, 'rgb': 1}} min required: 0.2** **synced frames: None** **right 0:01:25.966328** **new minimum: {'ts': 0.09893300000000238, 'indicies': {'left': 0, 'right': 0, 'rgb': 0}} min required: 0.2** **new minimum: {'ts': 0.0010949999999922966, 'indicies': {'left': 0, 'right': 0, 'rgb': 1}} min required: 0.2** **synced frames: None** **rgb 0:01:25.867395** **new minimum: {'ts': 0.09893300000000238, 'indicies': {'left': 0, 'right': 0, 'rgb': 0}} min required: 0.2** **new minimum: {'ts': 0.0010949999999922966, 'indicies': {'left': 0, 'right': 0, 'rgb': 1}} min required: 0.2** **synced frames: None** **rgb 0:01:25.967395** **new minimum: {'ts': 0.09893300000000238, 'indicies': {'left': 0, 'right': 0, 'rgb': 0}} min required: 0.2** **new minimum: {'ts': 0.0010949999999922966, 'indicies': {'left': 0, 'right': 0, 'rgb': 1}} min required: 0.2** **synced frames: None** **right 0:01:26.066337** **left 0:01:26.066318** **new minimum: {'ts': 0.09893300000000238, 'indicies': {'left': 0, 'right': 0, 'rgb': 0}} min required: 0.2** **new minimum: {'ts': 0.0010949999999922966, 'indicies': {'left': 0, 'right': 0, 'rgb': 1}} min required: 0.2** **synced frames: None** **left 0:01:26.166316** **new minimum: {'ts': 0.09893300000000238, 'indicies': {'left': 0, 'right': 0, 'rgb': 0}} min required: 0.2** **new minimum: {'ts': 0.0010949999999922966, 'indicies': {'left': 0, 'right': 0, 'rgb': 1}} min required: 0.2** **synced frames: None** **right 0:01:26.166329** **new minimum: {'ts': 0.09893300000000238, 'indicies': {'left': 0, 'right': 0, 'rgb': 0}} min required: 0.2** **new minimum: {'ts': 0.0010949999999922966, 'indicies': {'left': 0, 'right': 0, 'rgb': 1}} min required: 0.2** **synced frames: None** **rgb 0:01:26.067393** **new minimum: {'ts': 0.09893300000000238, 'indicies': {'left': 0, 'right': 0, 'rgb': 0}} min required: 0.2** **new minimum: {'ts': 0.0010949999999922966, 'indicies': {'left': 0, 'right': 0, 'rgb': 1}} min required: 0.2** **Returning synced messages with error: 0.0010949999999922966 {'left': 0, 'right': 0, 'rgb': 1}** **synced frames: {'left': , 'right': , 'rgb': }** **Timestamp of left is 0:01:25.766316** **Timestamp of right is 0:01:25.766330** **Timestamp of rgb is 0:01:25.767397** **Traceback (most recent call last):** **File "S:\\test_run\\depthai\\calibrate.py", line 1207, in ** **Main().run()** **File "S:\\test_run\\depthai\\calibrate.py", line 1196, in run** **self.capture_images_sync()** **File "S:\\DEPT\\SVM4\\Shared\\Crossfunctional_Work\\Projects\\DepthCameras\\LuxonisDepthAI\\test_run\\depthai\\calibrate.py", line 590, in capture_images_sync** **gray_frame = cv2.cvtColor(frameMsg.getCvFrame(), cv2.COLOR_BGR2GRAY)** **^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^** **cv2.error: OpenCV(4.5.5) d:\\a\\opencv-python\\opencv-python\\opencv\\modules\\imgproc\\src\\color.simd_helpers.hpp:92: error: (-2:Unspecified error) in function '__cdecl cv::impl::\`anonymous-namespace'::CvtHelper Invalid number of channels in input image:** **> 'VScn::contains(scn)'** **> where** **> 'scn' is 1**

@"gdeanrexroth"#p25858 Update: Disregard the above comments. I followed the git submodule command that you mentioned. It created the "new*_tof_calib" branch on my local computer. As i was going through the code and debugging it. Most of the methods and functions are compatible with different python libraries that are not installed on my computer. After going through various documentation, I changed my conda environment based by the updated script. Now that it is updated, I was able to run the code and display the interface as coded to do. But whenever I click the space bar to capture the images, it prompt this**"python calibrate.py -db charuco_24inch_13x7 -nx 13 -ny 7 -c 1 -cd 0 -s 4 -ms 3 -brd OAK-D-SR-POE*** ***Cam: rgb and focus: False*** ***Cam: left and focus: False*** ***Cam: right and focus: False*** ***Saving dataset to: dataset*** ***Sensor name for left is OV9782*** ***Sensor name for right is OV9782*** ***Camera type ToF*** ***Starting image capture. Press the [ESC] key to abort.*** ***Will take 13 total images.*** ***Start capturing...*** ***new minimum: {'ts': 0.09893999999999892, 'indicies': {'left': 0, 'right': 0, 'rgb': 0}} min required: 0.2*** ***new minimum: {'ts': 0.0010839999999987526, 'indicies': {'left': 0, 'right': 0, 'rgb': 1}} min required: 0.2*** ***Time stamp of left is 3:23:16.393060*** ***Markers count ... 39*** ***Total markers needed -> 18*** ***py: Saved image as: dataset\\left\\p0_0.png*** ***Status of left is True*** ***Time stamp of right is 3:23:16.393072*** ***Markers count ... 25*** ***Total markers needed -> 18*** ***py: Saved image as: dataset\\right\\p0_0.png*** ***Status of right is True*** ***Time stamp of rgb is 3:23:16.394132*** ***Markers count ... 0*** ***Total markers needed -> 18*** ***Status of rgb is False*** ***py: Capture failed, unable to find chessboard! Fix position and press spacebar again"***

@"gdeanrexroth"#p25870 https://github.com/luxonis/depthai/tree/tof_calib

@"jakaskerl"#p25900 With previous documentation, users have had issues with this error: **import depthai_calibration.calibration_utils as calibUtils** **ModuleNotFoundError: No module named 'depthai_calibration.calibration_utils'** \*\*I am having the same error, on one post you recommended this "\*\*Change .git/config under a given submodule from ssh path to https path: $$ active = true - url = git@github.com:luxonis/depthai-calibration.git + url = https://github.com/luxonis/depthai-calibration.git $$ **" I have tried this method but it prompted me this message:fatal: no submodule mapping found in .gitmodules for path 'depthai-calibration' To add onto this, i have changed the submodule url to test if it would do anything different. I then ran the command "git submodule update --init --recursive" and it resulted in "fatal: No url found for submodule path 'resources/depthai-boards' in .gitmodules". My main error is the module not being found.**

@"gdeanrexroth"#p25939 Go to main branch, init the submodules, then switch to tof_calib. Should work then. Just tried by purging the whole clone. Thanks, Jaka

@"jakaskerl"#p25950 Before you made your comment I have tried to update my submodule through the depthai_calibration. then i ran the command "python calibrate.py -db 24_charuco*13x7 -nx 13 -ny 7 -c 1 -cd 0 -s 4 -ms 3 -brd OAK-D-SR-POE". But my results are below. I am on the tof*_calib branch. The interface which displays the info popped up and it allowed me to click space. But the errors prompted at the bottom. Is there a loop within the code that could be causing this? Cam: rgb and focus: False Cam: left and focus: False Cam: right and focus: False Saving dataset to: dataset Camera type ToF Sensor name for right is OV9782 Camera type ToF Starting image capture. Press the [ESC] key to abort. Will take 13 total images. [14442C10A1C6AECF00] [169.254.1.222] [7.072] [ToF(1)] [error] Unexpected input image size 1280 x 800, maybe connected camera is not ToF?

@"gdeanrexroth"#p25953 I think you might still be using outdated submodule for boards where there are two tof sensors inside SR-POE.json. Correct config. **When** calibrating the SR POE, make sure to use a physical board, not a monitor. Monitors have IR filters and will not reflect the light - so charuco pattern will not be seen. ``` { "board_config": { "name": "OAK-D-SR-POE", "revision": "R0M0E0", "cameras":{ "CAM_B": { "name": "left", "hfov": 71.86, "type": "color", "extrinsics": { "to_cam": "CAM_C", "specTranslation": { "x": -2.0, "y": 0, "z": 0 }, "rotation":{ "r": 0, "p": 0, "y": 0 } } }, "CAM_C": { "name": "right", "hfov": 71.86, "type": "color", "extrinsics": { "to_cam": "CAM_A", "specTranslation": { "x": -1.7382, "y": 0, "z": 0 }, "rotation":{ "r": 0, "p": 0, "y": 0 } } }, "CAM_A": { "name": "tof", "hfov": 71.86, "type": "tof" } }, "stereo_config":{ "left_cam": "CAM_B", "right_cam": "CAM_C" }, "imuExtrinsics": { "sensors":{ "BNO": { "name" : "BNO086", "extrinsics": { "to_cam": "LEFT", "specTranslation": { "x": 0.2915, "y": -0.069, "z": 0.2832 }, "rotation":{ "r": 180, "p": 0, "y": 270 } } } } } } } ```

Oak-d-sr-poe calibration

jakaskerl

gdeanrexroth
External lights could contribute to the noise. From our tests (let me quote): more light makes more outliers and also it make bigger variance of the error.

What do you mean by unnecessary noise? How much noise are we talking about. I'm afraid you are reaching HW limits. The image you sent looks pretty good.

Thanks,
Jaka

gdeanrexroth

jakaskerl
Can you explain the HW limits you are referring to?

It may not be unnecessary noise, but what I am trying to express is more so the formality of the objects at hand. Here are examples of what I mean. I am providing the tof depth and rgb window to show you depth and raw color image. The provide screenshot shows the side of the point cloud. I have used the global registration method to combine two point clouds into one. It does look good as you have said. I agree with that. I wondering why the sides are struggling to form.

gdeanrexroth

jakaskerl
The issue I am coming in with is that with both filtration and just displaying the point cloud as it is. I am still experiencing lingering point clouds. Versus getting the pcd to look like this. So again the point cloud that I have is cleaned. Yet I am trying to get it to form like the example below. I took this example from the link:https://www.open3d.org/docs/latest/tutorial/Advanced/pointcloud_outlier_removal.html
Also i have applied global, icp registration to the pcd and the outcome has been nearly the same as it without the filters. Could this be a calculation issue or maybe something that I am not directly doing right?

gdeanrexroth

gdeanrexroth
I have 287685 points within the pcd file. Is this too much to capture at one time?

jakaskerl

gdeanrexroth

gdeanrexroth I have 287685 points within the pcd file. Is this too much to capture at one time?

As long as you PC is capable enough, no.

gdeanrexroth I have used the global registration method to combine two point clouds into one.

The pointclouds don't look fully aligned.. I know you don't need alignment to perform global registration but this seems off.
How do stereo and tof images look like? I would suspect that if the perspectives are too different, the registration won't work.

Thanks,
Jaka

gdeanrexroth

jakaskerl Yes the points are not aligning. I have tried both icp and global registration. I will try to find a way to give you both stereo and tof. I have already given you the tof in the above image. The registration process works, it does remove the outlier points and remove back points. My last goal is simply having it combine into one, and having the objects points aligned together as one.

« Previous Page