Hi from Pollen Robotics :) We just received our ToF module. Using [this example](https://github.com/luxonis/depthai-python/blob/main/examples/ToF/tof_depth.py) we get a pretty nice looking depthmap [upl-image-preview url=https://discuss.luxonis.com/assets/files/2023-10-18/1697632040-137414-capture-decran-du-2023-10-18-14-27-14.png] (Except for the looping colors, but I assume that over a few meters the sensor is not accurate anymore) (We are using an OAK FFC-4P btw) However, when I try to mesure distances, they are pretty far off most of the time. For example Real distance (black circle) \~= 380 [upl-image-preview url=https://discuss.luxonis.com/assets/files/2023-10-18/1697635664-666344-capture-decran-du-2023-10-18-15-23-55.png] Real distance \~= 340 [upl-image-preview url=https://discuss.luxonis.com/assets/files/2023-10-18/1697635721-65842-capture-decran-du-2023-10-18-15-25-52.png] Real distance \~= 195 [upl-image-preview url=https://discuss.luxonis.com/assets/files/2023-10-18/1697635742-552214-capture-decran-du-2023-10-18-15-26-09.png] Real distance \~= 880 [upl-image-preview url=https://discuss.luxonis.com/assets/files/2023-10-18/1697635754-323551-capture-decran-du-2023-10-18-15-27-03.png] A few questions : * I don't understand what the different parameters are doing, and the [documentation](https://docs.luxonis.com/projects/api/en/latest/components/messages/tof_config/#tofconfig) is not very helpful to me (this is the first time I work with a ToF sensor). Could you explain what **freqModUsed, avgPhaseShuffle, and minimumAmplitude** are doing ? * Am I right to expect millimeters out of the depth map ? * Is there any calibration needed ? Could that have been degraded during transport ? Here is my code for depth mesurement in the image (at the position of the cursor), based on the provided example ``` #!/usr/bin/env python3 import cv2 import depthai as dai import numpy as np pipeline = dai.Pipeline() cam_a = pipeline.create(dai.node.Camera) cam_a.setBoardSocket(dai.CameraBoardSocket.CAM_A) tof = pipeline.create(dai.node.ToF) # Configure the ToF node tofConfig = tof.initialConfig.get() tofConfig.depthParams.freqModUsed = dai.RawToFConfig.DepthParams.TypeFMod.MAX tofConfig.depthParams.avgPhaseShuffle = False tofConfig.depthParams.minimumAmplitude = 3.0 tof.initialConfig.set(tofConfig) cam_a.raw.link(tof.input) xout = pipeline.create(dai.node.XLinkOut) xout.setStreamName("depth") tof.depth.link(xout.input) last_mouse_pos = (0, 0) last_depth_mesurement = 0 def mouse_callback(event, x, y, flags, params): global last_mouse_pos last_mouse_pos = (x, y) cv2.namedWindow("colorized depth") cv2.setMouseCallback("colorized depth", mouse_callback) time_averaged_depth = [] # Connect to device and start pipeline with dai.Device(pipeline) as device: print('Connected cameras:', device.getConnectedCameraFeatures()) q = device.getOutputQueue(name="depth") while True: imgFrame = q.get() depth_map = imgFrame.getFrame() depth_downscaled = depth_map[::4] non_zero_depth = depth_downscaled[depth_downscaled != 0] # Remove invalid depth values if len(non_zero_depth) == 0: min_depth, max_depth = 0, 0 else: min_depth = np.percentile(non_zero_depth, 3) max_depth = np.percentile(non_zero_depth, 97) last_depth_mesurement = depth_map[last_mouse_pos[1]-5:last_mouse_pos[1]+5, last_mouse_pos[0]-5:last_mouse_pos[0]+5] last_depth_mesurement = np.mean(last_depth_mesurement) time_averaged_depth.append(last_depth_mesurement) time_averaged_depth = time_averaged_depth[20:] if len(time_averaged_depth) > 20 else time_averaged_depth print("mean :", np.mean(time_averaged_depth), "(std :", np.std(time_averaged_depth), ")") depth_colorized = np.interp(depth_map, (min_depth, max_depth), (0, 255)).astype(np.uint8) depth_colorized = cv2.applyColorMap(depth_colorized, cv2.COLORMAP_JET) depth_colorized = cv2.circle(depth_colorized, last_mouse_pos, 5, (0, 0, 0), -1) cv2.imshow("colorized depth", depth_colorized) if cv2.waitKey(1) == ord('q'): break ``` Thanks ! Antoine

Hi @"apirrone"#p13338 , We apologize for the inconvenience, but the full TOF decoding is not yet implemented, and we are working on it at the moment. It should be ready in firmware in a few week, so you'll be able to just update the library (pip install depthai -U) and have accurate depth. Kind regards, Erik

Thanks @"erik"#p13367 for the reply, please keep me updated when the update is ready! Just curious, how is it that it "kind of works" right now, but with bad accuracy? The colorized depth map looks really good and continuous 😅

Hi @"apirrone"#p13369 , That's the thing - we get _some_ data out, but full decoding and error correction aren't implemented. The depth does look decent (at least the corners are very sharp), but if you'd visualize the pointcloud it's completely off.

Hi @"erik"#175, Any news on the firmware ? Thanks ! Antoine

Hi @"apirrone"#1318 , We got the decoding mostly working, there's just some offset left that we haven't figured out. Below is an A4 paper sheet and its size - about 10% dimension mismatch. We believe we'll have it ready this month. Thanks, Erik [upl-image-preview url=https://discuss.luxonis.com/assets/files/2023-11-06/1699279674-367947-a4-take3.png] [upl-image-preview url=https://discuss.luxonis.com/assets/files/2023-11-06/1699279674-443357-a4-take2.png] [upl-image-preview url=https://discuss.luxonis.com/assets/files/2023-11-06/1699279674-578178-a4-take1.png]

Hi @"erik"#175, any news ? We are very eager to try it out ! :) Thanks, Antoine

Hi @"apirrone"#p14695 We need to finalize a few things, I'd say roughly about 2 weeks.. Thanks, Jaka

@"jakaskerl"#p14717 hey any update on the ToF sensor finalization so its accurate and ready to be used in depthai?

ToF bad accuracy

erik

ToF is already calibrated, the only thing that's missing is the decoding of the ToF results. Current estimations are end of the month, we apologize for the inconvenience.

apirrone

Hi @erik @jakaskerl any news ? 🙂

tetsuro_okuyama

Hi, @erik When is the Firmware update?

Best regards,

erik

Hi @apirrone @tetsuro_okuyama ,
Sorry about the delay - we have had a bunch of challenges, and are now working on 3 different routines to get this decoding done asap (contractor, vendor, and in-house). We just got some promising results last week, and I think we'll be able to share initial lib version (and some pointclouds) in a week or two. More info to come, we'll likely make a blog post about it.
Thanks, Erik

tetsuro_okuyama

@erik Thanks for the quick reply. I'll look forward to it!

Nearpoint

erik

is the code for this available anywhere? it would be a nice test and a quick demo of its accuracy once it's ready, as we all have A4 sized paper

Nearpoint

maybe the demo script could adjusted to recognize a quarter highlight it and provide its x, y, and z dimensions as a demo anyone can quickly run with the tof camera once calibrated.

Nearpoint

@erik any update on the progress for completing the ToF camera code so the accuracy is fixed and we can begin utilizing the component. The ToF camera integration into our product is on hold since the software you guys are working on isnt complete causing invalid accuracy. But once I get word that the software is complete I will be able to start the integration! Would be exciting to start working with the ToF module this week, fingers crossed, lmk! thank u!

erik

@Nearpoint not yet, let me check with the team

tetsuro_okuyama

Hi @erik, Any update?

erik

Hi @tetsuro_okuyama @Nearpoint

Thank you for patience. We have some exciting updates 🙂 We got everything ready, the only missing part was calibration together with the front glass (reflections..), but we just got it figured out as well.

At 75cm, depth error across FOV (below 2% everywhere):

At 1m, depth error across FOV (below 2% everywhere as well):

So now it's only the manufacturing (and procuring the new front glass) + calibration. If you already received an OAK-D-SR-POE, I'd suggest sending an email to support@luxonis.com and we can arrange replacing the unit with one that was freshly calibrated.

Thanks, Erik

tetsuro_okuyama

erik Thank you for your reply!
I'll mail you gave me.

Best regards,

Nearpoint

erik

The front glass, is that only related to the Enclosed OAK Cameras not the modules? I have the ToF sensor sold as an embedded module to integrate into our product. Does the lens of the module need to be replaced as the front glass or is the front glass only related to the Enclosure for the OAK camera devices?

Otherwise, as far as the ToF 33D module is concerned, I would just need to calibrate it with the new updates correct?

apirrone

Hi @erik, great news !

We are pretty much in the same situation as @Nearpoint. We have the ToF module that we want to integrate in our product. What should we do now to be able to use it properly ?

Thanks !

Antoine

erik

Hi @apirrone ,
If you already have the module/camera, for best accuracy you'd need to replace it with a newly calibrated one (email support@luxonis.com). We will be updating calibration scripts that will let you self-calibrate, but have not yet done that.
Thanks, Erik

Matthieu

erik We have ordered recently a new ToF module, it has been shipped today (order 16554). Is it the new version already ?

erik

@Matthieu best to ask support team🙂

apirrone

Hi @erik @jakaskerl

We received our calibrated ToF modules !

I updated the depthai library to the latest (2.27.0.0) and tried to run these examples on my FFC-4P :

I get the following outputs :

[2024-07-01 11:55:21.392] [depthai] [warning] [18443010F19A5B1200] [1.2] Flashed bootloader version 0.0.21, less than 0.0.28 is susceptible to bootup/restart failure. Upgrading is advised, flashing main/factory (not user) bootloader. Available: 0.0.28
Connected cameras: [{socket: CAM_B, sensorName: S5K33D, width: 640, height: 480, orientation: AUTO, supportedTypes: [MONO], hasAutofocus: 0, hasAutofocusIC: 0, name: left}, {socket: CAM_C, sensorName: OV9282, width: 1280, height: 800, orientation: AUTO, supportedTypes: [MONO], hasAutofocus: 0, hasAutofocusIC: 0, name: right}]
[18443010F19A5B1200] [1.2] [5.894] [ToF(0)] [error] ERROR unexpected frequency code: 70

[18443010F19A5B1200] [1.2] [5.894] [ToF(0)] [error] ERROR unexpected phase code: 15

[18443010F19A5B1200] [1.2] [5.894] [ToF(0)] [error] Orientation not supported! Only normal and 180 flip(not mirrored) can be used.
[18443010F19A5B1200] [1.2] [5.894] [ToF(0)] [error] ERROR unexpected frequency code: 105

[18443010F19A5B1200] [1.2] [5.894] [ToF(0)] [error] ERROR unexpected phase code: 21

[18443010F19A5B1200] [1.2] [5.894] [ToF(0)] [error] Orientation not supported! Only normal and 180 flip(not mirrored) can be used.
[18443010F19A5B1200] [1.2] [7.787] [ToF(0)] [warning] Temperature coefficients are not adequate in the EEPROM. Disabling temperature correction.

The script still runs while outputting these errors and I get a nonsensical output frame

tof-2024-07-01-115654.mp4

2MB

Do I have to flash the latest version of the bootloader as suggested by the first warning ?

Am I doing something wrong ?

Thanks !

apirrone

FYI, I tried with a newer FFC-4P and got a different result :

[2024-07-01 15:55:40.541] [depthai] [warning] [14442C10019342D200] [1.2] Flashed bootloader version 0.0.26, less than 0.0.28 is susceptible to bootup/restart failure. Upgrading is advised, flashing main/factory (not user) bootloader. Available: 0.0.28
Connected cameras: [{socket: CAM_A, sensorName: S5K33D, width: 640, height: 480, orientation: AUTO, supportedTypes: [TOF], hasAutofocus: 0, hasAutofocusIC: 0, name: color}, {socket: CAM_B, sensorName: IMX296, width: 1440, height: 1080, orientation: AUTO, supportedTypes: [MONO], hasAutofocus: 0, hasAutofocusIC: 0, name: left}, {socket: CAM_C, sensorName: IMX296, width: 1440, height: 1080, orientation: AUTO, supportedTypes: [MONO], hasAutofocus: 0, hasAutofocusIC: 0, name: right}]
[14442C10019342D200] [1.2] [5.762] [ToF(0)] [error] Couldn't read CCM EEPROM: I/O error
[14442C10019342D200] [1.2] [5.762] [ToF(0)] [error] Error: fppn_80 Could not read header data.

With an image that looks like this

apirrone

EDIT : this was with the second ToF that we received, I still have the same issue with the first one. Sorry for the confusion

apirrone

Ok, by playing around with the parameters I get this :

tof-2024-07-01-161528.mp4

10MB

It's better, but something weird is happening on the right half of the image (or the left)

« Previous Page Next Page »