Hi from Pollen Robotics :) We just received our ToF module. Using [this example](https://github.com/luxonis/depthai-python/blob/main/examples/ToF/tof_depth.py) we get a pretty nice looking depthmap [upl-image-preview url=https://discuss.luxonis.com/assets/files/2023-10-18/1697632040-137414-capture-decran-du-2023-10-18-14-27-14.png] (Except for the looping colors, but I assume that over a few meters the sensor is not accurate anymore) (We are using an OAK FFC-4P btw) However, when I try to mesure distances, they are pretty far off most of the time. For example Real distance (black circle) \~= 380 [upl-image-preview url=https://discuss.luxonis.com/assets/files/2023-10-18/1697635664-666344-capture-decran-du-2023-10-18-15-23-55.png] Real distance \~= 340 [upl-image-preview url=https://discuss.luxonis.com/assets/files/2023-10-18/1697635721-65842-capture-decran-du-2023-10-18-15-25-52.png] Real distance \~= 195 [upl-image-preview url=https://discuss.luxonis.com/assets/files/2023-10-18/1697635742-552214-capture-decran-du-2023-10-18-15-26-09.png] Real distance \~= 880 [upl-image-preview url=https://discuss.luxonis.com/assets/files/2023-10-18/1697635754-323551-capture-decran-du-2023-10-18-15-27-03.png] A few questions : * I don't understand what the different parameters are doing, and the [documentation](https://docs.luxonis.com/projects/api/en/latest/components/messages/tof_config/#tofconfig) is not very helpful to me (this is the first time I work with a ToF sensor). Could you explain what **freqModUsed, avgPhaseShuffle, and minimumAmplitude** are doing ? * Am I right to expect millimeters out of the depth map ? * Is there any calibration needed ? Could that have been degraded during transport ? Here is my code for depth mesurement in the image (at the position of the cursor), based on the provided example ``` #!/usr/bin/env python3 import cv2 import depthai as dai import numpy as np pipeline = dai.Pipeline() cam_a = pipeline.create(dai.node.Camera) cam_a.setBoardSocket(dai.CameraBoardSocket.CAM_A) tof = pipeline.create(dai.node.ToF) # Configure the ToF node tofConfig = tof.initialConfig.get() tofConfig.depthParams.freqModUsed = dai.RawToFConfig.DepthParams.TypeFMod.MAX tofConfig.depthParams.avgPhaseShuffle = False tofConfig.depthParams.minimumAmplitude = 3.0 tof.initialConfig.set(tofConfig) cam_a.raw.link(tof.input) xout = pipeline.create(dai.node.XLinkOut) xout.setStreamName("depth") tof.depth.link(xout.input) last_mouse_pos = (0, 0) last_depth_mesurement = 0 def mouse_callback(event, x, y, flags, params): global last_mouse_pos last_mouse_pos = (x, y) cv2.namedWindow("colorized depth") cv2.setMouseCallback("colorized depth", mouse_callback) time_averaged_depth = [] # Connect to device and start pipeline with dai.Device(pipeline) as device: print('Connected cameras:', device.getConnectedCameraFeatures()) q = device.getOutputQueue(name="depth") while True: imgFrame = q.get() depth_map = imgFrame.getFrame() depth_downscaled = depth_map[::4] non_zero_depth = depth_downscaled[depth_downscaled != 0] # Remove invalid depth values if len(non_zero_depth) == 0: min_depth, max_depth = 0, 0 else: min_depth = np.percentile(non_zero_depth, 3) max_depth = np.percentile(non_zero_depth, 97) last_depth_mesurement = depth_map[last_mouse_pos[1]-5:last_mouse_pos[1]+5, last_mouse_pos[0]-5:last_mouse_pos[0]+5] last_depth_mesurement = np.mean(last_depth_mesurement) time_averaged_depth.append(last_depth_mesurement) time_averaged_depth = time_averaged_depth[20:] if len(time_averaged_depth) > 20 else time_averaged_depth print("mean :", np.mean(time_averaged_depth), "(std :", np.std(time_averaged_depth), ")") depth_colorized = np.interp(depth_map, (min_depth, max_depth), (0, 255)).astype(np.uint8) depth_colorized = cv2.applyColorMap(depth_colorized, cv2.COLORMAP_JET) depth_colorized = cv2.circle(depth_colorized, last_mouse_pos, 5, (0, 0, 0), -1) cv2.imshow("colorized depth", depth_colorized) if cv2.waitKey(1) == ord('q'): break ``` Thanks ! Antoine

Hi @"apirrone"#p13338 , We apologize for the inconvenience, but the full TOF decoding is not yet implemented, and we are working on it at the moment. It should be ready in firmware in a few week, so you'll be able to just update the library (pip install depthai -U) and have accurate depth. Kind regards, Erik

Thanks @"erik"#p13367 for the reply, please keep me updated when the update is ready! Just curious, how is it that it "kind of works" right now, but with bad accuracy? The colorized depth map looks really good and continuous 😅

Hi @"apirrone"#p13369 , That's the thing - we get _some_ data out, but full decoding and error correction aren't implemented. The depth does look decent (at least the corners are very sharp), but if you'd visualize the pointcloud it's completely off.

Hi @"erik"#175, Any news on the firmware ? Thanks ! Antoine

Hi @"apirrone"#1318 , We got the decoding mostly working, there's just some offset left that we haven't figured out. Below is an A4 paper sheet and its size - about 10% dimension mismatch. We believe we'll have it ready this month. Thanks, Erik [upl-image-preview url=https://discuss.luxonis.com/assets/files/2023-11-06/1699279674-367947-a4-take3.png] [upl-image-preview url=https://discuss.luxonis.com/assets/files/2023-11-06/1699279674-443357-a4-take2.png] [upl-image-preview url=https://discuss.luxonis.com/assets/files/2023-11-06/1699279674-578178-a4-take1.png]

Hi @"erik"#175, any news ? We are very eager to try it out ! :) Thanks, Antoine

ToF bad accuracy

apirrone

Hi @erik, any news ? We are very eager to try it out ! 🙂

Thanks, Antoine

jakaskerl

Hi apirrone
We need to finalize a few things, I'd say roughly about 2 weeks..

Thanks,
Jaka

Nearpoint

jakaskerl

hey any update on the ToF sensor finalization so its accurate and ready to be used in depthai?

Nearpoint

jakaskerl

Also do you have the instructions for how to calibrate the ToF sensor? Do you calibrate the ToF sensor with each camera in the setup?

apirrone

Thanks for the update!

Antoine

erik

ToF is already calibrated, the only thing that's missing is the decoding of the ToF results. Current estimations are end of the month, we apologize for the inconvenience.

apirrone

Hi @erik @jakaskerl any news ? 🙂

tetsuro_okuyama

Hi, @erik When is the Firmware update?

Best regards,

erik

Hi @apirrone @tetsuro_okuyama ,
Sorry about the delay - we have had a bunch of challenges, and are now working on 3 different routines to get this decoding done asap (contractor, vendor, and in-house). We just got some promising results last week, and I think we'll be able to share initial lib version (and some pointclouds) in a week or two. More info to come, we'll likely make a blog post about it.
Thanks, Erik

tetsuro_okuyama

@erik Thanks for the quick reply. I'll look forward to it!

Nearpoint

erik

is the code for this available anywhere? it would be a nice test and a quick demo of its accuracy once it's ready, as we all have A4 sized paper

Nearpoint

maybe the demo script could adjusted to recognize a quarter highlight it and provide its x, y, and z dimensions as a demo anyone can quickly run with the tof camera once calibrated.

Nearpoint

@erik any update on the progress for completing the ToF camera code so the accuracy is fixed and we can begin utilizing the component. The ToF camera integration into our product is on hold since the software you guys are working on isnt complete causing invalid accuracy. But once I get word that the software is complete I will be able to start the integration! Would be exciting to start working with the ToF module this week, fingers crossed, lmk! thank u!

erik

@Nearpoint not yet, let me check with the team

tetsuro_okuyama

Hi @erik, Any update?

erik

Hi @tetsuro_okuyama @Nearpoint

Thank you for patience. We have some exciting updates 🙂 We got everything ready, the only missing part was calibration together with the front glass (reflections..), but we just got it figured out as well.

At 75cm, depth error across FOV (below 2% everywhere):

At 1m, depth error across FOV (below 2% everywhere as well):

So now it's only the manufacturing (and procuring the new front glass) + calibration. If you already received an OAK-D-SR-POE, I'd suggest sending an email to support@luxonis.com and we can arrange replacing the unit with one that was freshly calibrated.

Thanks, Erik

tetsuro_okuyama

erik Thank you for your reply!
I'll mail you gave me.

Best regards,

Nearpoint

erik

The front glass, is that only related to the Enclosed OAK Cameras not the modules? I have the ToF sensor sold as an embedded module to integrate into our product. Does the lens of the module need to be replaced as the front glass or is the front glass only related to the Enclosure for the OAK camera devices?

Otherwise, as far as the ToF 33D module is concerned, I would just need to calibrate it with the new updates correct?

apirrone

Hi @erik, great news !

We are pretty much in the same situation as @Nearpoint. We have the ToF module that we want to integrate in our product. What should we do now to be able to use it properly ?

Thanks !

Antoine

erik

Hi @apirrone ,
If you already have the module/camera, for best accuracy you'd need to replace it with a newly calibrated one (email support@luxonis.com). We will be updating calibration scripts that will let you self-calibrate, but have not yet done that.
Thanks, Erik

Matthieu

erik We have ordered recently a new ToF module, it has been shipped today (order 16554). Is it the new version already ?