Hi from Pollen Robotics :) We just received our ToF module. Using [this example](https://github.com/luxonis/depthai-python/blob/main/examples/ToF/tof_depth.py) we get a pretty nice looking depthmap [upl-image-preview url=https://discuss.luxonis.com/assets/files/2023-10-18/1697632040-137414-capture-decran-du-2023-10-18-14-27-14.png] (Except for the looping colors, but I assume that over a few meters the sensor is not accurate anymore) (We are using an OAK FFC-4P btw) However, when I try to mesure distances, they are pretty far off most of the time. For example Real distance (black circle) \~= 380 [upl-image-preview url=https://discuss.luxonis.com/assets/files/2023-10-18/1697635664-666344-capture-decran-du-2023-10-18-15-23-55.png] Real distance \~= 340 [upl-image-preview url=https://discuss.luxonis.com/assets/files/2023-10-18/1697635721-65842-capture-decran-du-2023-10-18-15-25-52.png] Real distance \~= 195 [upl-image-preview url=https://discuss.luxonis.com/assets/files/2023-10-18/1697635742-552214-capture-decran-du-2023-10-18-15-26-09.png] Real distance \~= 880 [upl-image-preview url=https://discuss.luxonis.com/assets/files/2023-10-18/1697635754-323551-capture-decran-du-2023-10-18-15-27-03.png] A few questions : * I don't understand what the different parameters are doing, and the [documentation](https://docs.luxonis.com/projects/api/en/latest/components/messages/tof_config/#tofconfig) is not very helpful to me (this is the first time I work with a ToF sensor). Could you explain what **freqModUsed, avgPhaseShuffle, and minimumAmplitude** are doing ? * Am I right to expect millimeters out of the depth map ? * Is there any calibration needed ? Could that have been degraded during transport ? Here is my code for depth mesurement in the image (at the position of the cursor), based on the provided example ``` #!/usr/bin/env python3 import cv2 import depthai as dai import numpy as np pipeline = dai.Pipeline() cam_a = pipeline.create(dai.node.Camera) cam_a.setBoardSocket(dai.CameraBoardSocket.CAM_A) tof = pipeline.create(dai.node.ToF) # Configure the ToF node tofConfig = tof.initialConfig.get() tofConfig.depthParams.freqModUsed = dai.RawToFConfig.DepthParams.TypeFMod.MAX tofConfig.depthParams.avgPhaseShuffle = False tofConfig.depthParams.minimumAmplitude = 3.0 tof.initialConfig.set(tofConfig) cam_a.raw.link(tof.input) xout = pipeline.create(dai.node.XLinkOut) xout.setStreamName("depth") tof.depth.link(xout.input) last_mouse_pos = (0, 0) last_depth_mesurement = 0 def mouse_callback(event, x, y, flags, params): global last_mouse_pos last_mouse_pos = (x, y) cv2.namedWindow("colorized depth") cv2.setMouseCallback("colorized depth", mouse_callback) time_averaged_depth = [] # Connect to device and start pipeline with dai.Device(pipeline) as device: print('Connected cameras:', device.getConnectedCameraFeatures()) q = device.getOutputQueue(name="depth") while True: imgFrame = q.get() depth_map = imgFrame.getFrame() depth_downscaled = depth_map[::4] non_zero_depth = depth_downscaled[depth_downscaled != 0] # Remove invalid depth values if len(non_zero_depth) == 0: min_depth, max_depth = 0, 0 else: min_depth = np.percentile(non_zero_depth, 3) max_depth = np.percentile(non_zero_depth, 97) last_depth_mesurement = depth_map[last_mouse_pos[1]-5:last_mouse_pos[1]+5, last_mouse_pos[0]-5:last_mouse_pos[0]+5] last_depth_mesurement = np.mean(last_depth_mesurement) time_averaged_depth.append(last_depth_mesurement) time_averaged_depth = time_averaged_depth[20:] if len(time_averaged_depth) > 20 else time_averaged_depth print("mean :", np.mean(time_averaged_depth), "(std :", np.std(time_averaged_depth), ")") depth_colorized = np.interp(depth_map, (min_depth, max_depth), (0, 255)).astype(np.uint8) depth_colorized = cv2.applyColorMap(depth_colorized, cv2.COLORMAP_JET) depth_colorized = cv2.circle(depth_colorized, last_mouse_pos, 5, (0, 0, 0), -1) cv2.imshow("colorized depth", depth_colorized) if cv2.waitKey(1) == ord('q'): break ``` Thanks ! Antoine

Hi @"apirrone"#p13338 , We apologize for the inconvenience, but the full TOF decoding is not yet implemented, and we are working on it at the moment. It should be ready in firmware in a few week, so you'll be able to just update the library (pip install depthai -U) and have accurate depth. Kind regards, Erik

Thanks @"erik"#p13367 for the reply, please keep me updated when the update is ready! Just curious, how is it that it "kind of works" right now, but with bad accuracy? The colorized depth map looks really good and continuous 😅

Hi @"apirrone"#p13369 , That's the thing - we get _some_ data out, but full decoding and error correction aren't implemented. The depth does look decent (at least the corners are very sharp), but if you'd visualize the pointcloud it's completely off.

Hi @"erik"#175, Any news on the firmware ? Thanks ! Antoine

Hi @"apirrone"#1318 , We got the decoding mostly working, there's just some offset left that we haven't figured out. Below is an A4 paper sheet and its size - about 10% dimension mismatch. We believe we'll have it ready this month. Thanks, Erik [upl-image-preview url=https://discuss.luxonis.com/assets/files/2023-11-06/1699279674-367947-a4-take3.png] [upl-image-preview url=https://discuss.luxonis.com/assets/files/2023-11-06/1699279674-443357-a4-take2.png] [upl-image-preview url=https://discuss.luxonis.com/assets/files/2023-11-06/1699279674-578178-a4-take1.png]

Hi @"erik"#175, any news ? We are very eager to try it out ! :) Thanks, Antoine

Hi @"apirrone"#p14695 We need to finalize a few things, I'd say roughly about 2 weeks.. Thanks, Jaka

@"jakaskerl"#p14717 hey any update on the ToF sensor finalization so its accurate and ready to be used in depthai?

ToF bad accuracy

erik

@Nearpoint not yet, let me check with the team

tetsuro_okuyama

Hi @erik, Any update?

erik

Hi @tetsuro_okuyama @Nearpoint

Thank you for patience. We have some exciting updates 🙂 We got everything ready, the only missing part was calibration together with the front glass (reflections..), but we just got it figured out as well.

At 75cm, depth error across FOV (below 2% everywhere):

At 1m, depth error across FOV (below 2% everywhere as well):

So now it's only the manufacturing (and procuring the new front glass) + calibration. If you already received an OAK-D-SR-POE, I'd suggest sending an email to support@luxonis.com and we can arrange replacing the unit with one that was freshly calibrated.

Thanks, Erik

tetsuro_okuyama

erik Thank you for your reply!
I'll mail you gave me.

Best regards,

Nearpoint

erik

The front glass, is that only related to the Enclosed OAK Cameras not the modules? I have the ToF sensor sold as an embedded module to integrate into our product. Does the lens of the module need to be replaced as the front glass or is the front glass only related to the Enclosure for the OAK camera devices?

Otherwise, as far as the ToF 33D module is concerned, I would just need to calibrate it with the new updates correct?

apirrone

Hi @erik, great news !

We are pretty much in the same situation as @Nearpoint. We have the ToF module that we want to integrate in our product. What should we do now to be able to use it properly ?

Thanks !

Antoine

erik

Hi @apirrone ,
If you already have the module/camera, for best accuracy you'd need to replace it with a newly calibrated one (email support@luxonis.com). We will be updating calibration scripts that will let you self-calibrate, but have not yet done that.
Thanks, Erik

Matthieu

erik We have ordered recently a new ToF module, it has been shipped today (order 16554). Is it the new version already ?

erik

@Matthieu best to ask support team🙂

apirrone

Hi @erik @jakaskerl

We received our calibrated ToF modules !

I updated the depthai library to the latest (2.27.0.0) and tried to run these examples on my FFC-4P :

I get the following outputs :

[2024-07-01 11:55:21.392] [depthai] [warning] [18443010F19A5B1200] [1.2] Flashed bootloader version 0.0.21, less than 0.0.28 is susceptible to bootup/restart failure. Upgrading is advised, flashing main/factory (not user) bootloader. Available: 0.0.28
Connected cameras: [{socket: CAM_B, sensorName: S5K33D, width: 640, height: 480, orientation: AUTO, supportedTypes: [MONO], hasAutofocus: 0, hasAutofocusIC: 0, name: left}, {socket: CAM_C, sensorName: OV9282, width: 1280, height: 800, orientation: AUTO, supportedTypes: [MONO], hasAutofocus: 0, hasAutofocusIC: 0, name: right}]
[18443010F19A5B1200] [1.2] [5.894] [ToF(0)] [error] ERROR unexpected frequency code: 70

[18443010F19A5B1200] [1.2] [5.894] [ToF(0)] [error] ERROR unexpected phase code: 15

[18443010F19A5B1200] [1.2] [5.894] [ToF(0)] [error] Orientation not supported! Only normal and 180 flip(not mirrored) can be used.
[18443010F19A5B1200] [1.2] [5.894] [ToF(0)] [error] ERROR unexpected frequency code: 105

[18443010F19A5B1200] [1.2] [5.894] [ToF(0)] [error] ERROR unexpected phase code: 21

[18443010F19A5B1200] [1.2] [5.894] [ToF(0)] [error] Orientation not supported! Only normal and 180 flip(not mirrored) can be used.
[18443010F19A5B1200] [1.2] [7.787] [ToF(0)] [warning] Temperature coefficients are not adequate in the EEPROM. Disabling temperature correction.

The script still runs while outputting these errors and I get a nonsensical output frame

tof-2024-07-01-115654.mp4

2MB

Do I have to flash the latest version of the bootloader as suggested by the first warning ?

Am I doing something wrong ?

Thanks !

apirrone

FYI, I tried with a newer FFC-4P and got a different result :

[2024-07-01 15:55:40.541] [depthai] [warning] [14442C10019342D200] [1.2] Flashed bootloader version 0.0.26, less than 0.0.28 is susceptible to bootup/restart failure. Upgrading is advised, flashing main/factory (not user) bootloader. Available: 0.0.28
Connected cameras: [{socket: CAM_A, sensorName: S5K33D, width: 640, height: 480, orientation: AUTO, supportedTypes: [TOF], hasAutofocus: 0, hasAutofocusIC: 0, name: color}, {socket: CAM_B, sensorName: IMX296, width: 1440, height: 1080, orientation: AUTO, supportedTypes: [MONO], hasAutofocus: 0, hasAutofocusIC: 0, name: left}, {socket: CAM_C, sensorName: IMX296, width: 1440, height: 1080, orientation: AUTO, supportedTypes: [MONO], hasAutofocus: 0, hasAutofocusIC: 0, name: right}]
[14442C10019342D200] [1.2] [5.762] [ToF(0)] [error] Couldn't read CCM EEPROM: I/O error
[14442C10019342D200] [1.2] [5.762] [ToF(0)] [error] Error: fppn_80 Could not read header data.

With an image that looks like this

apirrone

EDIT : this was with the second ToF that we received, I still have the same issue with the first one. Sorry for the confusion

apirrone

Ok, by playing around with the parameters I get this :

tof-2024-07-01-161528.mp4

10MB

It's better, but something weird is happening on the right half of the image (or the left)

apirrone

I powered the board with an external power supply and things seem better, but I still have issues.

With the following parameters, I get the result in the video below

tofConfig.enableOpticalCorrection False
tofConfig.enablePhaseShuffleTemporalFilter True
tofConfig.phaseUnwrappingLevel 1
tofConfig.phaseUnwrapErrorThreshold 300
tofConfig.enableFPPNCorrection None
tofConfig.median MedianFilter.MEDIAN_OFF
tofConfig.enableTemperatureCorrection False

tof-2024-07-03-110518.mp4

5MB

I still get errors at the start of the script, but it still runs:

[2024-07-03 11:06:34.215] [depthai] [warning] [14442C10019342D200] [1.2] Flashed bootloader version 0.0.26, less than 0.0.28 is susceptible to bootup/restart failure. Upgrading is advised, flashing main/factory (not user) bootloader. Available: 0.0.28
Connected cameras: [{socket: CAM_A, sensorName: S5K33D, width: 640, height: 480, orientation: AUTO, supportedTypes: [TOF], hasAutofocus: 0, hasAutofocusIC: 0, name: color}, {socket: CAM_B, sensorName: IMX296, width: 1440, height: 1080, orientation: AUTO, supportedTypes: [MONO], hasAutofocus: 0, hasAutofocusIC: 0, name: left}, {socket: CAM_C, sensorName: IMX296, width: 1440, height: 1080, orientation: AUTO, supportedTypes: [MONO], hasAutofocus: 0, hasAutofocusIC: 0, name: right}]
[14442C10019342D200] [1.2] [5.728] [ToF(0)] [error] Couldn't read CCM EEPROM: I/O error
[14442C10019342D200] [1.2] [5.728] [ToF(0)] [error] Error: fppn_80 Could not read header data.

Here is what I get by playing with some parameters

(Here it flickers a lot)

Also here is what I get looking at the raw values of the depth map

tof-2024-07-03-111117.mp4

4MB

Would love to get some help on this !

Thanks,

Antoine

jakaskerl

Hi apirrone
We believe this was calibrated at the very beginning where things were still being finalized and FW changed since then and their current version of calibration is not supported. The best solution in my opinion is to replace the unit with a newly calibrated one.

Please write to support@luxonis.com.

Thanks and sorry for the inconvenience.
Jaka

apirrone

Hello,

We received our newly calibrated ToF sensors, and at first I had the exact same issues than before, mainly the error messages :

[14442C10019342D200] [1.2] [5.728] [ToF(0)] [error] Couldn't read CCM EEPROM: I/O error
[14442C10019342D200] [1.2] [5.728] [ToF(0)] [error] Error: fppn_80 Could not read header data.

And the same weird results in depthmaps.

Looking through the docs, I found this on the shop page:

We are using only one ToF sensor (CAM_A), but we have two other cameras plugged on CAM_B and CAM_C. These are raspberry pi IMX296 cameras. When unplugging them, I don't get the errors anymore, and no more weird vertical splitting in the depthmap (I have to set enableOpticalCorrection to False though)

Could it be that there is the same issue with the shared I2C bus with these cameras as what you describe for multiple ToFs ? If so, how can we fix this ? We would like to have only one OAK-FFC-4P for our vision system

Thanks !

erik

@apirrone that's strange, what code are you running, could we get an MRE?

apirrone

I'm running this code example as is : https://docs.luxonis.com/software/depthai/examples/tof_depth/ so I guess that's the MRE 🙂

erik

@apirrone but you modified it so it reads cam_b and cam_c as well?

apirrone

No it doesn't read them, I execute the script as is without any modifications. If the other cameras are plugged to the board, I get the errors, if not I don't.

« Previous Page Next Page »