Getting pointcloud data using 33D ToF

Schotchaicharin

Hi,

I learned from this page ToF (luxonis.com) that I can use the depth from ToF node instead StereoDepth node, so I refer to example code: pointcloud_visualization.py as reference and implemented a simple script below to get the pointcloud data using ToF camera.

I confirmed that the Depth is correctly taken, however, each point in the output pointcloud include "nan" or "Inf" in X, Y channel while the depth info was parsed to Z channel correctly.
Could you please tell me what is wrong?

import depthai as dai

import numpy as np

pipeline = dai.Pipeline()

cam_tof = pipeline.create(dai.node.Camera)

tof = pipeline.create(dai.node.ToF)

pointcloud = pipeline.create(dai.node.PointCloud)

xOut_pcl = pipeline.create(dai.node.XLinkOut)

xOut_depth = pipeline.create(dai.node.XLinkOut)

cam_tof.raw.link(tof.input)

tof.depth.link(pointcloud.inputDepth)

tof.depth.link(xOut_depth.input)

pointcloud.outputPointCloud.link(xOut_pcl.input)

xOut_depth.setStreamName("depth")

xOut_pcl.setStreamName("pcl")

with dai.Device(pipeline) as device:

qDepth = device.getOutputQueue(name="depth", maxSize=4, blocking=False)

qPci = device.getOutputQueue(name="pcl", maxSize=4, blocking=False)

inDepth = qDepth.get()

inPointCloud = qPci.get()

depth = inDepth.getFrame()

points = inPointCloud.getPoints().astype(np.float64)

print(depth)

print(points)

jakaskerl

Schotchaicharin
Does TOF depth image look ok? Could you send the calibrartion dump as well, maybe the module is not calibrated.

Thanks,
Jaka

Schotchaicharin

Thank you for your reply.

I follow the calibration method in this link Calibration (luxonis.com) and conduct the calibration using tof_calib branch in DepthAI github with the following custom board config json file (for single ToF camera).

{

"board_config":

{

"name": "TOF-FFC",

"revision": "R0M0E0",

"cameras":{

"CAM_A": {

"name": "tof",

"hfov": 71.86,

"type": "tof"

}

}

}

}

The calibration process was success. (The "EEPROM written successful" window poped up)

And the result Depth image looks OK to me.

The pointcloud in the DepthAI viewer look flat when capturing the ceiling (check image below)

And since the .json file could not be uploaded on this site, I will share part of the calibration result file.

{

"batchName": "",

"batchTime": 1679712869,

"boardConf": "IR-C00M05-00",

"boardCustom": "",

"boardName": "DM1090",

"boardOptions": 0,

"boardRev": "R3M0E3",

"cameraData": [

[

0,

{

"cameraType": 0,

"distortionCoeff": [

-9.475359916687012,

29.59166717529297,

-0.0012394796358421445,

-0.0005019600503146648,

17.674795150756836,

-9.451038360595703,

29.16131591796875,

19.195999145507813,

0.0,

0.0,

0.0,

0.0,

0.0,

0.0

],

………..

"intrinsicMatrix": [

[

472.4834899902344,

0.0,

319.69110107421875

],

[

0.0,

473.01629638671875,

227.20944213867188

],

[

0.0,

0.0,

1.0

]

Thank you

jakaskerl

Schotchaicharin
Align it to some image. Then you should be getting the points in the frame (to which tof was aligned) coordinates.

https://gist.githubusercontent.com/jakaskerl/feaf19bd277b6a78a2b4a01ccf216a17/raw/fb21fde9ebdd1b524f9fa7b57adad35685521e1d/tof_depth_pointcloud.py

Thanks,
Jaka