ToF confidence not working

santelelle

Hi,

I am having trouble getting the ToF camera confidence map. When trying to access the confidence map, the code just hangs and nothing happen. Here a minimal example with depthai 3.0.0

`import depthai as dai

GET_CONFIDENCE = False

pipeline = dai.Pipeline()
tof = pipeline.create(dai.node.ToF)
sync = pipeline.create(dai.node.Sync)

if GET_CONFIDENCE:
tof.confidence.link(sync.inputs['confidence'])
tof.amplitude.link(sync.inputs['amplitude'])

queue = sync.out.createOutputQueue()

with pipeline:
print('Starting pipeline...')
pipeline.start()
print('Pipeline started!')
while True:
print('loop')
message = queue.get()
if GET_CONFIDENCE:
confidence = message['confidence']
print(confidence.shape)

    amplitude = message['amplitude'].getFrame()
    print(amplitude.shape)`

OskarSonc

Hey @santelelle

Looks like the hang happens because GET_CONFIDENCE = False, so nothing is linked into sync and queue.get() just blocks forever. Try setting GET_CONFIDENCE = True (and use .getFrame() for confidence as well), or skip Sync and read directly from tof.confidence / tof.amplitude output queues.

Thanks
Oskar

santelelle

The boolean GET_CONFIDENCE is there to test the two configurations. I've updated the code removing the sync and it still doesn't work when GET_CONFIDENCE is True

import depthai as dai

GET_CONFIDENCE = True

pipeline = dai.Pipeline()
tof = pipeline.create(dai.node.ToF)

if GET_CONFIDENCE:
    queue_conf = tof.confidence.createOutputQueue()
queue_amplitude = tof.amplitude.createOutputQueue()

with pipeline:
    print('Starting pipeline...')
    pipeline.start()
    print('Pipeline started!')
    while True:
        print('loop')

        if GET_CONFIDENCE:
            confidence = queue_conf.get()
            if confidence is not None:
                img_confidence = confidence.getCvFrame()
                print(img_confidence.shape)

        amplitude = queue_amplitude.get()
        if amplitude is not None:
            img_amplitude = amplitude.getCvFrame()
            print(img_amplitude.shape)

OskarSonc

You are not building the node:
tof = pipeline.create(dai.node.ToF).build()

santelelle

You are right! Unfortunately even with the build() it doesn't work. Setting the GET_CONFIDENCE = False works as usual

import depthai as dai

GET_CONFIDENCE = True

pipeline = dai.Pipeline()
tof = pipeline.create(dai.node.ToF).build()

if GET_CONFIDENCE:
    queue_conf = tof.confidence.createOutputQueue()
queue_amplitude = tof.amplitude.createOutputQueue()

with pipeline:
    print('Starting pipeline...')
    pipeline.start()
    print('Pipeline started!')
    while True:
        print('loop')

        if GET_CONFIDENCE:
            confidence = queue_conf.get()
            if confidence is not None:
                img_confidence = confidence.getCvFrame()
                print(img_confidence.shape)

        amplitude = queue_amplitude.get()
        if amplitude is not None:
            img_amplitude = amplitude.getCvFrame()
            print(img_amplitude.shape)

OskarSonc

@santelelle Sorry for the late reply — I didn’t have a ToF device on hand to test. The ToF node does expose .confidence, but the internal depth-confidence filter hasn’t been implemented yet, so that queue remains empty.

As a workaround, you can create a ToFDepthConfidenceFilter yourself and link the raw depth + amplitude outputs before reading from the queues. Once you add those lines, both confidence and filtered depth should start streaming as expected. This worked for me:

import time
import depthai as dai

GET_CONFIDENCE = True

pipeline = dai.Pipeline()
socket = dai.CameraBoardSocket.AUTO
preset = dai.ImageFiltersPresetMode.TOF_MID_RANGE
tof = pipeline.create(dai.node.ToF).build(socket, preset)

queues = []
if GET_CONFIDENCE:
    conf_filter = pipeline.create(dai.node.ToFDepthConfidenceFilter)
    conf_filter.build(tof.rawDepth, tof.amplitude, preset)
    queues += [
        ("confidence", conf_filter.confidence.createOutputQueue(maxSize=4, blocking=False)),
        ("depth (confidence filtered)", conf_filter.filteredDepth.createOutputQueue(maxSize=4, blocking=False)),
    ]
queues += [
    ("amplitude", tof.amplitude.createOutputQueue(maxSize=4, blocking=False)),
    ("depth", tof.depth.createOutputQueue(maxSize=4, blocking=False)),
]

def print_if_available(label, queue):
    if (msg := queue.tryGet()) is not None:
        frame = msg.getFrame()
        print(f"{label}: {frame.shape} {frame.dtype}")

with pipeline as p:
    print("Starting pipeline...")
    p.start()
    print("Pipeline started!")
    try:
        while p.isRunning():
            for label, queue in queues:
                print_if_available(label, queue)
            time.sleep(0.01)
    except KeyboardInterrupt:
        pass

santelelle

OskarSonc Great! That works. Thanks!