Hi, I'm trying to trigger the TOF sensor using an external trigger. Using the **[oak_deptahi_external_trigger_fsync.py ](https://gist.github.com/Erol444/0138af63378dc8de5b3f7d80db1ea1a5#file-oak_deptahi_external_trigger_fsync-py)**script, I get zero frames. It does not matter if I try one of the color cameras or the TOF sensor. I'm applying (and disconnecting for trigger on LOW) 24 or 12VDC on pins 2 and 7 according to the pinout diagram at https://docs.luxonis.com/hardware/platform/deploy/m8_m12_connectors/ . I've double checked that my cable is good and measuring the voltage at the cable end on the correct pins. Is there a (firmware) fix for this? Or does this mean my device has a hardware issue?

@"Hans"#p25847 The external FSIN input is 5V on the SR POE. I tried triggering the camera and can't get it working either. Will have to sync with the team. Thanks, Jaka

Thanks for the quick response. Hoping the team will get back with a fix (or timeline for a fix) soon. The documentation reads: "We are using an optocoupler and ESD protection diode, so the input trigger voltage should be 12V (up to 24V) and the trigger logic is reversed, so trigger event happens at 0V on the FSYNC line.", so I hope I didn't fry the circuit with 24V. I do have another camera I can test with, so please let me know if there is something I can try.

Another update; it seems the TOF sensor completely stopped responding. Can't get a single frame out of it either through the DepthAI Viewer or tof_depth.py example. RGB cameras are still working. Tried to force it back to streaming mode by passing `camTof->initialControl.setFrameSyncMode(dai::CameraControl::FrameSyncMode::OFF);` But that made no difference. Can you please confirm whether your device has the same issue after trying the trigger?

@"Hans"#p25872 Probably not, the Zener diodes in the inputs are 5V rated and probably only heated up when higher voltage was supplied. To trigger the cameras, pull pin 2 low (to pin 8) then switch it to produce frame pulses. The frames will be triggered when pin 2 is HIGH, with rate set by `setFps()` - set it to the highest value to get the most accurate triggers. NOTE: Pin7 is not ISO_GND anymore. > @"Hans"#p25888 Can you please confirm whether your device has the same issue after trying the trigger? Didn't set the TOF sync mode to either and I am not experiencing any issues. Reflash the bootloader to possibly fix the issue. Thanks, Jaka

Hi Jaka, That means the online documentation has some serious errors. Switching pin 2 to pin 8 is not working (nothing either from the TOF or RGB) and neither did reflashing or updating the bootloader make a difference in getting the TOF to work without triggers again. Seems like something did get fried after all? I will get a different device to test with tomorrow and if that one works fine I will make a request to have the broken one replaced. Your suggestion to "`setFps()` - set it to the highest value to get the most accurate triggers." means the frame is not captured within a fixed time interval from the trigger, but rather the exposure will start up to a 1/60th second random deviation? An additional issue/bug report: Installing the requirements for device_manager.py completely broke my Python enviroment and it took me half a day to recover. Basically numpy 2 is installed which is not compatible with the OpenCV version used by all the examples. I had to pip uninstall numpy and opencv and reinstall specifically numpy-1.26.4 and opencv-python-4.5.5.62 to get things running again. There are several people on the forum who ran into this, so I think this should get fixed. `RuntimeError: module compiled against ABI version 0x1000009 but this version of numpy is 0x2000000`

Ok, picked up and tested with another camera and regular capture and triggered capture of RGB works perfectly fine on that one. I will contact sales for a replacement of the broken device, unless you have other suggestions? I seems the trigger does not actually work on the TOF sensor, but I need to trigger one of the other cameras and when it does get the latest available TOF frame, correct? I guess that is why you were telling me to set the FPS to the max? Even when setting the output buffer to a single frame `device.getOutputQueue("depth", maxSize=1,blocking=False)` , I still usually get some (several seconds) old frames from before the trigger. This must mean there is another buffer on the device side.. is there a way to clear/limit that buffer as well?

> @"Hans"#p25920 Your suggestion to "setFps() - set it to the highest value to get the most accurate triggers." means the frame is not captured within a fixed time interval from the trigger, but rather the exposure will start up to a 1/60th second random deviation? Set to 129.6 FPS (for 800p) and exposure should start immediately 1/129.6th of a second governed by when in the trigger was active. Note that in this mode exposure is started immediately after trigger, then MIPI readout follows. In the other mode (frame sync input), exposure of next frame and readout of current are internally overlapped. > @"Hans"#p25934 Ok, picked up and tested with another camera and regular capture and triggered capture of RGB works perfectly fine on that one. I will contact sales for a replacement of the broken device, unless you have other suggestions? Could be fried from 24V.. > @"Hans"#p25934 I seems the trigger does not actually work on the TOF sensor, but I need to trigger one of the other cameras and when it does get the latest available TOF frame, correct? I guess that is why you were telling me to set the FPS to the max? Not sure I fully understand the question. The TOF triggering should work the same way, it shares the same FSIN line. > @"Hans"#p25934 Even when setting the output buffer to a single frame device.getOutputQueue("depth", maxSize=1,blocking=False) , I still usually get some (several seconds) old frames from before the trigger. This must mean there is another buffer on the device side.. is there a way to clear/limit that buffer as well? You can set the size and blocking behavior on any link that has the `.input`. So XLINK OUT: `depth.out.link(depth_xout.input)` can have `depth_xout.input.setBlocking(False)` and same for queue size. Thanks, Jaka

Hi Jaka, Thanks for the help. Will try the suggestions on the FPS and queue size. I still can't get triggering the TOF sensor directly to work. Can you share the code you used to test yourself? Or please check the code below; I tried to make it the most minimal combination of the tof_depth example and the external trigger example. This code for me just gives me continuous output with a warning "[warning] Device configured to generate sync, but no camera with sync output capability was started" The triggering works if I change TOF to a MonoCamera. ``` #include #include #include #include #include #include #include #include "depthai-shared/datatype/RawToFConfig.hpp" #include "depthai/pipeline/datatype/ToFConfig.hpp" #include #include #include // Define the shared pointers and other variables std::shared_ptr qDepth; dai::Pipeline pipeline; std::shared_ptr tof; dai::RawToFConfig tofConfig; std::shared_ptr device; std::shared_ptr createConfig(dai::RawToFConfig configRaw) { auto config = std::make_shared(); config->set(std::move(configRaw)); return config; } cv::Mat getDepth() { auto imgFrame = qDepth->get(); // blocking call, will wait until new data has arrived cv::Mat depthFrame = imgFrame->getFrame(true); return depthFrame; } cv::Mat convertToByte(const cv::Mat& frameDepth) { cv::Mat invalidMask = (frameDepth == 0); cv::Mat depthFrameScaled; try { double minDepth = 100.0; double maxDepth = minDepth + 255; cv::Mat rawDepth; frameDepth.convertTo(rawDepth, CV_32F); cv::medianBlur(rawDepth, rawDepth, 5); rawDepth = (rawDepth - minDepth) / (maxDepth - minDepth) * 255; rawDepth.convertTo(depthFrameScaled, CV_8UC1); depthFrameScaled.setTo(cv::Scalar(0, 0, 0), invalidMask); } catch (const std::exception& e) { depthFrameScaled = cv::Mat::zeros(frameDepth.size(), CV_8UC1); } return depthFrameScaled; } bool Init() { printf("Initializing DepthAI\n"); tof = pipeline.create(); // Configure the ToF node tofConfig = tof->initialConfig.get(); tofConfig.enableFPPNCorrection = true; tofConfig.enableOpticalCorrection = true; tofConfig.enableWiggleCorrection = false; tofConfig.enablePhaseShuffleTemporalFilter = false; tofConfig.enablePhaseUnwrapping = true; tofConfig.phaseUnwrappingLevel = 1; tofConfig.phaseUnwrapErrorThreshold = 300; tofConfig.enableTemperatureCorrection = true; printf("Configuring the ToF node\n"); auto xinTofConfig = pipeline.create(); xinTofConfig->setStreamName("tofConfig"); xinTofConfig->out.link(tof->inputConfig); tof->initialConfig.set(tofConfig); auto camTof = pipeline.create(); camTof->setFps(60); camTof->setBoardSocket(dai::CameraBoardSocket::CAM_A); camTof->initialControl.setFrameSyncMode(dai::CameraControl::FrameSyncMode::INPUT); camTof->initialControl.setExternalTrigger(1, 0); camTof->raw.link(tof->input); auto xoutDepth = pipeline.create(); xoutDepth->setStreamName("depth"); tof->depth.link(xoutDepth->input); xoutDepth->input.setBlocking(false); xoutDepth->input.queueSize = 1; tofConfig = tof->initialConfig.get(); device = std::make_shared(pipeline); std::cout has()) { cv::Mat depthFrameScaled = convertToByte(getDepth()); cv::imshow("Depth", depthFrameScaled); } else { printf("."); } int key = cv::waitKey(1); if (key == 'q') { break; } counter++; } return 0; } ```

Hi @"Hans"#p25966 My mistake; the tof can not be triggered as other cameras. From my tests, it just constantly runs and is not affected by any triggering. Though I guess you could achieve essentially the same thing with a sync node between a camera (which you trigger) and the ToF. They should both trigger at once anyway, other tof frames will be discarded. Thanks, Jaka

Hi Jaka, After weeks of trying to get everything to work reliably using your suggestions, I still have issues remaining. I'm triggering one of the RGB camera's with a sync node to the ToF sensor. I only got this to work by setting them both to 60FPS. Using a PLC and SSR I am providing a 5ms trigger. Main issue is that >50% of the time, my depth image seems to consist of 2 exposures. As if the ToF exposure was interrupted by the RGB exposure or something like that? Please look at the sample images. These are of the same container moving at the same low speed. I randomly get the first or the second depth result. The color image is always fine. Hope you can help with this. Regards, Hans [upl-image-preview url=https://discuss.luxonis.com/assets/files/2024-10-18/1729264515-190372-depth-issue.png] [upl-image-preview url=https://discuss.luxonis.com/assets/files/2024-10-18/1729264525-169282-color-issue.png] [upl-image-preview url=https://discuss.luxonis.com/assets/files/2024-10-18/1729264531-999631-depth-ok.png] [upl-image-preview url=https://discuss.luxonis.com/assets/files/2024-10-18/1729264541-585687-color-ok.png]

External trigger not working on OAK-D-SR-POE

jakaskerl

Hi @toon,
What seems to be the issue? I don't have any such examples on hand.
The general idea:

set both B and C as input + setExternalTrigger(1,0) or (3,2)
set TOF to the highest FPS (80 i think)
Get frames back from all three and match them by timestamps.

Thanks,
Jaka

toon

jakaskerl Thanks for the response.

If I setExternalTrigger(3,2) (1,0 seems to block at some point after 1 minute). And set TOF to the highest framerate, I still don't get any synced frames if I set the sync threshold lever than 0.8s. While I would expect with the TOF running at 80FPS, there should always be a frame within 0.0125s right?

jakaskerl

Hi toon
What about if you print the getTimestamp() for each received message on the host.. That would tell you the FPS and timestamps to make sure there isn't an issue in how sync node is configured.

Thanks,
Jaka

SimonvE

I have a similar issue as discussed in this topic.
I simply want a hardware triggered TOF image and I am attempting it using this warkaround.

TOF at 60FPS streaming continuously (OAK runs out of memory if 80FPS and if RGB camera is also 60FPS)
RGB cam_c with .initialControl.setExternalTrigger(1,0)
To trigger, short pin 2 to pin 8

The plan:

Read TOF queue continuously and write to host-side FIFO queue, this should prevent any issues with device-side buffers/queues.
Try to read RGB queue.
When a packet is found in the RGB queue, this is the result of a trigger. Get the RGB timestamp with color_packet.getTimestamp()
Collect some more TOF packets such that there are TOF packets before and after the trigger, guaranteeing an optimal TOF frame has been collected
Compare RGB timestamp with TOF timestamps in the host-side FIFO queue, find one with shortest time difference.

My simplified code to reproduce the main issue:

trigger_received = False
while pipeline.isRunning():
    time.sleep(1)  # delay to allow new packets to arrive, delay increased to 1 sec for better logs in minimal code.
    color_packet: dai.ADatatype = output_queues_dict["color"].tryGet()
    if isinstance(color_packet, dai.ImgFrame) and not trigger_received: # skip color packet processing if trigger already received
        color_timestamp: datetime.timedelta = color_packet.getTimestamp()
        time_of_trigger = time.time()        
        print(f"Received color packet at time {time_of_trigger}")        
        print(f"Timestamp color: {color_timestamp}")
        trigger_received = True
     
    tof_packets: list[dai.ADatatype] = output_queues_dict["tof"].tryGetAll()
    print(f"Retrieved {len(tof_packets)} new tof packets from device")
    for tof_packet in tof_packets:
        tof_packets_queue_hostside.append(tof_packet)

    ...
    ...

As can be seen in the code, both RGB and TOF packets should always be read out

My strange problem(s):

After boot, the TOF first does not start streaming. Only after an initial trigger, the TOF streams at ~30FPS
Main issue: When a trigger is received, some TOF packets come in but only until the color trigger. After that, no TOF packets arrive for serveral seconds.
The last TOF packet that is received is right before the RGB trigger

logs of the main issue:

Retrieved 36 new tof packets from device
Retrieved 29 new tof packets from device
Retrieved 28 new tof packets from device
Received color packet at time 1768380198.2891834
Timestamp color: 2:00:53.587175
Retrieved 18 new tof packets from device
Retrieved 0 new tof packets from device
Retrieved 0 new tof packets from device
Retrieved 0 new tof packets from device
Retrieved 0 new tof packets from device
Retrieved 0 new tof packets from device
Retrieved 0 new tof packets from device
Retrieved 0 new tof packets from device
Retrieved 0 new tof packets from device
Retrieved 0 new tof packets from device
Retrieved 23 new tof packets from device
Retrieved 35 new tof packets from device
Retrieved 37 new tof packets from device

What causes this strange behaviour? Is the TOF somehow dependent on the RGB stream?

jakaskerl

SimonvE
Can you give a full mre please? It's hard to say from the code you have sent.

Thanks,
Jaka

« Previous Page