I am trying to trigger my Oakd-s2 (AF) on ubuntu24 with the example provided here. I managed to get this script working, however, the RGB image is not shown. In the following thread, it was said that they got the trigger working included RGB image, unfortunately any example is missing. Could this be shared?

@"toon"#1264 RGB camera won't work because the sensor doesn't support "snapshot" triggering, only "continuous" triggering at a very fixed intervals (eg. 30.00Hz). Pleas see link below for details: https://docs.luxonis.com/hardware/platform/deploy/frame-sync/#Sensor%20FSYNC%20support Thanks, Erik

@"toon"#p25710 You are using an external trigger right? The code is meant for OAK-FFC (or POE M8) devices where the FSIN is exposed. This is not possible on S2 unless you open it and connect it to a FSIN line (which is not easily accessible). Thanks, Jaka

@"jakaskerl"#p25718 sorry for the misconfusion, I am using a **OAK-D S2 PoE** with autofocus, so I have already implemented the M8 connector. What I would like to do is to trigger multiple devices (3) with a external trigger pulse. Furthermore I want to output an rgb image as close as possible with the triggered mono-images, but they don't have to be fully hardware synchronised.

@"toon"#p25721 Then the code is fine, just set the color camera FPS to the exactly the same FPS at which you are triggering the other two cameras. As Erik said, snapshot mode does not work. > Continuous streaming with external syncing, configured with CameraControl.setFrameSyncMode(). In this mode, the FSIN signal is expected to arrive at a continuous rate matching the configured sensor FPS, and trigger can't arrive at arbitrary times as that would disrupt internal sensor operations (leading to bad frames, etc). It can only correct for very small amounts of drift over time. Thanks, Jaka

Dear Jakaskerl, Indeed the color camera in following script does not work [example](https://gist.github.com/Erol444/a9189a8215371ff9f4cf4472960e1d66). Instead I wrote the following script including sync node, but still I do not get an image, what am I doing wrong? ``` #!/usr/bin/env python3 import depthai as dai import cv2 import time from datetime import timedelta # FPS_tof = 10 FPS_rgb = 10 with_trigger = True pipeline = dai.Pipeline() # RGB camera camRgb = pipeline.create(dai.node.ColorCamera) camRgb.setColorOrder(dai.ColorCameraProperties.ColorOrder.RGB) camRgb.setIspScale(2, 3) camRgb.setBoardSocket(dai.CameraBoardSocket.RGB) camRgb.setFps(FPS_rgb) if with_trigger: camRgb.initialControl.setFrameSyncMode(dai.CameraControl.FrameSyncMode.INPUT) camRgb.initialControl.setExternalTrigger(4, 3) xout = pipeline.create(dai.node.XLinkOut) xout.setStreamName("out") # camRgb.isp.link(xout.input) # Link the RGB output to XLinkOut # Mono left camera monoLeft = pipeline.create(dai.node.MonoCamera) monoLeft.setResolution(dai.MonoCameraProperties.SensorResolution.THE_720_P) monoLeft.setBoardSocket(dai.CameraBoardSocket.LEFT) monoLeft.setFps(FPS_rgb) if with_trigger: monoLeft.initialControl.setFrameSyncMode(dai.CameraControl.FrameSyncMode.INPUT) monoLeft.initialControl.setExternalTrigger(4, 3) # Mono right camera monoRight = pipeline.create(dai.node.MonoCamera) monoRight.setResolution(dai.MonoCameraProperties.SensorResolution.THE_720_P) monoRight.setBoardSocket(dai.CameraBoardSocket.RIGHT) monoRight.setFps(FPS_rgb) if with_trigger: monoRight.initialControl.setFrameSyncMode(dai.CameraControl.FrameSyncMode.INPUT) monoRight.initialControl.setExternalTrigger(4, 3) xoutRight = pipeline.create(dai.node.XLinkOut) xoutRight.setStreamName("right") monoRight.out.link(xoutRight.input) # Sync node (if you want to sync all cameras) sync = pipeline.create(dai.node.Sync) # sync.setSyncThreshold(50) # Sync threshold in milliseconds sync.setSyncThreshold(timedelta(milliseconds=1000)) # Linking RGB, left, and right to the sync node camRgb.isp.link(sync.inputs["color"]) # Link RGB to sync monoLeft.out.link(sync.inputs["left"]) monoRight.out.link(sync.inputs["right"]) # You can link the sync output to XLinkOut for all cameras sync.out.link(xout.input) # sync.out.link(xoutLeft.input) # sync.out.link(xoutRight.input) # Device connection device_id = "169.254.1.103" with dai.Device(pipeline, dai.DeviceInfo(device_id)) as device: queue = device.getOutputQueue("out", 8, False) queueright=device.getOutputQueue("right") print("Starting...", queue) while True: # if queueright["right"].has(): # test=queueright["right"].get().getCvFrame() # print("x") messageGroup: dai.MessageGroup = queue.get() frameMono_right: dai.ImgFrame = messageGroup["right"] frameMono_left: dai.ImgFrame = messageGroup["left"] # frameDepth_tof: dai.ImgFrame = messageGroup["depth_tof"] frameRgb: dai.ImgFrame = messageGroup["color"] # Blend when both received if frameMono_right is not None: if frameRgb is not None: print(frameMono_right.getSequenceNum()) print(frameRgb.getSequenceNum()) # print(messageGroup["depth"].getTimestamp()-messageGroup["rgb"].getTimestamp()) # print(messageGroup["rgb"].getSequenceNum()) # print(messageGroup["depth"].getSequenceNum()) monoRight_img = frameMono_right.getCvFrame() # # cv2.imshow("rgb", frameRgb) # # cv2.imshow("depth", colorizeDepth(frameDepth.getCvFrame())) test=frameRgb.getCvFrame() print("It is working") else: print("no depth frame") else: print("no rgb frame") key = cv2.waitKey(1) if key == ord("q"): break cv2.destroyAllWindows() ```

@"toon"#p25969 How does the trigger signal look? Thanks, Jaka

The trigger signal is just a signal triggered by a pushbutton (for test and verification). Same setup as in [externally trigger fsync](https://youtu.be/q9JZHluu2Gc?feature=shared). In the end we would like to trigger with a PLC based on the encoder value (every x cm).

@"toon"#p25988 You need to trigger at exactly 30Hz for this to work. as mentioned in @"jakaskerl"#p25733

Ok, so if I understand correctly, there is no way to hardware-trigger the stereo frames and then find the most close rgb-frame softwarematticaly (even if we leave it streaming at 30FPS)?

@"toon"#p26111 Correct, the frame triggers are expected (by IMX378) to come at a fixed rate. The issue is the the fsin lines are connected together and only triggering the OV strereo cameras wouldn't work. If the lines were separate this should be possible. RGB will not stream if both left and right are set to input mode. Thanks, Jaka

Hardware trigger synchronization including RGB image (Oakd-s2)

bartvm

@jakaskerl any update on this? Thanks in advance. Kind regards, Bart

jakaskerl

bartvm
team is looking into it.

Thanks for patience,
Jaka

bartvm

jakaskerl Thanks for the update, just let me know if you need anything from my side.

« Previous Page