Best syncing approach

Hi,

I'm using downscaled LQ frames (cam_rgb.setPreviewSize(320, 320)) as input for a YOLOv5n model to detect and track objects. The tracker output (+ passthrough detections) is then synchronized in a script node with 1080p HQ frames by using the respective sequence numbers, to crop the detections (= bounding boxes) from the HQ frames. This approach is based on the device-nn-sync.py example and uses the following code:

# Create script node and define inputs for synchronization
script = pipeline.create(dai.node.Script)
script.setProcessor(dai.ProcessorType.LEON_CSS)
cam_rgb.video.link(script.inputs["frames"])  # HQ frames
script.inputs["frames"].setBlocking(False)
tracker.out.link(script.inputs["tracker"])   # tracker output
script.inputs["tracker"].setBlocking(False)

# Set script that will be run on OAK device to synchronize tracker output with HQ frames
script.setScript('''
def get_synced_frame(frames_list, tracks_seq):
    """Compare sequence numbers of tracklets and frame, return frame if equal."""
    for i, frame in enumerate(frames_list):
        if tracks_seq == frame.getSequenceNum():
            return frames_list.pop(i)

# Sync tracker output with HQ frame and send both to output queues
frames_list = []
while True:
    frames_list.append(node.io["frames"].get())
    tracks = node.io["tracker"].tryGet()
    if tracks is not None:
        tracks_seq = tracks.getSequenceNum()
        frame_synced = get_synced_frame(frames_list, tracks_seq)
        if frame_synced is not None:
            node.io["frame_out"].send(frame_synced)
            node.io["track_out"].send(tracks)
''')

# Define script node outputs
xout_rgb = pipeline.create(dai.node.XLinkOut)
xout_rgb.setStreamName("frame")
script.outputs["frame_out"].link(xout_rgb.input)  # synced HQ frames

xout_tracker = pipeline.create(dai.node.XLinkOut)
xout_tracker.setStreamName("track")
script.outputs["track_out"].link(xout_tracker.input)  # synced tracker output

With the Sync node introduced in Release v2.24.0.0, it seems to be possible to simplify this approach. I would now like to use the following code, based on the demux_message_group.py example, to achieve the same result:

# Create sync node and define inputs
sync = pipeline.create(dai.node.Sync)
sync.setSyncThreshold(timedelta(milliseconds=200))

cam_rgb.video.link(sync.inputs["frames"])   # HQ frames
tracker.out.link(sync.inputs["tracklets"])  # tracker output

# Create message demux node and define input + outputs
demux = pipeline.create(dai.node.MessageDemux)
sync.out.link(demux.input)

xout_rgb = pipeline.create(dai.node.XLinkOut)
xout_rgb.setStreamName("frame")
demux.outputs["frames"].link(xout_rgb.input)  # synced HQ frames

xout_tracker = pipeline.create(dai.node.XLinkOut)
xout_tracker.setStreamName("track")
demux.outputs["tracklets"].link(xout_tracker.input)  # synced tracker output

From my tests, the frames and tracker output (synchronized with the Sync node) always have the same sequence number, even though the Timestamp is used for synchronization. So there seems to be no difference in the final output between both approaches. The Sync node approach is slightly faster, which is another advantage besides better readability/simplicity.

I'm now wondering if there are any potential risks or unforeseen downsides by using the Sync node approach (timestamp syncing) compared to the script node approach (sequence number syncing)?

I would appreciate any information or insights into these two approaches and if the Sync node approach is now "officially" the way to go, also for syncing messages with the same source fps. In this case, it would be good to update the documentation for Message syncing and highlight the benefits of the Sync node more clearly.

Thanks!