I have a use case where I'd like to execute a synchronized still capture of 800P depth along with 12MP color. Importantly, I want to be certain that the still capture occurs at or after the time when I send the still capture control input from the host.

I was able to achieve a naive still capture solution, but due to the stereo pipeline latency there is some unwanted delay. That is, if I move my arm in front of the camera a bit before I send the still capture control signal, I get an synchronized depth + color pair that does not include my arm.

My current idea is to try to coerce the SyncNode to synchronize all three messages: the color image, the depth image, and the control input. The color and depth images should be arriving on their respective queues within the SyncNode at 30Hz, but the control input only arrives on still capture events. As the SyncNode synchronizes on timestamps, and as the control input is stamped on the host with the time I want to capture at, this should achieve exactly what I want.

    // Stereo depth. Left and right inputs are configured to be non-blocking and have a queue size of 1
    cam_left->out.link(stereo_depth->left);
    cam_right->out.link(stereo_depth->right);

    // Message sync. All inputs are configured to be non-blocking and have a queue size of 1.
    cam_color->isp.link(sync->inputs["color"]);
    stereo_depth->out.link(sync->inputs["depth"]);
    control_xin->out.link(sync->inputs["control"]);

    sync->out.link(sync_xout);

However, the sync node fails on every control input, and no images get published. I've tested with a number of different timestamp thresholds and sync attempts, but nothing seems to work.

Could someone provide a bit more details as to the internals of the SyncNode, and how I might be able to achieve what I'm seeking?

    Hi amusco-outrider
    May I see the rest of the code. Theoretically the still should capture your hand, but there might be other issues such as syncing and some buffering.
    Also make sure to run on the develop branch, since main still has a bug where 12MP rgb won't work with Sync node.

    Thanks,
    Jaka

      Also make sure to run on the develop branch, since main still has a bug where 12MP rgb won't work with Sync node.

      Yep, I caught that from another thread. I'm running on the commit that supports 12MP rgb support for Sync node.

      I'll work on getting you a minimum failing example so you can try it on your end.