I have a use case where I'd like to execute a synchronized still capture of 800P depth along with 12MP color. Importantly, I want to be certain that the still capture occurs at or after the time when I send the still capture control input from the host.
I was able to achieve a naive still capture solution, but due to the stereo pipeline latency there is some unwanted delay. That is, if I move my arm in front of the camera a bit before I send the still capture control signal, I get an synchronized depth + color pair that does not include my arm.
My current idea is to try to coerce the SyncNode
to synchronize all three messages: the color image, the depth image, and the control input. The color and depth images should be arriving on their respective queues within the SyncNode at 30Hz, but the control input only arrives on still capture events. As the SyncNode
synchronizes on timestamps, and as the control input is stamped on the host with the time I want to capture at, this should achieve exactly what I want.
// Stereo depth. Left and right inputs are configured to be non-blocking and have a queue size of 1
cam_left->out.link(stereo_depth->left);
cam_right->out.link(stereo_depth->right);
// Message sync. All inputs are configured to be non-blocking and have a queue size of 1.
cam_color->isp.link(sync->inputs["color"]);
stereo_depth->out.link(sync->inputs["depth"]);
control_xin->out.link(sync->inputs["control"]);
sync->out.link(sync_xout);
However, the sync node fails on every control input, and no images get published. I've tested with a number of different timestamp thresholds and sync attempts, but nothing seems to work.
Could someone provide a bit more details as to the internals of the SyncNode
, and how I might be able to achieve what I'm seeking?