I like to get synchronised IMU data and stereo images from the OAK-D hardware.
I am using 30fps left and right for the stereo images at 640x400 and 500Hz for the IMU. I am checking timestamps on both stereo images, accel and gyro in an IMU packet on the host side and ensure that they line up. I have noticed that even though accel and gyro timestamps are pretty close (within a few miliseconds) their sequence numbers are quite different sometimes by 20 counts or so- which should result in a timestamp diffence of 40ms or so. For instance I might receive a very first IMU packet, with sequence number of 20 for accel and seuence number of 0 for gyro but their respective timestamps are within 1 milisecond!
Do sequence numbers in IMU suppose to line up with timestamps?
Also any pointers as to the best method for ensuing synchronisation of gyro, accel, left and right images other than checking and comparing timestamp associated with each one of them?
Thanks for clarification.