I could not figure out why when I was trying to synch frames on host it would skip many sequence numbers causing big jumps.
To debug it I put script nodes that just pass along frames and output the sequence number of the frame
camPreview ->logScriptNode -> imageManip -> logScriptNode -> spatialImageDetections -> logScriptNode
stereoDepth -> logScriptNode - >
I set setNumPoolFrames
to 10 for: camPreview, stereoDepth, imageManip, spatialImageDetections
You can see the output for the log nodes and the sequence numbers and the jumps in the imageManip, stereoDepth, and spatialImageDetections. Ultimately causing big synching issues on host which leads to large frame jumps.
$$
[18443010C16C0F0900] [192.168.10.43] [12.754] [Script(9)] [warning] CamRgb Out SeqNum: 0
[18443010C16C0F0900] [192.168.10.43] [12.917] [Script(11)] [warning] Depth Output SeqNum: 0
[18443010C16C0F0900] [192.168.10.43] [12.919] [Script(9)] [warning] CamRgb Out SeqNum: 1
[18443010C16C0F0900] [192.168.10.43] [13.218] [Script(9)] [warning] CamRgb Out SeqNum: 2
[18443010C16C0F0900] [192.168.10.43] [13.226] [Script(9)] [warning] CamRgb Out SeqNum: 3
[18443010C16C0F0900] [192.168.10.43] [13.228] [Script(11)] [warning] Depth Output SeqNum: 1
[18443010C16C0F0900] [192.168.10.43] [13.231] [Script(10)] [warning] ImageManip Output SeqNum: 0
[18443010C16C0F0900] [192.168.10.43] [13.249] [Script(11)] [warning] Depth Output SeqNum: 2
[18443010C16C0F0900] [192.168.10.43] [13.256] [Script(11)] [warning] Depth Output SeqNum: 3
[18443010C16C0F0900] [192.168.10.43] [13.261] [Script(9)] [warning] CamRgb Out SeqNum: 4
[18443010C16C0F0900] [192.168.10.43] [13.265] [Script(10)] [warning] ImageManip Output SeqNum: 1
[18443010C16C0F0900] [192.168.10.43] [13.272] [Script(9)] [warning] CamRgb Out SeqNum: 5
[18443010C16C0F0900] [192.168.10.43] [13.273] [Script(11)] [warning] Depth Output SeqNum: 4
[18443010C16C0F0900] [192.168.10.43] [13.277] [Script(10)] [warning] ImageManip Output SeqNum: 2
[18443010C16C0F0900] [192.168.10.43] [13.279] [Script(9)] [warning] CamRgb Out SeqNum: 6
[18443010C16C0F0900] [192.168.10.43] [13.283] [Script(10)] [warning] ImageManip Output SeqNum: 3
[18443010C16C0F0900] [192.168.10.43] [13.285] [Script(9)] [warning] CamRgb Out SeqNum: 7
[18443010C16C0F0900] [192.168.10.43] [13.292] [Script(10)] [warning] ImageManip Output SeqNum: 4
[18443010C16C0F0900] [192.168.10.43] [13.294] [Script(9)] [warning] CamRgb Out SeqNum: 8
[18443010C16C0F0900] [192.168.10.43] [13.298] [Script(10)] [warning] ImageManip Output SeqNum: 5
[18443010C16C0F0900] [192.168.10.43] [13.302] [Script(10)] [warning] ImageManip Output SeqNum: 6
[18443010C16C0F0900] [192.168.10.43] [13.303] [Script(9)] [warning] CamRgb Out SeqNum: 9
[18443010C16C0F0900] [192.168.10.43] [13.315] [Script(10)] [warning] ImageManip Output SeqNum: 7
[18443010C16C0F0900] [192.168.10.43] [13.491] [Script(13)] [warning] Person Detection Output SeqNum: 0
[18443010C16C0F0900] [192.168.10.43] [13.499] [Script(10)] [warning] ImageManip Output SeqNum: 8
[18443010C16C0F0900] [192.168.10.43] [13.503] [Script(11)] [warning] Depth Output SeqNum: 5
[18443010C16C0F0900] [192.168.10.43] [13.610] [Script(13)] [warning] Person Detection Output SeqNum: 1
[18443010C16C0F0900] [192.168.10.43] [13.621] [Script(10)] [warning] ImageManip Output SeqNum: 9
[18443010C16C0F0900] [192.168.10.43] [13.622] [Script(11)] [warning] Depth Output SeqNum: 6
[18443010C16C0F0900] [192.168.10.43] [13.645] [Script(9)] [warning] CamRgb Out SeqNum: 13
[18443010C16C0F0900] [192.168.10.43] [13.721] [Script(13)] [warning] Person Detection Output SeqNum: 2
[18443010C16C0F0900] [192.168.10.43] [13.727] [Script(10)] [warning] ImageManip Output SeqNum: 13
[18443010C16C0F0900] [192.168.10.43] [13.732] [Script(9)] [warning] CamRgb Out SeqNum: 14
[18443010C16C0F0900] [192.168.10.43] [13.743] [Script(11)] [warning] Depth Output SeqNum: 7
[18443010C16C0F0900] [192.168.10.43] [13.942] [Script(13)] [warning] Person Detection Output SeqNum: 3
[18443010C16C0F0900] [192.168.10.43] [13.950] [Script(13)] [warning] Person Detection Output SeqNum: 4
[18443010C16C0F0900] [192.168.10.43] [13.966] [Script(11)] [warning] Depth Output SeqNum: 8
[18443010C16C0F0900] [192.168.10.43] [13.968] [Script(10)] [warning] ImageManip Output SeqNum: 14
[18443010C16C0F0900] [192.168.10.43] [13.969] [Script(9)] [warning] CamRgb Out SeqNum: 16
[18443010C16C0F0900] [192.168.10.43] [13.978] [Script(9)] [warning] CamRgb Out SeqNum: 17
[18443010C16C0F0900] [192.168.10.43] [13.993] [Script(10)] [warning] ImageManip Output SeqNum: 16
[18443010C16C0F0900] [192.168.10.43] [13.998] [Script(11)] [warning] Depth Output SeqNum: 9
[18443010C16C0F0900] [192.168.10.43] [14.173] [Script(13)] [warning] Person Detection Output SeqNum: 5
[18443010C16C0F0900] [192.168.10.43] [14.175] [Script(10)] [warning] ImageManip Output SeqNum: 17
[18443010C16C0F0900] [192.168.10.43] [14.184] [Script(11)] [warning] Depth Output SeqNum: 10
[18443010C16C0F0900] [192.168.10.43] [14.187] [Script(13)] [warning] Person Detection Output SeqNum: 6
[18443010C16C0F0900] [192.168.10.43] [14.195] [Script(11)] [warning] Depth Output SeqNum: 11
[18443010C16C0F0900] [192.168.10.43] [14.200] [Script(9)] [warning] CamRgb Out SeqNum: 18
[18443010C16C0F0900] [192.168.10.43] [14.211] [Script(9)] [warning] CamRgb Out SeqNum: 19
[18443010C16C0F0900] [192.168.10.43] [14.215] [Script(10)] [warning] ImageManip Output SeqNum: 18
[18443010C16C0F0900] [192.168.10.43] [14.398] [Script(13)] [warning] Person Detection Output SeqNum: 7
[18443010C16C0F0900] [192.168.10.43] [14.400] [Script(11)] [warning] Depth Output SeqNum: 12
[18443010C16C0F0900] [192.168.10.43] [14.406] [Script(13)] [warning] Person Detection Output SeqNum: 8
[18443010C16C0F0900] [192.168.10.43] [14.418] [Script(9)] [warning] CamRgb Out SeqNum: 20
[18443010C16C0F0900] [192.168.10.43] [14.422] [Script(11)] [warning] Depth Output SeqNum: 13
[18443010C16C0F0900] [192.168.10.43] [14.444] [Script(9)] [warning] CamRgb Out SeqNum: 21
[18443010C16C0F0900] [192.168.10.43] [14.619] [Script(13)] [warning] Person Detection Output SeqNum: 9
[18443010C16C0F0900] [192.168.10.43] [14.628] [Script(11)] [warning] Depth Output SeqNum: 14
[18443010C16C0F0900] [192.168.10.43] [14.630] [Script(13)] [warning] Person Detection Output SeqNum: 13
[18443010C16C0F0900] [192.168.10.43] [14.632] [Script(11)] [warning] Depth Output SeqNum: 15
[18443010C16C0F0900] [192.168.10.43] [14.634] [Script(9)] [warning] CamRgb Out SeqNum: 22
[18443010C16C0F0900] [192.168.10.43] [14.651] [Script(11)] [warning] Depth Output SeqNum: 16
[18443010C16C0F0900] [192.168.10.43] [14.663] [Script(11)] [warning] Depth Output SeqNum: 17
[18443010C16C0F0900] [192.168.10.43] [14.680] [Script(11)] [warning] Depth Output SeqNum: 18
[18443010C16C0F0900] [192.168.10.43] [14.692] [Script(9)] [warning] CamRgb Out SeqNum: 23
[18443010C16C0F0900] [192.168.10.43] [14.699] [Script(10)] [warning] ImageManip Output SeqNum: 19
[18443010C16C0F0900] [192.168.10.43] [14.830] [Script(13)] [warning] Person Detection Output SeqNum: 14
[18443010C16C0F0900] [192.168.10.43] [14.862] [Script(9)] [warning] CamRgb Out SeqNum: 24
[18443010C16C0F0900] [192.168.10.43] [14.864] [Script(11)] [warning] Depth Output SeqNum: 19
[18443010C16C0F0900] [192.168.10.43] [14.889] [Script(10)] [warning] ImageManip Output SeqNum: 20
[18443010C16C0F0900] [192.168.10.43] [14.894] [Script(10)] [warning] ImageManip Output SeqNum: 21
[18443010C16C0F0900] [192.168.10.43] [14.935] [Script(10)] [warning] ImageManip Output SeqNum: 22
[18443010C16C0F0900] [192.168.10.43] [15.158] [Script(13)] [warning] Person Detection Output SeqNum: 16
[18443010C16C0F0900] [192.168.10.43] [15.179] [Script(11)] [warning] Depth Output SeqNum: 20
[18443010C16C0F0900] [192.168.10.43] [15.182] [Script(13)] [warning] Person Detection Output SeqNum: 17
[18443010C16C0F0900] [192.168.10.43] [15.192] [Script(11)] [warning] Depth Output SeqNum: 21
[18443010C16C0F0900] [192.168.10.43] [15.199] [Script(11)] [warning] Depth Output SeqNum: 22
[18443010C16C0F0900] [192.168.10.43] [15.210] [Script(9)] [warning] CamRgb Out SeqNum: 25
[18443010C16C0F0900] [192.168.10.43] [15.229] [Script(9)] [warning] CamRgb Out SeqNum: 26
[18443010C16C0F0900] [192.168.10.43] [15.374] [Script(13)] [warning] Person Detection Output SeqNum: 18
[18443010C16C0F0900] [192.168.10.43] [15.403] [Script(11)] [warning] Depth Output SeqNum: 23
[18443010C16C0F0900] [192.168.10.43] [15.404] [Script(9)] [warning] CamRgb Out SeqNum: 27
[18443010C16C0F0900] [192.168.10.43] [15.415] [Script(13)] [warning] Person Detection Output SeqNum: 19
[18443010C16C0F0900] [192.168.10.43] [15.479] [Script(9)] [warning] CamRgb Out SeqNum: 28
[18443010C16C0F0900] [192.168.10.43] [15.494] [Script(11)] [warning] Depth Output SeqNum: 24
[18443010C16C0F0900] [192.168.10.43] [15.596] [Script(13)] [warning] Person Detection Output SeqNum: 20
[18443010C16C0F0900] [192.168.10.43] [15.621] [Script(9)] [warning] CamRgb Out SeqNum: 29
[18443010C16C0F0900] [192.168.10.43] [15.624] [Script(13)] [warning] Person Detection Output SeqNum: 21
[18443010C16C0F0900] [192.168.10.43] [15.628] [Script(11)] [warning] Depth Output SeqNum: 25
[18443010C16C0F0900] [192.168.10.43] [15.765] [Script(10)] [warning] ImageManip Output SeqNum: 23
[18443010C16C0F0900] [192.168.10.43] [15.768] [Script(11)] [warning] Depth Output SeqNum: 26
[18443010C16C0F0900] [192.168.10.43] [15.825] [Script(9)] [warning] CamRgb Out SeqNum: 30
[18443010C16C0F0900] [192.168.10.43] [15.827] [Script(13)] [warning] Person Detection Output SeqNum: 22
[18443010C16C0F0900] [192.168.10.43] [15.885] [Script(9)] [warning] CamRgb Out SeqNum: 31
[18443010C16C0F0900] [192.168.10.43] [15.922] [Script(11)] [warning] Depth Output SeqNum: 27
[18443010C16C0F0900] [192.168.10.43] [15.934] [Script(10)] [warning] ImageManip Output SeqNum: 24
[18443010C16C0F0900] [192.168.10.43] [16.047] [Script(10)] [warning] ImageManip Output SeqNum: 25
[18443010C16C0F0900] [192.168.10.43] [16.051] [Script(10)] [warning] ImageManip Output SeqNum: 26
[18443010C16C0F0900] [192.168.10.43] [16.087] [Script(10)] [warning] ImageManip Output SeqNum: 27
[18443010C16C0F0900] [192.168.10.43] [16.239] [Script(13)] [warning] Person Detection Output SeqNum: 23
[18443010C16C0F0900] [192.168.10.43] [16.246] [Script(11)] [warning] Depth Output SeqNum: 28
[18443010C16C0F0900] [192.168.10.43] [16.269] [Script(9)] [warning] CamRgb Out SeqNum: 32
[18443010C16C0F0900] [192.168.10.43] [16.537] [Script(13)] [warning] Person Detection Output SeqNum: 24
[18443010C16C0F0900] [192.168.10.43] [16.539] [Script(10)] [warning] ImageManip Output SeqNum: 28
[18443010C16C0F0900] [192.168.10.43] [16.543] [Script(11)] [warning] Depth Output SeqNum: 29
[18443010C16C0F0900] [192.168.10.43] [16.544] [Script(13)] [warning] Person Detection Output SeqNum: 25
[18443010C16C0F0900] [192.168.10.43] [16.544] [Script(9)] [warning] CamRgb Out SeqNum: 33
[18443010C16C0F0900] [192.168.10.43] [16.555] [Script(10)] [warning] ImageManip Output SeqNum: 29
[18443010C16C0F0900] [192.168.10.43] [16.558] [Script(11)] [warning] Depth Output SeqNum: 30
[18443010C16C0F0900] [192.168.10.43] [16.560] [Script(9)] [warning] CamRgb Out SeqNum: 35
[18443010C16C0F0900] [192.168.10.43] [16.605] [Script(10)] [warning] ImageManip Output SeqNum: 30
[18443010C16C0F0900] [192.168.10.43] [16.742] [Script(13)] [warning] Person Detection Output SeqNum: 26
[18443010C16C0F0900] [192.168.10.43] [16.744] [Script(9)] [warning] CamRgb Out SeqNum: 37
[18443010C16C0F0900] [192.168.10.43] [16.748] [Script(11)] [warning] Depth Output SeqNum: 31
[18443010C16C0F0900] [192.168.10.43] [16.992] [Script(10)] [warning] ImageManip Output SeqNum: 31
[18443010C16C0F0900] [192.168.10.43] [17.001] [Script(10)] [warning] ImageManip Output SeqNum: 32
[18443010C16C0F0900] [192.168.10.43] [17.041] [Script(10)] [warning] ImageManip Output SeqNum: 33
[18443010C16C0F0900] [192.168.10.43] [17.217] [Script(13)] [warning] Person Detection Output SeqNum: 27
[18443010C16C0F0900] [192.168.10.43] [17.219] [Script(11)] [warning] Depth Output SeqNum: 32
[18443010C16C0F0900] [192.168.10.43] [17.220] [Script(9)] [warning] CamRgb Out SeqNum: 38
[18443010C16C0F0900] [192.168.10.43] [17.229] [Script(11)] [warning] Depth Output SeqNum: 33
[18443010C16C0F0900] [192.168.10.43] [17.237] [Script(13)] [warning] Person Detection Output SeqNum: 28
[18443010C16C0F0900] [192.168.10.43] [17.249] [Script(9)] [warning] CamRgb Out SeqNum: 39
[18443010C16C0F0900] [192.168.10.43] [17.274] [Script(10)] [warning] ImageManip Output SeqNum: 35
[18443010C16C0F0900] [192.168.10.43] [17.281] [Script(10)] [warning] ImageManip Output SeqNum: 37
[18443010C16C0F0900] [192.168.10.43] [17.318] [Script(10)] [warning] ImageManip Output SeqNum: 38
[18443010C16C0F0900] [192.168.10.43] [17.440] [Script(13)] [warning] Person Detection Output SeqNum: 29
[18443010C16C0F0900] [192.168.10.43] [17.445] [Script(11)] [warning] Depth Output SeqNum: 34
[18443010C16C0F0900] [192.168.10.43] [17.454] [Script(9)] [warning] CamRgb Out SeqNum: 41
[18443010C16C0F0900] [192.168.10.43] [17.656] [Script(10)] [warning] ImageManip Output SeqNum: 39
[18443010C16C0F0900] [192.168.10.43] [17.783] [Script(13)] [warning] Person Detection Output SeqNum: 30
[18443010C16C0F0900] [192.168.10.43] [17.790] [Script(10)] [warning] ImageManip Output SeqNum: 41
[18443010C16C0F0900] [192.168.10.43] [17.803] [Script(9)] [warning] CamRgb Out SeqNum: 42
$$
You can see camRgb jumps from 9 -> 13, 14->16, 33->35, 35->37,
Further on (didn't want to post everything) depth jumps, and then spatial location also jumps.
This happens the most when someone is moving through the frame.
Is this a known issue? Why would these nodes skip sequence numbers?