Has anyone seen random non normalised bounding box values ?

Thor

The ImgDetections message contains a list of detections, which
contains label, confidence, and the bounding box information (xmin,
ymin, xmax, ymax).
This is the code we used for testing:
........
for detection in detections:
print(f"{detection.xmin} {detection.ymin} {detection.xmax} {detection.ymax}")
bbox = frameNorm(frame, (detection.xmin, detection.ymin,
detection.xmax, detection.ymax))
.......

The box coordinates are supposed to be normalized, however we
sometimes get short sections where the xmin value is either negative
or > 0
The NN node is the YoloDetectionNetwork, the model is a gold_yolo blob that
was obtained by passing Gold_n_dist.pt to the Luxonis blob conversion
tool
The issue is not limited to the gold_yolo I just mentioned

-4.883110523223877e-05 0.1005859375 0.17231445014476776 0.408203125
4.883110523223877e-05 0.1059570461511612 0.188183590769 0.4212890863418579
0.5125976204872131 0.740429699420929 0.8064452409744263 0.9996093511581421
0.0 0.11318360269069672 0.19277343153953552 0.42802733182907104
0.529296875 0.7547851800918579 0.82958984375 0.99951171875
4.883110523223877e-05 0.1191406399011612 0.19921875 0.4291015863418579
0.5438476800918579 0.7804198861122131 0.8353515863418579 0.9993163347244263
9.766221046447754e-05 0.12275391817092896 0.20654296875 0.44111329317092896
0.0 0.13076172769069672 0.2197265625 0.45732420682907104
0.5897461175918579 0.826416015625 0.891308605670929 0.99951171875
0.0 0.13417969644069672 0.2275390625 0.46611326932907104

Thor

wrong values (<0 or >1) randomly on
xmin only
ymin only
both xmin an ymin
It has not been observed so far on x/ymax values

We need to know what is going on, and fix it, please

erik

Hi @Thor ,
That is because some models don't have 0..1 bounding boxes - which is a feature of the model, not a bug. It effectively means it can predict how far out of the image the object will extend. One can easily remove this by clipping the values to 0..1 before denormalization, so values wouldn't be outside the image anymore. Thoughts?
Thanks. Erik

Thor

Thank you, Erik
The explanation makes sense if the object is borderline within reason.
We are talking about seeing odd values for object that are pretty much centered.
To that regard, when plotting trajectories we can see that a straight moving object sometimes has a number of random frames where it's position has an offset, looks like it's "jumping places" when in fact it is not

erik

Hi @Thor ,
Could you share a video of that happening? It might be that the model wasn't trained enough, and thus gives inaccurate results. It could also be the fp16 conversion issue (IMO small chance), but to confirm that we'd need the model, an image that gives false prediction on OAK and correct on eg. CPU, and the script to test that, so we could repro locally.

Thor

Hi Erik,
taking into consideration what you mentioned, we did more tests today. So far our findings are:
1) positive/negative values are indeed happing at edges, as you said most probably the model extends the bbox outside the the frame boundaries
2) trajectories not being always linear as they should be, it is probably related to the quality of the camera. Most probably the NN on some frames takes some of the background as an extension of the object, which becomes larger therefore making the trajectory look like jumping places. This might also be related to some positive/negative values seen with objects that are not yet near the edge.
The reason we never seen the issue on the other systems, we use a high quality Axis camera that most probably performs better.
We'll try the other sensor and report back.
In the meantime, once more you proved to be very experienced and knowledgable, and we thank you very much for the help provided over the past few days