ObjectTracker mis-classifying

KaranSingh

I used this blog to add object Tracking to my program. Its behaving in a weird manner. I put the object in frame and its tracking fine, but when I take it off the frame, its still tracking it.

When I put the object in the frame and remove it quickly, its still tracking the object and the bounding box keeps floating for some time.

Code:

jakaskerl

Hi KaranSingh
Tracklets remain in the lost state for a few frames. Typically you would have 4 states: NEW, TRACKED, LOST, REMOVED. When object is lost(out of frame) the tracker still tries to asses the objects position using some tracking algorithm. That's why it persist even though the camera can not see the object anymore.

You can set it so the bounding box disappears when a tracklet is given the status LOST.

Thanks,
Jaka

KaranSingh

Hi jakaskerl,

Thanks for your reply.

I set the bounding box to disappear with the status LOST. However, then there are several frames where there is no detection. And the reason I'm applying a tracker is to fill in those non-detection frames. So it doesnt help.

Is there any way to make it stop tracking as soon as the object is out of frame?

Thanks,
Karan

Edit: Can I not control how how many frames the LOST state should persist?

jakaskerl

KaranSingh Is there any way to make it stop tracking as soon as the object is out of frame?

No way to do that since there really is no difference between object not being detected and object being out of frame. Perhaps you can implement your own logic that looks at the tracklet data - if the tracklet is very close to the edge and the status is LOST, just remove it.

Thanks,
Jaka

KaranSingh

That is not the only issue. The corners are easy to deal with. The main issue with floating boxes in the middle. The object will be LOST and the bounding box will keep floating. But instantly after it draws another box around the same object while the previous one exists.

There are so many issues with this tracking.

Also I can't seem to integrate the yolo depthai detections with BYTETracker. Are they compatible? Can you provide a reference to this if exists?

jakaskerl

Hi @KaranSingh
The depthai trackers are not very configurable yet. You should be able to integrate BYTETracker if that fulfills your needs.

From BYTETracker repo readme:

from yolox.tracker.byte_tracker import BYTETracker
tracker = BYTETracker(args)
for image in images:
   dets = detector(image)
   online_targets = tracker.update(dets, info_imgs, img_size)

The detector (Yolo Node) should return detections (bbox, confidence, class), which you can easily input into BYTETracker. Detections have to be sent to host side since the VPU does not support external tracking solutions.

Thanks,
Jaka

KaranSingh

Thats the problem actually. You cannot put the detections from depthai directly into the tracker. It first needs to be converted to a format that is accepted by the tracked.

From depthai the detections are in the class depthai.detections something which is not directly accepted. It will throw an error

jakaskerl

Hi @KaranSingh
Yes, you need to parse the results so the can be passed to the tracker. Tracking is done on host side, so this shouldn't be a problem. Just check the input structure required by the ByteTracker.

Thanks,
Jaka