Limitation in object tracking for oak-1

raviks

Hi, i am planning to use the OAK-1 for object tracking specifically for people related use-case in an office premises. I see in one of the documentations on OAK-1 its mentioned that it can track only upto 20 objects with unique id's (https://store.opencv.ai/products/oak-1).

Is there a solution to increase this limit to for example upto 50 unique track id's ?

Thanks,
Ravi.K

GergelySzabolcs

raviks

Maximum number of tracked objects is 60 by default, for ZERO_TERM_COLOR_HISTOGRAM tracking mode.
Can be configured using objectTracker.setMaxObjectsToTrack(value) up to a theoretical 1000 using depthai library 2.12.0.0 or newer.
Here is an example:
https://github.com/luxonis/depthai-python/blob/main/examples/ObjectTracker/object_tracker.py

theDesertMoon

GergelySzabolcs Hi, I was playing around with this and continuously encounter this
critical error:

[18443010C1A5FA0F00] [10.0.14.31] [21.922] [system] [critical] Fatal error. Please report to developers. Log: 'ShaveHandler' '172'.

Running this from a Windows Command Prompt does yield a little more information:

[2022-09-21 12:53:07.380] [warning] Monitor thread (device: 18443010C1A5FA0F00 [10.0.14.31]) - ping was missed, closing the device connection

# possible tracking types: ZERO_TERM_COLOR_HISTOGRAM, ZERO_TERM_IMAGELESS, SHORT_TERM_IMAGELESS, SHORT_TERM_KCF
objectTracker.setTrackerType(dai.TrackerType.ZERO_TERM_COLOR_HISTOGRAM)
objectTracker.setMaxObjectsToTrack(100) #Maxes at 1000 but setting it to match object detector.
objectTracker.setTrackerIdAssignmentPolicy(dai.TrackerIdAssignmentPolicy.SMALLEST_ID)

With the 'setMaxObjectsToTrack' line commented out it runs with no issue, but tracking does max at 60 objects. The model I'm using is currently a 6 shave model though I have a 5 shave version that runs into the same issue (and it nannies me that I should use the 6 shave model. 🙂 )

I've also tried the first three tracking types as your Object Tracking page says that Short_Term_KCF maxes at 60 objects. I'm currently at code level 2.17.4.0.

While you and the page say that theoretically we can track up to 1000 items the method description further down says 60 is the max.

setMaxObjectsToTrack(self: depthai.node.ObjectTracker, maxObjectsToTrack: int) → None
Specify maximum number of object to track.

Parameter maxObjectsToTrack:
Maximum number of object to track. Maximum 60.

Can more than 60 objects be tracked? What am I missing that may be causing this to fail?

Thank you for any assistance you can provide,

Deon T

raviks

Thanks for the quick update. Since the ZERO_TERM_IMAGELESS achieves higher tracking compared to ZERO_TERM_COLOR_HISTOGRAM, may i know for which trackerType we can achieve only 20 track id's ? Or is this number 20 a mistake that needs to be corrected at https://store.opencv.ai/products/oak-1 ?

Thanks,
Ravi.K

GergelySzabolcs

raviks it's a mistake, thanks for pointing out.
Cc: @erik

GergelySzabolcs

@theDesertMoon looks like the limit is indeed 60. Not sure where it comes from.

theDesertMoon

GergelySzabolcs Ah, that sucks.

Does the 'theoretical' on this page refer to another method of object tracking that the OAK camera's have. further down the pipeline?
[https://docs.luxonis.com/projects/api/en/latest/components/nodes/object_tracker/]

Maximum number of tracked objects
SHORT_TERM_KCF can track up to 60 objects at once, while all other trackers can (theoretically) track up to 1000 objects at once.

Even though it crashes the camera/pipeline with a critical error it did momentarily track up to 91 items in a frame and I can see tracklet ID's in the 60's, 70's and 80's.

Is there some resource on the host the camera is looking for? It seems like the error mentions "ShaveHandler". Is there an issue with compiling the model with too many/too few shaves?

Thanks for any information you can provide.

Deon T

GergelySzabolcs

theDesertMoon
It's an issue with the object tracker implementation in firmware.

theDesertMoon

GergelySzabolcs
Okay. 🙂 👍️

Any chance of it being addressed in a future firmware update or is it a closed issue until some future series re-implementation?

GergelySzabolcs

theDesertMoon
What's your volume?

erik

theDesertMoon in general, we don't have nearly enough engineers to fix all non-critical issues (such as this one) in firmware. That said, if it's a large opportunity, we could do it for an NRE cost, or even for free if it's a large enough opportunity. Thoughts?
Thanks, Erik

theDesertMoon

GergelySzabolcs erik Sorry, off on the weekends.

We're doing low volume at the moment for PoC purposes across the line, a few OAK-1-Lites, OAK-D-Lite and now we've moved to your POE offerings. We're envisioning fitting machines with 3-6 PoE cameras each, so eventually we're anticipating ordering in larger quantities of around 100 or so camera’s a year.

My interest lies mainly in the fact that object tracking makes a more accurate speed determination possible since there's no guarantee the cameras will identify the objects in an identical order each time.

Gergely's description of it as "...an issue...in the firmware" made me curious if its a good enough for now or if its being worked also.

I would be interested if there's a quote for the work required that I can present to my manager.

Thanks,
-Deon T

theDesertMoon

erik I have another question: I see back in May on Discord you asked Luxonis-Szabi a question similar to mine and he replied not at 30 fps. Is there a FPS scaling chart where I can increase the number of tracked objects based on how much I reduce the FPS by?

erik

Hi theDesertMoon , very likely not, but if you do create one feel free to share, so we can add it on docs🙂

theDesertMoon

erik
Ha, I think Szabi was just joking. 😃

camRgb.setFps(5)

# testing MobileNet DetectionNetwork
detectionNetwork.setBlobPath(nnPath)
detectionNetwork.setConfidenceThreshold(0.50)
detectionNetwork.input.setBlocking(False)

# possible tracking types: ZERO_TERM_COLOR_HISTOGRAM, ZERO_TERM_IMAGELESS, SHORT_TERM_IMAGELESS, SHORT_TERM_KCF
objectTracker.setTrackerType(dai.TrackerType.ZERO_TERM_COLOR_HISTOGRAM)
objectTracker.setMaxObjectsToTrack(61)

Even down to an unusable 5 fps, even trying to get one more tracklet fails. 😛

I'm not sure how I missed your reply from last year though about other Object Tracker functionality:

sorry about the inconvenience, this shouldn't happen, but we, unfortunately, can't do anything about it. Object tracker logic is proprietary from Intel and is compiled so we can't change/fix anything.

I'm am still interested in either of the options you mentioned earlier if something has changed however.

Thank you,
-Deon T

GergelySzabolcs

theDesertMoon

I looked into it and doesn't look like an easy-to-fix BUG.
Will look more this weekend if time permits. Will keep you updated.

theDesertMoon

GergelySzabolcs Thank you, sir!