Help needed with "08 - RGB & MobilenetSSD" example

eugenek

Hi all!
I used the following example, as is, ie with pictures of bananas, apples and oranges.
I then downloaded the .blob, inserted the path to it into "08 - RGB & MobilenetSSD" example code and replaced labelMap array with:
labelMap = ["apple","banana","orange"]
When I run the script and point my OAK-D at some pics, four things happen:
1) It totally ignores oranges. Always.
2) It reliably detects bananas, but calls them oranges.
3) It reliably detects apples, but calls them bananas.
4) Often, when there are several objects in the pic, script fails with:

Traceback (most recent call last):
  File "08test.py", line 103, in <module>
    displayFrame("rgb", frame)
  File "08test.py", line 77, in displayFrame
    cv2.putText(frame, labelMap[detection.label], (bbox[0] + 10, bbox[1] + 20), cv2.FONT_HERSHEY_TRIPLEX, 0.5, 255)
IndexError: list index out of range

What can be wrong and how to fix?

Gratefully,
Eugene.

eugenek

I kept playing with that stuff and ended up changing labelMap to
labelMap = ["DUNNO","apple","banana","orange"]
Now it doesn't throw error, detects apples and bananas, but ignores 95% of oranges. It is despite the fact that Colab section "Running Inference: Checking what the trained model can detect" did detect oranges well.
And I still don't get why the indexes are shifted.... label_map.pbtxt is as follows:

item {
    id: 1
    name: 'apple'
}

item {
    id: 2
    name: 'banana'
}

item {
    id: 3
    name: 'orange'
}

Is it because it is 1-based, and python demo script uses 0-based indexing? Still won't explain what's the oranges problem..
Thanks.

erik

So I reproduced it and got the same results. Never detects oranges, but is accurate with apples/bananas. Since the detections are accurate (even for oranges) after the training inside notebook, I wonder if there is a problem with our createMobileNetDetectionNetwork(). Could also be something with ids, as you pointed out. I will re-train the model with shifted labels (so orange isn't last) and try again. Thank you for noticing this issue!

eugenek

erik I look forward to hearing what's the final verdict. Thanks.

eugenek

Any news, @erik?
Cheers.

erik

Hello @eugenek,
I apologize for not circling back; so there's probably something with our mobilenet detection network. I tried using NeuralNetwork node (instead of MobilenetDetectionNetwork) and it works better (detects oranges without a problem). See the code below:

eugenek

erik
thanks for that. It works on my end as well.

Cheers,
Eugene.

erik

To circle back: as we suspected, there was a problem with the MobileNetDetectionNetwork on the device side. Something with max number of detections, and oranges were out of range. That's why it worked when manually decoding the result (using NeuralNetwork node). We already fixed the issue and it should be out soon.
And thanks again for reporting this!