Correlating image sent to neural network with neural network response

abelard

Hi,

I want to track faces and for each tracklet returned on a face, submit it to a neural network so that I can get some analysis. When the network responds I want to show that information for the same tracklet where the face came from.

How can I match the faces being sent to the neural network with the neural network responses? Is there any kind of assurance that the neural network puts responses in its output queue by the same order they are requested?
What do you think should be the best way to implement this scenario?

TIA

Brandon

abelard Great question. I don't immediately know how to do this but I'm confident it's possible.

Will need the teams help here. We'll circle back.

erik

Hello abelard ,
this seems like a similar flow as with age/gender recognition demo - so it detects faces (with mobilenet) and with the help of Script node it crops faces and send them to 2nd NN for analysis (in our case age/gender recognition). To actually match faces with NN responses I would rely on sequence - so the first frame you send to the NN will result in the first NN response. Another option (if this for some reason wouldn't work as expected) would be to get passthrough output of the NN node and cross-check if it's the same image (like hash check for performance reasons).
Thanks, Erik

abelard

Hi Brandon, Hi Erik,

Thanks for your replies!

I took some days off...catching up now!

I did took age & gender as example, but taking a look at your link it seems you have a different version of it. The one I used, uses a python queue to synchronize detected faces being sent to age & gender neural network with the network's analysis responses. Actually this is kind of my original question...can I trust this python queue to return me the matching age & gender analysis to each face image being submitted?

Anyway I'm going to take a look at this other version of age & gender.

About your suggestion for using the passthrough output of the NN node. As far as I understand that output would be available at the Host in a different queue than the age & gender analysis. My question...How could I be assured to get them both at the same time, or in any other way that guarantees they are correlated?

Hope to hear from you soon!

Best regards

erik

Hello abelard,
sorry about the delay. So the python queue version you are talking about is the old version of age-gender demo. It sends frames/NN detections to the host where it crops the frame and then sends it back to the device. The new version uses Script node to crop the original image (with ImageManip) on the device itself, so you avoid this back-and-forth communication between the device and the host.
In the example I mentioned, the script node handles the passthrough frames and does frame/NN output sync, so everything is nicely in-sycn🙂 They are correlated based on the timestamp.
Thanks, Erik

abelard

Hi Erik,

Yes, I took a brief look at the new version and kind of conceptually understood what the code is doing. Still need some effort to fully grasp it.
If you don't mind I'll get back to you if I have some doubts 🙂

Again, thank you for your support and congratulations on your product!

Best regards

erik

Hello @abelard , sounds good and thank you for the kind words🙂