Hi all. I have a VERY large unlabeled image set, but luckily it is well classified by folders as per following: ``` | - data/ | - train/ | - class 0/ | - image.jpg .... | - class 1/ .... | - class n/ | - validate/ | - class 0/ | - class 1/ .... | - class n/ ``` I trained MobileNet model using this dataset and I am very satisfied with results of running ``` preds=model.predict(x) pred=np.argmax(preds) print(pred) ``` which returns me a class number (and x obviously is an image in form of an array). PLEASE, give me a hint of how to make this run on OAK-D. All I've got is .pb file which contains model, no pipelines, no labels, nothing like example manuals I can find online. Thanks, Eugene.

Use this to convert MobileNet SSD to blob - http://69.164.214.171:8084/ Note- Use MobileNet SSD v2. Use the model with the known labels in example - https://github.com/luxonis/depthai-python/blob/main/examples/08_rgb_mobilenet.py

@"dhruvsheth-ai"#p975 Thanks for you reply. I use MobileNet v2. I should have mentioned this. I will amend my fist post if it's still possible. http://69.164.214.171:8084/ is saying "Converting..." for the last 30 minutes, and I have tried restarting the conversion several times. I even tried example the model from one of the Colab notebooks, and it's getting stuck as well. I'm not sure why, maybe the server is a bit too busy. In any case, I tried reading OpenVino docs, and they say I need frozen inference model, whilst my .pb file is not (I think...). It's been created by running `model.save('model\modeltf2', save_format='tf')` I tried googling how to create frozen inference model from "unfrozen" one, and all I can see is some sort of black magic.... Am I headed in the right direction? Thanks, Eugene.

I'm not sure why it's not converting. @"erik"#175 any idea about this? Could you also try - http://69.164.214.171:8083/

@"eugenek"#224 Thing with the 8084 is that it's in development. When error occurs (when optimizing/compiling), it won't show the error modal. You can view the request inside the "Network" tab (web developer tools). I would assume you are using wrong arguments (**advance** drawer) thus an error occurs.

@"dhruvsheth-ai"#p977 @"erik"#p978 I wasn't really able to get 8084 to work. I do see POST request going out in web dev tools, but nothing informative gets back. I used 8083, and at first it complained that my model doesn't contain a frozen TensorFlow graph in the text or binary format. I then used [this](https://leimao.github.io/blog/Save-Load-Inference-From-TF2-Frozen-Graph/) code to freeze it and use the resulting .pb file in 8083. It worked, but upon plugging to "08 - RGB & MobilenetSSD" example and populating labelMap with relevant entries, the script crashes and burns immediately with the following: ``` Traceback (most recent call last): File "08.py", line 106, in displayFrame("rgb", frame) File "08.py", line 80, in displayFrame cv2.putText(frame, labelMap[detection.label], (bbox[0] + 10, bbox[1] + 20), cv2.FONT_HERSHEY_TRIPLEX, 0.5, 255) IndexError: list index out of range ``` I tried just commenting line 80 out. Then it runs and draws a lot of "detection" rectangles on the video, none of which seem plausible. So something is very wrong with the model or conversion. But the damn thing works perfectly fine on the PC.... Additionally, when I run inference on PC, I scale the colours (dividing by 255). Should I do something like this in the camera script? I can't find anything resebmbling colour scaling in the Python API. Additionally, I use --input_shape=[1,224,224,3] argument when running mo.py... Maybe it needs to be taken into account in camera script as well, I don't know. Need some guidance here... Any help will be much appreciated.

@"eugenek"#p980 it usually takes some time, did you give it a few minutes? Anyways, it's great that you got 8083 to work. Oh and when compiling for OpenVINO version that's not the latest (used by `depthai` lib), you have to specify the version in your python script (eg. `pipeline.setOpenVINOVersion(version = dai.OpenVINO.Version.VERSION_2021_1)`). So it could be your model actually outputs more "labels" than you populated in `labelMap`. Printing the output from the NN node can provide info about it. If you specified `224x224` as the input you should set the camera preview size to `224x224` (otherwise you will get an error every time NN node tries to do an inference)

@"erik"#p982 Yep, I gave 8084 some time. So, I specified OpenVino version and printed out some stuff from NN. Indeed, it sometimes outputs absolutely tremendous number of labels, so I guess I got it all wrong while converting TF2 model to blob. As I mentioned in my previous post, I think I'm failing to export frozen graph. I found a few ways to do that online, all quite different, tried them all with no luck. I used basic TF2 stuff and bare MobileNet v2 model in my project. Vast majority of examples which eventually work with DepthAI use TF API a lot and pretrained models from https://github.com/tensorflow/models.git. But they all require labelled data set, with these xml files sitting next to each and every image. There's no example which shows a way to make it happen with unlabeled data sorted by folders (like I described in my first post in this thread or as shown here https://www.tensorflow.org/api_docs/python/tf/keras/preprocessing/image_dataset_from_directory). Is that possible? How can I make it work? Thanks, Eugene.

@"eugenek"#p985 Hi and sorry for the delay. I'm sort of clueless here. @"erik"#175 do you have any docs or suggestions here?

Hello, high number of labels/bounding boxes can also be because of low (or non-existent) threshold confidence. So to get a blob, you first have to convert from TF2 to OpenVINO IR. Intel has many resources on how to perform this, eg: https://github.com/openvinotoolkit/openvino_notebooks. After getting the IR (.bin and .xml) you can compile to .blob. To label a dataset there are many tools to help you with that, for example [Roboflow](https://roboflow.com/) and [SuperAnnotate](https://superannotate.com/). I believe you can then export the labels in the desired format. Thanks, Erik

OpenVino'ing trained MobileNet to run on OAK-D

stephansturges

MartinL do you have any suggestions or is the choice of model less relevant when it comes to complexity development/procedure?

Mobilenet_v2 is going to be a great choice for running your model in realtime on the OAK-D, so that's a good one to continue on. There's really no right or wrong answer there though... you might find a YOLO model is easier to start with if you are just starting with DNNs, because you can train the model with Darknet and don't need to learn the tensorflow API which is somewhat messy to be honest (although it can do a lot more than Darknet once you get the hang of it). It's also a lot easier to set up your data for ingestion into Darknet compared to tensorflow in most cases 🙂

This tutorial from Luxonis is excellent if you just want to get started with an SSD and run it on the OAK:
https://colab.research.google.com/github/luxonis/depthai-ml-training/blob/master/colab-notebooks/YoloV3_V4_tiny_training.ipynb

« Previous Page