We’re excited to announce a new feature in our Open-Vocabulary Object Detection demo — Mark & Detect.
With Mark & Detect, you can simply draw a bounding box around any object in an image, and the neural network will automatically detect and highlight similar objects across the scene. No predefined labels, no retraining — just intuitive, visual AI interaction.
What makes this even more powerful is that it runs directly on our OAK4-D cameras, delivering around 10 FPS detection speed right on the device — no cloud, no latency. This is a great example of how neural networks on the edge are shaping the future of computer vision: fast, private, and efficient AI that understands the world around it in real time.
Why Mark & Detect?
The Mark & Detect feature is designed for flexibility and speed, especially in environments where objects or setups change frequently. For example:
Fast prototyping - Test and visualize new ideas quickly without the need to label data or train a new model.
Dynamic production lines - Detect new or changing parts, tools, or products on the fly, even when the system hasn’t seen them before.
Ad-hoc object search - Instantly find all similar items in a scene by marking just one example.
Instead of training a model for every new object, you simply show it what you mean - and it finds the rest.
Try it Yourself
You can experience the Mark & Detect feature today by running the example from our OAK examples repo:
git clone https://github.com/luxonis/oak-examples.git
cd custom-frontend/open-vocabulary-object-detection/
oakctl connect
oakctl app run .
Once the app starts, just draw a box around any object in the video feed — and watch as the system finds all similar items automatically.
