People Tracking & Counting with SDK

erik

The DepthAI SDK makes it significantly faster and easier to develop applications for OAK cameras. In many cases, complex tasks that would otherwise take hundreds of lines of code can be done in a fraction of the time and space.

Take a look at the following people tracking and counting application, which was created from only 8 lines of code:

from depthai_sdk import OakCamera, BboxStyle

with OakCamera(replay='people-tracking-above-04') as oak:
  camera = oak.create_camera('color', fps=30)

  nn = oak.create_nn('person-detection-retail-0013', camera, tracker=True)
  nn.config_nn(conf_threshold=0.2)

  visualizer = oak.visualize(nn)
  visualizer.detections(bbox_style=BboxStyle.CORNERS, color=(232, 36, 87), hide_label=True)

  oak.start(blocking=True)

Feel free to head over to DepthAI SDK documentation page to get started.

Charles

Hi, how could I change people to pig? I am looking for pig tracking and counting application algorithm. Thanks.

Charles

jakaskerl

Hi Charles
You would have to train a custom neural network that is capable of detecting pigs. Here's a study that uses prepared dataset (downloadable): https://www.hindawi.com/journals/mpe/2022/7078670/#abstract.

We also have some collab notebooks to aid in model training: https://github.com/luxonis/depthai-ml-training/tree/master/colab-notebooks

Hope this helps,
Jaka

Craftonix

Hi and thanks for this great example!

How can I change it to:

Add callback to read x,y coords of bbox?
Can I store the value of a variable on the OAK-D-PoE to retrieve it later? (IOW, can I store to flash and not to RAM?)
Run stand-alone (I'm only interested in detections, and not to visualize every frame)
Add webserver on the OAK-D-PoE to display a camera capture (one image per button click or per URL refresh, i.e. no need for high FPS)

I would like to use the latest version of the SDK, but I'm finding that the docs are a bit lagging behind the features, so any help by providing short snippets would be greatly appreciated.

Thanks!

erik

Hi Craftonix ,

You can use callbacks; oak.callback(nn, callback=my_function), which will call the function with the Detection Packet, which also contains bounding box.

def my_function(packet: DetectionPacket):
    for det in packet.img_detections.detections:
        print(det.xmax, det.ymax, det.xmin, det.ymin, det.label, det.confidence)

Note that for such custom app, you might rather use depthai API (instead of SDK), which is a bit more low-level, but provides more flexibility.

You can save to eMMC persistently using the script node, example here: https://docs.luxonis.com/projects/api/en/latest/samples/Script/script_emmc_access/#script-emmc-access
You can, docs here: https://docs.luxonis.com/projects/api/en/latest/tutorials/standalone_mode/
Yes, you can spin up http/tcp/.. server within the Script node, we have quite a few examples listed here: https://docs.luxonis.com/projects/api/en/latest/components/nodes/script/#examples-of-functionality

Thoughts?

Craftonix

Thanks Erik for the fast response!

I'm using the tracker because I need this functionality. I did try the callback, but it seems that when I do OakCamera.create_nn(…, …, tracker=True), my callback

   def cb(packet: DetectionPacket):

needs to be changed to

    def cb(packet: TrackerPacket):

So, according to the docs:

TrackerPacket

classdepthai_sdk.classes.packets.TrackerPacket(name, msg, tracklets, visualizer=None)

Output of Object Tracker node. Tracklets + Image frame. Inherits FramePacket.

But when I try to access tracklets, I get

AttributeError: 'TrackerPacket' object has no attribute 'tracklets'

This is what I meant by "I'm finding that the docs are a bit lagging behind the features"

As for the eMMC, following the instructions from here, I ran the prerequisite code to enable the eMMC, but I don't get "Flash OK". So, I printed both r and errmsg, so I got:

Flashing progress: 0.0%
False
No eMMC memory available

So, I thought maybe the eMMC is already enabled on my OAK-D-PoE (series 2), so I ran scriptemmcaccess.py but I got:

[1844301021A2970F00] [192.168.2.221] [6.090] [Script(2)] [warning] Saving to EMMC: /media/mmcsd-0-0/1000.jpg
[1844301021A2970F00] [192.168.2.221] [6.090] [Script(2)] [critical] FileNotFoundError: [Errno 2] N: '/media/mmcsd-0-0/1000.jpg'

At:
  <script>(15): <module>

Traceback (most recent call last):
  File "/Users/alain/Luxonis/depthai-python/examples/Script/script_emmc_access.py", line 92, in <module>
    inRgb = qRgb.tryGet()
RuntimeError: Communication exception - possible device error/misconfiguration. Original message 'Couldn't read data from stream: 'rgb' (X_LINK_ERROR)'

Finally, just wondering if I can run all the above tasks together, i.e. do I need one script node for the webserver and one for writing to the eMMC, and if yes, how do I communicate between script nodes? (This is all standalone and nothing running on the host)

Thanks!

Craftonix

and in case it's helpful:

python3 bootloader_version.py
Found device with name: 192.168.2.221
Version: 0.0.22
NETWORK Bootloader, is User Bootloader: False
Memory 'Memory.FLASH' size: 33554432, info: JEDEC ID: 01 02 19
Memory 'EMMC' not available...

Craftonix

Any answer regarding the above error?

AttributeError: 'TrackerPacket' object has no attribute 'tracklets'

Fakhrullo

hey guys, it seems a cool project, I also tried to run this project, but it's so laggy somehow, how can I get the exact FPS from it, and is there a possible way to define the ROI(detection area)?

jakaskerl

Hi Fakhrullo
FPS is obtained by passing fps=True to oak.visualize(). In regards to ROI, I'd would advise you to use the API since this is not implemented in the SDK yet and would likely be very confusing to incorporate.

Thanks,
Jaka

Fakhrullo

Hi jakaskerl
Thank you for the information

Fakhrullo

jakaskerl
One more question, I'm getting really low FPS like it's not even reaching 2 FPS

How can I solve it? What are the problems causing such a low FPS?

SamiUddin

first try fetching 60 fps on line 4, instead of 30.

erik

It might also be USB2 mode, or POE camera, and streaming high-res frames isn't possible through such link, therefore the FPS is low.