OAK-D POE S2: Multi-model + Standalone mode (TCP Streming)

In my attempt to follow the FireDetection example, I'm currently facing some uncertainty regarding the process of transitioning from the "lpb.NNData" object, which represents the results obtained from the fire_detection neural network, to a tensor result. Specifically, I'm looking to access the correct layer, "final_result" onto script node, which will be flashed on the device.

script.setScript("""
import socket
import time

server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(("0.0.0.0", 5000))
server.listen()
node.warn("Server up")

labelMap_SSD = ["background", "aeroplane", "bicycle", "bird", "boat", "bottle", "bus", "car", "cat", "chair", "cow",
                "diningtable", "dog", "horse", "motorbike", "person", "pottedplant", "sheep", "sofa", "train", "tvmonitor"]

label_fire = ["fire", "normal", "smoke"]

while True:
    conn, client = server.accept()
    node.warn(f"Connected to client IP: {client}")
    try:
        while True:
            pck = node.io["frame"].get()
            data = pck.getData()
            ts = pck.getTimestamp()
            
            # --------MobilenetSSD
            detections_ssd = node.io["detSSD"].tryGet()
            if detections_ssd:
                dets = detections_ssd.detections
                
                data_ssd = []
                for det in dets:
                    label = labelMap_SSD[det.label]
                    if label == "person":
                        det_bb = [det.label, det.xmin, det.ymin, det.xmax, det.ymax]
                        data_ssd.append(det_bb)

	    # --------FireDetection
	    data_fire = node.io["detFire"].tryGet()
            # node.warn(f"data_fire: {data_fire}")
            # TODO: extract tensor data ???


            # now to send data we need to encode it (whole header is 256 characters long)
            header = f"ABCDE " + str(ts.total_seconds()).ljust(18) + str(len(data)).ljust(8) + str(data_ssd).ljust(224)
            conn.send(bytes(header, encoding='ascii'))
            conn.send(data)

    except Exception as e:
        node.warn(f"Error oak: {e}")
        node.warn("Client disconnected")
""")

Any help will be appreciated.
Irena

Hi jakaskerl !

Result for OAK-D S2 PoE FF:

Found device with name: 10.1.1.102
Version: 0.0.22
NETWORK Bootloader, is User Bootloader: False
Memory 'Memory.FLASH' size: 33554432, info: JEDEC ID: 01 02 19
Memory 'EMMC' not available...

Here you can check script which I'm tryinf to flash on the device. I would greatly appreciate any assistance.

Thank you in advance, Jaka!
Irena

Hi @jakaskerl !!

Thanks for your quick response.

I tried using other models, and it turns out they don't work. This is a major issue for us because our priority that the cameras can work in standalone mode.

In response to your feedback and our tests, I have a series of questions regarding possible solutions and implementation on our devices.

As you mentioned, now we are trying to avoid standalone mode. With that in mind, and considering our goal of operating 12 cameras with a single PC, we've noticed that when we execute the script (which you can check oak_ssd_yolov5.py) on the host, it launches 8 threads per camera. 7 of these threads correspond to the created nodes, plus the main script.

Logs:

I understand it's a complex issue, but is there any way to reduce or encapsulate this behavior?

Secondly, can we run a single script for all our cameras? We are thinking about the scalability of our development, so it was important to us that the cameras can work in standalone mode. Since that's not possible, our question is whether we can execute a single .py to all of our devices.

Lastly, looking ahead to our next projects, do you think if this dives, OAK-1 POE FF fit to us and our aims (custom multimodel standalone mode + TCP streming) also it's possible to incorporate an M12 connector into this device? If so, who can I speak to about it?

I appreciate all the help and look forward to your response.
Irena

Hi erik !!

Thank you for your response. I'm eager to know if there's any information available regarding the release date and potential cost of these devices. This information holds significant relevance for our upcoming projects.

Thanks once again!
Irena