Hi there,
I'm a little unclear on the standalone of the ESP32-enabled devices and I'm hoping someone here can help me out!
With a device like the OAK-D-IoT-75: do I understand correctly that I can connect this thing to a local Wifi network and stream the video output of my DNN over wifi to a host without any other device connected to the camera? Meaning the camera is only powered over USB?

If so, since there is no host device connected locally I can only rely on the video processing enabled by the depthai library I guess, so I don't understand: if I output frames of segmentation maps (numpy arrays I guess) locally can I overlay these frames with the "image frame"-stream from the camera locally before transmitting over ESP32 or do I need to transmit two bit streams of data (image_frame and DNN_output_frame) and recombine on the host at the other end of the WiFi network?

Thanks!

  • erik replied to this.

    Hello stephansturges ,
    Regarding first question; yes, you can stream (preferably encoded) video to a server via local WiFi (demo here).

    Locally, do you mean on the VPU (see img below)? If so, you could decode and overlay the segmentation results on the VPU itself, but would need to create custom NN model in order to do that, see docs here. You could also stream both segmentation results and video streams to the server and do overlay there, which would be much less complicated.

    (from docs)

    Thanks, Erik

        erik

        erik Locally, do you mean on the VPU (see img below)? If so, you could decode and overlay the segmentation results on the VPU itself, but would need to create custom NN model in order to do that, see docs here.

        Yep that's what I meant... ok got it now!

        erik You could also stream both segmentation results and video streams to the server and do overlay there, which would be much less complicated.

        Ahhh ok that makes sense. I did not see any examples of this in the luxonis repos so it was not clear to me whether it's possible to have two concurrent bitstreams from the device that would carry (1) preview frames (2) inference arrays.

        So I guess it's as simple as setting up two streams with a similar structure as the example below from here

            print("Creating SPI pipeline: ")
    print("COLOR CAM -> ENCODER -> SPI OUT")
    pipeline = dai.Pipeline()

    cam_color         = pipeline.create(dai.node.ColorCamera)
    spiout_preview    = pipeline.create(dai.node.SPIOut)
    videnc            = pipeline.create(dai.node.VideoEncoder)

    # set up color camera and link to NN node
    cam_color.setPreviewSize(300, 300);
    cam_color.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1080_P);
    cam_color.setInterleaved(False);
    cam_color.setCamId(0);
    cam_color.setColorOrder(dai.ColorCameraProperties.ColorOrder.BGR);

    # VideoEncoder
    videnc.setDefaultProfilePreset(1920, 1080, 30, dai.VideoEncoderProperties.Profile.MJPEG);

    # Link plugins CAM -> ENCODER -> SPI OUT
    cam_color.video.link(videnc.input);
    spiout_preview.setStreamName("spipreview");
    spiout_preview.setBusId(0);
    videnc.bitstream.link(spiout_preview.input);

    return pipeline

        Would two such streams work concurrently (I mean bandwidth notwithstanding)?
        Thanks!

        • erik replied to this.