Hi there,
I'm a little unclear on the standalone of the ESP32-enabled devices and I'm hoping someone here can help me out!
With a device like the OAK-D-IoT-75: do I understand correctly that I can connect this thing to a local Wifi network and stream the video output of my DNN over wifi to a host without any other device connected to the camera? Meaning the camera is only powered over USB?

If so, since there is no host device connected locally I can only rely on the video processing enabled by the depthai library I guess, so I don't understand: if I output frames of segmentation maps (numpy arrays I guess) locally can I overlay these frames with the "image frame"-stream from the camera locally before transmitting over ESP32 or do I need to transmit two bit streams of data (image_frame and DNN_output_frame) and recombine on the host at the other end of the WiFi network?

Thanks!

  • erik replied to this.

    Hello stephansturges ,
    Regarding first question; yes, you can stream (preferably encoded) video to a server via local WiFi (demo here).

    Locally, do you mean on the VPU (see img below)? If so, you could decode and overlay the segmentation results on the VPU itself, but would need to create custom NN model in order to do that, see docs here. You could also stream both segmentation results and video streams to the server and do overlay there, which would be much less complicated.

    (from docs)

    Thanks, Erik

      erik

      erik Locally, do you mean on the VPU (see img below)? If so, you could decode and overlay the segmentation results on the VPU itself, but would need to create custom NN model in order to do that, see docs here.

      Yep that's what I meant... ok got it now!

      erik You could also stream both segmentation results and video streams to the server and do overlay there, which would be much less complicated.

      Ahhh ok that makes sense. I did not see any examples of this in the luxonis repos so it was not clear to me whether it's possible to have two concurrent bitstreams from the device that would carry (1) preview frames (2) inference arrays.

      So I guess it's as simple as setting up two streams with a similar structure as the example below from here

          print("Creating SPI pipeline: ")
          print("COLOR CAM -> ENCODER -> SPI OUT")
          pipeline = dai.Pipeline()
      
          cam_color         = pipeline.create(dai.node.ColorCamera)
          spiout_preview    = pipeline.create(dai.node.SPIOut)
          videnc            = pipeline.create(dai.node.VideoEncoder)
      
          # set up color camera and link to NN node
          cam_color.setPreviewSize(300, 300);
          cam_color.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1080_P);
          cam_color.setInterleaved(False);
          cam_color.setCamId(0);
          cam_color.setColorOrder(dai.ColorCameraProperties.ColorOrder.BGR);
      
          # VideoEncoder
          videnc.setDefaultProfilePreset(1920, 1080, 30, dai.VideoEncoderProperties.Profile.MJPEG);
      
          # Link plugins CAM -> ENCODER -> SPI OUT
          cam_color.video.link(videnc.input);
          spiout_preview.setStreamName("spipreview");
          spiout_preview.setBusId(0);
          videnc.bitstream.link(spiout_preview.input);
      
          return pipeline

      Would two such streams work concurrently (I mean bandwidth notwithstanding)?
      Thanks!

      • erik replied to this.