Accessing OAK-D-WiFi over WiFi

nalzok

Could someone point me to a tutorial that describes how to access an OAK-D-WiFi over WiFi? The official guide seems way too advanced and low-level. I'm looking for something like an out-of-the-box demo script, just like First steps with DepthAI, but over WiFi instead of a USB Type-C cable.

Brandon

Hi nalzok,

So I made a mistake in naming OAK-D-WiFi, well, OAK-D-WiFi. What we should have named it was OAK-D-IoT (which is what we now call it).

The why is that although it has Bluetooth and WiFi, the use-case of both is not for the standard DepthAI API to work over BT or WiFi, but rather for the API to work inside the onboard ESP32, and then metadata results can be returned over BT or WiFi. So in this case, it's acting like a classic IoT device.

So since this is our fault in naming (really, mine) we're more than happy to refund you on this device. As we discovered after shipping that naming it OAK-D-WiFi made everyone (rightfully-so) think they could use the standard DepthAI API over WiFi. (Which is not the case, unfortunately - it's more so to have an embedded/standalone unit that can communicate IoT results/etc. over WiFi, and the DepthAI API actually runs onboard the included ESP32 in the OAK-D-IoT/OAK-D-WiFi).

Even more confusing about naming it OAK-D-WiFi, which makes everyone think the same DepthAI API would run over WiFi is that OAK-D-PoE does in fact use the same exact API actually over Ethernet (and then can be used over WiFi w/ an Ethernet to WiFi converter/etc.).

Again, more than happy to refund here - sorry about the poor naming choice here.

-Brandon

Brandon

Oh and for what we envisioned for OAK-D-IoT (which should have been it's name, but I dumbly named it OAK-D-WiFi), was being able to get all the results (e.g. where a strawberry is in physical space) or some other metric and then with the onboard ESP32, get these results up to the cloud with something like this:
https://docs.microsoft.com/en-us/samples/azure-samples/esp32-iot-devkit-get-started/sample/

(Which should run unmodified on the OAK-D-IoT.)

nalzok

Brandon Sorry, I just saw your reply! I wonder if it is possible to make the standard DepthAI API work over BT or WiFi through a future firmware upgrade, probably by receiving commands from the computer with the ESP32 and then dispatch them to the DepthAI device? If not, why can't the hardware support it? I'm not blaming you or anyone; just curious about why the design decision was made as it is.

Anyway, this doesn't seem like horribly bad news, because (if I understand correctly) the only time you need to speak the standard DepthAI API with the device is during development. Once the device is deployed, it typically runs standalone and doesn't require a computer to tell it that to do.

Brandon

Zero worries of course. So likely the API would not work in a usable way over BT/WiFi as the BT/WiFi interface on the ESP32 is not meant for substantial throughput. The who ESP32 chip is architected more for IoT application - where most of the information that is sent/received is metadata in nature (so like JSON files at most, or more typically MQTT). Probably the highest throughput the OAK-D-IoT can support over WiFi is about 8mbps or so - so attempting to run the API over that would likely just never work right.

We don't currently have a high-throughput WiFi DepthAI device, although after learning the hard lesson of the mis-naming here, we have contemplated making one - as we have full TCP/IP support already working natively for DepthAI.

If we did that solution, it would likely still be relatively slow - likely 150mbps or so. Which means nearly all streams would have to be compressed.

So the why of OAK-D-IoT was actually for this:
https://github.com/luxonis/depthai-hardware/issues/9

The idea is more this IoT application, but in this case, allowing a visually impaired person to get feedback from audio or some other (haptic, for example) about their surroundings - what is where, trajectories, etc. So the idea is exactly that of all the processing and API back/forth happens between DepthAI and the ESP32. And then just results like "door is at 3pm, 5 meters" or <activate wrist buzz pattern 5> are the outputs from the device.

So as you say - not for interacting at a high-data-rate way with a computer - but rather running totally standalone, generating insights as a result of spatial AI + CV, and then taking action with those insights (by driving an actuator, making noise, or sending some insight out over BT/WiFi).

So yes, the better name for "OAK-D-WiFi" should have been "OAK-D-Embedded" or "OAK-D-AIoT" or "OAK-D-IoT" as the whole purpose is about having some relatively-small, completely-self-contained spatial AI device.

Thoughts?

Thanks,
Brandon

nalzok

Brandon OK, I get it. I actually think the name "OAK-D-WiFi" is fair enough given that we can still control the device through Wi-Fi by sending high-level instructions as long as they don't take a lot of bandwidth. For example, suppose the IP address of the onboard ESP32 is 192.168.6.100, then we can probably use curl -X POST -d 'cmd=sleep' 192.168.6.100 or curl -X POST -d 'cmd=wakeup' 192.168.6.100 to instruct the DepthAI device to enter/exit low-power mode? From my understanding, the only issue is that we cannot do crazy things like streaming video over the air due to bandwidth limitations.

What I'm trying to say is that you don't have to feel bad for choosing the name "OAK-D-WiFi". I would choose the same name if I were you, because hey, you can interact with the device in a meaningful way through WiFi!

Brandon

Thanks nalzok for the kind words!

And yes, exactly right. You can still interact with the device over WiFi using all the standard ESP32 techniques. What we've effectively made with OAK-D-IoT is a device that has a built-in embedded host. Like the OAK-D-CM4 has a built in actual host (Raspberry Pi CM4).

So then the DepthAI library resides on the ESP32, and beyond that we rely on the ESP32 community for all the things that folks would want to do (similar to Raspberry Pi). That said, we intended to make tutorials of how to get data to Azure/etc. We're just behind.

For folks who are used to the ESP32, this is a big advantage - as they're like "Sweet - this thing has an ESP32 onboard? I can use all my existing ESP32 code/etc.". For those who aren't it's a bit annoying as it means there's another toolset to learn.

So in terms of getting the ESP32 on WiFi, I don't think we have instructions - but I'll ask. I think any/all of the examples from below should work, although I have not personally used ESP32s. So I can't really speak well to it:

https://github.com/espressif/esp-idf/tree/220590d/examples/wifi

I know that we're also working to add Arduino Support (PlatformIO) support directly to ESP32 onboard OAK-D-IoT as well (with a working proof of concept, which is just painful to use, otherwise we would have released it already).

Best,
Brandon

jiapei100

@Brandon

Hi, Brandon, I just noticed this message, and I also purchased such a OAK-D-Wifi from https://www.kickstarter.com/projects/opencv/opencv-ai-kit .

Please also take a look at my blog: https://longervision.github.io/2021/06/29/MachineVision/Stereo/OpenCV-AI-KIT/ , which was initialed in 2021, and further edited just today. ONLY until just now when I was trying to SSH into this OAK-D-Wifi, I noticed your this message. I wonder if

either there is a replacement of this kit?
or can I ask for refund as well??

Thank you very much…

Cheers

jakaskerl

Hi @jiapei100
Can't say for sure, best to ask over at support@luxonis.com. Though the models like IOT have been community support only for some time now.

Thanks,
Jaka

jiapei100

@jakaskerl Thank you very much… Writing an Email to Luxonis…