Video stream over WiFi?

Sindel

For a performance project I want to have a bodycamera that is able to detect distance of other human bodies in low light conditions (even darkness) and send this data over WiFi to a computer for data mapping to sound processing.
I did some research and I found the BW1092 board, which has an ESP32 with which I worked before.
I have a few questions about the feasibility of the project with this technology:
Is the BW1092 board and a battery pack the only things needed for a mobile project?
Is it possible apart from the data to stream the video from one of the cameras (color, or depth) via WiFi?
Is this possible for a maker with moderate experience?
Is there another commercial device that could do that?
Any help is appreciated!

Brandon

Hi Sindel ,

Thanks for the interest, and sounds great!

On the darkness side - so the cameras on all current DepthAI models actually block IR. I mention this because a common way to allow computer vision in complete darkness is actually to illuminate with either blanket IR (like here) or with an IR pattern projector (best for depth, like here).

So the existing DepthAI models (including BW1092) do fairly well in low-light, but are not no-light capable.

To your questions:

Is the BW1092 board and a battery pack the only things needed for a mobile project?

Yes, that's right. With the caveat on no-light capability above. If you want no-light (and perhaps for low-light), I'd recommending this setup instead:

1x https://shop.luxonis.com/collections/all/products/usb3c-ffc-body
2x https://www.arducam.com/product/arducam-1mp-ov9282-global-shutter-mono-mipi-camera-module-20pin-for-depthai/
A Raspberry Pi host for sending out the video stream and metadata.
An IR illuminator like here

Is it possible apart from the data to stream the video from one of the cameras (color, or depth) via WiFi?

We are hoping that the BW1092 will be able to stream video over WiFi eventually. But right now the onboard SPI interface between DepthAI and ESP32 is limited to 100kbps (quite slow!) because of some unkown issue. So right now video is not possible, just low-resolution thumbnailing. For reference, 1080p/HD video with h.265 encoding is about 3,000kbps, so about 30x what the BW1092 can do out over SPI (and therefore, through WiFi, as the ESP32 communicates with DepthAI over SPI).

Is this possible for a maker with moderate experience?

Besides the 100kbps limit above, it for sure would be.

Is there another commercial device that could do that?

Aside form the modular-camera recommendation above (which supports low-light), actually our DM1097 is quite nice to work on as it has an integrated Pi 4CM, and integrated DepthAI:

https://shop.luxonis.com/collections/all/products/depthai-rpi-compute-module-4-edition

So it would be very easy to do what you want with it. The only rub is that it doesn't have IR-capable cameras on it. But here is an example of locating people in physical space with it:
https://github.com/luxonis/depthai-experiments/tree/master/social-distancing

Thoughts?

Thanks,
Brandon

Sindel

Hey Brandon, thank you for the info, and my apologies for not following up earlier!
Good that you mention the SPI limitation on the BW1092. I was definitively not aware of that and it is quite an important factor.
The modular combination you recommended has an increased overall complexity and size due to the number of parts involved, which goes against the main feature of portability (it needs to be a bodycam).
I mainly need the metadata extracted from the depth cameras. The wifi streaming would have been an awesome addition, but not strictly necessary all the times. I guess I can access the video feed if I connect it to a host, when needed?
I asked some time ago in the discord channel about low-light capabilities and Erik said that they should react well to IR. If this is not correct would it be possible to attach the IR-capable cameras you mentioned to the BW1092?
Thanks!

Brandon

Hi Sindel ,

Thanks. And yes I discussed with the team on the SPI limitation, and we think we should be able to get it fast enough such that at least 3mbps should be possible over SPI - which is enough for encoded 1080p video.

The only question about getting video off then is if the ESP32 can do 1080p video over WiFi. I'm not immediately sure.

Searching... It seems possible? https://www.instructables.com/ESP-32-Camera-Streaming-Video-Over-WiFi-Getting-St/

Once we get the SPI throughput increased, we'll be able to test.

So I think once this SPI throughput limitation is fixed you'll be able to stream 1080p video over WiFi from the BW1092.

I asked some time ago in the discord channel about low-light capabilities and Erik said that they should react well to IR. If this is not correct would it be possible to attach the IR-capable cameras you mentioned to the BW1092?
It is possible to attach IR-capable cameras. The default cameras are IR-blocking. But ArduCam sells direct replacements. And in the future we will use these.

https://www.arducam.com/product/arducam-1mp-ov9282-ccm-drop-in-replacement-for-oak-d/

So the only trick is that existing BW1092 have the modules glued down. So you'd have to use IPA (isopropyl alcohol) to soak that off and remove them. So not super easy.

But in the future we will have these ArduCam IR-capable ones on by default.

Thoughts?

Thanks,
Brandon

Sindel

Hey Brandon, thank for the help! And great that you are already improving the transmission rate.
Wether the ESP can do 1080p streaming is a good question. I don't have an hard answer, but ready made code of ESP32 camera based modules can stream reliably only at lower resolutions (640x480 If I remember correctly). But I believe that it might be because the esp must do a bunch of stuff like convert to JPG. If the conversion is done on your module maybe it could run better qualities?
Regarding the camera replacement: I am quite handy, if you say that the operation is not extremely risky I could try that.
Let me know!

Brandon

Sindel

Thanks. Yes, I have the same ponderance. Not 100% sure, but I'm guessing that what you describe is correct - that the lower load from now having to do on-ESP conversion will allow higher resolution (or frame-rate).

We're a bit swamped right now, but I'm guessing by July we'll have higher SPI throughput.

Thanks,
Brandon

Sindel

Hey Brandon, thanks for the reply.
I ordered the BW1092 today but unfortunately the replacement cameras you suggested are out of stock, with no indication of a lead time. Are they available elsewhere?
Please keep up the work on the SPI improvement! I'm sure many more like me will need a proper bandwidth between the chips.

Brandon

Thanks. So these do exist. I think ArduCam is in the process of switching between revisions. Will ask offline and circle back.

Thanks,
Brandon