Hi BetaMax Thanks for all the great info here (and also the kind words!). I don't know how I hadn't heard of Antmicro before, but some how I managed! They have some really neat stuff. I kind of want to meet these guys actually! So similar to them we'll be releasing our carrier boards to Github as well (it's on our docket, and actually may do it this weekend... the only reason we hadn't yet is because, well, they weren't fully tested/verified until now, but probably should have just posted w/ the caveat anyways).
Anyways, will be up soon. I just took a look at their Github and they use Altium as well! So that makes collaboration easy. And I even saw that they have an Edge TPU carrier board design in Altium, which is nice. We're planning on making a PoE-power carrier board for the Edge TPU SoM, for our own purposes and also we figure it will be useful for others.
It's actually kind of funny how similar our uses of tech are. So they are clearly using the Jetson series (they have nifty open-source carrier boards for it), and so are we, say for the drone stuff here, we're also working w/ the Edge TPU (making the PoE carrier for the SoM soon here), and of course the Myriad X stuff (although it looks like they are using Myriad 2) in their design. Cool to see! Also the mining application makes sense... hadn't thought of that.
Anyways, your project looks really neat! Need to do more reading on it. Exciting!
So for the trade-offs between say what Antmicro has made and what we're doing. They both have advantages and disadvantages.
Antmicro's advantages:
- Works with existing SoMs and hardware designs for the Myriad and also the i.MX 8M.
- So this means a lot of software will work in 'plug in play' mode, increased modularity and a 'just works' factor.
DepthAI advantages:
- Data path is optimized. That's the main
why of this approach.
- Allows stereo and other vision processing hardware in Myriad to be used, also unloads the host a lot. On RPi host it takes CPU from 220% use to 35% use... so allows your 'application code' to have room to breathe. :-)
The disadvantage of doing our approach is it's a TON more work (for us, not you). We have to do a bunch of custom hardware, firmware, software, etc... What this work nets (and what we needed for CommuteGuardian) is a system that's lower power, lower latency, and enables more use of the Myriad X. So it delivers a pretty-efficient Depth+AI platform which can then be embedded in products.
The disadvantage of Antmicro's approach is just cost, latency, power - they're all going to be higher than what we're doing. BUT, it gets you a lot of the capability of what we're doing and less effort into custom firmware, hardware, etc. and is more flexible for immediate prototyping/etc. as all the standard OpenVINO stuff is default supported, etc.
We're optimized to do one thing really efficiently: depth + AI, and specifically real-time object localization - i.e. what objects are in the field of view, where are they in pixel space, and where are they in physical space, and neural inference of those objects.
Here's a good example: Strawberry picking (similar example probably exists in mining)
An from DepthAI from in that application would be meta data of:
- All strawberries in your field of view.
- Each's position in physical space (x, y, and z), and
- A map of each's shape (for easier grasping)
- An estimate of each's ripeness (from output of neural model)
Then the application on top of this can say have simple logic that commands a robotic arm to pick only strawberries above X% ripeness, and then to say sort by ripeness into different bins (to account for ripening upon shipment, and varying ship distances).
And the host that is making that decision is free from all the computer vision tasks... it's just meta data back of strawberry location, ripeness of each strawberry, and size information. So then you could say have 1, 3, or 30 DepthAI hooked to that one host, and that single embedded host would handle it fine.
Whereas when the video has to -flow- through the host, there's no way even an extremely capable system would keep up w/ 30 sets of 3 cameras (90 cameras total). Whereas w/ DepthAI, it's just metadata of locations... a couple KB/s coming out, so 30 sets is doable even on a Raspberry Pi host! And a single camera w/ a NCS2, for reference, running on a Raspberry Pi takes 220% CPU.
So that's a situation where having all the image processing done before the host is important... in not all cases does that matter, but we're building DepthAI to cover the cases where it does matter.
Similarly, with the DepthAI approach, it also allows use w/out a host at all, where you just want some action, actuator, IO, etc. to be taken as a result of the information it's getting. So it allows a much smaller, lower power, and reduced-latency solution - as it's just a Myriad X running the whole show. One example of that is CommuteGuardian (honk at the guy who's on course to run you over) and another example of that is say a Myriad X used as a disaster-recovery system, like here, where it runs the whole show, and just does the direct-command of the craft in the case where other systems have failed.
Also, we'd be more than happy to get you some units to play with. And feel free to reach out to me at brandon at luxonis dot com. If/when we get the unit to play with... bear with us though as we're pretty early on firmware (as above, doing this approach makes it faster, smaller, and lower power - but requires a bunch of work!). We actually just today got the light-weight completely-Myriad-contained neural inference directly from the MIPI camera output working, as below:
So we'll soon be integrated this w/ our already working depth (see here).
Thoughts?
Thanks again!
-Brandon & The Luxonis Team
