Hi. I'm greatly interested in DepthAI USB3 FFC Edition.
And I'm also interested in both the color camera and the shutter pair cameras.

I intend to use these for an open source, DIY 3D printed pair of eyeglasses that I've been developing for the blind.
I've been completely blind myself for the past 17 years.
Here's a video of my preliminary prototype in action, though it's qquite overtly bulky at the moment:

Anyway, I'd greatly appreciate some help regarding my questions here:

  1. Would it be possible for the lens of the color camera and the shutter pair cameras to be separate from its board?
    • Currently, these MIPI CSI-2 cameras have the same dimensions as RPi Camera v2.1 (25x25x9mm) ;
    • This means the lens of these cameras are attached to a 25x25mm board ; and
    • It'd be really helpful if there are versions of these cameras where the lens is separate from the 25x25mm board via a flex PBC, something similar to this USB camera here, though with a 2x or so longer flex PBC >> http://www.camera-module.com/product/8mpcameramodule/8mp-usb-camera-module-sony-imx179-auto-fixed-focus.html

  2. As a follow-up to my first question — Are there other MIPI CSI-2 cameras (color camera and shutter pair cameras) that are designed this way, which would most likely work with DepthAI USB3 FFC Edition?
    • I can't seem to find any.

  3. And lastly, what are the dimensions (length x width x thickness) of the DepthAI USB3 FFC Edition?
    • I can't seem to find this anywhere.

I'd greatly appreciate some assistance. Thanks in advance! 🙂

So this thread ended up in email as well (where we've gone back/forth some). Cross-posting from email to here:

Hi. We're building a pair of 3D printed eyeglasses for the blind (I'm
completely blind myself).
And we're interested in the DepthAI USB3 FFC Edition.
Plus, we want to use it along with both your 12MP color camera and the
global-shutter stereo pair.
But we have the following questions:

  1. What are the dimensions (length x width x thickness) of the entire board?
  2. All cameras (color camera and shutter pair) each has a similar dimension to
    the RPi Camera V2.1, correct? (25 x 25 x 9 mm)
  3. So this means the lens of each camera is attached on top of a 25x25x9mm
    board, correct?
  4. If yes, then would it be possible to have the lens of each MIPI CSI-2
    camera separate from its board through flex PBC? Something similar to this USB
    camera, though with a 2x or so longer flex PBC? >>
    http://www.camera-module.com/product/8mpcameramodule/8mp-usb-camera-module-sony-imx179-auto-fixed-focus.html
    Our purpose in asking is that we want to integrate the cameras into the front
    frame of a 3D printed pair of eyeglasses.
    We want to position the lens of the left and right cameras of the shutter pair
    either at the ends of the front frame, or at the start of the left and right
    arms of the glasses.
    Meanwhile, we want to position the lens of the color camera at the center of
    the front frame (the area of the user's nose bridge).
    Please advise. Thanks! 🙂
  1. So the BW1098FFC kit is composed of 4 boards total to allow modular installation/prototyping like this:
    a. The Myriad X central board. This is the BW1098FFC itself. It is 60mm x 40mm, with mounting holes spaced at 34mm x 54mm. The hardware design for this is here.
    b. The Color camera board. This is the BG0249. It is 25mm x 21.86mm. The board dimensions, hole pattern and sizes, and center of the image sensor are identical to the Raspberry Pi v2 dimensions, here. We used a different FFC though as the camera/control is very different. The hardware design for this is here.
    c. The grayscale camera boards. Two of them for stereo depth. These are the BG0250TG. Also 25mm x 21.86mm. The board dimensions, hole pattern and sizes, and center of the image sensor are identical to the Raspberry Pi v2 dimensions, here. We used a different FFC though as the camera/control is very different. The hardware design for this is here.
  2. Yes, that's right. The dimensions of the Raspberry Pi camera are here. Our thickness is different though because we use different camera modules. The 3D STEP files for our cameras are available in the links above. And all our hardware is here.
  3. The Raspberry Pi dimension document (here) specifies where the center of the images sensor is located. So from that, the center of the lens is 9.462mm from the edge of the board with the FFC connector and 12.5mm from the other side.
  4. This may be possible, but we are very constrained based on supply chain permutations. So right now we are using a camera module on these modular boards that has a 0.75 inch long FFC. There may be others with longer FFC, but we are unaware of them. The easiest thing for a prototype would be to use our modular boards above with our FFC. It's wider than what you show by probably 2.5x (so about 1/2" wide). But for a prototype should be doable. The frames of the glasses could just be thick to accommodate. Since we are open source, one could take our design and change it to use a narrower FFC as well. And if that's of interested, we are happy to help facilitate. There is the blind institute of technology close to us in Denver. We would be happy to work with you and them to help make this happen. Still it would be better to prototype initially with what we have now, even though the glasses would be bigger than they could be after such optimizations.

So what you mention you want to do is doable now with the BW1098FFC kit. As the cameras are modular and come with a 6" flexible flat cable each. So it might look ugly to start... but it should make a viable proof of concept.

And we can help make custom versions of these boards to make this fit better in glasses. What would likely be best is a single PCB that holds all 3 cameras on the front of the glasses. So that way not cables to wrangle.

Thoughts?

Thanks,
Brandon

I like the back-of-neck idea for processing to start with. In terms of keeping costs down, I would say the B1098FFC kit itself is what should be used. https://shop.luxonis.com/products/depthai-usb3-edition Longer FFC cables can be used up to 12 inches... just let us know if you plan on ordering that and we'll order the extra-long FFC (12" instead of 6" cables) and included them in the box. With those I think nape of neck would be supported.

I'm going to introduce you to blind institute of technology now. Maybe they can help with funding.

Hi Brandon. Good morning! 🙂

This is just simply amazing — Thanks for supporting the project! 🙂
• Hopefully, we'd be able to get additional support for the custom
PCB from the Blind Institute of Technology and others ...

Yes, DepthAI USB3 Onboard Camera Edition sounds promising! 🙂
• 7.5cm spacing for the grayscale cameras, and the color camera in
the middle, look just about right ; and
• My blind friends and I also discovered through field tests that
cameras equipped into eyeglasses need to be tilted downwards at around
12 to 15° ...
— This tends to correctly simulate what people with eyesight see (and
what they naturally focus on) in front of them ; and
— Not doing this though tends to capture a lot of things above the
normal field of view ...

Oh also, are the cameras of BW1098OBC embedded unto the board?
Or does it have long FFC connectors running to the board's MIPI CSI-2 ports?
• I'm just thinking if it does have long FFC connectors, then the
board itself (and the heatsink) can hang somewhere at the user's nape.
And this in turn can be connected via a long USB cable to a
battery-powered host, which can be say clipped unto the user's belt /
waist, or inside the user's pocket ...

Eitherway, I'm really energized to be able to significantly enhance
the product with DepthAI! 🙂
• Right now, I'm using a single color camera and an ultrasonic sonar
ranger ; and
• Unfortunately, this only detects distances of objects right in
front of the user, as the sonar sensor sits below the color camera in
the middle of the glasses ...
— This also forced me to do a workaround in OpenCV where 2D X and Y
coordinates of detected objects are used to tell the user where the
object is in 6 quadrants (3 upper quadrants [left, center and right],
and 3 lower quadrants [left, center and right]) ; and
— Another thing I found interesting though is that blind users (well,
me and almost all of my blind peers in the communities I've been
participating in) prefer clockface directions when being told the
location of objects, i.e. 10 o'clock for objects in upper left
quadrant, 2 o'clock for upper right quadrant, etc. (we're also
actually trained this way during mobility rehab after I lost my
eyesight) ...

I really do hope we can continue working together on this. 🙂
Thanks again! This is really awesome! 🙂

Very good to know on the 12 to 15 degree down-tilt!

The cameras on the BW1098OBC are embedded directly on it. The OBC actually stands for 'On Board Cameras'.

And that's good to know as well for clockface directions. That makes a ton of sense! I think in tactical situations that is the preferred terminology as well. It's fast/simple and easy to understand.

Thanks again!

Hi again Brandon!

As a working proof of concept for this, I was just thinking about the complete DepthAI edition with the RPi Compute Module 3+ and 3 cameras ...

• Are the dimensions of these the same as the RPi Compute Module 3+ (671x310mm)?

I can perhaps do the following things:

  1. Connect a LiPoly battery and 3D print a case with something to attach it to the front neckline of a shirt ;
  2. Connect Bluetooth earbuds ; and
  3. Code something up to deliver audible info to the blind.

Thoughts? Thanks for your time! Greatly appreciate it. 🙂


Hi @MarxMelencio ,

Very happy to help. Really want to make this happen! I pinged some other to see if they'd be interested in helping on this as well.

So the dimensions of the Raspberry Pi Compute Module Edition of DepthAI (Part Number: BW1097) are 106mm x 60mm (and 37mm tall, from the tallest component on the bottom to the tallest on the top, both of which are heatsinks). So it holds on the back of it a RPi Compute Module 3+ which has the dimensions you mention.

  1. Yes. This is a great idea as then the whole computer is integrated.
  2. So this model does come with WiFi but not bluetooth. I'm going to look if there is a USB bluetooth that you cold plug in to give bluetooth audio. Any one that works with the Raspberry Pi should work.
  3. Yes! Would be so awesome.

Overall I think this is a great idea for getting off the ground. So actually a GoPro chest-strap would be awesome for holding it. That way it doesn't bob up/down when walking.

We bought this and although we haven't tested the chest strap, it looks great - and the whole kit is only $25.99 with good reviews. So Martin Nossel made a case for the DepthAI Compute Module Edition that could be used and make a GoPro mount for the back of it. Right now it is designed to mount to a seat post.

https://docs.google.com/document/d/1Ds81P4aJZK22VdSS5-GB5c5KEfIcZAO6Q5mYQqgXFFc/edit

I just asked a mechanical engineer friend if he would be able to help make this.

Thoughts?

Thanks,
Brandon

@Brandon,

Sounds really cool! 🙂

  1. Yes. Also, I don't think the device bobbing up and down while walking would be a show stopper. Though of course, it would be fantastic to at least significantly minimize this from happening ...

• That's because when my blind friends and I tested my current implementation, we found out that we mostly need options to be on-demand ;
• So I used a single Bluetooth button and coded something that can allow the user to select an option on demand, i.e. Detect objects / persons, detect central distance (using the sonar sensor), read text, go down each line of recognized text, request for manual visual assistance through Skype / Facebook, play multimedia like an audio book or so, etc. ; and
• So this time, I'm hinking of connecting to the device itself a tiny programmable USB or GPIO button, in order for the user to select / deactivate an on-demand option ; and
• But a GoPro chest strap, or something at the back of the DepthAI Compute Module Edition that can securely fasten it to the neckline of a user's shirt, would be awesome ...

  1. Yes, connecting a tiny USB Bluetooth dongle is a good solution.

  2. Yes. Audible output is quite straightforward through the Pico TTS library in Python (there are other libraries, though this is the one that my blind peers and I really like) ...

Overall, a case to hold the shutter camera pair (left and right), the center color camera, the DepthAI board, the RPi Compute Module 3+, a rechargeable LiPoly battery, the USB Bluetooth dongle, a tiny programmable USB / GPIO button / switch, and clips at the back to attach it to the neckline of a user's shirt would be really cool! 🙂

All sounds great. I think this is a good plan and agreed with what you propose for the prototype. Will see if someone would be interested in making this case.

On the Bluetooth adapter for Pi, this one on Amazon says it works with the Pi, here

Also have you read about Unfolding Space by Jakob Kilian? It uses a depth sensor and a glove to provide feedback:
https://www.designboom.com/technology/jakob-kilian-unfolding-space-see-with-your-hands-09-14-2019/

Hi @Brandon,

Yeah, that thing in Amazon looks good. 🙂
• Yes. I think that's small enough. But still efficient ; and
• I have a Logitech K375 Bluetooth keyboard. And it came with a USB dongle. I think just about the same size (can't find it now, though I remember) ...

And yes. Though I haven't used Unfolding Space yet. But I think it's really cool! 🙂
• I really think haptic feedback in this use case should be something explored more ; and
• I've experimented a little about this, i.e. Coordinating Peter Meijer'S vOICe (also known as SeeingWithSound) through distance data of central objects from a Devantech SRF10 ultrasonic ranger and an ERM vibrating disk connected to a GPIO microcontroller called DRV2605L from Texas Instruments, running in an RPi Zero, and also in an X86 SBC called Liva Q ...


Thanks. All looks good to me. :-) One thing we haven't made run yet is OCR. I think it's doable we just haven't done it yet. The default OpenVINI model is humungous for some reason, so will need some looking/iterating.

Thanks andredavidbrown ! Really like your work there. Super fun! Definitely we're excited to help out on all of this. So the System on Module (here) allows integrating the depth and AI capability into a custom formfactor easily.

And all the baseboards for it are opensource so are a great starting point for doing a custom formfactor:
https://github.com/luxonis/depthai-hardware

    Hi @andredavidbrown,

    Thanks so much for helping out! 🙂
    • I along with my blind friends greatly, and sincerely, appreciate all the valuable help we can get to hit the ground running with this project!

    Amazing work on WAAIV! 🙂
    • The OCR and gesture recognition stuff here is sure to help blind users like myself, along with my deaf friends ; and
    • I also hope you're interested in sharing your pre-trained models for these functions. 🙂

    Yes please, kindly help us with the prototype case discussed here; and
    I do hope you're also interested in supporting the concept custom PCB for the frontal frame of a 3D printed / plastic-molded eyeglasses. 🙂

    Really exciting stuff here! 🙂

    @MarxMelencio ,

    We've made a mount system for the BW1097 (Raspberry Pi Compute Module Edition). It's on our Github now and open-source:
    https://github.com/luxonis/depthai-hardware/tree/master/BW1097_DepthAI_Compute_Module/Mechanical


    So it mounts to any GoPro adapter so you allows you to mount the BW1097 to any GoPro compatible mounting system. We like this kit here as it allows mounting to pretty much everything anyone could think of and it’s only $25.99.

    • Need to mount your BW1097 to a Tripod? Done.
    • Need to head mount your BW1097? Done.
    • Need to put your BW1097 on a telescoping pole? Done.
    • Need to chest-mount your BW1097? Done.
    • Need to suction-cup your BW1097 to a window? Done.
    • Need to wrist-mount your BW1097? Done.

    Hi @Brandon,

    Awesome stuff! 🙂 Thanks so much for this!
    • Would I be able to order the mount along with the BW1097?
    • I'd like to start coding and testing it! 🙂
    • If yes, then how much should I prepare? I think the BW1097 is $399 plus $25 or so for shipping here to the Philippines, but how much for the mount, along with other components you might've added like the Bluetooth dongle and a LiPoly battery, if any? My purpose in asking is that I have to write a bunch of articles first for my client, so I can prepare it. 😃

    Also, I'll get from Amazon in 10 days or so the GoPro accessory that you mentioned, once I order it.

    Please advise. Thanks again! 🙂


    Hi @MarxMelencio ,

    Thanks. Yes, definitely. So you can buy the BW1097 here for $399 and the mount for it here for $19.99 here.

    So also I just got off the phone with Intel about sponsoring a competition around good uses of this technology (and your use is a fantastic one... I at least can't think of a better one). So Intel are planning on sponsoring a competition where the top 15 best ideas submitted get a free development kit. Then there will be a period of at least a month for implementation, and from those 15 selected, those who make the most compelling thing will also win a cash prize.

    So I don't know when this will launch... but this could be a fantastic way to not have to save up to buy it, but would require waiting a little bit.

    Thoughts?

    Thanks,
    Brandon

    Thanks so much, @Brandon!

    Yes, the Intel competition sounds really cool! 🙂
    • But I think in a week or less [ I write content quite fast 🙂 ], I'll be able to raise the money for the BW1097, the mount, the GoPro accessory and the Bluetooth USB dongle (or its equivalent since Amazon says the vendor doesn't deliver the dongle to the Philippines).
    • Which I think would be good, since this gives me time to be well-prepared for the competition. 🙂

    Oh also, I can't find the link in your post regarding the battery that you like. 🙂

    Cheers! 🙂

      Awesome, thanks MarxMelencio !

      So yes since we're in the US probably some of the products we find are not source-able local to you. The key for the USB battery is that it needs to support 2A output. The Raspberry Pi + DepthAI together regularly pulls above 1A, so that's why the 2A output is needed (and these batteries either do 1 or 2, usually nothing in-between).

      So in this video you can here me describe what power the unit is using when running depthai with object detection:
      [https://photos.google.com/photo/AF1QipOCD1QiztgtxhBhVp3NurVXzTVW_RSHFh8BV215](https://photos.google.com/photo/AF1QipOCD1QiztgtxhBhVp3NurVXzTVW_RSHFh8BV215

        Hi @Brandon. Good morning! 🙂

        Regarding the Bluetooth USB dongle — I found in Amazon this tiny Bluetooth USB dongle. It ships to the Philippines. And supports Raspberry Pi. 🙂

        And as for battery — Here are promising options:

        1. I already have this PiJuice hat. I've tested this to work with RPi Zero W / WH and RPi 3 Model B+. So I believe it likely works with the RPi CM3+.
          • And what's good about this is it has an 1860 mAh rechargeable battery and outputs 2a. Plus, it allows coding something for warning the user when battery level drops to a configurable percentage. And to shutdown the RPi safely when it drops to an even lower percentage ; and
          • However, what I don't like about this is its price ($90 here in the Philippines). This could be restricting for some blind users. And I'm not quite fond of its footprint (a little on the chunkier side) ...

        2. I'll also be testing the PiSugar units below. I've already contacted the manufacturer to add the Philippines to their shipping destinations. Both have a much smaller footprint. The latest variant has a 5000 mAh battery (up to 6a output). While earlier variant's have 900 and 1200 mAh options, both up to 2.4a. And both variants also provide programmatic access to battery levels. The latest variant even comes with a tiny programmable button, which I think can be used to also provide user access to non-battery-related functions, i.e. Can likely be programmed for recognition options.
          • PiSugar2 Pro — Latest variant. Works with RPi 3B+. So I think this'll likely work with RPi CM3+. But it's currently out of stock, though hopefully they'll have units soon ; and
          • PiSugar2 — Earlier variant. Works with RPi Zero W / WH. So I think it might work with RPi CM3+. This variant is currently available ...

        3. Another option is this battery hat from Waveshare. I like its small footprint. And it has a battery holder for 14500 Li-ion. But I still need to check if it outputs 2a (though I believe it does).

        Really exciting stuff here! 🙂

        Hi @MarxMelencio ,

        So let me get back to you with more details on Monday, but want to mention that HATs don't work on the BW1097 because the Myriad X module (the DepthAI System on Module) has a tall heatsink that gets in the way.

        I think there are like extenders that would allow this to still work, but it would have to be very tall. So for now it might make the most sense to use a USB battery that could say go in your pocked and then connect up to power the system.

        Later I think we could make a mount that includes the battery as well, once we down-select one and have that shipped out. I think a mechanical engineer friend is willing to work on this for free.

        The PiJuice version looks interesting. (Correction, sorry - referenced the wrong one.)

        Thanks again and will circle back more on Monday.

        -Brandon