• Hardware

  • Development a camera based on oak-som-pro

Hello,

Previously, we used oak-d-iot-75 cameras to do object counting and send the data to our servers.
But we have some problems with this camera and we need to add features.
So we plan to develop a camera based on oak-som-pro. We need to use it in standalone mode.
We will develop a custom PCB to assemble all the components we need. We will add an esp32 which will allow us to recover data from the SOM and then send it through different wireless protocols (wifi, LoRaWAN, 4G).

We are thinking of basing our PCB largely on the oak-ffc-poe-4p whose design is available in your resources.

For communication between the SOM and the ESP32, we are considering either using UART which has the advantage of being easy to set up (there is a script on the SOM side, and on the ESP32 side, this is the protocol "basic" communication), or use Ethernet, which is a little more complicated to set up in terms of hardware and software but which would allow us to do more complex things than with UART. We will perhaps integrate the two at the hardware level and perhaps make a choice during software development or perhaps we will find a complementary use for the two.

We don't need 4 cameras, we are only going to keep 2. We are thinking of using 2 IMX378 cameras (not for stereo, but rather to have a wider field of view).
However, to make our IoT as compact as possible, we want to avoid using complete camera modules like https://shop.luxonis.com/collections/oak-modules/products/oak-ffc-imx378. We would like to integrate the Arducam camera https://www.arducam.com/product/imx378-12mp-color-autofocus-oak/ directly on the PCB. I was able to find in the resources the schematic and the components that you use for the camera module. This doesn't seem very complex to me, do you think we could encounter difficulties at this level?
We would also like to have the ability to have night vision. We are simply thinking of integrating an SFH 4715AS IR LED that you use into your OAK-FFC IR module. As we do not want to measure depth, the laser seems useless to us. Are we wrong?
For the diode, is it really the strobe signal found on the OAK-FFC 4P diagram which manages its activation? Or is there also a digital component that manages its activation (I saw that on the module connector there was I2C for example)?
Can the imx378 camera give good results for night vision with an IR diode, or is it better to use another type of camera (for example monochrome)?


Very important, our cameras are outside, potentially in direct sunlight. Which with previous versions has led several times to overheating of the module and its shutdown. With this new camera, we will try to improve heat dissipation, but it is possible that even during very hot days the module will go into safety mode. We must be able to be sure that we can restart the module automatically using esp32. We are thinking of using one of the GPIOs of the esp32 and connecting it to the _RST pin of the SOM. Does the passage of the latter to the low logic state guarantee us in all circumstances to be able to restart the SOM or should we also plan to be able to cut and restart the power supply of the SOM to restart it?

More generally regarding the design of the card, do you see any particular difficulties that we could face?
I have looked through the diagrams of the modules on which we intend to base it quite a bit, done research on the different components, and apart from the differential pairs which I will have to pay attention to, I do not see any major difficulty.

We plan to delete some components of the OAK-FFC 4P schematic which are not useful for our project. Are there parts which according to you will be essential to allow in particular the debugging and testing of our card (I think for example of JTAG, or indicator LEDs)?



Also where can we know which logic level is used on the different GPIOs? I know that by default the logic level seems to be 1V8, this is the case for UART, _RST for example, but some GPIO also seems to be 3V3.



We do our PCB designs with Kicad. I know that your designs are made with Altium, but by any chance, do you not also have resources available under KiCad (libraries of the components used, SOM footprint, etc.)?

Thank you very much!

Hi @GuillaumeHupi ,

that you use for the camera module. This doesn't seem very complex to me, do you think we could encounter difficulties at this level?

I don't think so. Common issues are just fixing the CCM to the PCB, and not getting glue into the connector.

As we do not want to measure depth, the laser seems useless to us. Are we wrong?

That's correct, laser dot projector is just for active stereo depth.

For the diode, is it really the strobe signal found on the OAK-FFC 4P diagram which manages its activation? Or is there also a digital component that manages its activation

So I2C is used to configure the settings (how much power goes thru), but STROBE will actually enable the output. You can check schematics for dot projector here.

Can the imx378 camera give good results for night vision with an IR diode, or is it better to use another type of camera (for example monochrome)

In general, monochrome cameras have better quantum efficiency (QE), as no light gets absorbed by the bayer filter (which enabled the camera sensor to "see" colors). But our OAK-D-SR-POE also uses color sensors (OV9782) and has dot projector/illumination LED, so it's possible.

We are thinking of using one of the GPIOs of the esp32 and connecting it to the _RST pin of the SOM. Does the passage of the latter to the low logic state guarantee us in all circumstances to be able to restart the SOM or should we also plan to be able to cut and restart the power supply of the SOM to restart it?

I'll get hardware team to elaborate on this.

do you see any particular difficulties that we could face?

Not really, the designs for baseboards aren't that complex, even some high-school electronic students were able to modify it successfully.

(I think for example of JTAG, or indicator LEDs)?



Leds can be useful, JTAG is useful for debugging the firmware, which is closed-source, so that wouldn't be helpful to you.

Also where can we know which logic level is used on the different GPIOs?

Will ask for HW opinion as well.

do you not also have resources available under KiCad (libraries of the components used, SOM footprint, etc.)?

Not entirely, I'm only aware of this folder: luxonis/depthai-hardwaretree/master/OSHW_KiCad_Community

Thanks, Erik

Hi @erik !

Thank you very much for all your answers !

So I2C is used to configure the settings (how much power goes thru), but STROBE will actually enable the output. You can check schematics for dot projector here.

Thank you for the documentation of the dot projector, it will help us.

In general, monochrome cameras have better quantum efficiency (QE), as no light gets absorbed by the bayer filter (which enabled the camera sensor to "see" colors). But our OAK-D-SR-POE also uses color sensors (OV9782) and has dot projector/illumination LED, so it's possible.

So we will first try with color sensor and if result is not good enough, we will maybe think about using monochrome sensor.

Not entirely, I'm only aware of this folder:
luxonis/depthai-hardware
tree/master/OSHW_KiCad_Community

Thank you for the library! I will try to add it to KiCad and see if some components are useful for us.

GPIOs: They are either 1v8 or 3v3, depending on the pin. We have datasheet here: luxonis/depthai-hardware
blob/master/SoMs/OAK-SoM-Pro/OAK-SoM-Pro_Datasheet.pdf

I had already looked at the datasheet, but I had not seen the information. Looking again, I just found the information spread out in different tables. Thanks !

Reset - if nRST line is pulled low, the PMIC on SoM goes into shutdown mode so it cuts the power to the whole RVC2 (Myriad X soc).

OK. Thank you ! This helps us a lot.
To detect the safety shutdown of the RVC2, we plan to use the fact that the latter no longer communicates data to the ESP32 (either in uart or in ethernet depending on what we have chosen). For example, if it no longer responds or if no more data has been sent for X minutes/seconds, then we force it to reset. But the solution is perhaps not 100% optimal (for example during the code upload phase, we may perhaps have problems, etc.). Is there another way to know if the RVC2 is safe? (for example a GPIO which would change state?)

Hi @GuillaumeHupi ,
You could also set some GPIO to high/low inside Script node, and with ESP32 detect whether the line is still high/low. I'd assume this would be faster than waiting for a message from the RVC2.

    erik

    I dont know how it is working for the RVC2, but on microcontroller, even if it freezes, the GPIO remains on the last state you put them even after a freeze. I mean, even if after booting we set the GPIO in high mode, we can't be sure it will go low after a safety shutdown, it could remain high. So I think we can't use it as a sigh of life of the RVC2.

    But maybe we could make the GPIO "blink" and use the esp32 as a watchdog. Or just use a hardware watchdog and do not use the esp32 for this at all.