Hi, We are considering AI project like below drawing. before starting we hope to have your help whether our idea is available or not. [URL=https://i.imgur.com/08O1UXN.jpg][IMG]https://i.imgur.com/08O1UXNh.jpg[/IMG][/URL] And our application is kind of car black box for rear side especially big trucks. and it detect people(pedestrian), walls or another cars with distance(depth). But we hope to support 4 channels at the same time. Here is our question to cross-check with your help. 1. We hope to use our custom camera. is it possible? how? and which interfaces are possible for us?? please guide us about it. Because we are camera company. 2. We hope to interface with DVR with 4 channels individually at the same time. which interface options are available?? 3. Can we operate 4 channels at the same time with pedestrian, walls or another cars detection with 30 fps with one model? or??. 4. We would expect our temperature expectation like below. is it ok with customized Opecv Ai kit?? Standing temperature (-40 ~ 85) Operating temperature (-30 ~ 70) Thanks.

Hi Ryan, 1. The camera interfaces we have exposed on our SoM are all MIPI (CSI-2 or DSI), with separate I2C channels for configuration. Up to 4 cameras can be connected with OAK-SoM-Pro: 2 of them with a maximum of 4 lanes, the other 2 with a maximum of 2 lanes. What cameras do you have? As we have a limited list of supported camera sensors, if they're not there, would need writing a custom driver by us. Also if the interface is not MIPI, it may work with a converter to MIPI, but likely also need custom firmware support. Also what resolutions and FPS you have in mind? 2. At the moment we have USB3 and Ethernet outputs possible. Ethernet requires the Realtek 8111 PCIe NIC to be placed on the baseboard. The system performance and throughput with USB3 is better though. 3. This sounds a bit much to process, it's dependent on the selected models. For more info, I'm letting our colleagues reply. Or may need more than 1 SoM per device to handle all processing. We're also working on next versions of OAK, that would feature the Intel Keem Bay VPU, more powerful than Myriad X at NN processing. But could be few more months until ready (both hardware and software). 4. We have the temperature ranges listed in the datasheet here: https://github.com/luxonis/depthai-hardware/tree/master/SoMs/OAK-SoM-Pro . For storage, -30 .. 150 °C is listed, while for operation a maximum of 50 °C is recommended, but higher is possible, provided the heat sink is good. The maximum operation junction temperature for Myriad X is 105 °C.

Hello Ryan, sorry for the delay. Our cameras (VPU itself) already has ISP there and we currently don't support any kind of "bridge" (that would translate to MIPI/CSI) - we only support [these camera sensors](https://docs.luxonis.com/projects/hardware/en/latest/pages/articles/supported_sensors.html). A possibility would be to stream the images through USB/ETH or SPI to the VPU - not perfect solution, but possible. The reason behind `why` we don't support bridges (yet) is that it's really not what we focus on - we create Spatial AI/CV cameras, not AI accelerators, which would be more suited in this application. Thoughts? Thanks, Erik

DethAI

RyanLee

Hi,

We are considering AI project like below drawing. before starting we hope to have your help whether our idea is available or not.

And our application is kind of car black box for rear side especially big trucks. and it detect people(pedestrian), walls or another cars with distance(depth). But we hope to support 4 channels at the same time.

Here is our question to cross-check with your help.

We hope to use our custom camera. is it possible? how? and which interfaces are possible for us?? please guide us about it. Because we are camera company.
We hope to interface with DVR with 4 channels individually at the same time. which interface options are available??
Can we operate 4 channels at the same time with pedestrian, walls or another cars detection with 30 fps with one model? or??.
We would expect our temperature expectation like below. is it ok with customized Opecv Ai kit??
Standing temperature (-40 ~ 85)
Operating temperature (-30 ~ 70)

Thanks.

erik

Thanks Alex for replying! ! RyanLee For number 3, Running object detection (eg. tiny yolo or mobilenet) can reach about almost 30FPS, but only on 1 stream. I don't think it would be possible to run 4x object detection on the Myriad X in real-time. I would suggest either using multiple Myriad X (so multiple SOMs) or to wait for the Keembay, next version of Myriad X that should have a significant AI performance improvement (eg. 5x).

Luxonis-Alex

Hi Ryan,

The camera interfaces we have exposed on our SoM are all MIPI (CSI-2 or DSI), with separate I2C channels for configuration. Up to 4 cameras can be connected with OAK-SoM-Pro: 2 of them with a maximum of 4 lanes, the other 2 with a maximum of 2 lanes. What cameras do you have? As we have a limited list of supported camera sensors, if they're not there, would need writing a custom driver by us. Also if the interface is not MIPI, it may work with a converter to MIPI, but likely also need custom firmware support. Also what resolutions and FPS you have in mind?
At the moment we have USB3 and Ethernet outputs possible. Ethernet requires the Realtek 8111 PCIe NIC to be placed on the baseboard. The system performance and throughput with USB3 is better though.
This sounds a bit much to process, it's dependent on the selected models. For more info, I'm letting our colleagues reply. Or may need more than 1 SoM per device to handle all processing. We're also working on next versions of OAK, that would feature the Intel Keem Bay VPU, more powerful than Myriad X at NN processing. But could be few more months until ready (both hardware and software).
We have the temperature ranges listed in the datasheet here: https://github.com/luxonis/depthai-hardware/tree/master/SoMs/OAK-SoM-Pro . For storage, -30 .. 150 °C is listed, while for operation a maximum of 50 °C is recommended, but higher is possible, provided the heat sink is good. The maximum operation junction temperature for Myriad X is 105 °C.

RyanLee

@Luxonis-Alex I will check the camera which is our customer use soon. and they would expect 1080p/30fps. But i think that we need to consider the point of trade-off.

@erik Yes, i understand that your new model with Keembay would be 5 times faster than Myriad X. So It would cover 4channel AI inferences in real time. ok. let us go back to Myriad X again. How about if you would agree with 10 fps each channel (total 4 channel) with pedestrian, walls and car detection model, how many Myriad X needed?? And one more question, if you would agree with 15 fps each channel with pedestrian, walls and car detection model, how many Myriad X needed for it??

Thanks again and if i misunderstand with anything, please correct me. that will help me a lot.
Best Regards,.

RyanLee

I have one more question. As i told you our target is inferencing each channel individually with the model. In this design, is it possible to recognize depth information for each channel?? i know like, oak-d or oak-d-lite can be detected the depth information from mono camera. So I am wondering whether it is possible to detect depth information with one RGB camera like oak-1 or not.

Thanks.

erik

Hello RyanLee ,
it would depend on the object detection that you would use, but if you were to use tiny-yolo-v4/mobilenet, you can expect to get _30FPS on the Myriad X, so for 4 streams you would need 2 Myriad X / SOMs / Cameras to get 15FPS on all streams. If you were to use only rgb camera, you can estimate depth with AI model, but they are quite large, see demo here. They also don't provide actual depth, just an estimation relative to other objects.
Thanks, Erik

RyanLee

Ok i understand so far. So I have the next questions.

Eventually we hope to use or connect our camera to OAK-SoM. So could you help us to cross-check its possibility?
Again to integrate our camera with OAK-SoM. We made two step like below.

First step: We would like to test and verify its possibility with one of OpenCV AI Kit before manufacturing. Could you suggest one model of OpenCV AI Kit for this purpose? below i added some more details please check it out. And if you need more information for it, please let us know. i will prepare as soon as possible.
Second step: if the first step is ok, then we will make our own board with your OAK-SoM. Please check its possibility out as well. and if you have anymore good idea for it. please let us know.

Again we hope to make 4channel AI inference device with our own camera. And it will be working with pedestrian, car and obstacle detection at the same time.

Here is the block diagrams and datasheets of our camera which will be connected with OpenCV AI Kit (At first as a design verification) and then OAK-SoM (the second for making our own product).

<datasheet for Camera Sensor and Specification>
https://drive.google.com/drive/folders/1utNOl2rIAm7Pl4cM_n09uKKwS2UZvVJ-?usp=sharing

Best regards,
Ryan.

erik

Hello Ryan,
sorry for the delay. Our cameras (VPU itself) already has ISP there and we currently don't support any kind of "bridge" (that would translate to MIPI/CSI) - we only support these camera sensors. A possibility would be to stream the images through USB/ETH or SPI to the VPU - not perfect solution, but possible.
The reason behind why we don't support bridges (yet) is that it's really not what we focus on - we create Spatial AI/CV cameras, not AI accelerators, which would be more suited in this application.
Thoughts?
Thanks, Erik