Support for On-Semi AR0144 camera.

PeterMilani

Hi Luxonis,

I'd like to see what is involved in integrating a AR0144 IAS module (https://www.onsemi.com/download/eval-board-manual/pdf/evbum2742-d.pdf) WFOV or normal FOV (https://www.onsemi.com/design/tools-software/evaluation-board/IAS1MOD-AR0144CSSM080210-GEVB) to be used instead of the OV9282 modules for stereo depth perception. Looking to connect it to a SOM board integration probably RCV3.

I assume there might be some changes to the memory addressing for the I2C lines control, is that something i might be able to provide a PR for?

any guidance would be great.

cheers

Peter

jakaskerl

Hi @PeterMilani
Impossible to do on RVC2 without FW access, probably the same on RVC3, but I don't have enough knowledge to say for certain. Not sure which parts are needed to make it work.
cc @Luxonis-Alex

Thanks,
Jaka

PeterMilani

Okay, as a suggestion, maybe a hardware abstraction layer might help? That way new hardware could be introduced and only requires a config file change to support. I assume most of the functionality is basically the same between cameras, just different addresses and maybe formats.

This way you can keep your firmware closed but still support new/different hardware, and make use of the community to widen the support base.

PeterMilani

@jakaskerl I checked that you guys support the AR0234 which would probably suffice instead (happy for your advice on this). In your OAK-FFC AR0234 product, it notes that there is no strobe or fsync lines available. As we're interested in using for stereo depth, what would be a solution here?

Is there a reference design for the camera hardware to aid in customizing a stereo board?

cheers

Peter

jakaskerl

Hi @PeterMilani
Best to drive the FSIN with a GPIO pin. I think MXIO64 should be PWM, which allows for precise timing.

Thanks,
Jaka

PeterMilani

Thanks @jakaskerl , Checking a couple of your reference designs, what is the STROBE pin for? From some other posts, it mentioned that it was an output, why is that from one of the cameras not used for syncing?

Is the usual plan to just sync all cameras using their FSIN pins via an external gpio signal?

jakaskerl

PeterMilani

PeterMilani what is the STROBE pin for?

STROBE signals image acquisition time. https://docs.luxonis.com/hardware/platform/deploy/frame-sync/#Frame%20capture%20graphs. It is used to trigger IR led on IR capable devices. And I think it's disabled on AR0234.

Example for rvc2 on FFC-4P.

Thanks,
Jaka

PeterMilani

@jakaskerl Thanks, I was looking through the datasheet for the AR0234, and it seems that the I2C addresses can either be 0x20 or 0x21, but in the SOM-Max datasheet, sec 6.2.3 only specifies that the addresses can be 0xC0 or 0xC1 for the OV9282 in stereo camera. What is the mechanism for changing the target I2C address for the cameras, and extending that to generically changing the target addresses?

cheers

Peter

jakaskerl

Hi @PeterMilani
Uh, this is a bit beyond my expertise; cc @Luxonis-Alex

Thanks,
Jaka

PeterMilani

@jakaskerl Thanks for considering it. I just realised that you guys are focusing on a RCV4 development. I was wondering why the RCV3 support was only community based and google to find out why and found RCV4. I assume there will be a SOM version for RCV4? Do you think that would support different camera configurations as discussed above?

cheers

Peter

jakaskerl

PeterMilani
There will be a modular version, but I can't say for sure about the sensors supported. If there is enough demand you can be sure it will be implemented.

Thanks,
Jaka

Luxonis-Alex

@PeterMilani I agree on the idea of a hardware abstraction layer for generic sensor support, we also thought about it, but unfortunately it isn't that easy to implement on RVC2. For the I2C config and list of registers to initialize and start streaming, that would be doable to add, but then the MIPI receiver settings have to be preconfigured as well, and that part is a bit hardcoded now per sensor mode. And some formulas may need to be described for setting the exposure, gain etc. With some sensors these may not be straightforward, not a simple scaling of the raw registers. But if I2C support would be exposed with a Script node, the user might be able to implement own logic.

So at the moment it's only possible for us to add FW support for a new sensor. Once the hardware and datasheet/documentation would be received, it may need up to a week of work, including testing, debugging etc.

Maybe on RVC4 it would be easier to add that generic support, not sure...

PeterMilani

Thanks @Luxonis-Alex

So when it says that a sensor is fully supported, such as the AR0234, and is used for depth in the OAK-D-LR (based on a read of one of the hardware schematics). How can that work be leveraged on a SOM, either RCV2 or 4?

Luxonis-Alex

If no calibration data is written to the device, all firmware supported sensors would be probed on RVC2. If calibration is written, as an optimization to not lengthen the initialization time unnecessarily on our standard OAK devices, it should contain the FFC string in the board name to probe all supported sensors (incl AR0234) on all 4 sockets (CAM_A, B, C, D). A small optimization could be done by having the string as FFC-3P, then probing is done only on the A, B, C sockets.

On RVC4 I'm sure yet on how it would be exposed.

PeterMilani

Does that mean the name in the board json, needs to include the string “FFC”?

What about I2C addresses, do specific cameras roles need specific addresses. It says so in the SOM datasheet, but does this mean the address needs to be changed on the sensor depending on the role the sensor takes on?

Luxonis-Alex

Yes boardName or productName needs to contain the substring FFC, otherwise it may be that only IMX378 and OV9282 are probed (common on most devices).

For the I2C addresses, it depends on the sensor driver, that mention from the SoM datasheet of 0xC0/0xC1 isn't relevant for other sensors. And usually the driver probes for both addresses depending on how SLASEL or SADDR pin is pulled. For instance, AR0234 communicates on either 0x20/0x21 or 0x30/0x31, and both are checked.

PeterMilani

Awesome, that answers my question. Thanks Alex.

Just another question regarding the AR0234 chip, which pin is used for fsin and which are used for strobe? These aren’t the signal names used in the datasheet. And just confirming for this camera the Fsin has to be from an external source?

Luxonis-Alex

Yes, FSIN (Frame Sync INput) is usually the name on Omnivision sensors. With AR0234 that's connected to the TRIGGER pin (input only). The signal can be generated by the RVC2 SoC (Script node, any GPIO with high/low toggle, or PWM on specific GPIOs that have this function), or can be external.

For STROBE, the A0234 FLASH pin is used. Currently it's not enabled by firmware, we should expose it

PeterMilani

Thanks for clarifying @Luxonis-Alex