We have an initial implementation for the DepthAI SPI communication (streaming of NN metadata for now). The SPI protocol that Martin described above is not integrated yet, for now the packet consists of a NULL-terminated string.
The DepthAI boot and initialization is still done by USB, and afterwards an SPI controller can pull the data off SPI. Tested with ESP32:
https://github.com/luxonis/depthai/compare/spi_esp32
https://github.com/luxonis/depthai-python/compare/master...spi_esp32
6:19
The ESP32 app is based on this example: https://github.com/espressif/esp-idf/tree/v4.0.1/examples/peripherals/spi_slave/sender
with these modifications: https://github.com/luxonis/depthai-python/commit/619d485
As DepthAI streams the NN metadata on its own (when the controller initiates communication), to test that the COPI (MOSI) line works well, the ESP32 app sends a NULL-terminated string like esp32-cnt:253 that is echoed back by DepthAI, together with the NN metadata. After DepthAI boots up (by running ./depthai.py), the prints in the ESP32 app should look like:
I (0) cpu_start: Starting scheduler on APP CPU.
I (305) gpio: GPIO[2]| InputEn: 1| OutputEn: 0| OpenDrain: 0| Pullup: 1| Pulldown: 0| Intr:1
[RECV-0]
[RECV-1]
[RECV-2]
[RECV-3]
[RECV-4]
...
[RECV-106]
[RECV-107]
[RECV-108]
[RECV-109]
[RECV-110]
[RECV-111]
[RECV-112] | DepthAI pkt 0:
[RECV-113] esp32-cnt:108 | DepthAI pkt 1:
[RECV-114] esp32-cnt:109 | DepthAI pkt 2:
[RECV-115] esp32-cnt:110 | DepthAI pkt 3:
[RECV-116] esp32-cnt:111 | DepthAI pkt 4:
[RECV-117] esp32-cnt:112 | DepthAI pkt 5:
[RECV-118] esp32-cnt:113 | DepthAI pkt 6:
[RECV-119] esp32-cnt:114 | DepthAI pkt 7:
[RECV-120] esp32-cnt:115 | DepthAI pkt 8:
[RECV-121] esp32-cnt:116 | DepthAI pkt 9:
[RECV-122] esp32-cnt:117 | DepthAI pkt 10:
[RECV-123] esp32-cnt:118 | DepthAI pkt 11:
[RECV-124] esp32-cnt:119 | DepthAI pkt 12:
[RECV-125] esp32-cnt:120 | DepthAI pkt 13:
[RECV-126] esp32-cnt:121 | DepthAI pkt 14:
[RECV-127] esp32-cnt:122 | DepthAI pkt 15:
[RECV-128] esp32-cnt:123 | DepthAI pkt 16:
[RECV-129] esp32-cnt:124 | DepthAI pkt 17:
[RECV-130] esp32-cnt:125 | DepthAI pkt 18:
[RECV-131] esp32-cnt:126 | DepthAI pkt 19:
[RECV-132] esp32-cnt:127 | DepthAI pkt 20:
[RECV-133] esp32-cnt:128 | DepthAI pkt 21:
[RECV-134] esp32-cnt:129 | DepthAI pkt 22:
[RECV-135] esp32-cnt:130 | DepthAI pkt 23:
[RECV-136] esp32-cnt:131 | DepthAI pkt 24:
[RECV-137] esp32-cnt:132 | DepthAI pkt 25:
[RECV-138] esp32-cnt:133 | DepthAI pkt 26:
[RECV-139] esp32-cnt:134 | DepthAI pkt 27:
[RECV-140] esp32-cnt:135 | DepthAI pkt 28:
[RECV-141] esp32-cnt:136 | DepthAI pkt 29:
[RECV-142] esp32-cnt:137 | DepthAI pkt 30:
[RECV-143] esp32-cnt:138 | DepthAI pkt 31:
[RECV-144] esp32-cnt:139 | DepthAI pkt 32:
[RECV-145] esp32-cnt:140 | DepthAI pkt 33:
Det0: tvmonitor 57.67% (-0.01,0.02)->(1.00,1.00)
[RECV-146] esp32-cnt:141 | DepthAI pkt 34:
[RECV-147] esp32-cnt:142 | DepthAI pkt 35:
[RECV-148] esp32-cnt:143 | DepthAI pkt 36:
[RECV-149] esp32-cnt:144 | DepthAI pkt 37:
Det0: person 78.27% (0.23,0.23)->(0.41,0.47)
[RECV-150] esp32-cnt:145 | DepthAI pkt 38:
Det0: person 83.30% (0.22,0.22)->(0.41,0.47)
[RECV-151] esp32-cnt:146 | DepthAI pkt 39:
Det0: person 92.43% (0.21,0.23)->(0.41,0.47)
[RECV-152] esp32-cnt:147 | DepthAI pkt 40:
Det0: person 95.61% (0.20,0.22)->(0.41,0.48)
Det1: person 62.06% (0.04,0.26)->(0.23,0.48)
[RECV-153] esp32-cnt:148 | DepthAI pkt 41:
An initial version of the USB flasher/bootloader is implemented: branch spi_esp32, these commits:
https://github.com/luxonis/depthai/compare/127ff77...c22bac6
Initial flashing procedure:
(during these steps it's handy to have a dmesg -wH terminal running, to check device state)
- Boot the 'loader.cmd' firmware over USB:
./depthai.py -boot loader.cmd. A new DFU device should appear: 03e7:f003.
- Download the 'loader.cmd' firmware, in order to be flashed:
dfu-util -D loader.cmd . At the end of the process, the DFU device should be re-enumerated by the host.
- At this point it is possible to switch the board to Flash boot mode, recommended boot mode 0x8 (Quad-SPI 50MHz) on DIP switches:
001000
123456
- Download the 'depthai.cmd' firmware, in order to be flashed:
dfu-util -D depthai.cmd. Note that the flashing process takes longer and is not finished by the time dfu-util exits (this will be fixed). At the end of the process, depthai should boot and create the usual 03e7:f63b device. The device will reboot every few seconds (because of the watchdog), until the USB host connects: ./depthai.py -boot '' -- this specifies that no firmware has to be loaded.
Special pin functions:
- header SPI2.5 (GPIO35_3V3) asserted low, followed by a hard-reset: the device enters in DFU mode, allowing to update the application (or the bootloader).
- header SPI2.6 (TAO_WAKE/GPIO34_3V3) asserted low, followed by a hard-reset: the device switches to USB-boot mode, allowing to run for example
./depthai.py. The device stays in USB-boot mode during soft-resets (depthai being closed - reopened), until it is hard reset.
TODO (WIP): add options to bootloader to write application sections to Flash (config, network blob), and modify depthai.cmd to use them, allowing no-USB operation. This will require switching the Flash part from 24-bit to 32-bit addressing, to access data past 16 MiB boundary. A non-volatile Flash config of 32-bit addressing would be preferred, to allow boot after a hard-reset without power-cycle.
