The ESP32s3-LLM-Cluster project connects seven ESP32-S3 microcontrollers in a daisy‑chain using the SPI bus. One board is designated as the master node and runs the BPE tokenizer and the INT4 embedding lookup, while the other six boards act as compute nodes that each host a contiguous slice of the transformer layers (e.g., nodes 1‑3 handle layers 0‑3, nodes 4‑5 handle layers 4‑11, and node 6 handles layers 20‑23). Hidden state vectors are passed from the master to the first node and then sequentially from node to node until the result returns to the master for the final RMS norm, language‑model head and greedy sampling.
This image depicts the actual SPI daisy‑chain interconnection of the seven ESP32‑S3 modules, matching the article’s description of master node and six compute nodes.
To make the model fit within the limited flash and RAM of each ESP32‑S3, the project applies 1.58‑bit (BitNet) ternary quantization to the weight matrices. Instead of 32‑bit floating‑point values, each weight is restricted to −1, 0, or 1, which can be stored with roughly 1.58 bits per parameter. The repository includes quantization‑aware training scripts (qat_158.py) and shows the use of bitlinear.cpp and bitlinear_forward.S for the ternary linear layers, as well as look‑up tables in lut_table.cpp for efficient multiplication.
While the README demonstrates that the clustered hardware can forward a token through the sliced model, it does not report any performance metrics such as tokens per second, power consumption, or latency, nor does it compare the approach to a single‑node ESP32‑S3 or other microcontroller platforms. The document also mentions both a 0.4 B and a 0.5 B parameter model in different sections, indicating that the specifications are still evolving. Consequently, the practical viability of the cluster for real‑world language‑model inference remains uncertain without further benchmarking.