Three years ago, Eileen Yoon began reverse-engineering the Apple Neural Engine (ANE) — the dedicated machine-learning accelerator inside Apple's M-series chips — and published a Linux driver. She paused, convinced the block was too specialized to be generally useful. Now she has returned, not to make the ANE do new workloads, but to map exactly how its internal architecture reveals what Apple bet on when it first committed neural acceleration to silicon in the A11 Bionic (2017).

The central finding is that the ANE was designed for convolutional neural networks (CNNs), the image-processing architecture that dominated pre-2018 ML, not for the transformer models that power today's LLMs. A CNN's dense, predictable data reuse maps neatly onto the ANE's dataflow; a transformer's autoregressive decode does not. Apple's own response, Yoon argues, was to fold the ANE cores inside GPU clusters in the M5 (2025) — effectively retiring the standalone NPU rather than adapting it.

Each of the M1 ANE's 16 compute cores contains 128 parallel multiply-accumulate (MAC) lanes — 2048 across the chip — that perform the recurrence s ← s + a × b. The MAC is a scalar reduction unit: it knows nothing about which dimension of a tensor it is reducing; that meaning comes from how operands are scheduled onto the core. The accumulator is a 32-bit fixed-point register in Q16.16 format, saturating at 2¹⁵, with the product read out as FP16. This choice — 16-bit multiply, 32-bit accumulate, 16-bit readout — is a direct consequence of the CNN workloads Apple targeted: bounded activations, predictable range, fixed-point friendly.

For nonlinearities, the ANE does not compute tanh in hardware. It uses a 33-entry piecewise-linear lookup table with linear interpolation, spacing knots at intervals of 1/8. A ReLU is mode 1 (identity), tanh is mode 2 (the LUT). CoreML's compiler folds bias and activation into the same post-MAC path at compile time — the bias register and the nonlinear mode are set per-layer, and the MAC output feeds directly into the lookup without an intermediate memory round-trip.

Diagram showing Core ML folding constant scale and offset into convolution weights and bias before the ReLU activation
Core ML compiler folds bias and activation into the post-MAC path at compile time, avoiding an intermediate memory round-tripeiln.github.io

The article explains that CoreML's compiler folds bias and activation into the same post-MAC path at compile time, with the bias register and nonlinear mode set per-layer, and the MAC output feeds directly into the lookup without an intermediate memory round-trip

One nonzero lookup-table entry produces two straight-line segments on the ANE's piecewise-linear tanh approximation
A single nonzero LUT entry generates two straight-line segments, illustrating the ANE's piecewise-linear interpolation schemeeiln.github.io

The article describes the ANE's nonlinearity approach: a 33-entry piecewise-linear lookup table with linear interpolation, spacing knots at intervals of 1/8, where tanh is mode 2 and ReLU is mode 1

Core ML tanh overlaid with double-precision tanh, followed by signed error plot
Core ML tanh output overlaid against double-precision tanh, with the signed error beloweiln.github.io

The article discusses the ANE's tanh implementation as mode 2 of the piecewise-linear lookup table; this figure validates the approximation's accuracy against a double-precision reference

The scheduler is, by Yoon's description, disappointingly simple. The driver never sends CONV or MATMUL opcodes. All neural operations are pre-compiled into a command stream of fixed-size task descriptors; the driver loads the stream into memory, points TM_ADDR/TM_SIZE at it, and rings TM_PUSH. The hardware then owns the submission until completion, raising an interrupt to the ARM64 core when done. Eight task queues (qid 0–7) provide ping-pong staging — one descriptor slot executes while software fills the other — and each queue carries a 32-entry base-address register table that maps tile-level work.

Yoon's GitHub repository (eiln/ane) contains the kernel module, userspace library, and Python bindings — 193 commits, 537 stars. The work is explicitly not about making the ANE useful today; it is about reading the architectural assumptions Apple etched into silicon in 2017 and tracing what happened when those assumptions met the transformer era.

What remains uncertain: whether the M4 and later ANE blocks expose capabilities beyond the M1 design Yoon documents (a discussion cue flags that Apple apparently doubled ANE blocks for M6/A20, and that the M5's "Neural Accelerators" inside GPU cores are a different subsystem). Yoon's source covers the M1 ANE only, and the article does not independently verify M4/M5 claims. Similarly, whether independent developers can access the ANE at all — versus Apple restricting it to computational-photography pipelines — is not resolved by the supplied material; one commenter asserts the restriction, but the supplied driver itself is evidence that Linux access exists, if only via reverse-engineering.