Standard deep learning relies on backpropagation: a forward pass computes predictions, then a backward pass propagates the error gradient from the output back through every layer to update weights. The problem is that backprop requires precise, synchronized coordination across the entire network—a neuron in an early layer must hold its activation while waiting for the error signal to arrive. Neuroscientists have long noted that the brain has no known mechanism for such strict timing, which motivates the search for local learning rules that can assign credit without a global backward pass.
Predictive coding (PC) is one such alternative. Each layer tries to predict the activity of the layer below it; only the prediction error—the mismatch—is passed upward. Mathematically, this avoids backprop's synchronized passes: layers iteratively adjust their activations to minimize a shared "free energy," and each update touches only neighboring layers. PC has worked on shallow or wide networks, but it suffers from a signal-decay problem. In deep, narrow networks, the supervision credit originating at the output becomes too weak to influence earlier layers before it dissipates through a chain of local compromises. Increasing the number of inference steps helps, but at the cost of more computation per update.
PC-ALM augments predictive coding with a classical optimization tool: the augmented Lagrangian. Each layer gets a dual variable (a Lagrange multiplier) the same size as its activation vector. During inference, the layer updates its activations to reduce prediction error, while the dual variable accumulates the residual constraint violation. In a linear network, these dual variables converge to the exact backpropagation credit signals using only local computation. In nonlinear networks, the dynamics become a form of primal-dual ascent: the primal variables (activations) descend on an energy, while the dual variables ascend on accumulated errors.
This figure illustrates the paper's key result that PC-ALM maintains accuracy within approximately two percentage points of backpropagation even at extreme depths of 1000 layers, validating the effectiveness of the augmented Lagrangian approach for credit propagation in deep networks.
The practical effect is a wavefront of credit that moves through the network faster than PC's diffusive, heat-flow-like propagation. The authors call this "ballistic" credit propagation: in PC-ALM, credit signals spread evenly across layers rather than decaying with depth. On MNIST, PC-ALM trains residual MLPs up to 1000 layers (width 32, ReLU) and stays within roughly two percentage points of backpropagation accuracy across the full depth range. On CIFAR-10 and Tiny ImageNet with ResNet-18, PC-ALM improves over PC on every benchmark and narrows the gap to backpropagation as depth increases.
This visualization demonstrates PC-ALM's robustness across multiple image classification datasets, confirming that the method improves over standard predictive coding and progressively closes the performance gap with backpropagation on CIFAR-10 and Tiny ImageNet as network depth grows, supporting claims of ballistic credit propagation.
Important caveats apply. The experiments use relatively simple feedforward tasks; the authors have not shown PC-ALM on large transformers or language modeling. The method matches backpropagation performance at matched inference budget, but it does not appear to be more computationally efficient—it simply achieves comparable accuracy with local updates where PC alone fails. The paper was submitted to arXiv on May 29, 2026, by Jeffrey Seely and Julian Gould of Sakana AI.
The result is best understood as a proof of principle: gradient-like supervision signals can be distributed through a deep network using only layer-local dynamics, which has implications for neuromorphic hardware where dynamical-system simulation is cheaper than GPU execution. Whether the approach scales to the large, discrete, or compositional architectures discussed in commentary remains open—the supplied sources do not address finetuning pretrained models, discrete components such as argmax or mixture-of-experts, or connections to Friston's free-energy principle.