The project doomformer explores a novel way to execute neural networks by mapping their mathematical operations to the game mechanics of DOOM. In this system, the marine's 'brain' is compiled directly into the game level. The core concept relies on 'voodoo dolls'—copies of the player character—to represent activation values. A number is represented by a crowd of these dolls on a conveyor belt, while an activation function is modeled as a staircase the dolls must climb. Matrix multiplications are achieved by dolls running across lava lines, where each line crossed increases a room's light level. The final output is read from these light levels and compared bit-for-bit against a standard integer reference to ensure exact precision.
This image illustrates the core concept of using game mechanics like conveyor belts and lava lines to represent neural network activations and matrix multiplications.
To achieve this, the developer had to account for the specific quirks of the DOOM engine. For example, the system handles the fact that two floor moves on one sector in the same game tick might be silently lost by the engine, or that teleport lines can cancel out previous line crossings. These constraints were mapped to ensure that every value—including hidden layers, score comparisons, and one-hot answers—remains bit-exact. The project supports various architectures, including Multi-Layer Perceptrons (MLP), Transformers, and Spiking Networks. For instance, a Transformer model can be compiled into a single map where dolls squeeze under ceiling gates (representing attention weights) to compute results without any external Python code running between layers.
This figure supports the claim that the system ensures every value remains bit-exact despite the engine's quirks.
While technically impressive, the method is significantly slower than traditional computing. One demonstration of a handwritten digit recognition network showed a 'READ' operation taking approximately 163 seconds of real-time DOOM gameplay, which is roughly 70,000 times slower than a standard NumPy calculation. Despite this performance trade-off, the project successfully demonstrates that complex logic, such as a 2-layer transformer capable of naming new DOOM levels or answering questions about cheat codes, can be executed entirely within the game's environment. The system even allows for 'lobotomies,' where a single shotgun blast to a specific 'neuron' in the map can change the marine's behavior from aggressive to pacifist by physically blocking the dolls' path.
The chart provides a visual representation of the massive performance trade-off, showing the 70,000x speed difference.