A new project, `tinyworldformer`, demonstrates a compact world model capable of generating and playing games directly on a Raspberry Pi Pico 2 W, a low-cost microcontroller. The system utilizes two small neural networks: a world model that predicts the next frame and reward based on the current state and an action, and an actor network that selects actions based on the generated frames. By running these models on a microcontroller with a floating-point unit (FPU), the project explores the limits of running complex generative tasks on extremely constrained hardware.

Animation showing the interaction between the world model and the actor network
The system architecture features a world model for state prediction and an actor network for action selection.github.com

This visual explains the dual-network architecture where the world model predicts frames and the actor network selects actions.

The core innovation lies in the use of a 'looped transformer' architecture. Instead of stacking multiple different layers, the model applies a single transformer block several times with the same weights. This approach significantly reduces memory overhead, which is critical for microcontrollers. To maintain performance on the Pico 2 W, the model uses two loops, trading a small amount of accuracy for a viable frame rate. The world model is further optimized through 8-bit quantization (QAT), where weights and activations are compressed to `int8` while keeping norms and embeddings in `float32` to leverage the chip's FPU. This allows the model to fit within the device's 520 KB of SRAM.

The project supports several games, including Breakout, Flappy Bird, and a raycast first-person view called 'Doom-lite.' For more complex environments like Pac-Man, the model requires more parameters (up to 131k) and more loops to achieve high accuracy. A notable technical challenge involved ensuring the model's outputs remained deterministic and logically consistent; for instance, the developers implemented a 'global singleton' rule to ensure that objects like the ball in Breakout are only rendered once. While the world model can 'dream' up new scenarios, the project also includes a streaming reinforcement learning agent that can learn to play within these dreamed environments, providing a foundation for advanced procedural generation."

A GIF showing the world model generating a game of Breakout
The world model 'dreams' and generates a playable version of Breakout.github.com

This demonstrates the project's ability to generate and play Breakout on the microcontroller.

A GIF showing the world model generating a first-person view of Doom-lite
The model generates a raycast first-person view known as 'Doom-lite'.github.com

This illustrates the more complex 'Doom-lite' environment supported by the tinyworldformer project.

A GIF showing the world model generating a game of Flappy Bird
The model generates and plays a version of Flappy Bird.github.com

This shows one of the primary games supported by the tinyworldformer system.