Strata brings a 125-billion-parameter language model to the desktop. The model is Qwen3.8-Flash-Next, released by the Qwen team as an experimental preview of the architecture that will underpin the next-generation Qwen4. Strata, an open-source inference engine under the MIT license, runs it on consumer hardware — a gaming PC with a recent NVIDIA or AMD graphics card — at speeds around 100 tokens per second on 24-gigabyte cards, with nothing leaving the local machine.
The opening paragraph introduces Strata as the open-source engine that brings the 125-billion-parameter model to a desktop gaming PC.
The model is large but unusual. It has 125 billion parameters in total, yet only about 6 billion are activated for any given token. It is a mixture-of-experts model: 512 experts per layer across its 48 layers, roughly 24,576 specialists in all, of which 10 routed experts plus 1 shared expert are selected for each word. The rest stay dormant. This is what makes a 125B model behave more like a 6B model per step — and what makes it feasible to run on a PC at all.
Beyond expert routing, the model introduces several architectural changes the Qwen team says are aimed at efficiency at scale. Qwen Sparse Attention (QSA) processes text in micro-blocks rather than individual tokens, cutting long-context latency. Gated Residual modulates information flowing through wider residual streams via data-dependent read and write gates. A large n-gram embedding table — 51 billion parameters' worth, indexed by short word sequences — scales parameter count with less computation than a standard embedding. Its native context length is 262,144 tokens, extensible to a million.
Strata's job is to make all of this fit on a machine built for games. The README uses a kitchen analogy: the experts you use constantly stay on the counter, and the rest wait in the pantry. Concretely, the GPU keeps the few thousand experts accessed most often; system RAM holds all of them; the processor works on the remaining experts while the GPU runs; and an SSD holds a large lookup table. The installer picks a quantized version of the model — 2 to 4 bits per weight — because the full model does not fit in 12 to 24 gigabytes of video memory.
Paragraph four explains the kitchen analogy and the concrete split of experts across GPU, RAM, and SSD that makes the 125B model fit on a gaming machine.
A second trick speeds up replies. A small helper model guesses the next few words, and the big model checks them all at once, keeping the right ones. The README calls this guess, then check and says it produces the same answer 1.6 to 1.8 times sooner. Long prompts are read in chunks of up to 8,192 tokens at a time, at over 1,000 tokens per second, so a 32,000-token document takes about a minute to ingest.
Paragraph five describes the guess-then-check trick that speeds up replies and the README's claim about how much sooner it produces the same answer.
The published benchmarks come from two ordinary gaming PCs, not the RTX 4090 in the headline. On an NVIDIA RTX 5070 with 12GB of VRAM, 64GB of RAM, and a Ryzen 5 7600, the engine writes answers at 94 tokens per second with the most aggressively compressed Q2_0 quant, falling to 53 tokens per second with the larger IQ3_S quant. Reading a 32,000-token prompt ran between 1,620 and 2,650 tokens per second on the same machine. On an AMD RX 9070 XT with 16GB, the same quants wrote 60 and 52 tokens per second respectively. The README notes that a card with more VRAM is faster, and estimates that an RTX 3090 with 24GB writes about 100 to 140 tokens per second — the ballpark the headline references. A token is roughly three-quarters of a word, so 100 tokens per second is well past reading speed.
The tradeoff is quality, and the README is candid about it. Smaller, faster quantized files are a bit smarter at larger sizes, and the installer recommends different quants depending on how much RAM a machine has. The Coder variant removes half the experts, reaches 91% of the full model's SWE-bench Verified score as measured by its authors, and fits in 32GB of RAM — but it is weaker outside of code, including Chinese and other CJK text. There are also community fine-tunes: a Swift 1.5 version that thinks for a shorter time before answering, and Unsloth's roughly 4-bit versions that sit between the recommended quants in quality, with the closest-to-full variant writing only 7 to 8.5 tokens per second on a 64GB PC because it is largely read from the SSD. The model's published benchmark scores, listed on its Hugging Face page, are the Qwen team's own measurements at full precision — a reference point for what the uncompressed model can do, not a guarantee about any particular quantized copy.
Running it is straightforward. The installer asks which model size to use, how much context to keep, and whether the model should read pictures, then downloads about 70 gigabytes and starts the engine. In a browser at localhost:8080 there is a chat interface and a live monitor of the model and the machine's GPU, CPU, and RAM usage. Apps that use an OpenAI-compatible API can point at http://127.0.0.1:8080/v1; apps that use Anthropic's API can use the messages endpoint, which lets tools like Claude Code route requests through the local model. The engine also exposes a Responses API for apps built on OpenAI's newer interface.
Paragraph eight describes the browser chat interface at localhost:8080 and its live monitor of the model and the machine's GPU, CPU, and RAM usage.
Part of the appeal is that a 125B model that previously needed a server farm can now run on a desktop, with no data going to a remote provider. For developers and hobbyists who want a powerful local model for coding, writing, or agentic tasks — and who may be sensitive about sending code or documents to a cloud service — that is a meaningful shift.
The caveats matter. Strata is experimental, and the README warns that the first start can freeze the machine for one to three minutes while the model loads into RAM, which is normal. The engine answers one request at a time by default, though it can be configured for several concurrent streams. Picture input works on most setups, but AMD cards can only read images on Linux, not Windows, for now. The speed figures come from a specific engine version on specific hardware — most rows in the published tables used engine 0.1.26, with the fastest Q2_0 row on 0.1.36 — and real-world numbers will vary with the card, the quant size, and the prompt.
What remains uncertain is how well the lower-bit quantizations hold up in practice, and whether the headline's roughly 100 tokens per second on an RTX 4090 matches the engine's own estimate for a comparable 24GB card. The RTX 3090 figure is an estimate, not a benchmark, and the published tables come from different cards. Independent testing on the exact configuration would be needed to confirm the number.