Janas is an open‑source family of tools for artificial intelligence written in pure C with no external dependencies beyond the standard C library. Its central component, Janas‑LLM, executes large mixture‑of‑experts language models by keeping the model weights on disk and streaming only the experts required for each token from an NVMe SSD, while keeping frequently used experts in a resident cache in memory. This design allows models whose parameter size exceeds the system’s RAM—such as a 48.4 billion‑parameter model—to run on a machine with only 32 GB of memory.
On the developer’s test machine (Debian 13 Linux, Intel Core Ultra 9 185H CPU with integrated Arc GPU, 32 GB DDR5 RAM, 1 TB NVMe SSD ~5.7 GB/s read speed), Janas‑LLM achieved 62 tokens per second when reading a prompt and 24 tokens per second when writing a reply for the Qwen3‑Next‑80B‑A3B model (48.4 GB). By comparison, llama.cpp on the same hardware managed only 16 tokens per second for reading and 2.8 tokens per second for writing the identical model. Similar advantages were reported across 13 different models evaluated in Q4_K_M quantization, with Janas‑LLM outpacing llama.cpp on every one.
This image illustrates the performance advantage of Janas-LLM over llama.cpp as discussed in the article's benchmark results section.
The project stresses that these results are early‑stage and hardware‑specific; performance may differ on other CPUs (especially those lacking AVX‑VNNI or AMD processors), discrete GPUs, or slower storage devices. Using the GPU has occasionally triggered driver resets that require a full system restart, and the expert cache consumes free memory when a model loads, potentially impacting other running applications—though the system degrades gracefully rather than crashing. Users are advised to run the janas‑bench benchmark on an idle machine first, as background software can reduce speed and increase disk wait times, and to share those results to help characterize Janas across broader hardware configurations.