Cactus has released Needle 3, an "automation foundation model" designed to run tool calls, structured data extraction, and text search on tiny devices — phones, wearables, smart-home hubs, robots, and microcontrollers. The whole model ships as a single 8–29 MB binary, quantized to roughly 2 bits per weight, and the company's headline claim is that a small, fine-tuned slice of it can match DeepSeek V4 Flash, a much larger frontier model hosted in the cloud, on the specific tasks of calling tools and pulling structured fields out of text.
The article explains the Laddered Simple Attention Network replaces the standard feed-forward network with a Monarch Hadamard MLP, pairs GQA with causal convolutions, and reads an engram n-gram memory through a gather operation.
The unusual part of the design is what Cactus calls the "intelligence ladder." Needle 3 is one set of weights, but every depth from 2 layers to 20 layers is itself a deployable model. A developer can ship the full 20-layer version or an 8-layer subnetwork for a Raspberry Pi, and each subnetwork can be fine-tuned independently. In the company's own benchmarks, a 4-layer subnetwork fine-tuned for one epoch on downstream tasks matches DeepSeek V4 Flash, the model beats models roughly 10 times its size on mobile tool-calling tasks, and it matches models 2 to 3 times its size on extraction.
The article describes the intelligence ladder where every depth from 2 to 20 layers is a deployable model that can be fine-tuned independently, and the 4-layer subnetwork matches DeepSeek V4 Flash after one epoch.
How does an 8–29 MB model attempt that? The answer is the Laddered Simple Attention Network, an architecture built to spend far less compute per token than a transformer of the same size. According to the GitHub repository, it replaces the standard feed-forward network (FFN) with a Monarch Hadamard MLP, pairs grouped-query attention (GQA) with causal convolutions, reads an "engram" n-gram memory through a gather operation, and adds multi-lane hyper-connections. Most of the model's parameters live in that engram, so the company says the 121M-parameter model does the arithmetic of a 50M-parameter one. The product page adds that the architecture spends over 2x fewer million floating-point operations (MFLOPs) per token than a transformer of the same configuration. The model was trained on 360B tokens of a proprietary structured dataset, and on a Raspberry Pi 5 it decodes at 400–4,000 tokens per second and pre-fills at 1,000–10,000 tokens per second.
Needle 3 is built for three narrow capabilities, not conversation. Given the functions an app exposes, it picks the right tool and fills every argument from what the user said: asked for two things, it returns two calls in order, and asked for something no tool covers, it returns an empty list rather than guessing. For structured extraction, a developer declares a shape — an invoice, a booking, a notification — hands over messy text, and gets typed fields back; the company says the decode grammar guarantees the output parses and that extraction generalized well to classification problems. The same model also returns a text embedding vector, so an app can search, match, and route locally without leaving the device. The product page lists the target domains: smart home, where "dim the bedroom and lock up" becomes two calls executed offline; robots; phones; wearables; AR glasses; automotive; and plain-English control of a computer.
The fine-tuning path is cheap by the standards of this field. Cactus offers three hosted fine-tuning runs for $19 once, with compute and training-data generation included, and the open-source package supports local LoRA (low-rank adaptation) fine-tuning on the frozen base at the full 20 layers, then exports a 4-bit .cact of any subnetwork that runs on the same engine:
needle finetune data.jsonl --epochs 10 --out adapter.safetensors needle build --lora adapter.safetensors --out tuned.cact
The Python loop is similarly short — decorate a function, and run() completes the loop by executing the function and returning its results:
The article spells out how the benchmarks should be read: exact-match accuracy for tool calling, field micro-F1 for extraction, and the comparison conditions including the CQ2-bit binary versus f16 baselines.
import needle
The article lists the broad deployment platforms including macOS, Linux on x86_64, arm64, armv7, riscv64, mipsel, Windows, Android, iOS, tvOS, watchOS, browser via WASM, and a WASI component.
@needle.tool def get_weather(city: str): "Get the current weather for a city." return {"city": city, "temp_c": 27, "sky": "clear"}
agent = needle.Needle(tools=[get_weather]) print(agent.run("what's it like in Lagos right now?")["results"]) # [{'city': 'Lagos', 'temp_c': 27, 'sky': 'clear'}]
Every turn returns one JSON object with the function calls, the model's reasoning, a calibrated confidence score, and timing and memory figures. That confidence score is the part of the design that most determines whether the model is reliable enough to use in a real product. The engine applies a floor of 0.1: below it, the call is withheld into a suppressed_calls field and function_calls is empty. Above it, the company's own guidance is to route on the score — act immediately when it is high, show the call and ask when it is middling, and treat an empty result as a refusal. A tool marked with triggers always produces a call for a matching request, so the score is what tells a developer whether to run it or confirm it first. The company is blunt that tool design is "the whole game": the model reads a schema literally, so a narrow tool with a plain description beats a broad one, and every extra tool in the set is a chance to misroute.
This is also where the model's honest limitations become clear, and the GitHub README states the biggest one plainly: the product "trades general chat capacity." Needle 3 is not a chatbot and is not meant to be. It is an automation model: call a tool, extract a field, embed a sentence. Its guidance for writing tools — one tool per action, names drawn from what a user says, value formats spelled out in descriptions, required arguments given defaults, and a small toolset per turn — is really a list of the ways the model can fail. The company's claim that extraction "generalized well to classification problems" is scoped to its own suites; the source material does not supply independent evidence that it generalizes to arbitrary classification tasks, and the design's narrowness is the whole point of its speed and size.
How the benchmarks should be read is worth spelling out, because "matches DeepSeek V4 Flash" is easy to over-interpret. The published figures come from the maker's own suites, run on the full test splits: tool calling is exact-match accuracy, extraction is field micro-F1. The tool suites are Mobile Actions (phone commands to Android intents), DroidCall (phone requests, some needing two calls), and BFCL v4 (Berkeley function calling, single turn); the extraction suites are DSTC8, SNIPS gold, and SNIPS 7-way. The comparison conditions are specified: the Needle 3 subnetworks run through the shipped CQ2-bit binary (Cactus Quants, the company's 2-bit quantization), while the baselines run at f16 precision under the vLLM inference engine, and DeepSeek V4 Flash is reached through its cloud API. So the claim is that a 2-bit, on-device subnetwork scores comparably to a much larger, cloud-served, full-precision model on a handful of tool-calling and extraction suites — not that it matches DeepSeek V4 Flash in general. The fine-tuning figure reinforces the same point: fine-tuning on DroidCall lifts every subnetwork by 18 to 36 points, and from 4 layers up, the tuned subnetwork passes DeepSeek V4 Flash, starting at 29M parameters.
Deployment is broad. Every target ships a prebuilt engine under 1 MB that loads the needle3.cact weights at startup, and needle build fetches the engine for a platform and places the weights beside it. Platforms include macOS, Linux on x86_64, arm64, armv7, riscv64, and mipsel, Windows, Android, iOS, tvOS, watchOS, browser via WebAssembly (WASM), and a WebAssembly System Interface (WASI) component. A developer can build any subnetwork — an 8-layer version for a Pi, for example — and the Cactus Platform offers the full path (datasets, the 2-bit quantization, evaluation, fine-tuning, and dataset management) on the company's infrastructure for teams that do not want to build their own pipeline.
What remains uncertain is the gap between the maker's own suites and real-world use. The published numbers are Cactus's measurements, on Cactus's benchmarks, against a specific set of comparison models run in specific conditions, and the source material includes no independent audit. The model's behavior on ambiguous natural language — the kind of phrasing that produces a confident wrong call rather than a refusal — depends on the quality of the tool descriptions and on the confidence score, which is calibrated on the maker's data. The model is openly available, from the Python package and the weights on Hugging Face to a GitHub repository with 12.9k stars, 875 forks, and 326 commits, so the claims are independently testable. For a team building on-device automation around a well-defined tool set, the ladder architecture and the $19 fine-tune path are worth evaluating on their merits.