STROT (Agentic Schema Intent + Compiler & State + Code + Execution + NLU) is designed to solve the trade-off between the high reasoning capabilities of large language models and the practical requirements of production systems, such as low latency, cost-efficiency, and determinism. Instead of calling an LLM API for every individual request, STROT uses a large model (such as Claude Opus) during a one-time 'compilation' phase. During this phase, the system synthesizes a local decision model, an input projection, and specific handlers based on a provided intent and sample data. The result is a frozen Python artifact that can be deployed on local hardware.
The image represents the Project STROT repository, which houses the framework for compiling LLM intent into local decision pipelines.
The mechanism functions by transforming a high-level task—such as triaging support tickets or scoring customer sentiment—into a self-routing NLU (Natural Language Understanding) runtime. The compiler evaluates the intent, generates a 'golden gate' of teacher labels, and calibrates the projection. At runtime, a much smaller, local model (such as the 144M parameter Julia-1) handles the actual records. This small model produces typed JSON outputs, including probabilities and confidence scores. If a record falls below a specific confidence threshold, the system is designed to abstain rather than guess, ensuring that the output remains bounded and auditable.
Benchmarks provided by the project suggest that this 'compiled' approach significantly outperforms naive implementations using the same small model. For example, in grounded routing accuracy tests on 398 records, the STROT-compiled model achieved 92.5% accuracy compared to 74.8% for a naive approach, representing an 18-percentage-point improvement. The system also demonstrates a significant reduction in operational overhead, achieving a latency of approximately 60ms per record with zero marginal cost and zero LLM calls during production execution. However, the effectiveness of the system relies on the quality of the initial 'compile' phase and the accuracy of the sample data used to calibrate the projection.