Amazon engineers recently swapped out Claude Sonnet 4.6 for a specialized 'decision model' called Strands Decider to label internal tickets, resulting in a 98% reduction in token costs. This shift allows the company to process massive volumes of customer escalations, security findings, and automated alarms at a fraction of the previous expense while maintaining comparable performance.
From Generative Text to Probability Scores
The original system used Claude Sonnet 4.6 to categorize tickets into 20 different types. Because Sonnet is a generative model, it was required to process a long system prompt for every ticket to produce a structured JSON response containing the reasoning and the final label.
The new approach utilizes a decision model, which cannot generate free-form text. Instead, the model is provided with the ticket state and a set of options, returning a probability score and a confidence level for each possible label. This architectural shift allows the system to bypass the overhead of generating complex JSON structures.
“This change will save millions of dollars of token costs in a year.”
— Amazon saves millions by replacing Sonnet 4.6 with Strands Decider
Performance and Infrastructure Optimization
The team deployed the Strands Decider E4B build on AWS SageMaker using ml.g5.xlarge instances. This setup handles requests in about 640 ms, a significant speedup over the previous 4,215 ms average. The model occupies roughly 19 GiB of memory, fitting comfortably within the 24 GB capacity of the A10G instances.
The table illustrates the significant speedup and efficiency gains achieved by switching to the decision model.
To optimize the input, engineers rewrote the 20 label descriptions into one- or two-sentence criteria. This ensured the necessary context fit within the model's 4,096-token window. On long tickets, the system also automatically drops the oldest comments to maintain this context limit.
Cost Analysis and Operational Trade-offs
The cost reduction is primarily driven by a 70% drop in total tokens per ticket. While Sonnet 4.6 required 6,724 tokens per ticket, Strands Decider requires only 2,010. This efficiency translates to a massive saving of millions of dollars in annual token costs.
This visual supports the cost analysis section by showing the specific dollar amounts saved per million units.
Chart drawn by HN Digest from figures reported by strandsagents.com; values are shown as written in the source.
However, this efficiency comes with specific constraints. The decision model is best suited for 'closed choice' tasks where the catalog is relatively stable. It may be less effective for tasks requiring detailed explanation text, high-frequency catalog changes, or scenarios where the required context exceeds the model's token limits.
What Hacker News readers are saying
- Experience: One commenter who previously built a ticket classification pipeline noted that the industry seems to be moving in a full circle, returning to faster and cheaper classifiers after a period of prioritizing large language models. comment ↗
- Alternative: Another user suggested that using a specific Rust crate could provide even more efficient results by allowing for greater control over logic and the use of incredibly cheap models. comment ↗
- Skeptical: A commenter questioned the scale of the savings, noting that if the new model costs 98% less than the original, the actual amount saved might be a small fraction of the millions mentioned. comment ↗
- Experience: One commenter observed that while building SVMs requires careful data labeling and expertise, LLMs offer a different trade-off by providing an easier development experience at the cost of higher resource consumption. comment ↗
Summarized opinions from the Hacker News thread, not verified facts. Read the full discussion ↗