AWS has released the AWS Emulator, a local program that serves AWS API endpoints and allows developers to test software without making actual calls to the cloud. Notably, every line of the emulator's code was written and tested by autonomous AI agents rather than human engineers. This development aims to solve the difficulty of testing complex distributed systems by providing a local environment where agents can iterate on code faster than human teams can verify it in production.
The Evolution of Distributed System Testing
Testing distributed systems at scale is notoriously difficult because concurrency bugs often only appear under specific network conditions or event orderings. To address this for early services like 'Alf' (a Paxos-based lock service), Amazon engineers built a simulation environment to drop packets and inject faults in a controlled JVM environment.
While simulation and formal reasoning (like TLA+) improved software quality, building these environments by hand became unsustainable as AWS grew to over 200 services. The AWS Emulator addresses this by using AI agents to automatically generate these test environments based on existing service specifications.
“When agents write the code and check their own work, the engineer’s job is to give them room to work on their own and to build the specs, tests, and merge gates that decide when that work is correct.”
— AWS Emulator and the engineer's job when agents write the code
The Agentic Development Workflow
The emulator was built using a three-team agent architecture. The first team converts Smithy API models and real-world AWS behavior into 'fidelity tests.' The second team replays these tests against the emulator to identify divergences, filing issues in a 'things-that-do-not-work' folder. The third team picks up these issues, writes the fixes, and submits them for review.
This diagram visualizes the three-team agent architecture described in the article, showing how different AI teams collaborate to build and maintain the emulator.
Chart drawn by HN Digest from figures reported by aboutamazon.com; values are shown as written in the source.
These agents operate within a 'merge gate'—a series of automated checks that verify the code compiles, meets coding standards, is secure, and maintains fidelity against the real AWS cloud. This allows the agents to iterate on the code in a loop, correcting their own work before a human ever sees the final commit.
This figure provides a more granular look at the data flow between the Smithy API models, real AWS behavior, and the specific folders used by the AI agents.
Shifting the Engineer's Role
As agents take over the bulk of the coding and testing, the role of the human engineer is shifting toward high-level oversight and infrastructure design. Engineers now focus on defining the 'coloring book' for the agents: the public API models, the fidelity tests, and the automated guardrails that ensure the agents do not 'color outside the lines.'
The project demonstrates that software quality in the AI era depends on these guardrails. By providing a local environment where agents can check their own work, engineers can supervise the process more effectively, moving from manual implementation to the design of the systems that govern automated production.
Usage and Limitations
The AWS Emulator is intended strictly for building and testing; it is not designed for production workloads or security testing. It runs offline on a user's machine, and while it provides a local copy of AWS APIs, it does not provide a 1:1 replica of every possible cloud behavior.
Individual service gaps are tracked in the repository's issues directory. While the emulator provides a significant leap in local testing capabilities, the 'fidelity scores' for each service indicate that some behaviors may still differ from the live AWS environment.