As OpenAI's development pace accelerated, the company's monorepo faced significant scaling challenges. The integration of Codex and other AI tools enabled engineers to perform more work in parallel, leading to a surge in concurrent pushes, merges, and fetches from research jobs and CI environments. These demands eventually caused the existing source control systems to struggle, with reliability occasionally falling below 90% due to slow fetches and failed merges. To address this, OpenAI engineers built Pando, a new backend for Git repositories designed to handle 'agentic-scale' work—where both human engineers and AI agents interact with the codebase simultaneously.
Pando is designed to be transparent to the user; it powers the monorepo without requiring engineers to change their existing Git workflows or tools. The system achieves horizontal scalability by rethinking two core constraints of traditional Git services. First, it coordinates updates to branches independently. In many standard systems, a single order is imposed on all writes to a repository, meaning a push to one branch might wait on a push to an unrelated branch. Pando removes this bottleneck by allowing unrelated branches to be updated in parallel. Second, Pando optimizes read throughput by reusing 'packfiles'—Git's compressed format for multiple objects—across different clients. Instead of generating a unique packfile for every fetch, the server serves the same bytes to multiple clients, significantly reducing CPU and disk I/O.
The article explains how Pando optimizes read throughput by reusing packfiles, which are Git's compressed format for multiple objects.
The article describes how Pando coordinates updates to branches independently to remove bottlenecks in traditional Git services.
To achieve this scale, Pando adopts a 'causal consistency' model, which involves a specific trade-off regarding repository-wide state. Because Pando coordinates branches independently, it does not provide a single, consistent point-in-time snapshot of all references across the entire repository. For example, if a client performs a full clone while two different branches are being updated, the client might see one branch at version A and the other at version B, even if they were technically updated 'simultaneously.' However, Pando maintains serializability for individual references, ensuring that updates to a single branch remain consistent. In production, this architecture has cut median push latency by nearly 5x and currently handles nearly a petabyte of Git data per day. While the system provides high throughput, the specific number of concurrent AI agents currently utilizing the system remains undisclosed.