As DNS zones grow in complexity, processing updates for massive domains presents a significant engineering challenge. Cloudflare reported that some of its zones now contain more than 20 million records, with expectations that these numbers will reach hundreds of millions. Previously, the system relied on 'full builds' to propagate changes. A full build reads the entire current database state for a zone, converts every record into a format used by authoritative servers, and compares them against the existing data in Quicksilver, Cloudflare's distributed key-value store. For massive zones, this process is inefficient because the entire zone must be held in memory, often exceeding available capacity and creating bottlenecks in the propagation pipeline.
To solve this, Cloudflare implemented a targeted build planner that identifies the smallest set of changes required to maintain a correct state. The system now classifies updates into five specific build types: 'Resources only' (standard A, AAAA, TXT, and CNAME changes), 'Resources and metadata' (nameserver and wildcard changes), 'Apex and metadata' (settings like Universal SSL), 'Purge' (removing a zone), and 'Full zone' (used only when a change cannot be proven safe to target). The planner ensures safety by tracing each setting through the code to determine its impact. If a change is ambiguous—such as a timestamp-only update that doesn't specify what changed—the system defaults to a full build to avoid serving stale or incorrect data.
Efficiency is further improved by a unified scheduler that combines multiple pending changes for the same zone into a single build operation. This prevents the system from performing redundant work when several updates arrive in quick succession. To maintain reliability, the system uses 'soft committing' with Kafka; while a build might finish out of order, the Kafka offset only advances once all preceding messages are complete. This ensures that if a worker restarts, no customer changes are lost. However, a specific trade-off exists for the largest zones: if a zone exceeds a fixed record-count threshold, the system refuses to perform a full build. This prevents a single massive zone from blocking an entire Kafka partition, even if it means the system must manually rebuild individual names in rare cases where a full build would be required but is technically impossible to complete within memory limits.