The concept of 'decoherence' describes a specific form of technical debt where a system becomes a patchwork of locally functional but mutually incompatible models. Using the analogy of a building's plumbing, the author explains that while replacing a single pipe as it fails is a low-disruption strategy, doing so repeatedly with different standards creates a complex web of unique forces and properties. Over time, this creates a system that requires significantly more expertise to modify because every 'patch' may have unintended consequences on the rest of the network. In software, this manifests as a 'maze' of code where the complexity of interactions grows combinatorially as the system expands.
This diagram illustrates the plumbing analogy used to explain how local patches create a complex web of incompatible models over time.
This phenomenon is currently being accelerated by the widespread use of artificial intelligence. The author posits that AI-generated 'slop'—content or code that is locally valid but lacks global coherence—is being integrated into systems at an unsustainable rate. While some fear a runaway recursive self-improvement (an intelligence explosion), the author argues that decoherence actually acts as a brake on such progress. Because the cost of change is dominated by the need to translate between incompatible models and prevent side-effects, decoherence prevents AI from improving rapidly within a bubble and limits its ability to improve outside of one.
This graph illustrates the author's argument that decoherence acts as a brake on rapid progress by increasing the cost of change.
However, the author warns of a 'Decoherence Collapse.' This occurs when the complexity of the system's maze grows exponentially, outpacing the ability of engineers to navigate it. The author uses a simulation to demonstrate that while low-coherence actors (those who make quick, local decisions) move faster initially, they eventually create a graph so tangled that even high-coherence actors are slowed down by the sheer volume of context required to make a single change. If the complexity grows exponentially rather than linearly, adding more compute or more autonomous agents may only provide a temporary reprieve before the maze consumes all available resources, potentially leading to a state where the system becomes impossible to maintain or modify safely.
This simulation demonstrates the relationship between low-coherence actors and the resulting complexity of the system's graph.