Apple researchers have developed a method to significantly reduce the computational cost of Retrieval-Augmented Generation (RAG) by replacing traditional geometric search with small, learned neural networks. By training models to predict the 'support function' of a database, the system can route queries to the correct data clusters more accurately than standard centroid-based methods, achieving up to 8x faster retrieval for high-recall results.
The Limitation of Geometric Search
Maximum Inner Product Search (MIPS) is the engine behind recommendation systems and RAG, where a system must find the best-matching 'key' vector for a given query. As databases grow to millions of keys and dimensions reach into the thousands, exhaustive comparison becomes computationally prohibitive.
Current approximate methods often use clustering to group keys around centroids. However, these methods assume queries will resemble the keys. In RAG, this is often false: user queries are typically short, while the keys are long, curated pieces of text, leading to routing errors where the best match is skipped because it sits far from its cluster's center.

This image explains the fundamental limitation of geometric search where queries might be routed to the wrong cluster because they are far from the centroid.
“The SupportNet model can answer a more useful question than a simple comparison to the centroid can: “How good is the best key inside this cluster for this query?”
— Faster Retrieval for Better, More Efficient RAG
SupportNet and KeyNet Architectures
The researchers identified that the MIPS value function is a 'support function' in convex analysis. They developed two specialized models to exploit this: SupportNet and KeyNet. SupportNet is an input-convex neural network trained to approximate the support function of each cluster, answering the question: 'How good is the best key inside this cluster for this query?'

This visual supports the transition from centroid-based routing to the SupportNet approach which considers the best key within a cluster.

This image illustrates the mechanism of SupportNet as described in the architecture section.
KeyNet models learn to directly output the best-matching key vector. By using a consistency loss derived from Euler’s theorem, these models remain geometrically grounded even without explicit convexity constraints. Both models are trained on the specific distribution of queries that actually matter, such as those found in the Natural Questions benchmark.
Performance Gains and Trade-offs
In testing, the SupportNet model hit 95% routing recall about 1.9x faster than traditional centroid routing. When using the largest model, the system achieved 97% recall in 1.74 ms, demonstrating that the small neural networks can outperform geometric baselines even when accounting for their own inference overhead.

This chart provides the empirical evidence for the performance gains and trade-offs discussed in the final section of the article.
Chart drawn by HN Digest from figures reported by machinelearning.apple.com; values are shown as written in the source.
KeyNet provides a different advantage by acting as a query rewriter. By transforming a raw query into a vector that is much closer to the target key, it can reduce the number of keys an index needs to scan to reach 99% recall by up to 11x. While these models require an upfront cost to train on a fixed database, they provide a more efficient path for production systems with stable query distributions.

This example demonstrates the 'query rewriter' capability of the KeyNet model mentioned in the performance section.