Jane Street Intern's Indexing Strategy Cuts Message Bus CPU Usage by 30%

Jane Street's internal messaging framework, Aria, processes multiple terabytes of data per day, providing live message streams to subscribed clients. As Aria's usage grew, the system encountered performance bottlenecks during two recovery processes: tip recovery, where clients that fall behind request recent messages from an in‑memory ring buffer, and initial recovery, where rebooting clients request historical messages from the beginning of the week. These operations became strained under increased load, prompting summer intern Theodor Totev to investigate ways to make message filtering more efficient.

To accelerate tip recovery, Totev replaced the naive linear scan of the entire stream with an index for each topic partition—a group of topics sharing the same two‑segment prefix (e.g., all topics under 'app/codestore'). When a client requests messages, Aria performs an n‑way merge of the relevant partition indexes using a min‑heap to reconstruct only the needed message stream in order. Benchmarking showed that swapping the standard heap implementation for a fast_heap_unboxed version doubled the performance of the indexed recovery code, contributing to an observed 30% drop in CPU usage on production workloads while preserving the system's strict correctness requirements.

For initial recovery, Totev addressed a different inefficiency: Aria's on‑disk storage originally split message segments into subtree stores using a simple heuristic based on the first three topic name segments, which could place a high‑volume topic together with low‑volume ones. This forced clients reading a small topic to filter out large amounts of irrelevant data, sometimes reading up to ten times more bytes than necessary. By analyzing actual message volumes per topic, Totev devised an adaptive splitting algorithm that combines a greedy clustering approach with binary search. The algorithm allocates its own subtree store to topics with large message volumes while bundling smaller topics into shared stores, thereby minimizing wasted bytes during topic‑specific reads.