Modern recommendation models have evolved to consume increasingly long and attribute-rich user histories. This shift creates a significant storage asymmetry: systems must handle small, continuous updates (streaming workloads) while simultaneously supporting repeated, large-scale reads of historical data (batch workloads). To address this, ByteDance developed USDB, a unified sequence storage practice designed to bridge the gap between these two conflicting requirements by reusing maintenance results across ingestion, inference, and training.
USDB achieves this through several architectural innovations. It employs a 'CascadeLSM' time-based compaction strategy to materialize application-specific histories. By using differentiated replicas that share persistent files, the system ensures that data is not redundantly stored across different workloads. Additionally, USDB utilizes a hybrid columnar format to accelerate projected reads, allowing the system to reconstruct inference inputs from a combination of captured recent states and versioned history. These designs allow the system to maintain a single source of truth for sequences while optimizing for the specific access patterns of different tasks.
Production migration data indicates that USDB significantly improves efficiency, recording an 85.8% reduction in mean write latency and a 42.1% reduction in infrastructure costs compared to previous production setups. In controlled comparisons, USDB outperformed an Optimized RocksDB sequence baseline, demonstrating faster long-history reads across various client loads and projections. Furthermore, the system's aggregation pushdown capability sustains 8.5x the throughput of ordinary reads under a constrained receive budget. While highly optimized for the specific asymmetry of recommender workloads, the system's performance is tailored to these specific use cases rather than general-purpose database operations.