The Nintendo 64 faced a significant technical hurdle regarding audio: cartridge space was extremely limited, and the 93 MHz CPU had restricted cycles for processing. Unlike the PlayStation, which could stream uncompressed redbook audio from a CD, the N64 required compressed formats to fit music onto cartridges. While the console's RSP (Reality Signal Processor) co-processor could handle some decompression, standard solutions were often inadequate. ADPCM (Adaptive Differential Pulse Code Modulation) saved space but was inefficient, often requiring too much storage for high-quality tracks. Conversely, modern formats like MP3 or Opus provided excellent compression but were too computationally expensive to run during active gameplay without severely impacting the frame rate.
The Ultra Low Complexity Codec (ULC) addresses these issues by functioning as a 'simpler' version of the MP3 format. Like MP3, ULC is a transform codec that uses the Modified Discrete Cosine Transform (MDCT) to convert PCM data into the frequency domain. However, it strips away the complex entropy encoding (such as Huffman coding) used in MP3. Instead, ULC uses a nibble-based syntax where each coefficient is stored in just 4 bits. This design allows the decoder to operate without arbitrary bit reads, making it highly efficient. ULC also includes a 'noise fill' feature, which is particularly effective for encoding percussive sounds like cymbals and snares—elements that MP3 struggles to represent accurately.
Implementing ULC on the N64 required navigating the RSP's severe memory limitations, specifically its 4096-byte data memory (DMEM) and instruction memory (IMEM). To fit the decoder into this space, the implementation utilizes 16-bit fixed-point math (Q1.14) rather than 32-bit floats. This choice reduces DMA traffic and speeds up computation, though it technically introduces a higher noise floor. In practice, the author found this noise to be negligible. The resulting decoder consumes only about 4% of the N64's CPU time and 5% of the RSP's time, allowing for 44100 Hz stereo audio at near-Opus bitrates. This efficiency is demonstrated in the Half-Life N64 port, where ULC enabled the inclusion of 2,500 sounds and 27 music tracks within a 64 MB cartridge image.
While ULC is a breakthrough for N64 hardware, it is important to note that its efficiency comes from its simplified syntax rather than a more advanced algorithm. By avoiding the 'cruft' of standard transform codecs, it achieves a balance of quality and performance that neither ADPCM nor Opus could provide on this specific hardware. However, because it relies on 16-bit fixed-point math to meet the RSP's memory constraints, it may not achieve the absolute peak fidelity of a float-based decoder, even if that difference is nearly imperceptible to the listener.