Dessa is a conversation model developed by a research lab at Block that distinguishes itself by being trained entirely on human speech. Unlike most modern large language models (LLMs), which ingest massive amounts of text to learn language, Dessa was trained from scratch using only audio data. This approach aims to replicate how humans learn to speak—by listening—allowing the model to capture information that is often lost when voice is converted into text, such as sarcasm, sighs, and specific inflections.
The image represents the Dessa model, which is trained exclusively on human speech audio to capture nuances lost in text conversion.
The model is designed for full-duplex interaction, meaning it can process and generate speech simultaneously rather than waiting for a user to finish speaking before responding. This capability allows for natural behaviors like 'backchanneling' (interjecting with brief acknowledgments like "yeah" or "got it") and reacting to emotional cues in real-time. According to the developers, the model was trained on a single NVIDIA GB200 NVL72 and is optimized to run locally on high-end hardware, such as a MacBook Pro with an M4 Max chip, with inference speeds faster than real-time.
While the model demonstrates high expressiveness, its training methodology presents unique technical challenges and research opportunities. Because it lacks a text baseline, it must learn facts and language from raw sound, which the developers note is significantly more difficult than learning from text. This creates a new category of deep learning models for researchers to study, specifically regarding how efficiently a model can learn new languages or perform in-context learning without a textual foundation. The developers have also invited AI safety researchers to evaluate the model, noting that its ability to perceive and replicate human emotion is a step toward more empathetic and safer AI interactions.