A 'Jev-like' model is defined by its ability to process a text input and a list of options, returning a probability for each option in a single pass. This contrasts with standard autoregressive generation, where a model predicts text one word at a time. By shifting to this single-pass approach, models can more efficiently perform classification tasks, such as yes/no or multiple-choice questions, by leveraging the intelligence already encoded in the model's latent space without the overhead of sequential token prediction.
The technical mechanism for this in the vLLM project involves repurposing the DiffusionGemma model to work with a fixed 'canvas' of token positions. In this framework, the requesting client provides a set of fixed token slots. The model then 'diffuses' the content of these slots while the structure remains constant. Key technical components introduced to support this include `diffusion_seed_canvas`, which pre-seeds the output with fixed tokens; `diffusion_pinned`, which ensures specific positions are held at their seed values during the denoising steps; and `diffusion_read_only`, which allows the model to emit logprobs at temperature 1 to provide a confidence score for each answer.
While this method offers a more direct path to structured outputs, it currently faces specific architectural constraints. The model requires that each choice in the output be represented by a single token to ensure the canvas does not shift during the diffusion process. Consequently, if a client needs to offer multi-token options, they must first map those options to single-token IDs (for example, mapping "moderation_spam" to "A"). Additionally, while the model provides logprobs to help clients calculate entropy and determine confidence, the current prototype does not yet include empirical benchmarks comparing this single-pass diffusion method against standard autoregressive models for complex, multi-token structured outputs.