The technique converts an LLM into a decision model by numbering each answer option, inserting the state, question and numbered options into a JSON prompt, and prefilling the assistant’s response with the string “choice_index:”. This causes the model to continue by outputting a token that is an index into the option list. By restricting the model’s vocabulary to only those option indices (using vLLM’s allowed_token_ids) and reading the log‑probability of exactly those indices (via logprob_token_ids), the normalized probabilities give a confidence for each choice, allowing a typed decision in one forward pass without fine‑tuning the base model.

Pale strands entering from the left at scattered heights and settling into one narrow band, one of them lime, on a dark petrol-to-green gradient
Illustration of the decision-making process showing multiple options converging to a single choicewww.privatemode.ai

This image depicts the core technique where the LLM treats answer options as strands that converge to a decision index, enabling typed decisions in one forward pass.

Benchmarking on 28 public text datasets covering intent routing, sentiment, topic classification and related tasks shows that GLM‑5.3‑Flash configured this way matches TypeSafe’s Jev in accuracy: each system is more accurate on 10 datasets, they are within one percentage point on 8 others, and the median gap is 0.7 percentage points in Jev’s favor, which is not statistically significant (p = 0.64). Latency measurements reveal a geographic split – from Germany the Privatemode‑hosted GLM‑5.3‑Flash averages 180 ms per decision versus Jev’s 264 ms, but from the United States the order reverses to 299 ms for the LLM approach and 164 ms for Jev. Cost per million decisions is about EUR 62 for the GLM‑5.3‑Flash setup and EUR 16 for Jev at each service’s list prices, reflecting differences in token pricing and prompt overhead.

A notable advantage of the LLM‑based method is that it accepts images alongside the text prompt, enabling typed decisions on visual inputs—a capability Jev does not provide. This is made possible by the same vLLM‑based pipeline that allows the prefilled assistant turn to continue when multimodal content is present. However, the approach still inherits the latency and cost characteristics of the underlying GLM‑5.3‑Flash model, which are higher than Jev’s in many regions, so users must trade the added multimodal flexibility against increased expense and variable response times.