Decision models such as TypeSafe's Jev were introduced as a middle‑ground approach for AI safety guardrails, promising the structured, type‑safe outputs of traditional classifiers together with the zero‑shot flexibility of large language models. By returning a fixed decision (e.g., a probability) rather than generating text, Jev claims to offer faster, cheaper inference while avoiding the need for task‑specific training data required by conventional text classifiers.
Benchmark tests run by Red Hat’s AI Safety team measured latency and accuracy for prompt‑injection and content‑safety guardrails. For prompt‑injection detection, Jev achieved 86.35 % accuracy with a median latency of 348.1 ms, compared with 89.01 % accuracy and 54.1 ms latency for the traditional deberta‑v3‑base‑prompt‑injection‑v2 classifier and 89.31 % accuracy and 312.5 ms latency for the Qwen3.6‑35B LLM‑as‑a‑judge model. In the toxicity/profanity safety test, Jev scored 86.20 % accurate at 360.4 ms latency, while the granite‑guardian‑hap‑125m traditional classifier reached 80.27 % accuracy at 33.2 ms and Qwen3.6‑35B reached 85.47 % accuracy at 307.6 ms. Similar latency gaps appeared when Jev was compared to other LLM‑as‑a‑judge options such as Shieldstral‑1.0 and Nemotron‑3.5 variants.
This image displays the Red Hat AI logo, representing the organization that performed the benchmark tests on decision models and traditional classifiers mentioned in the article.
These results show that Jev does not deliver on its promised speed advantage; its latency is roughly six times higher than that of traditional classifiers and 10‑20 % higher than leading LLM‑as‑a‑judge models. While Jev’s accuracy remains competitive and its zero‑shot capability can be useful for novel guardrail scenarios without training data, the added decision‑making delay may outweigh the schema‑safety benefits in high‑throughput AI applications where millisecond‑level response times are expected. Consequently, adopters must evaluate Jev against specific latency requirements rather than assuming superior efficiency.
The chart shows Jev achieving 83.8% accuracy, lower than GLiDE and higher than GPT-6 Luna and Kev-9B, illustrating the trade-off between accuracy and latency discussed in the article.