Decision Models Are Eating Chat Models' Lunch

My agent calls tools. Every single turn. Sometimes the tool call is correct. Sometimes it’s the right intent in the slightly wrong place, floating in the model’s text where the runtime doesn’t look. Sometimes the model describes doing the work without actually calling a tool, and the description reads just fine.

For months the solution was to stretch chat models into every role: routing, gating, ranking, action selection, triage. They work because you can prompt them to output JSON. They work because you can parse JSON from their text. They work until they don’t, and then you spend three days debugging a prompt drift that nobody can reproduce.

In October 2026, the decision model class — purpose-built models that output probabilities over typed questions instead of free-form text — went from research curiosity to real infrastructure. Cloudflare released Clef and Clef-flash. Amazon released Strands Decider. Jev, the category founder, released Jev-2 alongside Jev-Flash. All three speak the same API. All three eat chat models’ lunch for routing tasks.

What a Decision Model Actually Does

A chat model takes text in, text out. A decision model takes a state blob and a list of typed questions, and returns a probability for every allowed answer. No prose. No parsing. No “maybe the model meant to call delete_file.”

Cloudflare’s Clef-Flash 9B takes a state (JSON, text, up to 4 images or video), a list of typed questions with criteria per option, and returns calibrated probabilities. One forward pass. Single logit per option. The model doesn’t generate text at all — it routes through a joint schema head.

Jev-Flash does the same thing but is proprietary API-only. Strands Decider from Amazon is Apache 2.0 open weights, also decision-class, also Jev-API compatible.

The difference from a chat model isn’t architecture — they’re all transformers. The difference is the training objective and the output head. Decision models are post-trained on routing tasks with label-smoothed cross-entropy and Brier loss for calibration. The result is that they don’t hallucinate answer choices. They don’t produce prose when you need a number. They don’t need prompt engineering to stay on format.

The Benchmarks

Cloudflare published Clef against Jev on 10 decision benchmarks. These are Cloudflare’s own numbers, so take them as a best-case comparison:

Clef leads on tool classification and intent detection. Jev leads on judgment tasks — When2Call (does the model call a tool, ask a follow-up, or decline?) and BRIGHT (reasoning-heavy retrieval). Clef-flash is fastest but collapses on out-of-scope detection (CLINC150: 66.77 vs Clef’s 97.43).

The independent test that matters most for agent builders: an autonomous coding agent running 42 real decisions across computer-use actions, subagent supervision, and inbox triage. Jev scored 71.4%. Clef-flash scored 66.7%. They agreed on 35 of 42 predictions. Where they diverged, they tended to fail for the same underlying reasons — the two models agree more on which tasks are ambiguous than on which answer is correct.

Latency is the other axis. Clef-flash delivered 38.8 ms median on an H200. Jev posted 524.1 ms median at the API. For the local user running Clef-flash on an RTX 3090, the independent test reported 315 ms fully local with no batching — still competitive with a fast API call, but the privacy win is real: log tails, file paths, and message contents never leave the network.

Why This Matters for Local Agents

I’ve run agents on chat models for tool routing for months. They work. They also need 8B-70B parameters, they produce prose between tool calls, and when the format slips you get a silent zero-tool-call turn that looks like success until you check the filesystem.

Decision models fix that. A 9B model doing tool routing produces no text, uses less VRAM than a chat model doing the same job, and has calibrated confidence — you can set a threshold and have the model defer to a fallback when uncertain.

The tradeoff is that decision models don’t do reasoning. They don’t write code. They don’t summarize. They route. Clef’s GPQA Diamond score is 48.0 versus Jev’s 78.3; MMLU-Pro is 65.9 versus 82.7. You’d never run a decision model as your primary agent model. But as a routing layer on top of any model — local or cloud — it’s the infrastructure you didn’t know you were running with prompt hacks.

The Open Weights Question

Both Clef (27B) and Clef-flash (9B) are Apache 2.0 on Hugging Face. Strands Decider is also Apache 2.0. Jev is proprietary API-only. This matters because the routing layer sits in your agent’s hot path — every turn goes through it. If you can self-host it under EU data residency, that’s a compliance win. If you need to fine-tune on your own domain data, open weights let you do that without a vendor contract.

Clef needs about 54 GB of VRAM in bf16. The community GGUF quantizations don’t include the decision head, so you need PyTorch and the custom model file. For a single GPU, Clef-flash (9B) runs at 18 GB in bf16, which fits on a 24 GB card. The full 27B model needs a second GPU or CPU offloading.

The Verdict

The decision model class is no longer a research demo. Cloudflare, Amazon, and Jev have converged on the same API shape, and the benchmarks show clear differentiation:

• Clef-flash 9B: fastest, smallest, best for high-volume routing on consumer GPU. Loses on nuance.
• Clef 27B: highest accuracy on routing tasks, needs serious hardware.
• Jev: still the best on judgment tasks (When2Call, BRIGHT). API-only. More expensive.
• Strands Decider: open weights, Apache 2.0, Amazon’s take on the space.

If you’re running a local agent today, you’re probably using a chat model for routing. It works. But it’s also running twice the parameters it needs to, producing unnecessary output, and drifting on format when the context gets long. A decision model is a dedicated routing layer that does one thing well: given a state and a set of questions, return calibrated probabilities.

The agent stack is becoming more modular. The chat model reasons. The decision model routes. The memory system remembers. Each layer has its own performance profile, and that’s the point — you stop asking every model to do everything.