Give Typical some state, a question, and the options your software can take. Get a probability for each one back, including none of the above.
On screen: Typical playing DOOM, reading one sentence a tick and picking one of five moves.
Open decision models. Generate an answer, or read a decision.
A week ago TypeSafe launched Jev and made a strong case for a different interface between models and software. Instead of generating a string that code then has to parse, the model returns typed probabilities over outcomes the program already knows. TypeSafe calls this a System One model. We think decision model is the more useful name for the category.
The reaction was quick and concrete. Within a week Cloudflare was serving it on Workers AI as typesafe/jev, Vercel had wired it into AI Gateway behind the AI SDK’s evaluation API, and an independent write-up had reconstructed the architecture from the published interface alone. A month ago the category had no name. It now has two clouds serving it and people reverse-engineering it in public.
The idea deserves an open implementation: weights you can inspect, run on your own hardware, adapt to your domain, and keep next to your own data. Today we're releasing Typical, two open-weight decision models: typical-small (1.7B) and typical-medium (4B).
You give Typical a state once. That can be a support ticket, a policy document, an agent trace, or a game frame rendered as text. You can then ask it as many questions as you like, each with options you define at call time. Every answer is a probability distribution over those options, plus an explicit "none of the above." Nothing is generated and nothing needs parsing. A decision takes tens of milliseconds on a single GPU, and each further question about the same state costs a few more.
Look at where language models sit inside production code and the same loop appears. A prompt asks a question, the model writes an answer, and a parser tries to turn that answer back into something the program can branch on.
That loop is a poor fit for what the program wants. The program already knows the possible outcomes: route to one of four queues, approve or don't, assign urgency 0 to 3. What it lacks is a calibrated sense of which outcome is right. The program should own the outputs, and the model should own the probabilities.
A generated string gets both halves wrong:
A generative model produces answers in an order, because each one is conditioned on the last. A decision model does not have an order to produce them in. The state is read once into a cache and every question is a short suffix scored against it in the same forward pass, so the fortieth answer does not wait behind the thirty-ninth. It is computed alongside it.
One support ticket: a broken integration and a double charge, from a customer with a renewal coming up. Twenty typed questions about it, asked in one request of us and of a hosted model, both pinned by the same JSON schema to the same options. Started together.
noul line prints P(yes), so under .5 is a no.Typical starts from a pretrained base model and changes what happens at the end.
The probabilities come from those scores directly. No token is ever sampled. The answer is always one of the options you passed or an explicit abstention, by construction rather than by validation.
The same model handles both kinds of question software asks. Some are grounded in the state ("does this contract include a termination clause?"). Others lean on what the model already knows ("which of these libraries is for plotting?").
Different decisions deserve different probability spaces.
Picks one of K options defined at call time, as a categorical distribution.
Answers yes or no with a single Bernoulli probability, from a head that never sees the labels. That makes it exactly invariant to which label you call "yes." When we treated yes/no as a two-way Choice instead, reversing the label order moved P(yes) by as much as .55. In a matched comparison, the dedicated head also generalized better: on PagerDuty incidents, a dataset never used in training, accuracy rose from .60 to .89.
Returns a distribution over ordered levels, trained so that probability concentrates near the true level instead of spreading over every wrong one. In a matched comparison this halved negative log-likelihood on held-out Score tasks (2.07 to 1.23) without changing any top answer.
Attaches a probability to every Choice and Score answer that none of the options fit. It is a judgment about the set as a whole, not an extra label you have to remember to add.
Most software doesn't make one decision about a piece of state; it makes many. Is this a refund? Is it still under warranty? How severe is it? Which team owns it?
45msone decision, cold state
2.7msevery question after it
129msthirty‑two on one ticket
Asking a generative model each of those means re-reading the ticket each time. With Typical, the state is paid for once. On typical-small, with the state cached and 32 questions batched against it, each question costs about 3 ms for up to 32 options. A single cold decision, including reading the state, is in the tens of milliseconds. Cost grows with the total length of the option text, so for hundreds of options, narrow the set first.
That per-question cost lets a decision model live inside a loop rather than at the end of a pipeline.
Games are a good stress test for a decision model. The state changes every tick, the questions repeat, and a slow answer is a wrong answer.
The road · Three.js
Snake · canvas
DOOM · E1M1 · live
REC · typical-small · E1M1
booting
Every row below answered the same 231 public JevBench ids. That makes the table comparable, and it does not flatter us: four open decision models and one closed one are ahead of typical‑medium on the standard tier, and so is our own 14B preview. What did change this week is the bottom of our range — typical‑small now leads everything at or below its size, including the 400M classifier that used to be ahead of it. We would rather print the field we sit in than the one we win.
site/data/run-requirements.json.The JevBench columns above look like a ranking. They are not one. The standard tier is 72 items drawn from only 36 independent states, each written twice, and the hard tier is 111 items. Cluster-bootstrapped over the paraphrase group, that is a 95% interval of roughly ±6 to 13 points on any standard-tier number and ±9 on any hard-tier one. typical-medium's standard-tier interval is [.778, .944] and its hard-tier interval is [.405, .595]. Put the same ±6 to 13 around typical-small's .792, and the same ±9 around its .441, and the two models' intervals overlap on both tiers.
JevBench cannot separate these two models. Nor can it separate any adjacent pair in our seven-checkpoint ladder, which spans 1.7B to 14B and four backbone generations. Two of those checkpoints score exactly 55 of 111 on the hard tier while disagreeing on 26 items outright. That is not a tie, it is a measurement that has run out of resolution. Do not read the standard-tier gap between Small and Medium as a result. We do not.
What does separate them is everything measured at a sample size that can answer the question. Every typical-small number below is the v3 checkpoint, published 24 September; it replaced a one-day-old v2 in place, at the same address, and the comparison that made us do it is at the end of this section.
| Evaluation set | typical-small | typical-medium |
|---|---|---|
| MMLU-Pro among-K | .316 | .429 |
| ANLI | .512 | .554 |
| BoolQ | .817 | .840 |
| 20 Newsgroups | .549 | .579 |
| SNLI | .903 | .900 |
| MNLI | .867 | .855 |
| JevBench standard Brier | .370 | .224 |
Medium leads by 11 points on MMLU-Pro among-K, 4 on ANLI, 3 on 20 Newsgroups and 2 on BoolQ, and its standard-tier Brier is 39% lower, which is what matters if you intend to threshold on the probability rather than take the argmax. Both inference sets now go the other way — Small takes SNLI by four thousandths and MNLI by a point — close enough to read as level, but the direction flipped with v3.
It is not a clean sweep, and three sets are held off the table above for space rather than for comfort. All three go to Small: CLINC-150 by 4 points, HWU64 by 7, and TREC-fine, the 50-way fine-grained topic set, by 1. CLINC is in the training mix anyway, so read that row as capacity and not as evidence about unseen labels. All three are on the research page.
One matched pass at the capped protocol covers both shipping checkpoints. On held-out DecisionMix-v2 families, against a .384 floor, Small reads .820 and Medium .889. On level-7 composition, the families our generator never produces, they read .499 and .540 against a .460 floor — four and eight points, where every in-distribution family is forty above its own. And on held-out yes/no, against a .583 floor, Medium reads .858 and Small reads .705. That gap is the one that should decide it: if your decisions are mostly yes/no at 1.7B, Medium is 15 points ahead and it is the pick.
Small's v3 checkpoint also brings a full-file pass of its own, at the protocol the superseded 1.7B was published under, so the two are directly comparable. Against that baseline it is level on yes/no — .705 against .715, under a point apart — where the v2 checkpoint shipped 7 points down and this page said so. It gains 4 points on PagerDuty, .857 against a .792 floor. The one number that went backwards has come back.
Take Medium for accuracy, Small for cost, and do not use the benchmark column to decide. If you are choosing between these two for real work, the number you want is a paired test on your own decisions, not a leaderboard row.
It is the fair question, and the answer is not that we are fast. It is that the two years are already in the backbone, and we did not have to spend them again.
Almost everything a decision model needs to know already sits inside a pretrained backbone: what a refund is, what a termination clause looks like, that “charged twice” and “duplicate charge” are the same complaint. Somebody spent those two years and a great deal of money putting it there. We took Qwen3-1.7B-Base and Qwen3.5-4B-Base frozen, cut the stack at about 71% of its depth, and trained a head to read what was already in the hidden states.
LoRA at rank 16 on the top eight kept layers. Everything below is frozen and everything above is never computed at all. There is no pretraining here, no reward model, no preference data, no reinforcement learning. The thing being learned is a readout, and a readout is small.
$121.7B, H100 time
$154B, H100 time
88training runs in six days
That is the whole trick. The expensive part was paid for by whoever trained the backbone, and what sits on top of it is small enough to train again from scratch in an afternoon when the first attempt is wrong.
A decision model that memorises "this kind of ticket gets that label" is useless the moment your policy changes. So a large share of the workflow training is counterfactual: the same state and the same option list appear under different rubrics with different correct answers, which forces the model to read the rule instead of pattern-matching the state.
It is the largest training-distribution effect we measured, and it is larger than anything we changed in the architecture. Adding the rule-engine corpus took held-out rubric-flip accuracy, where the rule is inverted and the state left alone, from .477 to .710. Held-out families went .481 to .827, held-out grammars .507 to .881, held-out styles .491 to .888. Same head, same steps, same backbone. Only the mixture changed.
Write the rule into the state. A changed policy does not need a retrain, it needs a different paragraph.
It works inside the rule grammar we generate and it stops at the edge of it. On level-7 composition, which mixes temporal, unit, expected-value and trade-off reasoning and which our generator never produces, the same change moved accuracy from .477 to .498 against a .441 uniform-guess rate. The curriculum that moved every in-distribution family by thirty points moved that one by two. What the generator produces is learned. What it does not produce is not.
Those four figures are from the mixture ablation, which is what isolates the effect. The shipping checkpoints carry it through: held-out rubric-flip reads .730 on typical-small and .740 on typical-medium, against a .400 floor. Shuffle the rubric and leave the state alone and they fall to .394 and .392. That is under the floor, which is what reading a rule you were not given looks like.
Each of these started as a bug or a failed run, and each one cost more time than the thing it taught us.
The first full-scale version read the backbone's last hidden state, which is what you do without thinking about it. The control that caught it was blanking the premise and re-scoring: a model that has learned entailment should collapse, and this one did not. It scored 58.6 on SNLI, which is the hypothesis-only baseline — the number you get by ignoring the premise entirely and guessing from the sentence you are asked about. The last layer had specialised into predicting the next token and had stopped carrying the evidence. Moving the tap to layer 20 of 28, so the state and the query interact inside the pretrained layers rather than above them, took SNLI from .863 to .906 on a matched run, MNLI from .752 to .865, and CLINC-150 from .740 to .862. Going from a 0.6B backbone to a 1.7B one had bought nothing until the tap moved.
The candidate-blind architecture scored above chance, and the shuffled-question control killed it. Replace the real question with an unrelated one, hold the candidate set fixed, and the listwise teacher loses .102 of accuracy while the students lose between −.026 and +.023, in both directions, at 1,200 items. Their entire above-chance score was candidate-set priors: which option strings look plausible together. Eight probes instead of one decision vector, cross-attention over the candidate tokens and multi-set supervision all failed to move that number, and multi-set training produced exactly .000. Rendering the options into the suffix, so the candidate text passes through the same pretrained layers as the question, produces .127 at 1.7B and .194 at 4B from the same backbone.
Rendering "none of the above" as an extra option makes the abstention rate a pure function of K. On CLINC with the gold label removed, the model abstained correctly .02 of the time at K=5 and 1.00 of the time at K=150. Not more often: always, on every item, which is a model that has stopped answering. AUROC read .97, .94 and .91 across that sweep and hid the pathology, because a model that abstains on everything ranks every positive above every negative. Moving abstention into its own head took MMLU-Pro false abstention from .692 to .003 and Banking77 at all 77 labels from .063 to .579, and it cost 2 to 7 points on intent and topic label spaces. A null score that means something different at every candidate count cannot be thresholded, and a threshold is the entire point.
The serving path encoded the state once and then deep-copied the entire prefix cache on every single decision, because mutating a shared cache is the kind of bug you only find in production and the copy made it impossible. Replacing the copy with a stride-0 view is bit-identical and allocates nothing, and with cached attention masks and position tensors it took warm p50 on the 1.7B from 21–27 ms to 15.5–17 ms; the Qwen3.5-4B stack typical-medium runs on went from 42–53 ms to 34–46 ms. A quarter to a third of serving latency, for a change that removes code. We tried torch.compile and CUDA graphs on top of it and rejected both: 8 ms on a single shape, but probabilities moved by up to .1 across shape buckets.
The controls behind all four, along with the ones that came back null, are in the research notes.
pip install typical-ai
from typical_ai import Typical
m = Typical.from_pretrained("OzLabs/typical-medium", device="auto")
m.choice(state, "What does the customer want?", ["refund", "replacement", "repair"])
m.noul(state, "Is the order still under warranty?")
m.score(state, "How urgent is this ticket?", ["0", "1", "2", "3"])state is a string or any JSON-serialisable dict. The package is typical‑ai on PyPI, Apache‑2.0, Python 3.11 or newer. The import is typical_ai: PyPI’s bare typical is an established typing toolkit and is not us.
Typical makes bounded decisions in one pass. Some decisions need more: dates to compare, quantities to combine, trade-offs to weigh. Every model we've trained still struggles with them. So the next question isn't only how to make a decision model more accurate.
When should a decision model decide, and when should it think?
The interface category is similar. Typical is open-weight and self-hostable. Jev's internal architecture isn't public, so we don't claim architectural equivalence. Jev currently scores higher, especially on the hardest decisions.
No. A classifier has a fixed output head. Typical's options arrive as text at call time, can change on every request, and take part in the decision pass itself.
You can. Typical is trained around the decision interface itself: typed probability semantics, explicit abstention, one state encoding shared across questions, and a direct readout instead of sampled tokens.
For bounded decisions it can apply evidence from the state and rules given at runtime. Hard multi-step composition is still weak. For genuinely serial reasoning, use a generative reasoning model.
No. The demo supplies a text description of the game state.
Good open base models at the sizes we wanted, in one family we could test consistently. typical-medium uses Qwen3.5-4B.
The per‑source training table and the exact flags are on each model card: typical‑small and typical‑medium.