Jev-Style Decisions from Local Models: Turning a Chat Model into a Decision Model

Here is something I noticed while building automation with language models: most of the time, I don’t actually need the model to write anything. I need it to decide. Which team gets this support ticket? Is this answer safe to send? Does this document belong in category A or category B?

A chat model can do all of that. But it answers in free text. Then I have to parse that text, the text can be malformed, and I have no idea how sure the model was. When I started, I thought the first two problems were the reason to do it differently. After measuring, I think the third one is: a decision is only useful in automation if you know when to trust it.

So this article is about reading decisions directly from a local model, and about what that really buys you, in accuracy, in trust and in speed:

1. What Jev is, and how a decision can be read from any local model.
2. How we do it in geistr, and what the community found.
3. Whether the decisions are right, measured on JevBench.
4. Whether you can trust the model’s confidence.
5. Whether the model can think only when it is unsure, and leave the clear decisions to the fast readout.
6. How fast it is, from reading the prompt to writing the answer.

Let’s get into it.

1. What is Jev?

Jev comes from a company called TypeSafe. They call it a “System One” model and describe it in one line: “unstructured state in, typed probabilistic decisions out”. You define the question and the allowed answers first; Jev returns a choice, a score or a yes/no probability. Jev is a hosted API in early access. TypeSafe has not published its weights, so you cannot run it yourself.

The independent developer guide at jevai.dev lists three output types: Choice (one option from a fixed list, with a probability per option), Score (a place on an ordered scale) and Noul (“a number from 0 (no) to 1 (yes) for a yes/no question”). Because the answer space is fixed before the call, the result always has the type you expect. But, as the same guide points out, “typed does not mean correct”.

2. Reading a decision instead of writing one

Think of a multiple-choice exam. You don’t ask the student to write an essay and then search the essay for the answer. You look at which box they tick.

A language model already does something like ticking boxes. At every position it computes a probability for every possible next token. So if the prompt lists the options with short keys, A, B, C, and ends exactly where the answer begins, the model’s probabilities for those keys are the decision. The model reads the prompt once, and nothing is generated. Two things follow:

• The answer can never be malformed. It is always one of the options.
• You get a probability per option for free. It tells you how sure the model is, which a written answer does not. Keep that in mind: it will turn out to be the most useful part.

The model itself is unchanged, so any open instruction model can be used this way. That is what makes the idea attractive for local hardware.

Quick recap. Instead of letting the model write, read its probabilities for the option keys where the answer would start. The result is always valid and comes with a probability per option.

3. How we do it in geistr

geistr is our open-source program for running language models on your own computer, built on our inference engine geistlib. You can install it from GitHub.

Decisions are an experimental feature. geistr decide renders the prompt in the model’s own chat template, with a fixed instruction, the optional context, the question and the options keyed A to Z, and stops right where the answer would start (for thinking models, with an empty thinking block). It checks that every key is exactly one token there and refuses the request otherwise; it never guesses. Then the engine reads the prompt once, and geistr turns the logits of the key tokens into probabilities over the options.

$ geistr decide gemma4-e2b "I was charged twice. Which team handles this?" billing technical sales
billing
  billing    1.00
  technical  0.00
  sales      0.00

In a terminal, geistr shows each option’s probability; piped into another program, it prints only the chosen option. In a chat, /decide question | option | option does the same, and the C API and the Python package return the probabilities. It works with every chat model geistr runs. In this article I use two: Gemma 4 E2B (Q4KM), a small model, and Bonsai 2 27B (ternary, PQ2_0), a large one.

What the community found

Since Jev launched, many people have tried to rebuild it in the open. The project list on jevai.dev shows two approaches: training a decision model (more than twenty projects fine-tune an open model, mostly Qwen3.5 between 0.8B and 9B; others train small encoder decision models, such as one on ModernBERT), and reading decisions from an unchanged model, as AnyJev (Nokia Applied Research), local-jev and geistr do. Many measure themselves on JevBench, an independent benchmark with 231 public items in three tiers. These are their own numbers, not reproduced by us:¹

Three things stand out. Readout alone gets close: an untrained 4B model is about six points behind Jev. The order of the options changes the answer: AnyJev reports that reversing the options flips 23.0 % of Qwen3-8B’s answers, and that reading every rotation and dividing out the model’s preference for some labels raises accuracy from 74.7 % to 80.3 % without a single label. Trained models can win, but the board moves fast: JevBench’s score blends accuracy, calibration, speed and cost. On its board of 27 September, three trained open 4B models ranked above Jev; under the changed method of v1.5.4, Jev 1.13 is ahead of those three again, and two systems built on a 12B model lead.² TypeSafe released Jev 1.14 on 14 October with only its own internal measurements and no JevBench result; every Jev number in this article is for 1.13.0.

So which route? Training gives the best model for one kind of decision, but it needs data and compute for every new base model, can overfit a public benchmark, and for geistr would mean new architecture (geistlib runs no encoder models). Reading out needs no training, works on any model on the day it is released, and uses the same weights for chat and decisions, which matters on a small device; its weaknesses are bias and overconfidence. For geisten, the readout is the foundation.

4. Are the decisions any good?

I ran JevBench’s 231 public items through geistr decide with both models and let JevBench’s runner and scoring method compute every number. Each item maps to one decision: instructions become the question, the item’s state the context, its labels with their criteria the options (yes/no items become no and yes, score items 0 to n). This is the raw readout: one prefill per item, no option rotation, no calibration.³ The raw data, the adapter and the method are in benchmarks/jevbench_geistr_2026-10-14/.

The Jev row is the published number from local-jev’s README. ECE is the expected calibration error: how far the stated probabilities are from the actual hit rate (lower is better).

• Bonsai 2 27B almost matches the hosted Jev on the public items: 197 of 231 against 200, with no training. And its probabilities are honest: of the 146 items where it was at least 90 % sure, it got 99 % right.
• Gemma 4 E2B is overconfident. It is at least 99 % sure on 85 % of the items, and right on only two thirds of those. Easy items are no problem; the hard ones are.
• What this does not show: these are only the public items (the official board also uses sealed ones), we cannot rule out that the models saw them in training, and the comparison with Jev compares different runs and machines.
• Under JevBench’s current time limit, Bonsai would score lower. Since late October the board counts a decision over 10 seconds as wrong. On this Mac, 50 of Bonsai’s 231 readouts took longer, all of them hard items; under that rule its 85.3 % becomes 70.6 %. Gemma never took longer than 7 seconds.⁴

5. Can I trust the confidence value?

Reading the decision does not make a model smarter. Asked in chat for exactly one label, the same models were about as accurate: Bonsai 84.4 % against 85.3 %, Gemma even 68.8 % against 61.9 %.⁵ What only the readout adds is a probability per option. If I can trust it, I can automate the sure decisions and send the rest somewhere else, for example from a small model to a larger one.

To measure that, I use one number throughout: automatable at ≤ 5 % error, the largest share of decisions, taken from the most confident down, whose error rate stays at or below 5 %.

Read the chart from left to right: the further a line stays flat at the bottom, the more decisions you can hand over without checking them.

For Bonsai, yes. It was at least 99 % sure in 42 % of the items and right in every one of them. Three out of four decisions can go through with at most one error in twenty.

For Gemma, no. A third of its “99 % sure” decisions were wrong. The two models differ by 23 points in accuracy, but by a factor of three in how much you can safely automate. Conformal prediction (Kumar et al.) gives the same picture: aiming to contain the right answer 90 % of the time (reached: Bonsai 91 %, Gemma 88 %), Gemma has to leave 2.1 of its 3.7 options open on average, Bonsai 1.2.⁶

But Gemma is not useless. Its own number cannot tell you when to trust it, but the kind of question can. JevBench’s tiers differ in how much work it takes to get from the text to the answer. Easy: the text names it (“The mug I received arrived smashed into pieces.” Intent? report_damage). Original: a short text and one rule to apply. Hard: a longer policy with several rules, exceptions and amendments to combine.

Gemma is reliable when it only has to recognize the answer in a short text. As soon as it has to apply a rule, it is wrong in a third of the cases, on long documents in more than half, and still about 95 % sure. The form of the answer does not matter (choice and yes/no questions came out the same, 62.6 % and 62.2 %); the thinking the question needs does. This matches the literature: after RLHF tuning, token probabilities are poorly calibrated and typically overconfident (Kadavath et al., Tian et al.), and 4-bit quantization makes calibration worse still (Proskurina et al.).

Make the question easier first

Gemma is reliable when it only has to recognize the answer. So why not rewrite a question until the answer is in the text? Because that rewriting is the thinking. The question is who does it: code (dates, thresholds and lookups computed into the input), a split into recognition questions whose answers code combines, or the model itself, writing notes first. Only the last can be measured fairly on JevBench. So Gemma wrote at most 256 tokens of notes,⁷ and then geistr decide read its decision:

This is the one change that makes Gemma clearly better: +14.3 points,⁸ on the original items (48 to 66) as much as on the hard ones (47 to 63), and above its own chat answer. It is still overconfident, though: what it keeps at p ≥ 0.99 is wrong in 19 % of the cases. The price is about 4 seconds per decision instead of under one, the same trade as thinking on a smaller scale.

Quick recap. A good model’s confidence can be trusted: Bonsai was never wrong at p ≥ 0.99. A small model’s cannot: Gemma is sure of answers it gets wrong as soon as it has to apply a rule. What does make Gemma better is to let it write notes first.

6. Think only when unsure

Thinking makes a model more accurate, but it is slow: a thinking answer takes seconds to minutes, a readout a fraction of that. If the model’s probability can be trusted, it can decide for itself when to think: take the readout where it is sure, and think only where it is not.

Bonsai’s probability can be trusted (section 5), so I tried exactly that. Its 85 decisions below p = 0.9 (37 % of the items) went back to Bonsai in chat mode:

Asking again without thinking adds nothing. With thinking on the uncertain ones and the readout for the rest, Bonsai got 228 of 231 items right (98.7 %), against 85.3 % for the readout alone. Thinking on the other 146 items as well changed not a single answer:⁹ the confidence picked exactly the decisions that needed it. The community saw the same direction: Tacit-9B passes 15.2 % of the items to its own reasoning and goes from 82.3 % to 88.7 %. One run on public items cannot rule out that the model has seen them, so I would not compare this number with Jev’s.

In geistr (0.4.0 or later), a threshold does this on the command line. Below it, decide either leaves the decision open (exit code 3, so a script can hand it to a person), or thinks:

geistr decide bonsai2-27b-pq2 "Which team handles this?" billing technical sales \
  --context "$TICKET" --threshold 0.9 --escalate chat

Pick the threshold on your own labelled examples, log the probabilities, and cap the share that goes to thinking, so that a flood of unclear inputs cannot stall the system.

What does not work: a small model first. The obvious variant, a small model deciding and a large one taking over where it is unsure, needs the small model’s “unsure” to be reliable, and Gemma’s is not (section 5). Tested on the stored runs, Gemma’s probability below 0.99 passed on only 15 % of the items, and what it kept was still wrong in a third of the cases. Only with three extra warning signs (option orders that disagree, a long prompt, a written chat answer that disagrees) did the combination come close to Bonsai’s accuracy (84.4 % against 85.3 %; I chose those signals on the same items, so on new data it will be worse), and on one machine it saved no time:¹⁰

A small model first could only pay off where the second step is much more expensive than both models here, such as a paid API or a person, and even then only with a trigger better than its own probability.

Quick recap. A model whose confidence can be trusted picks its own hard cases: Bonsai thinking on just its uncertain 37 % went from 85.3 % to 98.7 %, the same as thinking everywhere. An overconfident small model cannot do that for a large one.

7. How fast is it?

Speed is the other half of the promise: a decision that needs no written answer should also be faster. Here is what I measured, and where the time goes.¹¹

Reading is only a little faster than a short answer. Writing costs one step through the whole model per token, and a readout writes nothing. But a chat model told to answer with only a label writes a handful of tokens; the time goes into reading the prompt, which both modes have to do:

The large speed-ups often shown for decision models compare against a model that explains itself in hundreds of tokens. Asking for a label only avoids most of that.

Where the readout saves time: deciding where to think. Thinking only below p = 0.9 gave the same 228 of 231 as thinking everywhere. How much time that saves depends on how many decisions are uncertain:¹²

On JevBench, built to be hard, the saving is small: the uncertain decisions are the long ones, and an escalated prompt is currently read twice. On a workload where most inputs are routine, the readout pays off quickly.

Several questions, one read. Jev answers several questions about the same input in one call. geistr can do the same, so far as an experimental build option:¹³ geistr decide-batch --shared-context reads a document once and asks each question on top of it. On eight long JevBench documents, two to four questions took 224 seconds this way and 544 seconds one by one, 2.4 times less.¹⁴ It also makes asking the same question in several option orders affordable.

Can the decision be read before the thinking is done? That is the readout idea applied to thinking: read the decision from the thinking so far and stop when it is clear. Recent research says reasoning models often know their answer long before they stop writing. Goodfire read the answers of DeepSeek-R1 and GPT-OSS-120B during their thinking, with trained probes and by forcing an answer. On easy questions the answer was readable almost from the start; on hard ones it formed gradually. Stopping when the probe was 95 % sure saved DeepSeek-R1 68 % of the tokens on easy questions (MMLU) and 33 % on hard ones (GPQA-Diamond), keeping over 95 % of the accuracy (GPT-OSS-120B: 61 % and 26 %; the arXiv version reports other operating points) (Goodfire, “Reasoning Theater”, 2026, arXiv). In math, a trained probe checking the model’s own intermediate answers saved 24 % (Zhang et al. 2025). A cost-aware study of stopping rules that, like mine below, work without the model’s hidden states adds two cautions: on Qwen3’s multiple-choice runs (MMLU-Pro) a calibrated confidence threshold beat the authors’ learned stopper, and the saving depends on how the model is served. On GSM8K with Qwen3-8B, the same rule saved 32 % of the tokens when the thinking stays in the cache and cost 121 % extra when it is read again for every check (Dong et al. 2026).

I tried the untrained version, the forced answer, on Bonsai: cut its thinking at a fraction of its length, close it, and read the decision.¹⁵ On 20 decisions Bonsai was sure about, the answer was there before the first word of thinking (20 of 20 right, at p = 0.999): the easy case from the research, and exactly what the readout already uses. On the 85 uncertain ones:

The answer forms late, as for the hard questions in the research, but the saving is smaller: keeping about 95 % of the accuracy saves about a quarter of the writing, against a third for Goodfire. Their trained probes, my selection of the hardest third of the items and the much smaller model may explain that. For decisions, the two methods add up: the readout sends only the hard cases to thinking, and reading along saves another 15 to 25 % of the writing on those, at the cost of a few answers.

Summary

• The readout gets a Jev-style decision from any local chat model: read the probabilities of the option keys where the answer would start. geistr does this as an experimental feature, for every model it runs.
• Accuracy: on JevBench’s public items, Bonsai 2 27B gets 85.3 %, against 86.6 % published for the hosted Jev; Gemma 4 E2B gets 61.9 %. Asked for one label, the same models do as well (Gemma even better, 68.8 %). Under JevBench’s new 10-second limit, Bonsai’s slow long documents on this Mac would count as wrong (70.6 %).
• Trust: what only the readout adds is a probability. Bonsai’s can be trusted: 74 % of its decisions can be automated at ≤ 5 % error. Gemma’s cannot. Letting Gemma write notes first raised its accuracy from 61.9 % to 76.2 %.
• Think only when unsure: a model with trustworthy confidence picks its own hard cases. Bonsai thinking only on its decisions below p = 0.9 got 228 of 231 right (98.7 %), as many as thinking everywhere. A small model first does not work: its confidence is not reliable enough to hand over.
• Speed: a readout is only a little faster than a one-label chat answer (Bonsai: 2.1 against 2.7 seconds). The gains come from thinking only where needed (30 % less time at 20 % uncertain decisions, by estimate), from several questions over one read (2.4 times faster), and from reading the decision during thinking (about a quarter of the writing saved on hard cases, at 94 % of the accuracy).

If you try this on your own hardware, or if you work on decision models yourself, I would love to hear what you find.

Sources

• TypeSafe: Introducing System One models and Jev, Jev 1.14
• Jev AI Dev (independent guide): What is the System One AI model?, AnyJev and Projects
• Nokia Applied Research: AnyJev (README, results of 22 September 2026)
• amithgc: local-jev (README, JevBench results)
• Benchmark Heaven: JevBench, methodology (v1.7, 10-second rule); benchlm.ai JevBench table
• geistr: source and installation on GitHub
• Calibration and confidence: Kadavath et al. 2022, Tian et al. 2023, Proskurina et al. 2024, Kumar et al. 2023 (conformal prediction for multiple choice)
• Reading answers during reasoning: Goodfire, Reasoning Theater (2026) (arXiv 2603.05488), Zhang et al. 2025 (Reasoning Models Know When They’re Right), Dong et al. 2026 (a cost-aware study of early exits)
• Our JevBench runs: benchmarks/jevbench_geistr_2026-10-14/
• geist-runtime: docs/DECISIONS.md, src/decision_profile.c