What it is
A 42-claim benchmark testing whether a cited passage supports a sentence as written, run against Jev and three frontier models, with raw outputs and scored results.
How it uses Jev
Jev answers one Choice question per claim: given a sentence and its cited passage, decide supported, unsupported, or not_addressed. Its typed answer and probability are used as a publishing-gate pre-filter, escalating low-confidence verdicts.
Primitives:choice
Technique worth stealing
Score calibration and risk-coverage separately from headline accuracy, since frontier models put all answers in one confidence bucket.
Try it
bash bench.sh --system jev --tier A, or python3 risk_coverage.py results/*.jsonl (no keys needed).
Evidence
Each line is one question put to Jev about the README. ≥ 0.60 reads as yes, ≤ 0.40 as no; in between Jev is not making a call.
- Jev-centricunclear0.47
- Shows a System One patternno0.18
- Handles uncertaintyno0.07
- Measuredno0.39
- Runnableno0.05
- Worth recommendingno0.30
- Model replicano0.05
- Problem scopescore on a 0–2 scale1.06
- About Jevyes0.98
Signals by Jev jev-1.13.0, card written by DeepSeek V4.1 Flash from the README on 21 Sept 2026.