What it is
A frozen-protocol benchmark comparing Jev 1.13, PrismNLI-0.4B and Laya on four public English classification datasets, with supervised reference models and a label-efficiency study for practitioners deciding between zero-shot and labelling.
How it uses Jev
Jev is reached through the OpenRouter Decisions API and given only the raw text, one instruction and the label strings. Its returned probabilities are used for accuracy, macro-F1 and ECE scoring against the other systems; no training or tuning is applied.
Primitives:choice
Technique worth stealing
Freeze the protocol before inference and report raw, unfitted ECE alongside accuracy.
Try it
Run benchmark.py; see results/summary.json and REPORT.md.
Evidence
Each line is one question put to Jev about the README. ≥ 0.60 reads as yes, ≤ 0.40 as no; in between Jev is not making a call.
- Jev-centricunclear0.46
- Shows a System One patternno0.16
- Handles uncertaintyno0.06
- Measuredno0.32
- Runnableno0.05
- Worth recommendingno0.23
- Model replicano0.05
- Problem scopescore on a 0–2 scale1.48
- About Jevyes0.98
Signals by Jev jev-1.13.0, card written by DeepSeek V4.1 Flash from the README on 20 Sept 2026.