What it is
A benchmark harness measuring whether SQL ORDER BY over Jev probabilities yields a defensible order, for developers building semantic sort on Jev. It reports pairwise inversion, Score ordinality, calibration, wording invariants and sort-key ties against pre-registered gates.
How it uses Jev
Jev supplies the probabilities that are sorted: jev_bool, jev_choice and jev_score are called on 360 labeled rows and on Amazon ESCI query-product pairs, and the returned values are measured for inversion, ordinality, calibration and ties rather than used to make a product decision.
Primitives:choicescorenoul
Technique worth stealing
Pre-register gate thresholds, then add a paraphrase control so a negation result cannot be overclaimed as negation-specific.
Try it
python3 harness/run_calibration.py with a key; --analyze-only recomputes from cache.
Evidence
Each line is one question put to Jev about the README. ≥ 0.60 reads as yes, ≤ 0.40 as no; in between Jev is not making a call.
- Jev-centricyes0.81
- Shows a System One patternno0.26
- Handles uncertaintyno0.04
- Measuredyes0.94
- Runnableno0.09
- Worth recommendingno0.29
- Model replicano0.05
- Problem scopescore on a 0–2 scale1.09
- About Jevyes0.99
Signals by Jev jev-1.13.0, card written by DeepSeek V4.1 Flash from the README on 20 Sept 2026.