What it is
Tests whether three local Ollama models can judge hostile-to-love-language rewrites on PoL criteria, using fully crossed judging, self-preference correction, and calibration. Validation labels are Claude-generated, not human.
How it uses Jev
This project does not call Jev. It reproduces Jev's Choice/Score/Noul pattern in jev_local.py using Ollama logprobs: the judge emits a single option token, read from top_logprobs, with forward/reversed scales and negated Noul statements to expose position bias.
Primitives:choicescorenoul
Technique worth stealing
Order-reversed and negated re-asks on logprobs to expose position bias, with coverage flagging out-of-vocabulary probability mass.
Try it
python run.py validate --run flowtest --tuned --set holdout; python run.py analyze --run flowtest --tuned --quick
Evidence
Each line is one question put to Jev about the README. ≥ 0.60 reads as yes, ≤ 0.40 as no; in between Jev is not making a call.
- Jev-centricno0.11
- Shows a System One patternunclear0.41
- Handles uncertaintyno0.03
- Measuredno0.10
- Runnableno0.07
- Worth recommendingno0.03
- Model replicaunclear0.47
- Problem scopescore on a 0–2 scale0.57
- About Jevyes0.89
Signals by Jev jev-1.13.0, card written by DeepSeek V4.1 Flash from the README on 22 Sept 2026.