What it is
A preregistered experiment and teaching bench measuring whether a frozen Qwen3-4B can answer closed-choice schema fields by reading next-token logits for option letters, instead of generating grammar-constrained JSON. Task: CLINC150 intent classification.
How it uses Jev
Each closed-choice field becomes a lettered multiple-choice question; one forward pass reads the model's scores for the option letters at the answer position. No token is generated for choices. Strings and numbers are still generated under a grammar. Results are compared against JSON generation.
Primitives:choicenoul
Technique worth stealing
Give the model a one-token identifier (a letter) to answer with, then read the logits instead of decoding.
Try it
uv sync; MINIJEV_DEVICE=mps uv run python demo/server.py (or cuda), then open http://127.0.0.1:8765/.
Evidence
Each line is one question put to Jev about the README. ≥ 0.60 reads as yes, ≤ 0.40 as no; in between Jev is not making a call.
- Jev-centricyes0.81
- Shows a System One patternunclear0.41
- Handles uncertaintyno0.07
- Measuredno0.18
- Runnableno0.12
- Worth recommendingno0.22
- Model replicayes0.93
- Problem scopescore on a 0–2 scale0.74
- About Jevyes0.97
Signals by Jev jev-1.13.0, card written by DeepSeek V4.1 Flash from the README on 20 Sept 2026.