What it is
ASSAY-001 is a pre-registered verification of TypeSafe Jev's calibration and type-safety claims on Banking77 and CLINC150. It reports a split verdict: calibrated on CLINC150, overconfident on Banking77, with zero type errors across 8,576 responses.
How it uses Jev
Jev answers one Choice question per item, with null option descriptions. The chosen-option probability is compared to correctness by a deterministic scorer under a frozen rule. Every request and response is logged and sealed.
Primitives:choice
Technique worth stealing
Pre-register the protocol and hash the corpus before any query, then seal every request and response.
Try it
python3 harness/controls.py; python3 harness/score.py banking77 --sum-tol 0.02
Evidence
Each line is one question put to Jev about the README. ≥ 0.60 reads as yes, ≤ 0.40 as no; in between Jev is not making a call.
- Jev-centricyes0.89
- Shows a System One patternno0.15
- Handles uncertaintyno0.05
- Measuredyes0.83
- Runnableno0.07
- Worth recommendingno0.24
- Model replicano0.04
- Problem scopescore on a 0–2 scale1.22
- About Jevyes0.98
Signals by Jev jev-1.13.0, card written by DeepSeek V4.1 Flash from the README on 21 Sept 2026.