What it is
A reproducibility package for an experiment that asks Jev (TypeSafe AI System One) to judge whether an AI-generated sentence about a local ordinance is actually grounded in the article it cites. For researchers testing a separate scoring gate on generated answers.
How it uses Jev
For each of 120 (article, answer) pairs, run.mjs asks Jev a Choice{지지/모순/무관} plus a Boolean{does the answer hold on the cited article alone}. The Boolean probability is read as 'grounded'; results are aggregated per pair and printed.
Primitives:choicescore
Technique worth stealing
Treat Jev's Boolean probability as a gate: below 0.5 routed to human review, above passed.
Try it
npm install, set AI_GATEWAY_API_KEY, then node --env-file=.env run.mjs (or run.mjs dataset100.json result100.json).
Evidence
Each line is one question put to Jev about the README. ≥ 0.60 reads as yes, ≤ 0.40 as no; in between Jev is not making a call.
- Jev-centricyes0.94
- Shows a System One patternyes0.70
- Handles uncertaintyno0.05
- Measuredyes0.98
- Runnableno0.11
- Worth recommendingunclear0.53
- Model replicano0.06
- Problem scopescore on a 0–2 scale1.10
- About Jevyes0.98
Signals by Jev jev-1.13.0, card written by DeepSeek V4.1 Flash from the README on 20 Sept 2026.