What it is
A benchmark racing Jev, TypeSafe's System One decision model, against six general-purpose LLMs on Snake. Each player gets sixty seconds and its own board; the repo measures decisions per minute and cost per point.
How it uses Jev
Jev answers Choice questions about the Snake board state, returning the option, a probability per option, and a confidence. The chosen option is the move. The repo also uses output_shape.py to probe how output tokens scale with question shape.
Primitives:choice
Technique worth stealing
Fix the answer space in the question so output size is constant and no parsing is needed.
Try it
python race.py --seed 1 --seconds 60 --lineup jev,haiku,opus,luna,sol,kimi-fast,kimi --out race_seven.jsonl
Evidence
Each line is one question put to Jev about the README. ≥ 0.60 reads as yes, ≤ 0.40 as no; in between Jev is not making a call.
- Jev-centricunclear0.46
- Shows a System One patternyes0.82
- Handles uncertaintyno0.04
- Measuredno0.10
- Runnableno0.06
- Worth recommendingno0.21
- Model replicano0.10
- Problem scopescore on a 0–2 scale0.86
- About Jevyes0.97
Signals by Jev jev-1.13.0, card written by DeepSeek V4.1 Flash from the README on 20 Sept 2026.