What it is
A RuneBench fork that lets TypeSafe's Jev, a non-generative model, play RuneScape through a bounded action catalog built on rs-sdk. It records tick-level results and provides a live dashboard for developers and benchmarkers.
How it uses Jev
Jev answers typed questions (choice, noul, score) with calibrated probabilities. In tick mode, each poll asks next_action (current or another catalog action) and this_tick (do_nothing, close_dialog, eat_food, restart_current). Jev also answers poll_again_in_ticks. Code executes the chosen action and reports game state.
Primitives:choicenoulscore
Technique worth stealing
Bounded action catalog with tick-mode polling; Jev decides, code executes.
Try it
Run agents/jev/live-stack.sh up ../rs-sdk --skill Fishing --minutes 30 --policy jev --bot skyvern --speed 1 --xprate 1 --mode tick --poll-every 3
Evidence
Each line is one question put to Jev about the README. ≥ 0.60 reads as yes, ≤ 0.40 as no; in between Jev is not making a call.
- Jev-centricyes0.94
- Shows a System One patternunclear0.49
- Handles uncertaintyno0.06
- Measuredno0.08
- Runnableno0.20
- Worth recommendingunclear0.40
- Model replicano0.04
- Problem scopescore on a 0–2 scale0.70
- About Jevyes0.98
Signals by Jev jev-1.13.0, card written by DeepSeek V4.1 Flash from the README on 20 Sept 2026.