jevbooks

← All projects

jev-typed-evaluation-collapse

mleyvaz/jev-typed-evaluation-collapse

Case study on Jev's typed evaluation collapse via six experiments

Evaluation & Benchmarking76%Runner-up: Calibration & Research

What it is

A field note and manuscript studying Jev (TypeSafe AI), a commercial 'System One model', through six experiments run via the Vercel AI Gateway, targeting the NCML journal.

How it uses Jev

The README does not specify the state Jev receives or how results are used in code. It states Jev produces different output patterns depending on the declared question type and schema: collapsing with boolean/Noul, separating with Choice, and reconstructing LPA2v by composing two boolean calls.

Primitives:choicescorenoul

Technique worth stealing

Composing two independent boolean calls (μ/λ) reconstructs the LPA2v lattice without the model computing it internally.

Try it

cd experiment && npm install; export AI_GATEWAY_API_KEY=vck_...; node --env-file=.env.local run_experiment.mjs

View on GitHub

judged by Jevjev-1.13.0

Evidence

Each line is one question put to Jev about the README. ≥ 0.60 reads as yes, ≤ 0.40 as no; in between Jev is not making a call.

  • Jev-centricunclear0.55
  • Shows a System One patternno0.15
  • Handles uncertaintyno0.09
  • Measuredno0.12
  • Runnableno0.04
  • Worth recommendingno0.18
  • Model replicano0.07
  • Problem scopescore on a 0–2 scale0.98
  • About Jevyes0.96

Signals by Jev jev-1.13.0, card written by DeepSeek V4.1 Flash from the README on 20 Sept 2026.