jevbooks

← All projects

typesafe-ai-benchmark

iammrduncan/typesafe-ai-benchmark · Homepage

Side-by-side benchmark of Jev vs Qwen on structured judgment tasks.

What it is

An LLM gateway and benchmark comparing TypeSafe Jev with Qwen 3.8 27B on Cerebras across seven synthetic workloads, recording latency, cost, and judgment quality for developers evaluating structured-output approaches.

How it uses Jev

Jev handles one application decision per request by batching native Choice/Noul questions, mapped to the same application output as Qwen's schema-constrained response. Results are validated before simulated actions; native probabilities remain in exports.

Primitives:choicenoul

Technique worth stealing

Compile application fields into compact numeric slots and batch native typed questions instead of one joint schema response.

Try it

npm ci; set CEREBRAS_API_KEY and JEV_KEY in .env; npm run build; npm run start:demos:live

View on GitHub

judged by Jevjev-1.13.0

Evidence

Each line is one question put to Jev about the README. ≥ 0.60 reads as yes, ≤ 0.40 as no; in between Jev is not making a call.

  • Jev-centricno0.31
  • Shows a System One patternno0.21
  • Handles uncertaintyno0.07
  • Measuredno0.17
  • Runnableno0.19
  • Worth recommendingno0.24
  • Model replicayes0.62
  • Problem scopescore on a 0–2 scale1.17
  • About Jevyes0.97

Signals by Jev jev-1.13.0, card written by DeepSeek V4.1 Flash from the README on 20 Sept 2026.