cu-Jev

What it is

cu-Jev is a CUDA-native inference engine that turns Qwen3.5 checkpoints into a decision function. It provides a Jev-compatible API for sending a state and typed questions, returning probabilities and confidence. For developers needing fast, GPU-resident decision inference.

How it uses Jev

cu-Jev implements the Jev System One API: it accepts a state and typed questions (noul, choice, score), runs the state once, then evaluates each question as a short branch attending to the state's KV cache. It reads logits of option labels to produce calibrated probabilities and confidence, returned in the response.

Primitives:choicescorenoul

Technique worth stealing

Shared state with isolated branches: state KV cache and recurrent state are computed once and reused read-only across all questions.

Try it

uv run cujev serve --model models/Qwen3.5-4B --port 8080

View on GitHub