Source walkthrough
Vision-Jev source walkthrough: captions, decisions and failure paths
The useful split is concrete: the browser produces a caption, Jev chooses typed labels, and game code executes speeds and timers. This walkthrough follows that boundary at a fixed commit; it does not test Jev’s direct image input.
1. Follow one image to an action
Pipeline.prefetch starts captioning while an image is dragged; captions are cached by image URL, and failed entries are removed. perceive sends a bitmap to a browser worker. That worker serializes captions and loads Florence-2-base-ft at revision e88a44ea, using the CAPTION task at 384 × 384 pixels. WebGPU model-load failure falls back to Wasm; a later warm-up or caption failure is reported as an error.
On drop, Pipeline sends {caption, zone} to /api/decide. The server turns these into object_seen and a location sentence, then asks action (7 labels), category (10) and speed_limit (5) in one Jev call. For example, a caption describing a person in own_lane gives Jev a lane-specific decision; the game, not Jev, implements the selected stop or speed change.
2. Input validation and fallback solve different problems
The Worker requires a session, limits request bodies to 8 KiB and accepts only caption plus one of four zones. Captions are at most 300 characters and are flattened to one line; unknown fields are rejected. The Jev adapter validates label membership and confidence in [0,1]. Missing or invalid per-label probabilities become zero; it does not check that probabilities sum to one or that the chosen label is their maximum.
Caption or request failure reaches a deterministic fallback: stop in own_lane, slow_down in oncoming_lane, continue on either sidewalk. A valid but low-confidence answer follows its selected action: resolveBehavior has no confidence threshold. change_speed plus speed_limit=none keeps the speed. Typed output therefore prevents an unknown action label, but does not make a mistaken caption or action safe.
3. Cancellation and timing need separate measurements
A new decision aborts the previous request for that object; the engine also checks requestSeq before applying an answer. That sequence check protects against a stale response after the object moves. It does not prove the server stopped work or avoided billing when a browser request was aborted.
Each /api/decide attempt gets a 10-second timeout after session verification. A 401 triggers one session refresh and retry, so this is not a 10-second end-to-end ceiling. visionMs measures caption execution, excluding queue and model loading; latencyMs measures ai.run; browser roundTripMs includes verification and retries. Add these stages deliberately when measuring reaction time.
4. What the author’s 151/156 measures
The README reports 151 acceptable actions among 156 scored caption/location cases, with four unscored exploratory cases excluded. Those same cases were used to tune the policy. This is in-sample policy agreement, not a held-out image benchmark or an end-to-end driving-safety result.
The code calls Cloudflare AI with typesafe/jev. The Worker logs result.model, Jev latency and input tokens, but the browser response omits model provenance. If the upstream envelope lacks model, the adapter records the alias itself. A reproduction should save the actual served model when available, the caption-model revision, policy commit, device/backend and separate stage timings; the alias alone cannot establish a Jev version.
Use this boundary in your own prototype
Keep perception, semantic choice and execution independently observable. A useful first check is to feed a known caption into the policy, inject a request error, then return a valid low-confidence action: these exercise three different paths. Test an object move while a request is pending as a separate stale-response case.
README checked on 2026-11-02; source walkthrough added on 2026-11-03 at the same commit. The package declares Node ≥22.18, pnpm 11.0.3 and Transformers.js 4.3.0. We inspected source, but have not run the browser/WebGPU/Cloudflare chain or reproduced the author’s measurements. This is a game-prototype design study.