Source walkthrough
Jev Tetris: candidate choices, code and meaningful controls
This is a useful teaching project for turning model judgments into actions. Its stronger lesson is how to check whether the model adds value: keep geometry in code, inspect the descriptions, and compare the same task with simpler players.
1. Geometry → prose → five questions → a move
The README describes a reachability search over moves, rotations and drops that enumerates legal placements. In describePlacements, code evaluates each placement and writes its consequences using one shared sentence template: newly trapped space, already trapped space, cleared lines, surface shape and stack height. The model receives these descriptions, not a board image.
The composite mode asks five Choice questions over the same candidate IDs: avoid new holes, clear rows, keep existing holes reachable, stay low and stay flat. Source weights are 0.55, 0.22, 0.15, 0.05 and 0.03 respectively. blend combines the distributions in code, then normalizes them; its default is a weighted sum, with a product alternative also implemented.
The README says the game plays the highest-ranked placement. Code still owns legality and execution. Score and Noul questions are also present for board danger and doom; the five Choice questions are the action-selection mechanism, not a claim that the repository only uses Choice.
2. Four controls answer different questions
- Random placements: does the complete policy perform better than an uninformed choice among legal candidates? This is a floor for this game, not an intelligence benchmark.
- Shuffled probabilities: keep model-returned numbers but change which candidate each belongs to. The author reports a collapse toward random play. That tests whether the mapping between candidate and probability contributes to behavior; it does not remove the work already encoded in candidate construction.
- Keyword rules on the same sentences: can deterministic text rules extract the useful features? The author reports that this baseline beats Jev. Because it reads the same descriptions, it is a particularly relevant check on the need for a model here.
- El-Tetris heuristic: compare with a stronger code-based player using game features. This is a practical baseline to beat, not a proven upper bound or an equivalent prose-only model comparison.
3. What the author’s results do not establish
Only three seeds were used, and the questions, wording, blending method and weights were tuned on those same seeds. These are in-sample results, not held-out performance. The author also reports that the blended pick often matches the most heavily weighted “sealed” question; five outputs do not establish five independent contributions.
The author’s replay of 1,578 logged turns makes that concentration measurable: the blended pick equals the sealed question’s own argmax 91% of the time (sealed has weight 0.55). Equal weights reduce the reported mean to about 11.7 lines, versus about 52 as built. These are author-reported ablations on the tuning seeds, not a jevbooks reproduction or evidence that five questions add five independent benefits.
The pinned README says a 300-piece cap; findings says 200 while its table includes 300. Resolve the run configuration and logs before quoting an aggregate ratio. We have not rerun the experiments and do not treat the published latency or scores as current measurements.
For a stronger comparison, hold board states and candidate sets fixed, fit weights on one set and evaluate on another. Then run complete games on unseen piece sequences. Per-turn ranking and survival answer different questions because an early mistake changes every later board.
Initial source review: 2026-11-02. These ablation details were added on 2026-11-03 from the same pinned findings.md; we have not rerun the games.
4. Keep API failure separate from a losing move
The client throws for a missing key, an unsuccessful HTTP response or non-JSON content. readChoice returns null when the answer is missing; otherwise it accepts several field names and defaults missing probabilities to an empty object. That extraction alone does not validate candidate membership or a complete probability distribution.
The README describes a UI test in which a dropped connection stalls play rather than ending the game. That is an author-reported behavior, not an end-to-end result verified here. When adapting the pattern, check that the returned ID belongs to your legal set, check numeric fields, and define an explicit incomplete-decision path before executing an action.
5. Start with a mock, then test one change
The pinned README requires Node 22 or later and says there are no package dependencies or build step. These commands check out the reviewed version and start the interface with fake answers; they are instructions to try, not a record of a run by jevbooks. A mock can exercise the interface, not establish Jev’s playing ability.
git clone https://github.com/thelau/jev-tetris.git
cd jev-tetris
git checkout 9869b602965cf002afff766013f8c068846d36aa
npm run dev:mock- In another terminal inside that checkout, npm test runs the documented engine and mock UI suites; check the repository for browser requirements.
- Run the documented keyword and heuristic baselines before paying for model runs. Fix the seed and piece cap, save logs, and change one thing at a time.
- For real calls, follow the README’s .env instructions. Record the actual returned model, not only the jev-latest alias; model-backed play and shuffle controls incur API usage.
Take this to your own decision problem
If code already knows how to calculate holes, height or a legal position, keep that calculation in code. Use a model where language supplies information your rules cannot cheaply capture. Compare that added value against a direct rule over the same information before building a more elaborate question ensemble.
A useful exercise is to write three legal actions, describe their consequences in the same format, and decide how to handle a missing answer. Compare one Choice question with a simple rule before adding weights. Log the descriptions as well as the returned numbers: the author found a consequential contradiction between two generated description sentences.