jevbooks

← All design patterns

Blind review

Cross-check · 盲审

The reviewer never sees the answer

Put a second question next to the main one to check it — and never let it see the main answer.

action?done?your code

The problem

You are building a browser agent — a program that drives a web page on its own. You give it a goal, say "put this book in the cart", and at every step it asks Jev which button to click next, Jev picks one of the buttons on the page, and your code clicks it. A step takes 300 ms, so a few dozen steps finish a task.

A flaw shows up quickly: the task is plainly done and it keeps clicking. The book is already in the cart and it presses Add to cart again, so now there are two.

The natural fix is to ask one more question before acting: is the goal reached? To help it judge, you also put in the action you just picked — "I am about to click Add to cart; is the goal reached?" Nothing changes. Reading the logs, you see why: the second question saw the first answer and will only defend it. Having just said it would click Buy, it is not going to admit the task is over.

The solution

The Blind review pattern says: the checking question rides in the same request as the main one, but it looks only at the page — not at the main question, and certainly not at its answer.

So the check asks only about facts on the page, and mentions no action at all: "judging only from this page, has the goal already been reached?", "have the last three steps made no visible progress?" When you write it, deliberately keep out any phrase like "about to click".

Your code reads the check first and the main answer second. Goal reached, stop. Stuck, hand it to a person. Only when neither fired does it click the button the main question chose. The gate sits in front of the action.

Ask · Answer · Act

  1. AskPut in the page text and the list of buttons, and ask three things at once: which button to click next, whether the goal is already reached judging only from this page, and whether the last three steps made any progress.
  2. AnswerClick 'Add to cart' 0.62 (the likeliest of the buttons, not a measure of certainty); goal reached 0.91; stuck 0.08.
  3. ActGoal reached is over 0.85, so the run stops and nothing is clicked.

Where the name comes from

Blind peer review. The reviewer does not know who the author is and does not get to see what the editor thinks, which is the only reason the review counts for anything. Show them "I intend to accept this" first and nine reviewers out of ten will agree with it.

Use it when

The main answer triggers an action — a click, a submit, a stop — and there is a fact on the page that can be judged without knowing the action: the goal is reached, it is stuck, the road is blocked, there really is a match here.

Not when

The check means the same thing as the main question. Requiring "is the implementation complete", "are the requirements satisfied" and "can we finish" to all pass is not triple safety, it is one number asked three times.

Projects that do this

  • jev-browser — Fast, cheap browser automation driven by TypeSafe's Jev model.
  • foreman — Agent supervisor and software factory foreman using decision models.
  • jev-axi — CLI for TypeSafe's Jev: fast calibrated judgments from the shell.

All projects with this pattern →

Read this code
Show me the code
const { answers } = await jev(state, {
  action:    choice("What should the agent do next?", buttons),
  goal_done: noul("Judging only from this page, has the goal already been reached?"),
  stuck:     noul("Have the last three steps made no visible progress?"),
});
if (answers.goal_done.p >= 0.85) return done();             // the check speaks first
if (step > 2 && answers.stuck.p >= 0.8) return escalate();
click(answers.action.choice);                               // only then the main answer
How jevbooks recognises it

How do we know a project does this? We put one question about its README to Jev and it returns a percentage; the higher, the more it looks like it. If you copy the question, copy one thing: ask whether the text mentions this one concrete thing, never whether the project is good.

questions.json · sent for every README
{
  "cross-check": {
    "type": "noul",
    "instructions": {
      "what": "Does the text describe a second question that independently verifies the main answer, such as goal_done or stuck alongside the action choice?",
      "not_for": "A single question, or verification done by code rather than by another question."
    },
    "criteria": {
      "true": "The text explicitly describes this. One sentence is enough; other content does not cancel it.",
      "false": "The text does not describe this."
    }
  }
}

Build this with Jev →