Check agent output without rereading every line

Agent output arrives fast and reads fluently, which makes it easy to accept without checking. The problem is not that agents produce nothing useful; it is that a person rereading every line of every artifact becomes the slowest and least reliable part of the pipeline. This page builds up one answer to that problem in four stages: why stacked, independent checks catch more than any single check; how to add a bounded build-review loop to a hank; when to replace an agent reviewer with a fixed, repeatable review surface; and how to repair a production review loop so each run benefits from the last. Throughout, the goal stays the same: decide what evidence is enough to accept a result, and make producing that evidence part of the work itself.

Stack weak checks that fail in different ways#

When one check misses a defect, repeating that check may repeat its blind spot. We can reduce how much a person needs to reread by combining checks that look for different failures: build-review loops, fresh-eyes verifiers, and automated review surfaces. A hank – a program file made of codons, each one an agent task – lets us make those checks part of the work rather than an afterthought. The judgment about what counts as enough evidence still belongs to us.

Put different checks on the same output#

Suppose three checks each catch a problem 80% of the time. If their misses are independent, all three miss it with probability 0.2³ = 0.008, so they catch it 99.2% of the time. One 80% check catches it only 80% of the time. This is an illustration of the arithmetic, not a measured catch rate for the checks in your hank.

The benefit depends on independence. Vary the perspective, not only the model's capability: three different 80% checks can beat one 95% check under that assumption. Changing models or starting fresh does not establish independence by itself. Model output remains stochastic; we use checks to catch and correct defects, not to make output bit-repeatable.

Do not pass blind trust down a sequence#

Avoid relying on continue-previous alone when each step must verify the preceding artifact. That mode preserves the previous codon's full conversation history, including assumptions we may want the next agent to question. In a simplified sequence where all three steps must succeed and each has an independent 80% success probability, run reliability is about 51%. That illustrates the risk of passing work on without checks; it is not a measured failure rate for continue-previous. See codon session semantics.

A dated planning-hank lineup used Gemini 2.5 Pro, Claude Opus, and GPT-5.2 to generate three plans. Opus merged them; Sonnet reviewed the result with fresh eyes; Opus updated it through several rounds. Treat that lineup as dated narrative, not as a model recommendation.

A sentinel – an observer that watches event patterns – supplies a third perspective when the main agent misses one. Together, these checks combine different model perspectives, fresh-eyes review loops, and sentinels. See sentinels for the sentinel contract.

Verification requires fresh context. A verifier that sees the conversation that produced an error can repeat its assumptions. Keep the builder and reviewer separate when that distinction matters, and check the reviewer's findings against the artifact. We work through that separation in designing codons and handoffs. The next section turns that principle into a concrete loop you can add to a hank.

Add a build-review loop to your hank#

A build-review loop separates the generator from its evaluator. Put agents in separate boxes with a context wall between them: one codon builds, another reviews, and only the output crosses the wall. A standalone evaluator is easier to tune for skepticism than a generator asked to criticize its own work, which may confidently praise its own output. The rest of this section walks through the loop's shape, its validation, its runtime evidence, and its failure policy, in the order you would set them up.

Give each reviewer a fresh context#

Set the review codon to fresh, and set the update codon to fresh too. The verifier must not receive the builder's conversation; pass files across the wall instead. Each loop iteration is a complete context wall. See loops for lazy expansion, runtime IDs such as id#iteration, zero-indexed limits, and fresh-in-loop legality.

Add a bounded loop, not an open-ended wish#

The published planning-hank shape uses loop id blind-reviews, the name Repeatedly improve the plan through blind iteration, and an iterationLimit of 3. Its two codons are step-5-blind-review and step-6-update: the review codon uses haiku, fresh context, and codons/codon-5-review/0-review.md; the update codon uses opus and fresh context. Both append codons/system-prompt-broad.md. Treat this as the dated published configuration.

The checked plan-review-update fixture normalizes the codon ids to review and update, uses haiku for both, uses unprefixed prompt paths, and sets onFailure: "retry" on both. Its surrounding plan codon is why validation shows three codons. Use the fixture for the checked shape; the shared system prompt is configuration that both codons append, not the builder's conversation.

The excerpt below shows the loop object from that fixture: the terminateOn limit, both codons with fresh continuation, and the shared appended system prompt.

Captured from the plan-review-update fixture.

      "type": "loop",
      "id": "blind-reviews",
      "name": "Repeatedly improve the plan through blind iteration",
      "terminateOn": {
        "type": "iterationLimit",
        "limit": 3
      },
      …
      "codons": [
        {
          "id": "review",
          "name": "Blind review of the plan",
          "model": "haiku",
          "continuationMode": "fresh",
          "promptFile": "prompts/review.md",
          "appendSystemPromptFile": "prompts/system-broad.md",
          …
        },
        {
          "id": "update",
          "name": "Apply the review to the plan",
          "model": "haiku",
          "continuationMode": "fresh",
          "promptFile": "prompts/update.md",
          "appendSystemPromptFile": "prompts/system-broad.md",
          …
          "onFailure": "retry",
          …
        }
      ]

At 0.10.0, the schema makes this shape real. A loop requires the loop discriminator, id, name, terminateOn, and codons. iterationLimit requires an integer limit of at least 1. Codons may carry model, continuationMode: "fresh", promptFile, and appendSystemPromptFile. The two terminateOn variants are:

contextExceeded is the alternative termination condition when the loop context is exceeded.

Check-it: inspect your loop's terminateOn object and compare its type with the two contract variants above.

Validate before the first iteration#

Treat the hank as the program file, and work from the directory containing hank.json, prompts/, and codons/. At 0.10.0, keep loop prompt files inside that directory and use strict relative paths; see hank JSON references for that contract. Validate before anything runs:

bunx hankweave@0.10.0 hank.json <data> --validate

The validator prints the structure rendering and ends with a GOOD TO RUN! box. The <data> positional is the data source: a directory the first codon reads. The checked fixture passes data/, which contains seed.md. Use the positional grammar it prints; a bare hankweave invocation launches the welcome wizard instead of loading this hank.

Check-it: run the validation command and confirm that its output ends with:

╭─ GOOD TO RUN! ─────────────────────────────────────────────────────╮

The capture below is the checked validation output for the fixture. It renders the three-codon plan tree, the loop and per-codon budget table, and one rig on update. Its budget cells use <cost> and <secs> placeholders, so use this capture to check the output shape rather than a particular budget value. A codon's checkpointedFiles is an array of file-matching patterns over the agent workspace; matching files are watched, streamed to the client, and sealed in the git-based checkpoint. The rendering counts them as checkpointedGlobs; see checkpoints and hank JSON references. The update codon gets its rig through rigSetup, which declares deterministic command or copy operations that run before the codon starts. In this fixture, that operation copies review-notes.md into notes/ between review and update; see rigs.

Captured from the plan-review-update fixture's validate-output.txt.

├─ [1] plan (Write the initial plan)
│     model: haiku │ mode: fresh │ prompts: 1 (11 lines)
│     checkpointedGlobs: 1
│     ↓
└─ [2] LOOP: blind-reviews (Repeatedly improve the plan through blind iterati...
  ╭────────────────────────────────────────────────────────────
  │
  ├─ [2.1] review (Blind review of the plan)
  │     model: haiku │ mode: fresh │ prompts: 1 (12 lines)
  │     checkpointedGlobs: 1
  │     ↓
  └─ [2.2] update (Apply the review to the plan)
        model: haiku │ mode: fresh │ prompts: 1 (8 lines)
        checkpointedGlobs: 2
        rigs: cmd: mkdir -p notes && cp review-notes.md "notes/review-notes...
  │
  ╰────────────────────────────────────────────────────────────


Budget
─────────────────────────────────────────────────────────────────
  No global budget. Loop and codon limits only.

  Codon             Model       Max Dollars                       Max Time          On exceeded
  ─────             ─────       ───────────                       ────────          ───────────
  plan              Claude Ha…  $<cost> (codon cap)                 <secs>s (cap)        ⚠ fails run
  blind-reviews                 $<cost> (loop budget)               <secs>s (loop)
    ├─ review       Claude Ha…  $<cost> (codon cap)                 <secs>s (cap)        completes
    └─ update       Claude Ha…  $<cost> (codon cap)                 <secs>s (cap)        completes

╭─ GOOD TO RUN! ─────────────────────────────────────────────────────╮
│  3 codons • 3 prompts • 3 system prompts • 1 rigs • 3 checkpoints  │
╰────────────────────────────────────────────────────────────────────╯

Watch the loop's journal, not a silent console#

Validation confirms the shape; the run itself produces the evidence. Use the same positional shape without --validate; pass the same data-source directory as <data_path>:

bunx hankweave@0.10.0 hank.json <data_path>

Each completed iteration emits loop.iteration.completed. In an execution with no interactive console, inspect <exec>/.hankweave/events/events.jsonl from the execution directory (<exec>); the event journal records the loop while the console stays silent. See events for the event contract and observe and debug for log details. The event's payload fields are:

Check-it: after a run, inspect its event journal and confirm that the final loop.iteration.completed row marks iteration 2 as final and uses iteration_limit.

The capture below shows the three rows from the checked fixture run. Notice the zero-indexed iteration values, the per-iteration cost and token counts, and terminationReason appearing only on the final row.

Captured from the plan-review-update fixture's final-events.txt.

{"id":"<id>","timestamp":"<ts>","type":"loop.iteration.completed","data":{"loopId":"blind-reviews","iteration":0,"durationMs":45311,"costUsd":0.0637166,"tokensUsed":4167,"isFinal":false}}
…
{"id":"<id>","timestamp":"<ts>","type":"loop.iteration.completed","data":{"loopId":"blind-reviews","iteration":1,"durationMs":68266,"costUsd":0.08828710000000001,"tokensUsed":6222,"isFinal":false}}
…
{"id":"<id>","timestamp":"<ts>","type":"loop.iteration.completed","data":{"loopId":"blind-reviews","iteration":2,"durationMs":81500,"costUsd":0.0972383,"tokensUsed":6642,"isFinal":true,"terminationReason":"iteration_limit"}}

Size failure policy before you freeze the loop#

Every iteration calls its provider again, and that repetition is where loops fail in ways a single codon does not. A same-provider five-hour usage quota tripped during one chain: the provider returned 429 code 1308, and the failure was classified as billing, non-retriable, and fail-fast with exit 1. Stagger codons that use one provider or split providers across the loop.

Since 0.7.3, --validate warns when loop codons retain the default onFailure: "abort". A transient failure on one iteration can halt the loop; set retry or ignore on loop codons when that policy fits. The checked fixture sets retry on both loop codons.

Budget caps are a second failure shape. In two recorded review runs, caps set at the estimate stopped the codons at turn boundaries after $1.58 and $3.52 of spend, with no artifact written. Leave margin; at least twice the estimate is a starting point, not a guarantee. Use onExceeded: "complete" only when partial review output is useful and checked before downstream use. That policy records completion after interruption; it does not give the reviewer extra time to finish or ensure a file exists. See testing and hardening for the cap decision and budgets for budget semantics.

Time caps are a fourth failure shape. At 0.10.0, maxTimeSeconds is checked on each cost update and by a 1000 ms watchdog independent of cost events. When the limit is exceeded, the budget tracker emits exceeded and the codon runner requests SIGTERM; that request does not prove that the running process will promptly tear down. An older C1-pilot capture from 2026-09-01 recorded a codon running about 24,923 seconds against a 2,400-second cap, with the budget error surfacing only when an unrelated idle-timeout failure did. Treat that as a dated observation with unresolved source/runtime cause, not as the intended watchdog behavior or a successful timeout experiment. For unattended work, set an external process deadline; do not assume the in-process cap recovers checkpoints.

Check-it: inspect the fixture's dated cost record and confirm $0.272 across its per-codon codon.completed entries, measured 2026-09-03 with haiku throughout.

Check-it: set maxTimeSeconds on a loop or codon and run bunx hankweave@0.10.0 hank.json <data> --validate. Confirm that the budget table's Max Time column shows <secs>s (cap) or <secs>s (loop); this checks the rendered limit, not process teardown.

Expand the pattern when the work needs it#

An implementation variant uses three codons: Sonnet implements, Opus critiques, and a third codon updates the plan. Attach a rig – a deterministic script for setup or evaluation – to the codon that consumes its results; it executes before that codon starts so the test suite runs before the next reviewer reads its fresh results. The blog reports that a smarter model can help on the critique side because it starts from fresh context and gets situated quickly.

Pasting the loop is the cheap part. Its real cost is limit × codons model runs per review round, so allocate the loop's budget before freezing it.

Check-it: make your estimate count every iteration and codon instead of counting the loop as one model call.

Replace the reviewer with a fixed review surface#

The loop above spends a model call on every review. Before assigning a review question to an agent, ask whether a fixed operation can answer it. Comic-panel generation and documentation fetching were examples that did not need agents. Where a fixed operation works, it is cheaper, repeatable, and catches a class of failure that prose review can miss.

Turn repeatable review into a fixed surface#

Where possible, replace an agent reviewer with a fixed evaluation suite run by a rig. Start each build iteration fresh with that suite's results. You lose flexibility and general intelligence; you gain repeatability, which is the useful trade when a loop iterates several times.

Execute the fixed suite first at every stage boundary and fail closed. Model review missed dist-scaffolds-5, bare-invocation wizard behavior, and a test command that exited 0; execution caught those cases.

Check-it: name the fixed operation that can answer the review question before assigning that question to an agent.

Ship the check instead of rewriting it each time#

Ship diff extractors and test runners as rig scripts. We should not need to ask an agent to recreate a web server each time it needs one.

A checked quote-template-unification corpus example shows what a shipped fixed surface looks like. Its checker re-parses the emitted corpus against an independently re-transcribed oracle, prints PASS or FAIL for each invariant, exits nonzero on a failure, and reports 76/76 checks passed in its green capture. The excerpt below is the checker's entry point; note that it re-parses the emitted corpus rather than trusting the generator's own constants, which is what makes it a real two-path check.

Captured from the quote-template-unification fixture.

#!/usr/bin/env bun
// Entry point: bun generator/check-invariants.ts <corpusDir>
//
// Re-parses the EMITTED corpus (not seed.ts) and recomputes every numbered
// invariant from corpus-spec.md §4. The "expected" numbers used for
// cross-checking are re-transcribed from the spec directly into SPEC_ORACLE
// below (independently of generator/seed.ts) so this is a real two-path
// check, not a tautology against the same constant the generator used.
// Exits nonzero if any invariant fails; prints PASS/FAIL per invariant.

import { readFile, readdir } from 'node:fs/promises';
import { join } from 'node:path';

const corpusDir = process.argv[2];
if (!corpusDir) {
  console.error('usage: bun generator/check-invariants.ts <corpusDir>');

Execute the checker from the quote-template-unification fixture root:

bun generator/check-invariants.ts <corpusDir>

The intact corpus produces:

76/76 checks passed.

Corrupt one corpus file and the checker exits nonzero. That is the review side of a loop: the fixed surface catches a repeatable contract failure without spending a model call.

Check-it: run bun generator/check-invariants.ts <corpusDir> from the quote-template-unification fixture root and confirm the intact corpus prints:

“76/76 checks passed.”

Keep an agent when the question is open-ended#

Fixed surfaces answer repeatable questions. Use an agent reviewer when the question needs flexibility or general judgment. A sentinel – a parallel observer – can review an event pattern: a QA-reviewer pattern observes file.updated events for files matching \.ts$, waits 10 seconds for activity to settle, then reviews the batched events rather than file contents. The checked sentinel configuration below is an observe-only completion-event shape.

These are two sentinel shapes: the QA reviewer observes a settled batch of file.updated events, while the anchor's observe-only sentinel watches a completion event. The anchor sentinel receives event fields rendered into its prompt, reports only the status and budget fields present there, and cannot read or verify output artifacts. Attach a sentinel to a codon through that codon's sentinels array of { sentinelConfig } wrappers. The anchor hank sets sentinelConfig to sentinels/quality-observer.json on validate-and-repair; see hank JSON references and sentinel configuration. The configuration below shows the observe-only contract: its own description states that it reports only fields present in the event.

Captured from the anchor-hank fixture.

{
  "description": "Observes the validate-and-repair completion event. Reports only status and budget fields present in that event; cannot read or verify output artifacts.",
  "trigger": {
    "type": "event",
    "on": [
      "codon.completed"
    ],
    …
  },
  "execution": {
    "strategy": "immediate"
  }
}

A dated blog report on 2026-03-26 compares two reported arrangements: (Agentic Build × Automated Review) × 5 + Agentic Review beat (Agentic Build + Agentic Review) × 4. It also reported comics-hank v2 at one-tenth of v1's overall cost with better output. Treat both as dated reported outcomes, not general multipliers.

A dated embed check shows why the fixed surface matters: on 2026-09-02, the embed verifier classified 237 directives and found 6 drift cases. Paraphrased captures were fidelity bugs, so the verifier caught a class of error that prose review can miss.

Check-it: run the checker on the intact corpus, corrupt one corpus file, and run it again. The first run exits 0 and the second exits nonzero; this checks the fixed surface without spending a model call.

Repair a production review loop#

The patterns so far describe how to build checks into a hank. Production loops add a second problem: when a review finds a defect, the next run should benefit from the fix. In a dated documentation-rebuild case, the first review-repair hank (C1) and second review-repair hank (C2) kept human judgment at the boundaries and put corrections in the durable source, not only the rendered prose.

Put judgment at the hank boundaries#

C1 moves from page-outline vetting to preservation mining, page-batched source-brief writers, and a source-brief fact-check loop; a human reviews those source briefs. C2 moves from diagrams to fresh writers, a tighten loop, a confusion panel, example coverage, crosslinking, and adversarial validation; a human reviews the resulting triage.

Check-it: point to the boundary where a human reviews the source artifact, rather than asking a later prose codon to recover the decision.

Feed the reviewer a fixed surface#

The dated C2 chain, recorded for its v0.2.0 run on 2026-09-01, used luna writers, a fresh-eyes review codon consuming fixed lint checks, a three-model confusion panel with independent reads from haiku, deepseek, and gemini-flash, a repair codon, and a haiku output-packing codon. The review codon remained an agent; its evidence came from the fixed surface. That is the hybrid pattern: agent judgment over repeatable evidence.

The small-model panel logged 198 confusion items for $0.14 on 2026-09-01; those confusions drove prose fixes. See designing codons and handoffs for the dumber-reviewer pattern's design home.

Check-it: inspect the review codon's inputs and confirm that it receives the lint result and starts fresh, instead of relying on the writer's conversation.

Land repairs where the next run can keep them#

When review finds a defect, put the fix in the durable source brief or page outline and render from that source. Avoid hand-editing prose; it must remain downstream of the durable source. In the lived run, nine review gaps were recorded in that durable source rather than being invented into prose.

Make the loop catch silent no-ops too. One packet-rewrite codon ended successfully after 15k thinking tokens but wrote no file. The repair was a mandatory incremental-write and self-verify protocol: write the page outline first, complete at least 4 write steps, and run wc -l before finishing. Use that discipline for output-packing codons.

Check-it: confirm that the durable review record gains rows and that each surviving claim in the rendered page still has a source.

Compare reviewer families and artifact boundaries#

The same source-brief writing task across three model setups – glm-5.3, DeepSeek-V4-Pro, and Kimi-K3 – produced 0/5, 5/5, and 3-pass/2-repair splits under one reviewer. A wave-1 reviewer, Sonnet, later retracted two of its own findings as false positives after re-verification. Separately, the C1 pilot review falsified a source brief's universal claim against index.ts. Treat these as attributed, directional observations, not a model-quality ranking.

Review across artifacts, not only within one. A dedicated cross-brief ownership pass found 9 issues, including two priority issues; one was a duplicated sentinel output-path statement that a per-page review could not see.

Check-it: re-verify a finding against the neighboring brief or artifact before calling it a real defect.

Let the first full run teach you#

Treat the first full run as a learning run. The C1 pilot review found 21 findings across three source briefs; all were repaired and re-verified. Do not stop the first run early. See observe and debug for that operating discipline.

Check-it: review the first run's final record and confirm that it includes repaired findings and re-verification, not an early stop after the first passing codon.