Constrain extraction variance with a preflight rig

Chapter 1 ended with a working extraction: one codon reads the Aster submissions and writes a normalized envelope. That hank works, but it trusts its inputs completely, and it only speaks one input dialect. This chapter hardens it before we add more. We will move integrity checks out of the agent and into an ordinary script that runs before any model call, then extend the same mechanical extraction to two more suppliers whose source data looks nothing like Aster's.

A word about what "deterministic" can mean here, because the chapter title could overpromise. The preflight rig is deterministic in the strict sense: same inputs, same report, no model involved. The extraction codons are not. Their prompts forbid invention, correction, merging and judgment, which narrows the variance considerably, but model output remains stochastic. So the chapter's standard of success is not byte-identical envelopes; it is a run where integrity failures stop the hank before a token is spent, and where two fresh runs agree on the fields that matter. By the end you will have three envelope files, a checked repeatability comparison, and a clear boundary where judgment enters in chapter 3.

Start from the sealed Aster baseline#

Chapter 2 is a shipped fixture directory, not a Git tag or checkpoint name. From the 0.10.0 fixture bundle, extract the archive from its parent directory with tar -xzf hankweave-fixtures-0.10.0.tar.gz, then enter hankweave-fixtures-0.10.0/chapters/ch2. Individual published files use /fixtures/0.10.0/files/<relative-file>. A hank is the JSON program in hank.json; a codon is one agent task in its sequence. This chapter's hank.json starts with normalize-aster, which produces envelope-aster.json containing both Aster submissions; the chapter then adds Beacon, Cedar, and the preflight rig.

The fixture's setup uses a non-root account, Bun for its rig command, Git, and ANTHROPIC_API_KEY. The shown version-pinned bunx/npx recipe launches the package's Node-shebang entrypoint, so Node >=22.19.0 is required; Bun/bunx is the fixture's launcher, and npx hankweave@0.10.0 is the equivalent spelling for a Node/npm user. The capture ran on Node v23.8.0. Prepare the task-only input directory once; it must not already exist:

python3 ../../verify.py prepare-data ../../quote-template-unification task-data

During execution setup, Hankweave exposes the task-data argument as <execution>/agentRoot/read_only_data_source/; rigs run from that agentRoot, so the preflight's cwd-relative read_only_data_source/ path points at the prepared inputs. Run validation before the paid execution. It checks the hank without running codons or provider health checks. Then run the three-codon chapter from the chapter directory:

bunx hankweave@0.10.0 hank.json task-data --validate
bunx hankweave@0.10.0 hank.json task-data --headless --start-new --execution exec --max-cost 9 --shim-idle-timeout 1800 --overwrite-output -o out
python3 ../../verify.py chapter 2 out

Preparation copies digitized, digitized-extracts, lookups, rfq, and source-quotes plus a filtered manifest. The held-out truth/, generator/, and planted fixtures/ failure specimens remain outside task-data/. Reuse of the same prepared directory is allowed for repeats; preparation refuses to overwrite it.

Throughout the chapter we keep the working hank and add one concern at a time rather than designing the complete program up front: first the rig, then the new dialects, then the repeatability check.

Check-it: After the run, out contains the chapter's three envelope outputs; verify.py chapter 2 out is the semantic check for that run.

Move checksum work out of the agent#

Verifying checksums, checking fixed shapes and validating watermarks are problems with exact answers, so paying a model to answer them buys nothing but variance. That work belongs in a script. Hankweave's mechanism for this is a rig: an explicit setup step that runs before a codon. A rig can execute a command or copy a file into the agent workspace, and because allowFailure defaults to false, a failing rig blocks its codon from starting at all.

The preflight script below reads read_only_data_source/corpus-manifest.json, recomputes sha256 for each declared file, checks the expected top-level keys for seven files, and verifies FICTIONAL TRAINING FIXTURE — NOT CUSTOMER DATA on the four quote documents. It makes no model call. Note the exit-on-failure contract in the header comment: that is what lets rigSetup treat any mismatch as a stop condition.

#!/usr/bin/env bun
// Preflight rig for the quote-template-unification anchor hank.
// Zero dependencies beyond node:crypto/node:fs, run under `bun`. Verifies that
// the corpus mounted at read_only_data_source/ (a) contains the files this
// hank depends on, (b) matches corpus-manifest.json's sha256 for each, and
// (c) has the expected top-level shape and watermark for each quote document.
// Exits nonzero on any failure so rigSetup (allowFailure: false) stops the
// run before any codon spends a token on corpus that doesn't match its
// manifest.

import { createHash } from "node:crypto";
import { existsSync, readFileSync, writeFileSync } from "node:fs";
import { join } from "node:path";

const DATA_ROOT = "read_only_data_source";
const MANIFEST_PATH = join(DATA_ROOT, "corpus-manifest.json");
const REPORT_PATH = "preflight-report.json";
const WATERMARK = "FICTIONAL TRAINING FIXTURE — NOT CUSTOMER DATA";

If a manifest hash no longer matches a corpus file, the script writes preflight-report.json with ok: false, prints FAIL lines to stderr, and exits nonzero. The blocking rig then stops normalize-aster before it starts. This is also why a checkpoint is not a corruption detector: if a codon had already run against bad input, resuming its sealed output would replay the wrong result.

Check-it: The clean fixture's preflight-report.json has ok: true, seven findings, and ok: true on every finding.

Copy the rig before the command runs#

The script from the previous section lives in the hank directory next to hank.json, but a command rig does not run there. It runs in the agent workspace, agentRoot/, inside the execution directory. Unless we stage the script into that workspace first, the command has nothing to execute.

The fix is three operations in order: create pipeline/, copy the rig from inside the hank directory to pipeline/preflight.ts, then execute the copied file. Here is the chapter's actual rigSetup block:

      "rigSetup": [
        {
          "type": "command",
          "command": {
            "run": "mkdir -p pipeline"
          }
        },
        {
          "type": "copy",
          "copy": {
            "from": "rigs/preflight.ts",
            "to": "pipeline/preflight.ts"
          }
        },
        {
          "type": "command",
          "command": {
            "run": "bun pipeline/preflight.ts"
          }
        }
      ],

A copy operation copies a file or directory tree from the hank directory into the workspace. For a command, workingDirectory: "project" resolves to the agent workspace and workingDirectory: "lastCopied" resolves to the previous copy target. At 0.10.0, copy.from must be a relative path inside the hank directory; .. escapes and symlinks are rejected during configuration loading. rigSetup is optional per codon, and each operation must succeed unless that operation sets allowFailure: true. The configuration rules recommend that override for loop-codon rigs that may run again. See rig operation semantics for the full execution model.

Before the hank runs, verify the rig by hand from an agent-root-shaped workspace where read_only_data_source/ is mounted; running it from the chapter directory alone will not provide that directory. For example, from the chapter directory, use mkdir -p manual-agent && ln -s ../task-data manual-agent/read_only_data_source && cp rigs/preflight.ts manual-agent/preflight.ts && (cd manual-agent && bun preflight.ts). The normal Hankweave run creates the equivalent <execution>/agentRoot/read_only_data_source mapping automatically. The script reads that cwd-relative directory and does not accept an arbitrary corpus-path argument. A local failure is a rig failure you can catch before codon startup.

Check-it: In the captured clean run, normalize-aster starts after the preflight and completes; the preflight has not blocked the codon.

Preserve three dialects under one contract#

With the corpus guarded, we can widen the input. Beacon and Cedar do not share Aster's source shape. Beacon is a Reducto-shaped submission–its digitizer data uses result.chunks[] rather than a flat fields[] array–while Cedar is a skewed-scan generic-OCR submission. The design choice is to preserve those source observations exactly as they appear, while giving all three normalizers one envelope contract to write into.

Append normalize-beacon and normalize-cedar after normalize-aster. Each codon uses the haiku model shortcut, continuationMode: "fresh", a prompt file, and one envelope output. fresh starts a new agent session with no memory of the preceding codon, so the handoff is explicit in envelope-aster.json, envelope-beacon.json, and envelope-cedar.json rather than hidden in conversation context.

Each prompt requires one envelope object with every key present: use null for a value absent from the source, never invent a value, and never merge or deduplicate. The source is copied, not judged. The excerpts below show the contract and the source quirks each prompt calls out; notice how every instruction tells the normalizer what to copy verbatim and explicitly assigns any decision to a later codon.

Produce one envelope object with exactly these fields (use `null` for anything the source doesn't have — never omit the key, never invent a value):

...

- Line 2 (`NC-1002-A`'s slot) is printed as **`BI-2002-X`** in `buyer_part_id_raw`, with a `substitution: {"sub_part_id": "BI-2002-X", "approved": true}` object. Copy `buyer_part_id_raw` exactly as printed (`BI-2002-X`) and copy the `substitution` object verbatim. Do not resolve the substitution yourself — that is `validate-and-repair`'s job.
- Line 3 (`NC-1003-B`) has `price_basis: {"kind": "PACK", "pack_size": 100, "pack_price_minor": 11500}`. Copy all three sub-fields.
- Line 5 (`NC-1005-A`) has `tooling_minor: 25000` (a one-time tooling charge) and its source chunk is on `page: 2` (the document is two pages). Copy `tooling_minor` verbatim; do not add it into `unit_price_minor` or any extension — that decision belongs downstream.
- The document's `quote_meta` has no top-level `freight_included: false` shortcut without the accompanying note — copy `freight_exclusion_note` alongside `freight_included` so the reason isn't lost.

Produce one envelope object with exactly these fields (use `null` for anything the source doesn't have — never omit the key, never invent a value):

...

- Line 4 (`NC-1004-A`'s slot) has OCR block text `"NO BID"`, `unit_price_minor: null`, `status_hint: "NO_BID"`. Copy this exactly. Never invent a price or treat it as `0`.
- Line 5's OCR block text is **`NC-1OO5-A`** (capital letter O, not zero — an OCR misread of `NC-1005-A`) at `ocr_confidence: 0.74`. Copy `buyer_part_id_raw` exactly as OCR rendered it (`NC-1OO5-A`) and copy its `ocr_confidence`. Do not correct the spelling yourself — alias resolution against `read_only_data_source/lookups/part-aliases.json` happens in `validate-and-repair`.
- There is one handwritten `annotations[]` entry: `{"type": "handwriting", "text": "lead time 45 days on pins", "applies_to_block": "doc-cedar-cw-77-b2", ...}`. Copy the full `annotations` array verbatim into the envelope (top-level `annotations` key, sibling of `lines`). This annotation is already mirrored onto line 2 (`NC-1002-A`) as `lead_time_days: 45` in the source — copy that line field too, but the annotation must **never** be applied to price. Annotations are a notes channel only.
- `quote_meta.valid_until` is `2026-01-20`. Copy it verbatim. Do not compare it to any award date or flag it yourself — that judgment call belongs to `validate-and-repair`.

Concretely, Beacon's BI-2002-X substitution object, pack pricing for NC-1003-B, and one-time tooling_minor: 25000 on line 5 are carried verbatim. The _minor value is in minor currency units; the normalizer does not fold it into unit_price_minor or another extension. Cedar's NO_BID line, handwritten lead-time annotation, OCR value NC-1OO5-A with a capital O, and valid_until: 2026-01-20 are likewise preserved without correction. Every one of these is a fact a downstream codon will need; the extraction layer's job is only to make sure none of them are lost or silently "fixed" on the way in.

Check-it: The chapter completion list contains normalize-aster, normalize-beacon, and normalize-cedar; all three are fresh mechanical extraction codons.

Constrain variance without claiming certainty#

It is worth being precise about what this chapter has and has not made repeatable. The preflight is deterministic: unchanged manifest entries and matching sha256 values produce the same ok: true report and findings, with zero model calls. That guarantee applies to the check, not to model-written envelopes.

The three normalizers remain agentic. Their prompts constrain the task by forbidding invented values, alias resolution, merging, deduplication, and judgment, but constrained is not identical. So compare two fresh runs semantically instead of treating byte identity as the definition of success: required identity, raw part, quantity, price or basis, status, and citation fields are compared, while full-JSON and byte differences are recorded rather than universally rejected. Model output is stochastic in general; later validate-and-repair and reconcile codons add judgment and can vary. The lesson is to push every deterministic operation that can be isolated into a rig and keep this chapter's model work mechanical.

Check-it: For independent attempts, retain --start-new, use distinct execution and output directories, check each output with verify.py chapter 2, and then run verify.py compare-json as shown below.

bunx hankweave@0.10.0 hank.json task-data --headless --start-new --execution exec-a --max-cost 9 --shim-idle-timeout 1800 --overwrite-output -o out-a
python3 ../../verify.py chapter 2 out-a
bunx hankweave@0.10.0 hank.json task-data --headless --start-new --execution exec-b --max-cost 9 --shim-idle-timeout 1800 --overwrite-output -o out-b
python3 ../../verify.py chapter 2 out-b
python3 ../../verify.py compare-json out-a out-b

Check the captured outputs#

Use the files, not a model's explanation of them, to check the chapter. These assertions are about the captured extraction contract:

1. Inspect preflight-report.json; it has ok: true, seven findings, and all findings are ok: true.
2. Inspect Beacon's line 2: its raw part is BI-2002-X and its substitution object remains. Confirm that line 3 (NC-1003-B) retains its pack-pricing fields unchanged.
3. Inspect Cedar's line 4 and line 5. The former has status_hint: "NO_BID"; the latter retains buyer_part_id_raw: "NC-1OO5-A" and the envelope retains valid_until: "2026-01-20".
4. Inspect Aster's envelope. It contains two objects sharing dedup_key; they are not merged.
5. Run the two fresh attempts above. compare-json checks the required extraction fields and reports full-JSON and byte variation rather than silently treating either as a universal pass/fail gate.

The -o out directory receives only the three envelope files declared in hank.json; preflight-report.json remains in the execution's agentRoot/ because the rig writes it there and the output contract does not copy it. Use the execution directory printed by Hankweave to inspect agentRoot/preflight-report.json. The following checks run from the chapter directory and inspect the copied files in out/. Each jq line corresponds to one of the assertions above: Beacon's substituted raw part, Cedar's NO_BID status, Cedar's misread OCR part id, and Aster's unmerged duplicate pair. jq is a separate command-line JSON viewer, so provide it in your shell or use another JSON viewer.

jq '.[0].lines[1].buyer_part_id_raw' out/envelope-beacon.json
jq '.[0].lines[3].status_hint' out/envelope-cedar.json
jq '.[0].lines[4].buyer_part_id_raw' out/envelope-cedar.json
jq 'length, .[].dedup_key' out/envelope-aster.json

The expected chapter contract names the three output files and the repeatability comparison:

- `normalize-aster`: `envelope-aster.json`
- `normalize-beacon`: `envelope-beacon.json`
- `normalize-cedar`: `envelope-cedar.json`

For repeatability, run twice with `--start-new --execution exec-a -o out-a` and `--start-new --execution exec-b -o out-b`, check both with `verify.py chapter 2`, then run `python3 ../../verify.py compare-json out-a out-b`. This checks required identity, raw part, quantity, price/basis, status and citation fields; full-JSON and byte differences are recorded, not universally forbidden. The automated chapters capture performs both fresh runs and retains their complete outputs.

The live capture records three completed codons, all on haiku, and a task-input-only run. The broader fresh captures show the model mix continuing in later chapters: ch3 adds validate-and-repair and a BASETEN_API_KEY, and ch3/ch4 use pi/baseten/deepseek-ai/DeepSeek-V4-Pro for the judgment codons.

# Capture manifest
- Captured: 2026-09-06T13:12:56.189980+00:00 · runtime: hankweave@0.10.0
- actual_service_output: true
- Scope: one complete live execution
- Raw execution (path normalized): ~/.hankweave-executions/chapters/1788700174373-bd72-6a377d
- Complete codons: 3/3
- Tracked codon cost: $0.16203200 (provider health checks and sentinel calls are separate)
- Input scope: task inputs only; oracle/generator/failure fixtures excluded

Stop before judgment enters the hank#

We stop at normalization, with three envelope files. We have not yet produced validated-records.json, exception-ledger.json, unified-records.csv, award-brief.md, exceptions.csv, or quality-observer.log. We will add validation and repair, reconciliation, and the observer layer in later chapters. There are no sentinels (parallel observers) or budget fields here.

The ch1 → ch2 change is deliberately small: add rigs/preflight.ts, add the Beacon and Cedar prompts, add the expected-output contract, attach the copy-rig setup to normalize-aster, and append two codons. The same codon shape now covers three input dialects without introducing judgment. The progression table below shows where that leaves the hank relative to the chapters around it.

# Chapter progression

| Chapter | Codons | Supplier scope | Keys | Result |
|---|---:|---|---|---|
| ch1 | 1 | AST | ANTHROPIC_API_KEY | two Aster envelopes |
| ch2 | 3 | AST, BCN, CDR | ANTHROPIC_API_KEY | three envelope files |
| ch3 | 4 | AST, BCN, CDR | ANTHROPIC_API_KEY, BASETEN_API_KEY | validated records and exceptions |
| ch4 | 5 | AST, BCN, CDR | ANTHROPIC_API_KEY, BASETEN_API_KEY | 15 supplier-part data rows |
| ch5 | 7 | AST, BCN, CDR, DVR, EMB, FJR, HBR, IRS | ANTHROPIC_API_KEY, BASETEN_API_KEY | full anchor: 40 data rows, cited award brief; GRN quarantined |

Each chapter's `EXPECTED.md` gives commands from its own directory and an executable semantic check. ch1–ch4 intentionally remain smaller checkpoints; the eight-supplier scope begins at ch5. The full-anchor prompt files are under ch5/prompts/, exactly where hank.json references them.

Use a non-root user, Bun and Git. Validation is not a live run. For an independent repeat, use `--start-new` and a new output directory; to resume after interruption, use the same data source without `--start-new`. Input checksums do not guarantee byte-identical model output.

Next we add validate-and-repair, the first codon that makes deduplication, alias-resolution, and exception-routing decisions. It uses pi/baseten/deepseek-ai/DeepSeek-V4-Pro, so we will need BASETEN_API_KEY alongside ANTHROPIC_API_KEY. The re-proven anchor run records the normalize, survey, and award codons on haiku, the judgment codons on that DeepSeek model, and a tracked codon cost of $0.88303367 rather than the approximately $2.41 recorded for the earlier sonnet run. For now, we leave those decisions out of the raw envelopes.

Check-it: The ch2 output contract stops at the three envelope files; no downstream validation or award artifacts are part of this chapter.