# Build and test a connector with a fresh agent This page walks through a complete, runnable Hankweave example: an agent builds a data connector from a fixed specification, a deterministic test suite checks every attempt, and a quality gate decides what gets exported. You will run the fixture yourself, read its captures, and then adapt the same loop to your own domain. Along the way the example exercises most of the machinery a hank offers – codons, loops, rigs, checkpoints, budgets, and output gates – so it doubles as a tour of how those pieces cooperate in one run. ## What the connector-build-test pattern solves The pattern comes from the Clausetta project's polymorphic-building approach: keep the specification fixed, check correctness with an evaluation suite, and use an agentic loop to discover the implementation. Clausetta itself builds integration shims for arbitrary agent systems; here we apply the same idea to a small connector rather than reproduce Clausetta. Two Hankweave terms carry the design. A **codon** is one task in the hank's sequence; a **rig** is the per-codon setup that copies inputs or executes commands before that task starts. The key decision is who judges the work. When correctness is objective, a deterministic rig can check pass or fail, so the agent is reserved for implementation and review stays non-agentic. That makes the loop repeatable and avoids asking an agent to judge another agent's work. The run follows four phases: 1. **Research** gathers the documentation and specification. 2. **Build-loop** has an agent implement against the specification, with fresh eyes on each iteration. 3. **Eval** runs deterministic tests through rigs. 4. **Document** has an agent write up the result. In this fixture, those phases map to the top-level `research` codon, the `build` loop whose `implement` codon runs the eval rig, and the `document` codon. The pattern is production-derived: it has been used for real-time connector generation. It also transfers to API client generation, data migration scripts, configuration generators, and test-suite generation. > **DeepDive:** When the correctness criterion is objective, a deterministic rig can check pass or fail without an agent judging quality. That is the useful separation between build-test and agentic build-review loops. ## How the project structure guides the loop Before running anything, it helps to see what the fixture ships and how the pieces refer to each other. Start with the versioned fixture bundle, then enter its `connector-build-test/` directory. The layout below is what you will find there; the rest of this section reads it top to bottom, from the specification and tests through the hank and prompts. Afterward we will run the fixture, compare the outputs and measured costs, then adapt the example. ```text hank.json hank.break-gate.json spec.md data/ input.json tests/ connector.test.ts tests-impossible/ connector.test.ts prompts/ research.md build.md document.md expected/ validate-output.txt run-transcript.txt final-events.txt costs.json ``` Two files define correctness. `spec.md` is a sanitized fictional weekly-records normalizer contract: it accepts records with `id`, `name`, `amount`, and `region`; `transform` preserves the identifying fields, computes `amountCents`, and returns records sorted by ascending `id`. `tests/connector.test.ts` is the zero-dependency deterministic eval suite. It checks the exact transformed output and separately checks sorting plus verbatim `name` and `region`; it imports `../connector`, so the initial workspace fails until the loop creates `connector.ts`. The fixture's input directory is `data/` (its records live in `data/input.json`); it is passed as the second positional argument, and the run transcript labels it `Source → data`. The hank below wires everything together: it copies the specification and tests into each agent workspace, the loop runs the test command there and leaves `test-output.txt` for the next fresh agent, and `checkpointedFiles` preserves the implementation between iterations. The document codon exports `IMPLEMENTATION.md` and `connector.ts` only after its `beforeCopy` test command passes; that command is the output gate. ```json { "$schema": "https://unpkg.com/hankweave@0.10.0/schemas/hank.schema.json", "meta": { "name": "connector-build-test", "version": "1.0.0", "description": "Clausetta-derived polymorphic build fixture: a fixed spec + deterministic bun test suite, an agentic build loop with fresh continuation and an eval rig between iterations, then a documentation codon with a beforeCopy quality gate." }, "hank": [ { "id": "research", "name": "Read the spec and test suite", "model": "haiku", "continuationMode": "fresh", "promptFile": "prompts/research.md", "rigSetup": [ { "type": "copy", "copy": { "from": "spec.md", "to": "spec.md" } }, { "type": "copy", "copy": { "from": "tests", "to": "tests" } } ], "checkpointedFiles": ["notes.md"], "onFailure": "abort", "budget": { "maxDollars": 0.2, "maxTimeSeconds": 240, "onExceeded": "fail" } }, { "type": "loop", "id": "build", "name": "Build the connector until the tests pass", "terminateOn": { "type": "iterationLimit", "limit": 3 }, "budget": { "maxDollars": 0.6, "maxTimeSeconds": 600, "onExceeded": "complete" }, "codons": [ { "id": "implement", "name": "Write the connector implementation", "model": "haiku", "continuationMode": "fresh", "promptFile": "prompts/build.md", "rigSetup": [ { "type": "copy", "copy": { "from": "spec.md", "to": "spec.md" } }, { "type": "copy", "copy": { "from": "tests", "to": "tests" } }, { "type": "command", "command": { "run": "bun test 2>&1 | tee test-output.txt" }, "allowFailure": true } ], "checkpointedFiles": ["connector.ts"], "onFailure": "retry", "retryConfig": { "maxAttempts": 2, "delayMs": 1000, "maxDelayMs": 60000 }, "budget": { "maxDollars": 0.2, "maxTimeSeconds": 300, "onExceeded": "complete" } } ] }, { "id": "document", "name": "Write the implementation note", "model": "haiku", "continuationMode": "fresh", "promptFile": "prompts/document.md", "checkpointedFiles": ["IMPLEMENTATION.md"], "outputFiles": [ { "beforeCopy": [ { "type": "command", "command": { "run": "bun test" } } ], "copy": ["IMPLEMENTATION.md", "connector.ts"] } ], "onFailure": "abort", "budget": { "maxDollars": 0.2, "maxTimeSeconds": 240, "onExceeded": "fail" } } ] } ``` Read the hank with the loop in mind. The loop wrapper has an explicit `iterationLimit` of three and one `implement` codon. `continuationMode: "fresh"` is on that codon, so the loop's memory comes from files and test output rather than conversation history. The loop schema summarizes the wrapper's fields: | field | type | default | required | constraints | description | | ------------------ | ----------------------------------- | ------- | -------- | ----------- | ------------------------------------------------------------------------------------- | | `type` | `string` | | yes | = `loop` | Type discriminator - required for loops | | `id` | `string` | | yes | minLength 1 | Unique identifier for this loop | | `name` | `string` | | yes | minLength 1 | Human-readable name displayed in UI and logs | | `description` | `string` | | no | | Optional description shown to users | | `terminateOn` | `iterationLimit \| contextExceeded` | | yes | | Termination condition for the loop | | `codons` | `array` | | yes | minItems 1 | Array of codons to execute in each iteration | | `budget` | `object` | | no | | Budget scope for this loop. | | `archiveOnSuccess` | `array` | | no | | Paths to archive when the loop terminates. Paths are relative to the agent workspace. | The `implement` codon's rig copies `spec.md` and `tests/`, then runs `bun test 2>&1 | tee test-output.txt`. The pipe writes the test result to the workspace for the next agent; because rig commands run through a shell, this pipeline reports `tee`'s exit status, so a red test still leaves the result available. The command retains `allowFailure: true` as the recommended loop-rig setting. The validator warning is broader than this command: it recommends that setting for loop setup generally, while the fixture's source copies succeed and the test command already carries the flag. The loop uses `iterationLimit` rather than `contextExceeded` because this example has a fixed number of attempts. The rig schema gives the exact shape of `copy` and `command` operations, including the path-safety constraints on `from`: | field | type | default | required | constraints | description | | ---------------------- | --------- | --------- | -------- | ------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `copy` | `object` | | | `type` = `copy` | | | ↳ `copy` | `object` | | yes | | | | ↳ ↳ `from` | `string` | | yes | minLength 1 | Source path, relative to the hank directory and inside it, using '/' separators; absolute paths, '..' escapes, and symlinks are rejected. | | ↳ ↳ `to` | `string` | | yes | minLength 1 | Target path relative to projectPath (parent directory must exist). Always specifies the full target path including name. Examples: from: 'templates/foo', to: 'src/foo' → copies directory foo to src/foo; from: 'templates… | | ↳ `allowFailure` | `boolean` | `false` | no | | If true, failure of this operation won't fail the codon (default: false). Recommended for rig setup in loop codons where operations might fail in some iterations (e.g., copying files that don't exist yet). | | `command` | `object` | | | `type` = `command` | | | ↳ `command` | `object` | | yes | | | | ↳ ↳ `run` | `string` | | yes | minLength 1 | Shell command to execute | | ↳ ↳ `workingDirectory` | `enum` | `project` | no | `project` \| `lastCopied` | Working directory for command execution (default: 'project') | | ↳ `allowFailure` | `boolean` | `false` | no | | If true, failure of this operation won't fail the codon (default: false). Recommended for rig setup in loop codons where operations might fail in some iterations (e.g., running commands that might not succeed initially). | The output contract has two relevant fields: `copy` names the files to export, and `beforeCopy` runs commands before that export. This fixture's `document` codon uses `beforeCopy` with `bun test`, so a failing test prevents `connector.ts` and `IMPLEMENTATION.md` from being copied to `out/`. | field | type | default | required | constraints | description | | ------------ | --------------- | ------- | -------- | ----------- | ------------------------------------------------------------------ | | `copy` | `array` | | yes | minItems 1 | Glob patterns to copy from execution directory to output directory | | `beforeCopy` | `array` | | no | | Optional commands to run before copying (run in executionPath) | ## How each build iteration starts fresh "Fresh" sounds like it throws work away; it does not. Each `implement` iteration gets a new agent session, but `connector.ts` is checkpointed and the rig writes the previous test result to `test-output.txt`. The next agent reads both before changing the implementation, so the conversation resets while the state of the work carries over. The build prompt makes that handoff explicit – note that it instructs the agent to check for existing work before writing anything: ```text CHECK FOR EXISTING WORK FIRST — before writing anything: 1. If `connector.ts` exists, read it (use `cat`). Do not start from scratch: preserve what already works and only fix what the test output says is wrong. 2. If `test-output.txt` exists, read it (use `cat`). It is the output of the last `bun test` run; the failures it lists are your to-fix list. 3. Read `spec.md` (the fixed contract) and `notes.md` (the research brief). Then write `connector.ts` in the current working directory so that it: … ``` Checkpoint `connector.ts` after every iteration. A sealed checkpoint is the recovery point for an interrupted loop; the checkpoints page explains the mechanics. The configuration calls these entries `checkpointedFiles`; the capture summarizes the single entry as `checkpointedGlobs: 1`. The build loop and its `implement` codon both use `budget.onExceeded: "complete"`, so a cap at a turn boundary can finish the current build rather than aborting it mid-build. The non-loop `research` and `document` codons use `"fail"`. Budget semantics belong to [concepts/budgets](/concepts/budgets). One runtime detail matters when you adapt this pattern: idle timeouts are harness-specific. At runtime, the live manager routes Anthropic models to the Claude Agent SDK and other models to the in-process Pi SDK; `ReplayProcessManager` is used only for replay, not as a third production provider harness. A harness is the manager that runs the agent session. For this fixture's Claude Agent SDK `haiku` codons, an unset `shimIdleTimeout` uses the SDK's flat 180-second stream-inactivity default. Set `shimIdleTimeout` at least as high as the longest expected build or test command when adapting the pattern. The fixture's test capture is about 11 ms and its three loop iterations take about 20–40 seconds, so the unset value is sufficient for this run. The 300-second busy-state allowance is a Pi SDK behavior, not a timeout guarantee for these Claude codons. > **Pitfall:** A command can run for minutes without producing agent events. Give the owning harness–the process managing the agent–enough idle time; do not transfer Pi's busy-state floor to a Claude Agent SDK codon. ## What a successful run leaves behind With the structure in place, we can run the fixture and check each stage against its capture. Download [the v0.10.0 fixture bundle](/content-assets/cf45dff5691c48c0/fixtures/0.10.0/hankweave-fixtures-0.10.0.tar.gz), keeping the archive in its parent directory. Individual assets are also served under `/fixtures/0.10.0/files/`; for example, [the fixture hank](/content-assets/cf45dff5691c48c0/fixtures/0.10.0/files/connector-build-test/hank.json). The bundle already contains one top-level `hankweave-fixtures-0.10.0/` directory; extract it from the parent and then enter the named fixture directory rather than creating a second directory with the same name: ```sh tar -xzf hankweave-fixtures-0.10.0.tar.gz cd hankweave-fixtures-0.10.0/connector-build-test ``` Check the prerequisites before launching. The documented launcher is `bunx`, but the package entrypoint is a Node program. Use Node `>=22.19.0` for this recipe, with Bun/bunx as the package launcher. Bun must also be installed and on `PATH`: the build loop's rig runs `bun test 2>&1 | tee test-output.txt`, and the document codon's `beforeCopy` gate runs `bun test`. If you use npm instead, the equivalent launcher is `npx hankweave@0.10.0`; that changes the package launcher, not the Bun prerequisite. All three codons use the `haiku` shortcut on the direct Anthropic route, so set `ANTHROPIC_API_KEY` before starting. Without it, startup fails its self-test before any codon runs; do not substitute a Claude Code login. The fixture's captured run shows Node rather than a Bun-only runtime. Validate first, then run the fixture in headless mode (without an interactive terminal), with a new execution and an explicit output directory: ```sh bunx hankweave@0.10.0 hank.json data/ --validate # prints the GOOD TO RUN! box bunx hankweave@0.10.0 hank.json data/ --headless --start-new -o out ``` The validation capture prints the three-codon/one-loop plan, its budget table, a `GOOD TO RUN!` box, and the loop-rig `allowFailure` warning: ```text ╭──────────────────────────────────────────────────────────────────────────────╮ │ connector-build-test v1.0.0 │ │ 3 codons • 1 loop │ ╰──────────────────────────────────────────────────────────────────────────────╯ … ╭─ GOOD TO RUN! ─────────────────────────────────────────────────────╮ │ 3 codons • 3 prompts • 0 system prompts • 5 rigs • 3 checkpoints │ ╰────────────────────────────────────────────────────────────────────╯ … Warnings: - Loop 'build' > Codon 1 (implement): rigSetup in loop codon should use 'allowFailure: true' to prevent loop termination on setup failures. This is especially important if subsequent iterations might fail (e.g., trying to copy files to where they already exist). ``` **Check-it:** After validation, confirm the plan tree, budget table, `GOOD TO RUN!` box, and the expected `allowFailure: true` warning are present. Validation is a preflight step; it is not the model-generation self-test for the run. The successful headless run writes the implementation and test feedback in the execution workspace. With `-o out`, the `document` codon's gated `outputFiles` copy places `IMPLEMENTATION.md` and `connector.ts` in `out/`. Execution-directory creation and output-directory behavior belong to [operate/runbook](/operate/runbook). Run-level `--max-cost` and `--max-time` flags belong to [reference/cli](/reference/cli). The transcript below shows the plan, the budget table, and the test output from inside the loop: ```text ╭────────────────────────────────────────────────────────────────────╮ │ Hankweave v0.10.0 │ │ darwin arm64 • node v23.8.0 │ ╰────────────────────────────────────────────────────────────────────╯ Created new execution directory: ~/.hankweave-executions/ New execution: Source → data Exec → ~/.hankweave-executions/ SDKs → Claude node_modules ✓ ╭──────────────────────────────────────────────────────────────────────────────╮ │ connector-build-test v1.0.0 │ │ 3 codons • 1 loop │ ╰──────────────────────────────────────────────────────────────────────────────╯ ├─ [1] research (Read the spec and test suite) │ model: haiku │ mode: fresh │ prompts: 1 (11 lines) │ checkpointedGlobs: 1 │ rigs: copy: ~/.hankweave-executions/1788419193951-ygag-... │ ↓ ├─ [2] LOOP: build (Build the connector until the tests pass) × 3 iterations │ ╭──────────────────────────────────────────────────────────── │ │ │ └─ [2.1] implement (Write the connector implementation) │ model: haiku │ mode: fresh │ prompts: 1 (14 lines) │ checkpointedGlobs: 1 │ rigs: copy: ~/.hankweave-executions/1788419193951-yga... │ │ │ ╰──────────────────────────────────────────────────────────── │ ↓ └─ [3] document (Write the implementation note) model: haiku │ mode: fresh │ prompts: 1 (11 lines) checkpointedGlobs: 1 Budget ───────────────────────────────────────────────────────────────── Global ceiling: $ (runtime/cli) Allocation: shared sequentially Codon Model Max Dollars Max Time On exceeded ───── ───── ─────────── ──────── ─────────── research Claude Ha… $ (codon cap) s (cap) ⚠ fails run build $ (loop budget) s (loop) └─ implement Claude Ha… $ (codon cap) s (cap) completes document Claude Ha… $ (codon cap) s (cap) ⚠ fails run Shared pool: codons run in order. Each uses what it needs; the remainder passes to the next. ! Configuration warnings: - Loop 'build' > Codon 1 (implement): rigSetup in loop codon should use 'allowFailure: true' to prevent loop termination on setup failures. This is especially important if subsequent iterations might fail (e.g., trying to copy files to where they already exist). ══════════════════════════════════════════════════ Hankweave Server Started WebSocket: ws://localhost: ══════════════════════════════════════════════════ Running in headless mode on port ➜ Listening on: http://localhost:/ (all interfaces) [] [ERROR] [DEBUG] Command stderr: tests/connector.test.ts: [] [ERROR] [DEBUG] Command stderr: (pass) transform produces the exact documented output for the fixture input [0.ms] [] [ERROR] [DEBUG] Command stderr: (pass) transform sorts by id ascending and preserves name/region verbatim [0.ms] 2 pass 0 fail 3 expect() calls Ran 2 tests across 1 file. [12.ms] ``` Don't let the `Command stderr` label obscure the test result: this capture reports `2 pass` and `0 fail`. The console capture does not show completion events; those appear in the final event stream, where each codon reports success and the run ends with `RunCompleted`: ```text {"id":"","timestamp":"","type":"codon.completed","data":{"codonId":"research","success":true,"cost":"","duration":"","exitStatus":{"type":"success"}}} … {"id":"","timestamp":"","type":"codon.completed","data":{"codonId":"implement#0","success":true,"cost":"","duration":"","exitStatus":{"type":"success"}}} … {"id":"","timestamp":"","type":"codon.completed","data":{"codonId":"implement#1","success":true,"cost":"","duration":"","exitStatus":{"type":"success"}}} … {"id":"","timestamp":"","type":"codon.completed","data":{"codonId":"implement#2","success":true,"cost":"","duration":"","exitStatus":{"type":"success"}}} … {"id":"","timestamp":"","type":"codon.completed","data":{"codonId":"document","success":true,"cost":"","duration":"","exitStatus":{"type":"success"}}} {"id":"","timestamp":"","type":"state.transition","data":{"transitionType":"RunCompleted","runId":"","transition":{"type":"RunCompleted","data":{"runId":""}},"resultingState":{"currentRunId":null,"runCount":1,"totalCost":"","currentRunCost":""}}} ``` ```text - Captured: 2026-09-03T07:25Z · runtime: hankweave@0.10.0 (published npm artifact, `bunx hankweave@0.10.0 --version` → 0.10.0) · model: haiku … - 5 codon completions: 1× `research` + 3× `implement` (`implement#0…#2`) + 1× `document`, then `RunCompleted`, exit 0. ``` The measured costs from that locked-version run are: ```json { "codons": [ { "codon": "research", "cost": 0.01747275 }, { "codon": "implement#0", "cost": 0.050346749999999996 }, { "codon": "implement#1", "cost": 0.022904149999999998 }, { "codon": "implement#2", "cost": 0.01768455 }, { "codon": "document", "cost": 0.0162796 } ], "total": 0.12468779999999999 } ``` **Check-it:** Confirm five successful codon completions – `research`, `implement#0`, `implement#1`, `implement#2`, and `document` – followed by `RunCompleted`, and confirm the run's exit status is 0. The costs above are measured and dated fixture results, not price estimates. A passing run does not by itself prove the gate works, so the fixture ships a variant that forces the gate to fail. Run the named break-gate hank from the same fixture directory: ```sh bunx hankweave@0.10.0 hank.break-gate.json data/ --headless --start-new --max-cost 0.5 -o out-break ``` This variant stages `tests-impossible/` for the `document` codon's rig and leaves its `beforeCopy` command as the bare `bun test`. Its `build` loop is configured for one iteration in the break-gate hank, rather than the normal three. The rig command is piped through `tee` and runs with `shell: true`, so that setup command resolves and the document checkpoint is sealed; the later `beforeCopy` test exits 1. The output stage records `codonOutputBeforeCopy`, `RunFailed`, and `Shutdown: codon failure (exit code: 1)`. The captures below show the failing test, the error event, and the shutdown lines: ```text ╭────────────────────────────────────────────────────────────────────╮ │ Hankweave v0.10.0 │ │ darwin arm64 • node v23.8.0 │ ╰────────────────────────────────────────────────────────────────────╯ Created new execution directory: ~/.hankweave-executions/ New execution: Source → data Exec → ~/.hankweave-executions/ SDKs → Claude node_modules ✓ ╭──────────────────────────────────────────────────────────────────────────────╮ │ connector-build-test (break: failing gate) v1.0.0 │ │ 3 codons • 1 loop │ ╰──────────────────────────────────────────────────────────────────────────────╯ ├─ [1] research (Read the spec and test suite) │ model: haiku │ mode: fresh │ prompts: 1 (11 lines) │ checkpointedGlobs: 1 │ rigs: copy: /... │ ↓ ├─ [2] LOOP: build (Build the connector until the tests pass) × 1 iteration │ ╭──────────────────────────────────────────────────────────── │ │ │ └─ [2.1] implement (Write the connector implementation) │ model: haiku │ mode: fresh │ prompts: 1 (14 lines) │ checkpointedGlobs: 1 │ rigs: copy: /... │ │ │ ╰──────────────────────────────────────────────────────────── │ ↓ └─ [3] document (Write the implementation note) model: haiku │ mode: fresh │ prompts: 1 (11 lines) checkpointedGlobs: 1 rigs: copy: /... Budget ───────────────────────────────────────────────────────────────── Global ceiling: $ (runtime/cli) Allocation: shared sequentially Codon Model Max Dollars Max Time On exceeded ───── ───── ─────────── ──────── ─────────── research Claude Ha… $ (codon cap) s (cap) ⚠ fails run build $ (loop, capped by hank) s (loop) └─ implement Claude Ha… $ (codon cap) s (cap) completes document Claude Ha… $ (codon cap) s (cap) ⚠ fails run Shared pool: codons run in order. Each uses what it needs; the remainder passes to the next. ! Configuration warnings: - Loop 'build' > Codon 1 (implement): rigSetup in loop codon should use 'allowFailure: true' to prevent loop termination on setup failures. This is especially important if subsequent iterations might fail (e.g., trying to copy files to where they already exist). ══════════════════════════════════════════════════ Hankweave Server Started WebSocket: ws://localhost: ══════════════════════════════════════════════════ Running in headless mode on port ➜ Listening on: http://localhost:/ (all interfaces) [] [ERROR] [DEBUG] Command stderr: tests/connector.test.ts: [] [ERROR] [DEBUG] Command stderr: 1 | import { test, expect } from "bun:test"; 2 | // Deliberately impossible: the spec's output shape can never satisfy this assertion. 3 | test("impossible gate", () => { expect(1).toBe(2); }); ^ error: expect(received).toBe(expected) Expected: 2 Received: 1 at (~/.hankweave-executions//agentRoot/tests/connector.test.ts:3:43) (fail) impossible gate [0.ms] 0 pass 1 fail 1 expect() calls Ran 1 test across 1 file. [5.ms] [] [ERROR] [DEBUG] Command failed with exit code 1 [] [ERROR] [DEBUG] Full stderr: tests/connector.test.ts: 1 | import { test, expect } from "bun:test"; 2 | // Deliberately impossible: the spec's output shape can never satisfy this assertion. 3 | test("impossible gate", () => { expect(1).toBe(2); }); ^ error: expect(received).toBe(expected) Expected: 2 Received: 1 at (~/.hankweave-executions//agentRoot/tests/connector.test.ts:3:43) (fail) impossible gate [0.ms] 0 pass 1 fail 1 expect() calls Ran 1 test across 1 file. [5.ms] exit=1 ``` ```text {"id":"","timestamp":"","type":"error","data":{"message":"Copy group 0 failed with: CommandError: Command failed with exit code 1","context":"codonOutputBeforeCopy","severity":"operation","codon":"document","fatal":false}} {"id":"","timestamp":"","type":"state.transition","data":{"transitionType":"RunFailed","runId":"","transition":{"type":"RunFailed","data":{"runId":""}},"resultingState":{"currentRunId":null,"runCount":1,"totalCost":"","currentRunCost":""}}} ``` ```text [] [ERROR] [DEBUG] Command stderr: tests/connector.test.ts: [] [ERROR] [DEBUG] Command stderr: 1 | import { test, expect } from "bun:test"; [] [ERROR] [DEBUG] Command failed with exit code 1 [] [ERROR] [DEBUG] Full stderr: [] [INFO] [operation] codonOutputBeforeCopy: Copy group 0 failed with: CommandError: Command failed with exit code 1 [] [INFO] State transition: RunFailed [] [INFO] Shutdown: codon failure (exit code: 1) ``` Three slices of the runtime source explain why the failure behaves this way. The command execution path uses a shell for rig commands: ```typescript const proc = spawn(cmd.command.run, { shell: true, ``` A command resolves on exit code 0 and rejects on a nonzero exit: ```typescript proc.on("exit", (code) => { if (flushInterval) clearInterval(flushInterval); if (code === 0) { this.logger.log(`[DEBUG] Command completed successfully`, "info"); resolve(); } else { // Handle null exit code (killed by signal) const exitCode = code ?? -1; this.logger.log(`[DEBUG] Command failed with exit code ${exitCode}`, "error"); this.logger.log(`[DEBUG] Full stdout: ${stdout}`, "info"); this.logger.log(`[DEBUG] Full stderr: ${stderr}`, "error"); // Create CommandError with exit code and output const error = new CommandError( `Command failed with exit code ${exitCode}`, exitCode, stdout, stderr, ); reject(error); ``` The output-stage error transitions the run to `RunFailed` and shuts it down: ```typescript beforeCopySuccess ? "codonOutputCopyFiles" : "codonOutputBeforeCopy", ); // An output-stage failure (beforeCopy validator or copy) must fail // the run. handleError at OPERATION severity only logs and notifies; // without the transition + shutdown below the runtime proceeds to // the "all codons completed" shutdown and exits 0 despite the // failure. Fail fast: don't run remaining output groups. if (this.currentRunId) { this.stateManager.transition({ type: "RunFailed", data: { runId: this.currentRunId }, }); await this.stateManager.waitForPendingTransitions(); } await this.shutdown("codon failure"); ``` **Check-it:** The break-gate capture must show the impossible `bun test` failing with exit 1, and the companion event/log excerpts must show `codonOutputBeforeCopy`, `RunFailed`, and the codon-failure shutdown. This is an output-copy gate failure, not a claim that the loop's eval rig stopped the run. Recovery from a failed run reuses checkpoints rather than starting cold. If a build loop fails, resume with `-e ` to reuse its execution directory, or use `--start-new` for a new one. The `research` codon has already sealed a completion checkpoint before `build` starts, so a connector resume logs `Execution thread failed, rolling back...` and `Found last successfully completed thread codon to rollback to`, rolls back to that completed checkpoint, and restarts the interrupted `implement#N` from scratch. Files in the failed codon's workspace may be lost. Do not use the pre-checkpoint `No checkpoints found in execution history` message from the separate kill-and-resume scenario as the connector resume check. The following diagnostic is borrowed from that scenario's **after-checkpoint** resume capture, not from this connector fixture; its `write-line` and `write-second` names are example-specific. The reusable observables are the reused execution directory, rollback to the last completed codon, and a fresh start for the interrupted codon: ```text Resuming execution in: ~/.hankweave-executions/ Resuming: … [] [ERROR] Execution thread failed, rolling back... … ``` **Check-it:** On a connector resume after a build-loop failure, confirm `Resuming execution in:`, the rollback message, `Found last successfully completed thread codon to rollback to`, and that the interrupted `implement#N` starts over. Change one specification requirement and run the loop again to let the eval suite catch implementation drift. ## How to adapt the loop to your domain The fixture is small, but its control flow is the part worth keeping: fixed specification → deterministic evaluation suite in rigs → fresh-agent build loop → `beforeCopy` quality gate. What changes per domain is the correctness check inside the eval rig: | Domain | Evaluation suite | | ------------------------ | --------------------------------------- | | API clients | Tests against a live endpoint | | Configuration generators | A schema validator | | Data migrations | A dry-run comparison | | Test-suite generation | Tests that exercise the generated suite | Two details decide whether an adaptation behaves like the fixture. This fixture does not set `appendSystemPromptFile`; its rig copies `spec.md` into the workspace, and the build prompt tells the agent to read it alongside `notes.md` and `test-output.txt`. If a different adaptation needs the fixed specification in the agent's system prompt, use `appendSystemPromptFile` on the build codon. Keep the evaluation output in the workspace context: this fixture writes `test-output.txt`, and the next fresh agent reads it before editing. The specification remains the source of truth while each iteration gets a fresh session over the current files. For a first adaptation, change one specification requirement, re-run, and inspect the evaluation result before accepting the exported files. The specification and checks stay explicit while the loop discovers a new implementation.