# Operate a Hankweave run

A Hankweave run is easy to start and easy to misread. Validation can pass while the launch command is wrong; a rerun can exit 0 having done nothing; a graceful stop can leave the run record showing `running`. This runbook walks one run through its full arc: validate the hank, launch the intended execution, confirm what actually ran, watch progress, stop cleanly, and wire the result into CI. Each step ends with a **Check-it** line naming the evidence that tells you the step did what you expected.

## Know where the run is

Before the steps, one piece of vocabulary and one picture. A **hank** is the configuration describing the work; each **codon** is one sealed agent task in its sequence. A run moves through four states, and most operator actions are about moving between them or recovering from them.

![Run lifecycle: before-run → preflight → running → completed, with fix and recovery branches](/content-assets/cf45dff5691c48c0/diagrams/operate-runbook/1.png)

Run lifecycle: before-run → preflight → running → completed, with fix and recovery branches

<details>
<summary>Diagram as text</summary>

```text
// Pseudocode: reviewable ASCII source for the run lifecycle.
before-run -> preflight -> running -> completed
                 ^             ^
                 | fix         | resume / rollback / retry
                 +-------------+
```

</details>

*Run lifecycle: before-run → preflight → running → completed, with fix and recovery branches.*

The diagram tracks the run as a whole. Individual codons have their own finer-grained transitions, visible in the event journal described in [Watch a run that says nothing](#watch-a-run-that-says-nothing); those labels are not additional run states. When a run lands on the fix branch, use [troubleshooting](/operate/troubleshooting). For the recovery branches, use [resume, rollback, and retry](/operate/resume-rollback-and-retry).

## Catch a broken hank before it spends money

**1. Validate before you launch.** Configuration-only validation is the cheapest check Hankweave offers, so it comes first:

```bash
bunx hankweave@0.10.0 --validate
```

Validation checks that the data source exists, computes its data signature, validates the hank including the strict-reference preflight, and runs a self-test for each unique model. It exits 0 when valid and 1 when invalid. At 0.10.0, the validation self-test checks SDK import, executable or credential presence, and the Pi model catalog; it does not run model generation or make a billable model call. It can still probe EC2 instance metadata for Bedrock credentials when no static AWS credential source is visible, so do not treat validation as a universal offline guarantee. A key must be present, but Hankweave does not call the model during this validation.

Validation runs before execution setup and creates no execution directory. Its printed `Execution path` is a would-be path: `validation-<dataHash6>` by default, the directory selected by `-e` when supplied, or a timestamp/random/data-hash name with `--start-new`. Validation is not side-effect-free: it writes a temporary validation log under the OS temp directory and can add a missing `$schema` key to the hank. A successful validation prints `✓ Configuration is valid!`, the hank structure, the `GOOD TO RUN!` box, and a `Run it:` line. When the hank or system configures environment variables, it also prints a masked environment-variable report. If a model is known to the registry but cannot be served by Pi, preflight fails while loading the configuration; follow [model-resolution](/reference/model-resolution) for those diagnostics. Runtime startup separately performs provider health checks; those checks can call `generateText("Hi", maxOutputTokens:16)`, unlike `--validate`.

The first capture shows a successful validation of a minimal one-codon hank; note the `GOOD TO RUN!` summary box and the `Run it:` line at the bottom, which gives the exact launch command for this fixture.

*Captured from the published `hankweave@0.10.0` fixture.*

```text
╭────────────────────────────────────────────────────────────────────╮
│  Hankweave v0.10.0                                                 │
│  darwin arm64 • node v23.8.0                                       │
╰────────────────────────────────────────────────────────────────────╯

Calculating data signature for validation...

> Validating configuration: <workspace>/fixtures/minimal-single-provider/hank.json

  Data source:    <workspace>/fixtures/minimal-single-provider/data
  Execution path: ~/.hankweave-executions/validation-<id>

✓ Configuration is valid!

╭──────────────────────────────────────────────────────────────────────────────╮
│  Minimal single provider v1.0.0                                              │
│  1 codon • 0 loops                                                           │
╰──────────────────────────────────────────────────────────────────────────────╯

└─ [1] summarize-notes (Summarize the notes)
      model: haiku │ mode: fresh │ prompts: 1 (13 lines)
      checkpointedGlobs: 1

╭─ GOOD TO RUN! ─────────────────────────────────────────────────────╮
│  1 codons • 1 prompts • 0 system prompts • 0 rigs • 1 checkpoints  │
╰────────────────────────────────────────────────────────────────────╯

Run it:  hankweave hank.json <data_path>
```

*From the env-vars fixture at 0.10.0.*

The second capture shows the masked environment-variable report that appears when the hank or system configures variables: values are redacted to their last characters with a length, so you can confirm the right variables arrived without exposing them.

```text
╭────────────────────────────────────────────────────────────────────╮
│  Hankweave v0.10.0                                                 │
│  darwin arm64 • node v23.8.0                                       │
╰────────────────────────────────────────────────────────────────────╯

Calculating data signature for validation...

> Validating configuration: <workspace>/fixtures/scenarios/env-vars/hank.json

  Data source:    <workspace>/fixtures/scenarios/env-vars/data
  Execution path: ~/.hankweave-executions/validation-<id>

✓ Configuration is valid!

╭──────────────────────────────────────────────────────────────────────────────╮
│  Environment variables v1.0.0                                                │
│  1 codon • 0 loops                                                           │
╰──────────────────────────────────────────────────────────────────────────────╯

└─ [1] write-line (Write one line)
      model: haiku │ mode: fresh │ prompts: 1 (2 lines)
      checkpointedGlobs: 1

╭─ GOOD TO RUN! ─────────────────────────────────────────────────────╮
│  1 codons • 1 prompts • 0 system prompts • 0 rigs • 1 checkpoints  │
╰────────────────────────────────────────────────────────────────────╯

Run it:  hankweave hank.json <data_path>

Environment Variables:

  From System (HANKWEAVE_ prefixed):
    - CAPTURE_VERSION: 0.10.0
    - FIXTURE_SECRET: ••••••ABCD (24 chars)
    - FIXTURE_SHORT: abc

  From Codon Configurations:
    Codon "Write one line" (write-line):
      - FIXTURE_MODE: docs
      - FIXTURE_TOKEN: ••••••cdef (20 chars)
exit=0
```

**Check-it:** The capture contains `✓ Configuration is valid!`, the `GOOD TO RUN!` box, and the `Run it:` line; the validation process exits 0.

## Launch the run you meant to launch

**2. Launch the intended path.** Hankweave has no `run` subcommand; the option grammar below is the whole interface. With one positional path, a path ending in `.json` is the hank path; otherwise it is the data path. The `Run it:` line in the validation capture shows a hank path and a data path; the rule above applies when only one positional path is supplied. Add `--headless`, the mode without the text interface (TUI), when a CI job or script should run without it.

```text
Usage: hankweave [options] [config-or-data-path]

Arguments:
  config-or-data-path       Path to hank.json or project directory
                            When only one argument provided:
                            - If ends with .json: treated as hank-path
                            - Otherwise: treated as data-path

Execution Control:
  -e, --execution <path>    Use specific execution directory
                            Creates if doesn't exist, resumes if has state
  -n, --new, --start-new    Start new execution, never resume
                            Use -n -f to overwrite existing state
  -f, --force               Override safety checks (hash mismatch, existing state)
  --no-wipe                 With --start-new --force, preserve the existing
                            agentRoot/ workspace instead of wiping it
  -y                        Non-interactive mode, skip confirmation prompts

Output:
  -o, --output <path>       Copy outputs to this path (default: stay in execution dir)
  --overwrite-output        Overwrite existing output files instead of renaming
```

```text
Server:
  -p, --port <port>         WebSocket server port (default: auto-select free port)
  --headless                Run without TUI (for CI/CD and scripts)
  --no-autostart            Don't automatically start codons
  --proxy                   Enable the LLM proxy server (disabled by default)
  --anthropic-base-url <url> Custom Anthropic API base URL
  --idle-timeout <seconds>  Idle timeout for WebSocket and proxy servers (0-255, default: 0)
  --shim-idle-timeout <seconds>   Harness idle timeout in seconds (default: 120, per-codon)

```

At startup, the process changes into the execution directory. By default, the data source appears inside it at `agentRoot/read_only_data_source` as a symlink; pass `--copy` to use a copy instead. There is no default output directory: outputs stay in the execution directory's `agentRoot/` workspace unless you pass `-o`. Hankweave copies those outputs after each codon's `outputFiles` step. At the `-o` destination, existing names are renamed unless you pass `--overwrite-output`; the rename form is `_N_timestamp`.

Startup prints a `New execution` or `Resuming` block with the source, execution directory, and SDKs, then the `Hankweave Server Started` box with its WebSocket – the control channel – URL. The capture below shows the new-execution form; the automatic fresh-selection and `--start-new` messages are distinguished in the execution-directory section below. Pin the WebSocket port with `-p, --port`. Without it, Hankweave asks the operating system for a free port, which allows concurrent instances on one machine.

*Captured from the published `hankweave@0.10.0` fixture; execution IDs and ports are run-specific.*

```text
Created new execution directory: ~/.hankweave-executions/<exec-id>
New execution: <exec-id>
  Source → data
  Exec   → ~/.hankweave-executions/<exec-id>
  SDKs   → Claude node_modules ✓
…
  Hankweave Server Started
  WebSocket: ws://localhost:<port>
…
Running in headless mode on port <port>
```

**Check-it:** The capture names the new execution and prints `Running in headless mode on port <port>` after the `Hankweave Server Started` box.

## Don't mistake a silent no-op for a fresh run

**3. Choose the execution directory deliberately.** This is the step where reruns go wrong, so check which execution you'll be using before launching again. Without `-e`, Hankweave hashes the data source, then resumes the newest matching execution directory under `~/.hankweave-executions/` whose `execution-meta.json` carries the same data hash. If no execution matches, Hankweave creates a fresh `<timestamp>-<random>-<dataHash6>` directory under that managed root.

A completed execution can therefore look successful while doing no new work: the console prints `Resuming execution in: <dir>`, `all codons completed` appears in `server.log` rather than on the console, and the process exits 0 with no new `codon.*` events in the same execution directory. Pass `--start-new` (`-n`) to force a fresh run.

> **Pitfall:** A rerun that exits 0 in seconds may have run nothing. Reuse is keyed on the data signature and can silently resume a completed run; pass `--start-new` when you need fresh work.

Use `-e, --execution <path>` to pin the directory. Hankweave creates it when missing and resumes it when it contains `.hankweave/` state. The safety checks have three tiers. A new explicit `-e` target under the managed root is refused when it lacks `execution-meta.json`; an existing explicit path containing both `/.hankweave-executions/` and `/data` is rejected by the nested-execution guard. Choose another explicit directory or omit `-e` to let Hankweave allocate one, and keep the input source separate from the execution directory. On existing state, `--start-new` requires `--force`; that backs up state to `.hankweave.backup-<timestamp>` and wipes `agentRoot/` unless `--no-wipe`. Another non-empty directory warns and prompts. Without `-e`, `--start-new` creates a fresh managed directory; `--force` does not select the previous workspace.

When resuming after a hank edit, Hankweave shows a hash-mismatch warning and asks for confirmation. In non-interactive mode, that prompt cancels by default, so CI needs `-y`. A changed data source is an error with remediation choices; override it with `--force` or the deprecated `--ignore-data-mismatch`.

The three captures below are the evidence for the silent no-op. The console shows a resume and a clean exit; the server log shows the run skipping straight to shutdown; the event count proves no codon ran.

*Captured from the published silent-reuse fixture; execution IDs and ports are run-specific.*

```text
Resuming execution in: ~/.hankweave-executions/<exec-id>
Resuming: <exec-id>
…
➜ Listening on: http://localhost:<port>/ (all interfaces)
exit=0
```

*Captured from the reused execution's `server.log`.*

```text
[<ts>] [INFO] Resuming from last completed codon: write-line in run <id>
…
[<ts>] [INFO] Shutdown: all codons completed (exit code: 0)
```

*Captured from the published silent-reuse fixture; event counts are normalized.*

```text
codon.* events in <exec-id>/.hankweave/events/events.jsonl: after first run 2, after reuse 2 (delta 0); execution resumed by the second run: the same directory
```

**Check-it:** The captures show `Resuming execution in: <dir>` and `exit=0` on the console, `all codons completed` in `server.log`, and zero new `codon.*` events in the same directory; fresh selection prints `Created execution directory: <dir>`, while `--start-new` prints `Created new execution directory:`.

## Watch a run that says nothing

**4. Observe the files, not the quiet console.** In headless mode, the console is quiet while codons run. Watch the event journal at `.hankweave/events/events.jsonl`, the human-readable server log at `.hankweave/logs/server.log`, and `.hankweave/state.json`. Also inspect files codons write under `agentRoot/`. These paths are inside the execution directory; for a resumed run, use the directory named by `Resuming execution in:` as their root, and for a new run start from the execution path printed in the startup block.

Completion has a specific shape: one `codon.completed` journal event per codon, followed in `server.log` by `Shutting down server: all codons completed`, `State transition: RunCompleted`, and `Shutdown: all codons completed (exit code: 0)`. The journal excerpt below shows that full sequence for the minimal fixture: run start, the codon's state transitions, cost and message increments, the checkpoint, `codon.completed`, and finally `RunCompleted`.

```jsonl
{"id": "<id>", "timestamp": "<ts>", "type": "state.transition", "data": {"transitionType": "RunStarted", "runId": "<id>", "transition": {"type": "RunStarted", "data": {"runId": "<id>", "runFolder": "~/.hankweave-executions/<exec-id>/.hankweave/runs/<id>", "gitBranch": "run-<id>", "startingConditions": {"type": "fresh"}, "serverPid": 50024}}, "resultingState": {"currentRunId": "<id>", "runCount": 1, "totalCost":"<n>", "currentRunCost":"<n>"}}}
{"id": "<id>", "timestamp": "<ts>", "type": "state.transition", "data": {"transitionType": "CodonStarted", "runId": "<id>", "codonId": "summarize-notes", "transition": {"type": "CodonStarted", "data": {"runId": "<id>", "codonId": "summarize-notes"}}, "resultingState": {"currentRunId": "<id>", "runCount": 1, "totalCost":"<n>", "currentRunCost":"<n>"}}}
{"id": "<id>", "timestamp": "<ts>", "type": "state.transition", "data": {"transitionType": "CodonTransitioned", "runId": "<id>", "codonId": "summarize-notes", "transition": {"type": "CodonTransitioned", "data": {"runId": "<id>", "codonId": "summarize-notes", "from": "preparing", "to": "starting", "metadata": {}}}, "resultingState": {"currentRunId": "<id>", "runCount": 1, "totalCost":"<n>", "currentRunCost":"<n>"}}}
{"id": "<id>", "timestamp": "<ts>", "type": "state.transition", "data": {"transitionType": "CodonTransitioned", "runId": "<id>", "codonId": "summarize-notes", "transition": {"type": "CodonTransitioned", "data": {"runId": "<id>", "codonId": "summarize-notes", "from": "starting", "to": "initializing", "metadata": {"claudePid": 962109, "claudeLogPath": ".hankweave/runs/<id>/summarize-notes-claude.log"}}}, "resultingState": {"currentRunId": "<id>", "runCount": 1, "totalCost":"<n>", "currentRunCost":"<n>"}}}
{"id": "<id>", "timestamp": "<ts>", "type": "state.transition", "data": {"transitionType": "CodonTransitioned", "runId": "<id>", "codonId": "summarize-notes", "transition": {"type": "CodonTransitioned", "data": {"runId": "<id>", "codonId": "summarize-notes", "from": "initializing", "to": "running", "metadata": {"claudeSessionId": "be7a1143-f289-4e61-9346-9c3bf0dc8ed2"}}}, "resultingState": {"currentRunId": "<id>", "runCount": 1, "totalCost":"<n>", "currentRunCost":"<n>"}}}
{"id": "<id>", "timestamp": "<ts>", "type": "state.transition", "data": {"transitionType": "CostsIncremented", "runId": "<id>", "codonId": "summarize-notes", "transition": {"type": "CostsIncremented", "data": {"runId": "<id>", "codonId": "summarize-notes", "costDelta":"<n>", "tokensDelta": {"inputTokens":"<n>", "outputTokens":"<n>", "cacheCreationTokens":"<n>", "cacheReadTokens":"<n>"}}}, "resultingState": {"currentRunId": "<id>", "runCount": 1, "totalCost":"<n>", "currentRunCost":"<n>"}}}
{"id": "<id>", "timestamp": "<ts>", "type": "state.transition", "data": {"transitionType": "AssistantMessageCountUpdated", "runId": "<id>", "codonId": "summarize-notes", "transition": {"type": "AssistantMessageCountUpdated", "data": {"runId": "<id>", "codonId": "summarize-notes", "newCount":"<n>"}}, "resultingState": {"currentRunId": "<id>", "runCount": 1, "totalCost":"<n>", "currentRunCost":"<n>"}}}
{"id": "<id>", "timestamp": "<ts>", "type": "state.transition", "data": {"transitionType": "CostsIncremented", "runId": "<id>", "codonId": "summarize-notes", "transition": {"type": "CostsIncremented", "data": {"runId": "<id>", "codonId": "summarize-notes", "costDelta":"<n>", "tokensDelta": {"inputTokens":"<n>", "outputTokens":"<n>", "cacheCreationTokens":"<n>", "cacheReadTokens":"<n>"}}}, "resultingState": {"currentRunId": "<id>", "runCount": 1, "totalCost":"<n>", "currentRunCost":"<n>"}}}
{"id": "<id>", "timestamp": "<ts>", "type": "state.transition", "data": {"transitionType": "AssistantMessageCountUpdated", "runId": "<id>", "codonId": "summarize-notes", "transition": {"type": "AssistantMessageCountUpdated", "data": {"runId": "<id>", "codonId": "summarize-notes", "newCount":"<n>"}}, "resultingState": {"currentRunId": "<id>", "runCount": 1, "totalCost":"<n>", "currentRunCost":"<n>"}}}
{"id": "<id>", "timestamp": "<ts>", "type": "state.transition", "data": {"transitionType": "CostsIncremented", "runId": "<id>", "codonId": "summarize-notes", "transition": {"type": "CostsIncremented", "data": {"runId": "<id>", "codonId": "summarize-notes", "costDelta":"<n>", "tokensDelta": {"inputTokens":"<n>", "outputTokens":"<n>", "cacheCreationTokens":"<n>", "cacheReadTokens":"<n>"}}}, "resultingState": {"currentRunId": "<id>", "runCount": 1, "totalCost":"<n>", "currentRunCost":"<n>"}}}
{"id": "<id>", "timestamp": "<ts>", "type": "state.transition", "data": {"transitionType": "AssistantMessageCountUpdated", "runId": "<id>", "codonId": "summarize-notes", "transition": {"type": "AssistantMessageCountUpdated", "data": {"runId": "<id>", "codonId": "summarize-notes", "newCount":"<n>"}}, "resultingState": {"currentRunId": "<id>", "runCount": 1, "totalCost":"<n>", "currentRunCost":"<n>"}}}
{"id": "<id>", "timestamp": "<ts>", "type": "state.transition", "data": {"transitionType": "CostsIncremented", "runId": "<id>", "codonId": "summarize-notes", "transition": {"type": "CostsIncremented", "data": {"runId": "<id>", "codonId": "summarize-notes", "costDelta":"<n>", "tokensDelta": {"inputTokens":"<n>", "outputTokens":"<n>", "cacheCreationTokens":"<n>", "cacheReadTokens":"<n>"}}}, "resultingState": {"currentRunId": "<id>", "runCount": 1, "totalCost":"<n>", "currentRunCost":"<n>"}}}
{"id": "<id>", "timestamp": "<ts>", "type": "state.transition", "data": {"transitionType": "AssistantMessageCountUpdated", "runId": "<id>", "codonId": "summarize-notes", "transition": {"type": "AssistantMessageCountUpdated", "data": {"runId": "<id>", "codonId": "summarize-notes", "newCount":"<n>"}}, "resultingState": {"currentRunId": "<id>", "runCount": 1, "totalCost":"<n>", "currentRunCost":"<n>"}}}
{"id": "<id>", "timestamp": "<ts>", "type": "state.transition", "data": {"transitionType": "CostsIncremented", "runId": "<id>", "codonId": "summarize-notes", "transition": {"type": "CostsIncremented", "data": {"runId": "<id>", "codonId": "summarize-notes", "costDelta":"<n>", "tokensDelta": {"inputTokens":"<n>", "outputTokens":"<n>", "cacheCreationTokens":"<n>", "cacheReadTokens":"<n>"}}}, "resultingState": {"currentRunId": "<id>", "runCount": 1, "totalCost":"<n>", "currentRunCost":"<n>"}}}
{"id": "<id>", "timestamp": "<ts>", "type": "state.transition", "data": {"transitionType": "AssistantMessageCountUpdated", "runId": "<id>", "codonId": "summarize-notes", "transition": {"type": "AssistantMessageCountUpdated", "data": {"runId": "<id>", "codonId": "summarize-notes", "newCount":"<n>"}}, "resultingState": {"currentRunId": "<id>", "runCount": 1, "totalCost":"<n>", "currentRunCost":"<n>"}}}
{"id": "<id>", "timestamp": "<ts>", "type": "state.transition", "data": {"transitionType": "CostsIncremented", "runId": "<id>", "codonId": "summarize-notes", "transition": {"type": "CostsIncremented", "data": {"runId": "<id>", "codonId": "summarize-notes", "costDelta":"<n>", "tokensDelta": {"inputTokens":"<n>", "outputTokens":"<n>", "cacheCreationTokens":"<n>", "cacheReadTokens":"<n>"}}}, "resultingState": {"currentRunId": "<id>", "runCount": 1, "totalCost":"<n>", "currentRunCost":"<n>"}}}
{"id": "<id>", "timestamp": "<ts>", "type": "state.transition", "data": {"transitionType": "AssistantMessageCountUpdated", "runId": "<id>", "codonId": "summarize-notes", "transition": {"type": "AssistantMessageCountUpdated", "data": {"runId": "<id>", "codonId": "summarize-notes", "newCount":"<n>"}}, "resultingState": {"currentRunId": "<id>", "runCount": 1, "totalCost":"<n>", "currentRunCost":"<n>"}}}
{"id": "<id>", "timestamp": "<ts>", "type": "state.transition", "data": {"transitionType": "CodonFinalCostSet", "runId": "<id>", "codonId": "summarize-notes", "transition": {"type": "CodonFinalCostSet", "data": {"runId": "<id>", "codonId": "summarize-notes", "finalCost":"<n>", "finalTokens": {"inputTokens":"<n>", "outputTokens":"<n>", "cacheCreationTokens":"<n>", "cacheReadTokens":"<n>"}}}, "resultingState": {"currentRunId": "<id>", "runCount": 1, "totalCost":"<n>", "currentRunCost":"<n>"}}}
{"id": "<id>", "timestamp": "<ts>", "type": "state.transition", "data": {"transitionType": "CheckpointCreated", "runId": "<id>", "codonId": "summarize-notes", "transition": {"type": "CheckpointCreated", "data": {"runId": "<id>", "codonId": "summarize-notes", "checkpointType": "completed", "sha": "<sha>", "branch": "run-<id>"}}, "resultingState": {"currentRunId": "<id>", "runCount": 1, "totalCost":"<n>", "currentRunCost":"<n>"}}}
{"id": "<id>", "timestamp": "<ts>", "type": "state.transition", "data": {"transitionType": "CodonTransitioned", "runId": "<id>", "codonId": "summarize-notes", "transition": {"type": "CodonTransitioned", "data": {"runId": "<id>", "codonId": "summarize-notes", "from": "running", "to": "completed", "metadata": {"exitCode": 0, "resultMessageReceived": true, "checkpointSha": "<sha>", "contextExceeded": false, "extensionCount": 0}}}, "resultingState": {"currentRunId": "<id>", "runCount": 1, "totalCost":"<n>", "currentRunCost":"<n>"}}}
{"id": "<id>", "timestamp": "<ts>", "type": "codon.completed", "data": {"codonId": "summarize-notes", "success": true, "cost":"<n>", "duration":"<n>", "exitStatus": {"type": "success"}}}
{"id": "<id>", "timestamp": "<ts>", "type": "state.transition", "data": {"transitionType": "RunCompleted", "runId": "<id>", "transition": {"type": "RunCompleted", "data": {"runId": "<id>"}}, "resultingState": {"currentRunId": null, "runCount": 1, "totalCost":"<n>", "currentRunCost":"<n>"}}}
```

Use `--attach` to connect a TUI to an already-running server. It is a boolean flag: the target comes from `--port`, the execution's `runtime.lock` when `--execution` is supplied, or the default port 7777; it does not take a URL argument. The attached UI displays `READ-ONLY: Commands are disabled` and guards its mutating `n`, `s`, `f`, and `r` keys, while `q` disconnects. Those are client-side guards, not server-enforced authority: the handshake requests `READANDWRITE`, the server accepts the requested mode without authentication, and even its read-only command set includes `server.shutdown` and `server.force_shutdown`. The control channel is not a security boundary. Restrict access to the WebSocket and use authenticated controls for remote exposure. See [resume, rollback, and retry](/operate/resume-rollback-and-retry) for the WebSocket envelopes. PostHog `ENOTFOUND hw-telemetry.southbridge.ai` stack traces on headless error output are a known 0.10.0 telemetry defect, not a run failure.

**Check-it:** Find a `codon.completed` event, then match the three shutdown lines in `server.log` and the final `RunCompleted` transition.

## Stop a run without losing state

**5. Stop gracefully first.** Send SIGINT with Ctrl+C or send SIGTERM to begin graceful shutdown. A second signal during shutdown escalates to force shutdown. Force shutdown sends `SIGKILL` to harness processes, the processes supporting codon execution. It performs minimal cleanup: checkpoints, telemetry, and state transitions are skipped, and it exits 1. A graceful signal-driven shutdown returns exit code 0 for the `SIGINT`, `SIGTERM`, and `client request` reasons. Since 0.7.3, a 30-second shutdown watchdog starts before awaited cleanup and force-exits with the computed exit code if graceful shutdown wedges.

If you control the server through WebSocket, `server.force_shutdown` is the deliberate-shutdown command. See [resume, rollback, and retry](/operate/resume-rollback-and-retry) before choosing a recovery action.

On SIGINT, the console prints `Shutting down server in 3s... (press Ctrl+C again to force close) Server closed.` and exits 0. `server.log` records `Shutting down server: SIGINT` and `Killing current codon runner for shutdown`; the interrupted run's `state.json` still shows the run and codon as `running`, with no terminal transition persisted. When checkpointing is enabled, a graceful interruption of a running codon still writes a checkpoint commit whose first line starts `exit:<codonId> [run:…]`, while the corresponding `CheckpointCreated` state/event uses `checkpointType: "skipped"`. Those checkpoint labels are separate from the process exit code: force shutdown skips checkpoints and exits 1, while graceful shutdown exits 0. A second signal during shutdown follows the source-level force-shutdown path and exits 1, but the captured double-SIGINT run delivered its second signal after graceful shutdown completed: it also exited 0 and contains no force-shutdown lines. Treat that capture as a race, not proof that force shutdown completed.

The captures show all three artifacts of a graceful interruption: the server-log lines, the double-signal attempt that arrived too late, and the persisted state still reading `running`.

*Captured from the published signal fixture's graceful-interruption `server.log` excerpt.*

```text
[<ts>] [INFO] Shutting down server: SIGINT
[<ts>] [INFO] Killing current codon runner for shutdown
```

*Captured from the published signal fixture's double-signal attempt; graceful shutdown completed before the second signal was observed.*

```text
[<ts>] [INFO] Shutting down server: SIGINT
[<ts>] [DEBUG] [SentinelManager] Shutdown complete
[<ts>] [INFO] Sentinel manager shutdown complete
```

*Captured from the published signal fixture's final state after the graceful interruption.*

```json
{
  "status": "running",
  "codons": [
    {
      "codonId": "write-line",
      "status": "running"
    }
  ]
}
```

**Check-it:** After Ctrl+C, match the console shutdown message and the `Shutting down server: SIGINT` / `Killing current codon runner for shutdown` lines in `server.log`, then confirm `.hankweave/state.json` still shows the run and codon as `running` as in the signal capture above. If checkpointing is enabled, distinguish an `exit:` commit prefix and `checkpointType: "skipped"` from the process's exit status. Do not infer a force exit from the double-signal capture; the runtime's force-shutdown path exits 1.

## Wire it into CI

**6. Gate the run on validation, then use headless mode.** In CI, stop before launch if validation fails:

```bash
bunx hankweave@0.10.0 --validate || exit 1
bunx hankweave@0.10.0 --headless
```

For a normal CI run, the contract is exit 0 for success and exit 1 for failure. The complete exit-code catalog belongs to [errors and exit codes](/reference/errors-and-exit-codes). `all codons completed` returns 0 unless the run is failed or crashed; `codon failure` returns 1. A failed `outputFiles` copy or `beforeCopy` step also fails the run and returns 1. The graceful signal path in [Stop a run without losing state](#stop-a-run-without-losing-state) is a separate exit-0 case: the process can return 0 while the interrupted run record still shows the run and codon as `running`.

> **VersionNote:** Since 0.9.0, a failed `outputFiles` copy or `beforeCopy` fails the run and returns 1, so CI can use that result as its gate.

In headless mode, a retriable codon failure under `onFailure: "abort"` fails fast because there is no interactive client to park for a retry. Headless mode autostarts by default; there is no `--autostart` flag. Use `--no-autostart` when the server should wait for WebSocket commands. If headless autostart fails, Hankweave prints `[FATAL] Headless autostart failed` and returns 1.

When a budget cap trips, the runner is interrupted and requests SIGTERM. With `onExceeded: "complete"` (the default), the attempt resolves by completing the codon and recording a `budget limit reached` information event; with `onExceeded: "fail"`, it fails the codon non-retriably. A `codon.completed` event may carry `budgetExceeded` with its `currency`, `limit`, and `used` values. Read [budgets](/concepts/budgets) for the two-party budget rules.

`--max-cost` and `--max-time` are parsed and validated as run-level caps, but neither appears in the shipped `--help`; see [CLI reference](/reference/cli) for the parity note. `maxTimeSeconds` has an active watchdog: each `costIncremented` event calls `checkTime()`, and a timer checks independently every 1000 ms; when the cap is exceeded, the runner requests SIGTERM. A timer tick and a kill request do not prove prompt harness teardown. One older observed attempt ran for 24,923 seconds against a 2,400-second cap; treat that as a historical unresolved capture, not the intended 0.10.0 mechanism. For unattended work, add an external process deadline. A mid-codon `onExceeded: "fail"` cost abort can forfeit the whole spend and can arrive after `codon.completed`; do not treat that event alone as proof that the final spend is settled. Use per-codon `onExceeded: "complete"` soft caps and hard caps at least twice the estimate for expensive codons. At 0.10.0, codons routed to `pi/zai/*` report no cost, so `maxDollars` does not bind for them; guard those codons with `maxTimeSeconds`.

**Check-it:** The normal CI process returns `$? = 0` for success and `$? = 1` for failure, including output-copy failure. The completion capture records the successful `RunCompleted` path.

## Find what the run left behind

When the run is over, the execution directory is the record of what happened, including work you may want to recover. See [Execution directory](/reference/execution-directory) for its full tree, including `~/.hankweave/`, replay copies, and archive shape.

Use `--cleanup` only when you intend to remove the selected execution directory itself. Cleanup recursively deletes that directory, including other files it contains, after `Proceed with cleanup? This cannot be undone! (y/N)`; `-y` skips the confirmation. With `-e`, the named directory is the deletion target; without it, the ordinary newest-match selection supplies the target. If the target contains `.hankweave/runtime.lock`, cleanup refuses with `Server is running`; that file-presence check does not prove a live process or silently remove a stale lock.

Continue with [Observe and debug](/operate/observe-and-debug) to diagnose failures by phase and find the agent log, including the `claudeLogPath` value and its `#`-to-`-` naming change. For recovery procedures and WebSocket envelopes, use [Resume, rollback, and retry](/operate/resume-rollback-and-retry).