Operations¶

Where kew stores data, how to back it up and restore it, how retention works, and what forward-only migrations mean for upgrades.

Where data lives¶

kew keeps its entire durable record in one SQLite file:

<logging.dir>/dispatch.db          # all tables (see below)
<logging.dir>/dispatch.db-wal      # WAL file — exists at runtime
<logging.dir>/dispatch.db-shm      # shared-memory index — exists at runtime

logging.dir defaults to ./logs/dispatch (relative to your repo root). Override it in kew.toml:

[logging]
dir = "./logs/dispatch"   # default

or at runtime with the KEW_LOG_DIR (or DISPATCH_LOG_DIR) environment variable.

dispatch.db holds five logical datasets in one file:

Table	Contents
`runs`	Every dispatch run — outcome, cost, tokens, model, branch
`audit_events`	Hash-chained governance event log (see Audit log & tamper-evidence)
`policy_snapshots`	Retained governance config snapshots bound to each dispatch
`loops`	Active and closed loop records
`interactive_usage`	Off-chain interactive usage from `kew ingest file` / `kew ingest serve` (schema v15, #256); surfaced in `kew report` but not covered by `kew audit verify`
`epic_graph_cache`	Off-chain per-epic cache of the last provider graph fetch (schema v28, #894); lets the Linear driver / `kew epic status` short-circuit the GraphQL fetch when tracker events show nothing changed. A disposable performance artifact, never covered by `kew audit verify`
`review_rounds`	Off-chain per-node governed-review state (round counter + per-round `{head_sha, verdict, sidecar_digest, sidecar_path}`, schema v29 + v32, #977/#1030); lets the driver resume the round count across a restart and locate a NEEDS_CHANGES round's stored findings to remediate against. Operational state — the attested per-round fact is the `adversarial_review_verdict` audit event — so it is never covered by `kew audit verify`

The WAL (-wal, -shm) sidecars exist only while kew is running. A clean shutdown checkpoints WAL into the main file. For backups, kew backup uses SQLite's online-backup API, so the WAL is always included consistently regardless of checkpoint state.

Per-run log files (dispatch-*.log, dispatch-*.events.jsonl) live alongside dispatch.db in logging.dir and are not included in a backup by default (opt-in with --include-logs).

Backup¶

kew backup snapshots dispatch.db and kew.toml into a single compressed archive using SQLite's online-backup API — consistent and safe to run against a live database.

kew backup                          # write kew-backup-<ts>.tar.gz to logging.dir
kew backup -o /backups/             # write to a different directory
kew backup -o /backups/daily.tar.gz # explicit archive name
kew backup --include-logs           # also bundle per-run .log/.events.jsonl files

The archive contains:

File	Always included
`dispatch.db`	Yes
`kew.toml`	Yes (if found)
`manifest.json`	Yes — schema version, chain head, run/event counts, chain integrity flag
`logs/`	Only with `--include-logs`

What is NOT captured: environment variable overrides (KEW_LOG_DIR, KEW_RUNNER, etc.) and any config values supplied only via env vars are not persisted in the snapshot. If you rely on env-var overrides in CI, document them alongside your backup cadence.

See kew backup for the full flag reference.

Restore¶

kew restore validates an archive's schema compatibility and chain integrity, then places the DB non-destructively.

# Restore into the configured log dir (reads logging.dir from kew.toml):
kew restore /backups/kew-backup-20260624T120000Z.tar.gz

# Already have a dispatch.db? Pass --force; the existing file is moved aside:
kew restore --force kew-backup-20260624T120000Z.tar.gz

# Also restore kew.toml (existing one moved aside):
kew restore --force --with-config kew-backup-20260624T120000Z.tar.gz

# Skip chain verification (e.g. archive itself records chain_ok=false):
kew restore --no-verify kew-backup-20260624T120000Z.tar.gz

Stop kew before restoring. kew restore refuses to overwrite a database that another kew process is holding open. The check is a short BEGIN IMMEDIATE attempt; if it fails, the restore aborts with exit code 1 and a clear message. The automatic lock check is best-effort — under WAL a read-only connection (e.g. an idle TUI) does not hold a write lock, so it may pass the check; stopping kew is the real guarantee.

Non-destructive guarantees:

If dispatch.db exists at the target, restore refuses unless --force is passed.
With --force, the existing DB is moved aside to dispatch.db.bak-<ts> (and the -wal/-shm sidecars likewise) before the restored file is placed.
With --with-config, the existing kew.toml is moved to kew.toml.bak-<ts> before the archived one is placed.
The restored DB is schema-migrated forward to the running kew's schema version on first open (the migration runs once at restore time).

See kew restore for the full flag reference.

Suggested cadence¶

A daily backup run is enough for most teams. Example cron or CI step:

# cron: daily at 03:00
0 3 * * * cd /path/to/your/repo && kew backup -o /backups/kew/

In a GitHub Actions workflow:

- name: kew backup
  run: kew backup -o artifacts/
- uses: actions/upload-artifact@v4
  with:
    name: kew-backup-${{ github.run_id }}
    path: artifacts/kew-backup-*.tar.gz
    retention-days: 30

Rotate old archives with standard tools (find /backups/kew/ -mtime +30 -delete).

Retention¶

kew never prunes run rows automatically. retention_days sets a cutoff, but it is only enforced when you explicitly ask for it:

[logging]
dir = "./logs/dispatch"
retention_days = 90   # 0 = keep forever (default)

With retention_days > 0, run kew export --purge to delete records older than the cutoff. The purge happens after the export completes, so you always keep an exported copy first. There is no background or on-dispatch pruning — nothing is deleted unless you run that command (and with retention_days = 0, --purge refuses and tells you to set a cutoff first).

Cache Amortization Falsification Probe (D3)¶

Decision D3 (spec docs/superpowers/specs/2026-07-23-context-memory-layers-design.md): "Falsification test ships in Phase 0: two dispatches with equal prefix_key <5 min apart; if run 2's first-turn cache_read_tokens ≈ 0, the amortization language dies and D becomes purely per-run accounting."

This probe answers whether kew's fleet-amortization claim is real: do two agents dispatched in quick succession actually share a prompt-cache warm line, or does each pay full input cost?

What the probe measures¶

prefix_pair_probe from kew.analytics finds all pairs of completed runs where:

both runs share the same prefix_key (identical stable prompt prefix), and
the second run started within the provider's cache TTL of the first (default 300 s for Anthropic; pass ttl_seconds= to override).

Still-pending runs are excluded: their token counts are placeholders that would inject a spurious cache-miss verdict.

For each qualifying pair it returns run2_first_turn_cache_read_tokens — the cache-read tokens on the first turn of the second run, parsed from that run's stream-json transcript (log_file). First-turn semantics are essential: every multi-turn run self-caches from turn 2 onward, so the run-level total is almost always large and would report "amortization is real" even when the first turn served zero cross-agent cache read — the exact D3 kill scenario. Only the first turn's cache reads reflect a warm line left by a sibling run. A warm hit produces a large value; ≈ 0 means no cross-agent cache was served. The value is None when the second run's transcript is unavailable (the first-turn value cannot be determined — do not fall back to the run total).

Operator procedure¶

Dispatch the same trivial issue twice, under 5 minutes apart, against the same runner and model so both runs share a prefix_key. A dummy issue that always takes < 2 min is ideal (e.g. "echo hello").
Wait for both runs to finish, then open a Python REPL or script in your repo root:

from kew.run_store import SqliteRunStore
from kew.analytics import prefix_pair_probe

store = SqliteRunStore("logs/dispatch/dispatch.db")
pairs = prefix_pair_probe(store, ttl_seconds=300)
for p in pairs:
    print(
        f"run1={p['run1_id']}  run2={p['run2_id']}"
        f"  delta={p['delta_seconds']:.0f}s"
        f"  run2_first_turn_cache_read={p['run2_first_turn_cache_read_tokens']}"
    )
store.close()

Interpret the result:

`run2_first_turn_cache_read_tokens`	Verdict
≫ 0 (e.g. tens of thousands)	Cache warm — amortization is real
≈ 0	Cache miss — delete fleet-amortization language from the spec (D3)
`None`	Run 2's transcript is missing — re-run the probe with a completed run that has a `log_file`

Verdict recording¶

Record the outcome as a comment on spec decision D3 in docs/superpowers/specs/2026-07-23-context-memory-layers-design.md. If the result is a cache miss, remove all fleet-amortization language from that spec; leave per-run accounting only.

Migration safety¶

kew uses forward-only SQLite migrations. Each release can only add schema steps, never remove them. Opening a database that was written by an older kew version applies any missing migration steps automatically — you never need to run a migration command manually.

Opening a database written by a newer kew is refused with a clear error:

SchemaTooNewError: dispatch.db is schema vN; this kew supports up to vM. Upgrade kew to open it.

kew never silently corrupts a database it cannot understand. To resolve this, upgrade the kew CLI to match or exceed the version that wrote the database.

The same schema guard applies during kew restore: if the archive's manifest records a schema version higher than the running kew supports, the restore is refused before any files are touched.