8000 log-execution usage and default ledger; SKILL states the init-block shape; e2e counts plugin reads precisely; live run 15 by adeelahmad · Pull Request #19 · adeelahmad/aloop · GitHub
[go: up one dir, main page]

Skip to content
  • Merged
    Show file tree
    Hide file tree
    Changes from all commits
    Commits
    File filter

    Filter by extension

    Filter by extension

    Conversations
    Failed to load comments.
    Loading
    Jump to
    Jump to file
    Failed to load files.
    Loading
    Diff view
    Diff view
    6 changes: 6 additions & 0 deletions CHANGELOG.md
    Original file line number Diff line number Diff line change
    Expand Up @@ -50,6 +50,12 @@ tag in lockstep. Tag releases as `vMAJOR.MINOR.PATCH`.
    overrides it (`breach:` line) and acts per class (SKILL.md "Breach classes").

    ### Added
    - **`log-execution --help`**, a default ledger (the latest `docs/agents/sprintN/execution.log`) and
    exit 64 with the usage on a wrong call. Reason: it had no usage, so the supervisor read its
    source twice per live run to learn that the last argument is the file. SKILL.md now gives the
    exact call and says the init-block section list is the whole shape (the supervisor also read
    `init.template.md` four times per run). `make e2e-live` counts plugin-source reads precisely
    (tool calls naming a plugin path, `--schema …/schemas/…` excluded): runs 13–15 were 9, 8, 6.
    - **`task-worktree merge <TASK_ID> <BRANCH>`** records the §4.7 lineage on Claude Code, where the
    host makes the worktree and the supervisor merges it (so `task-worktree remove` never runs): a
    `--no-ff` merge, then `worktree.merged` + one `commit.created` per merged commit. SKILL.md tells the
    Expand Down
    1 change: 1 addition & 0 deletions Makefile
    Original file line number Diff line number Diff line change
    Expand Up @@ -111,6 +111,7 @@ test-hosts: ## Budget/worktree regression + Codex hooks emulation + OpenCode plu
    python3 scripts/test/supervisor-scope.test.py
    python3 scripts/test/hook-matchers.test.py
    python3 scripts/test/dispatch.test.py
    python3 scripts/test/log-execution.test.py
    python3 scripts/test/ensure-tools.test.py
    python3 scripts/test/fail-closed.test.py
    @if command -v node >/dev/null; then node scripts/test/opencode-plugin.test.mjs; else echo "node not found — skipped OpenCode test (CI runs it)"; fi
    Expand Down
    5 changes: 3 additions & 2 deletions docs/OVERVIEW.md
    Original file line number Diff line number Diff line change
    Expand Up @@ -103,8 +103,9 @@ The model can ignore a skill; it cannot ignore an exit code.
    - The north-star run, `make e2e-live YES=1 [INTENT=wordfreq]`: a one-line intent becomes
    planned, test-first, gate-verified, merged code in a real Claude Code session with the
    approval given up front. Two intents pass every check (semver parser: 11/11, ~$0.65–1.15,
    11–19 min; word frequencies with two dependent stories: 12/12, $0.86–1.56, 13–45 min; the last run 13m47s, $0.88, one host-run block). Every run so
    far has found a gate false positive that unit tests missed; ROADMAP M0.8 lists them.
    11–19 min; word frequencies with two dependent stories: 13/13, $0.86–1.56, 13–45 min; the last run 13m13s, $1.42, no planning write
    block, every merged commit traced to its task by `events trace`). Every run so far has found a gate false positive or a
    dead path that unit tests missed; ROADMAP M0.8 lists them.
    - One measured real run (*TheNoteBook*, a Go backend): ~17K LOC, 322 tests, 74 commits in 6
    sprints for ~1.55M output tokens, 76% of it on Sonnet. See the
    [README](../README.md#token-efficiency-a-measured-run).
    Expand Down
    2 changes: 1 addition & 1 deletion docs/ROADMAP.md
    Original file line number Diff line number Diff line change
    Expand Up @@ -14,7 +14,7 @@ spec; this table is the plan. Versions are targets, not promises.
    | Milestone | Target | Contents | Status |
    |---|---|---|---|
    | **M0 — Hygiene** | v0.10.x | Mark `DESIGN` / `ARCHITECTURE` / `KICKSTART` historical; fix stale counts (8 → 9 roles) and the README CI section; add the `CONTRIBUTING.md` the README badge links to; one live-session smoke test of the hook wiring (open since `DEVLOG` v0.1) | done: live smoke test `make smoke-live YES=1` (6 checks, ~$0.004 on haiku); its first run found the Linux fail-open in `gate-supervisor-scope` (fixed, regression test added) |
    | **M0.8 — North-star run** | v0.11.x | `make e2e-live YES=1`: a one-line intent (semver parser) goes through retrospective, intake, standards, planner, RED/SCAFFOLD/GREEN per task, structural review and the final gate to merged, tested code in a real Claude Code session, with the approval given up front. Reason: nothing had ever run the whole loop live; the first run found that no role gate fired on plugin installs | run 1: done end to end, 19 min, $1.15 (sonnet supervisor, haiku workers), 22 tests on main, 9/11 checks; found and fixed the `aloop:<role>` matcher bug, a selfcheck crash and copy-merging. Run 2: **11/11**, 12m46s, $0.92, host gates on every role, worktree branches merged with `--no-ff`; found gates looping on the supervisor's contract slips (fixed). Run 3: 11/11, 12m00s, **$0.79**; found GREEN looping on RED's unformatted tests (fixed: RED/SCAFFOLD run the format check) and selfcheck missing the contract in Claude Code worktrees (fixed: the contract is copied in). Run 4 (all fixes): 11/11, 17m56s, $1.10, 12 gate blocks and no loops (28 → 17 → 12). Cost varies $0.79–1.15 per run; next: cut it (scaffold symbol parsing slips such as `(enum,`, supervisor turns), then a second intent beyond the semver fixture. Run 5: 11/11 but 29 min, $1.83, 44 blocks (the planner made 5 tasks; 27 false isolation blocks after workers `cd`'d to the main tree, fixed; subagents also read the plugin's own source, which costs tokens). Run 6: 11/11, **11m10s, $0.65**, plugin-source reads 30 → 2 after the agent prompts said not to; found a dispatch-before-contract loop (25 blocks, fixed by `gate-dispatch`) and annotated scaffold-symbol lines (fixed). Run 7 (after #9): 11/11, 13m46s, $0.98, 20 blocks, no dispatch loop; found RED blocked for listing an earlier task's passing tests, scaffold-symbols tables with a header row and trait-impl lines, and a guessed memory.md frontmatter (all fixed; `aloop init` now seeds memory.md). The re-plan coverage loss is now caught by `gate-final` (every planned test must be on main). Run 8, **second intent** (`INTENT=wordfreq`, two dependent stories): 12/12, 18m05s, $1.53, 54 blocks; found scope globs expanded against the worktree (29 GREEN blocks) and `mod common;` across test crates reported as a HIGH duplicate (57 review halts), both fixed. Run 9 (wordfreq again): 12/12, 19m42s, $1.56, 29 blocks; the ctx-symbols fix had not reached the run (`ensure-tools` rebuilt only on a version bump, fixed) and the planner named story dirs `S1-01/` (block message now names `s1-01-<slug>/`). Run 10 (rebuilt tools): 12/12, $1.20, but 44m48s and 45 blocks: the duplicate false positive is gone (0 HIGH); a RED worker's Write/Edit to the story's output.md was refused by Claude Code's worktree isolation and it stopped 29 times without trying a shell append (fixed: the block and the personas say how), plus 3 stale-`ATTEMPT` tamper blocks (fixed). Fixed after run 10: a worker that repeats an identical block 3 times is released unverified with a `STUCK` marker the supervisor acts on (reason: 29 minutes lost in run 10). Run 11 (all run-10 fixes): 12/12, **12m56s, $0.86, 6 blocks** (from 44m48s, $1.20, 45): no repeated block, one RED report-append block fixed on the first retry, one scaffold fmt block, two structural-reviewer comms blocks (STORY_DIR unset for a sprint-wide review, then a stale block) and 12 `gate-validate-artifact` write blocks on the planner's artifacts. Fixed after run 11: the artifact-write block carries md-db's diagnostics (the planner guessed twice per artifact: 12 of 25 verdicts), `gate-dispatch` requires an existing `STORY_DIR` and names the sprint dir for a whole-wave structural review (2 blocks), and OpenCode reports a stuck release as unverified. Run 12 (those fixes): 12/12, 15m22s, $1.10, 21 blocks: the planner fixed each artifact on the first retry (8 write blocks, from 12) and the reviewer's contract had `STORY_DIR`, but a stale worktree checkout under `.claude/worktrees/` made `ctx-symbols conflicts` report every symbol of main as a duplicate and the reviewer was blocked 10 times on a HIGH finding it could not fix (fixed: the walk skips host checkouts, `.agentic/`, `.transcripts/`). The stuck release now covers the structural reviewer and the final gate too (reason: 10 identical blocks, run 12). Gap: `targets check` blocked the planner twice because a context target named a planned symbol not yet in `src/lib.rs` (reason: planned symbols do not exist before SCAFFOLD; §4.6 owner to decide). Run 13 (those fixes): 12/12, 16m31s, $1.33, 16 blocks; the first build of the walk exclusion also excluded a worktree that IS a host checkout, so the scaffold gate counted every symbol 0 times (9 blocks; the worker was then released by the stuck release and the run still finished); fixed before the run ended (the exclusion applies below the walk root) and covered by a conformance case. Open: `targets check` blocks on the planner's first `targets.json` (2 per run, fixed on retry; §4.6 owner), and sub-agents reading the plugin's own source rose to 17 (from 2–10). Run 14 (all fixes): 12/12, **13m47s, $0.88, 11 blocks**, one host-run block in all (RED touched `src/lib.rs`, fixed on retry), no stuck worker; the rest are planning write blocks fixed on the first retry and workers' own self-checks. The supervisor's GREEN mandates in runs 11–14 named one symbol and `src/lib.rs only`, and the diff scope (from `tasks.md` or `targets scope`) bounds them anyway, so the "GREEN instructions overreaching" item is closed as not reproduced. The planner persona now carries the schema's artifact shapes and the `targets check` rules (to measure). Next: sub-agents reading the plugin's own source (4–17 reads per run: tokens), and a live check that every commit on main traces to its task and intent (`events trace`, §4.7, never measured live) |
    | **M0.8 — North-star run** | v0.11.x | `make e2e-live YES=1`: a one-line intent (semver parser) goes through retrospective, intake, standards, planner, RED/SCAFFOLD/GREEN per task, structural review and the final gate to merged, tested code in a real Claude Code session, with the approval given up front. Reason: nothing had ever run the whole loop live; the first run found that no role gate fired on plugin installs | run 1: done end to end, 19 min, $1.15 (sonnet supervisor, haiku workers), 22 tests on main, 9/11 checks; found and fixed the `aloop:<role>` matcher bug, a selfcheck crash and copy-merging. Run 2: **11/11**, 12m46s, $0.92, host gates on every role, worktree branches merged with `--no-ff`; found gates looping on the supervisor's contract slips (fixed). Run 3: 11/11, 12m00s, **$0.79**; found GREEN looping on RED's unformatted tests (fixed: RED/SCAFFOLD run the format check) and selfcheck missing the contract in Claude Code worktrees (fixed: the contract is copied in). Run 4 (all fixes): 11/11, 17m56s, $1.10, 12 gate blocks and no loops (28 → 17 → 12). Cost varies $0.79–1.15 per run; next: cut it (scaffold symbol parsing slips such as `(enum,`, supervisor turns), then a second intent beyond the semver fixture. Run 5: 11/11 but 29 min, $1.83, 44 blocks (the planner made 5 tasks; 27 false isolation blocks after workers `cd`'d to the main tree, fixed; subagents also read the plugin's own source, which costs tokens). Run 6: 11/11, **11m10s, $0.65**, plugin-source reads 30 → 2 after the agent prompts said not to; found a dispatch-before-contract loop (25 blocks, fixed by `gate-dispatch`) and annotated scaffold-symbol lines (fixed). Run 7 (after #9): 11/11, 13m46s, $0.98, 20 blocks, no dispatch loop; found RED blocked for listing an earlier task's passing tests, scaffold-symbols tables with a header row and trait-impl lines, and a guessed memory.md frontmatter (all fixed; `aloop init` now seeds memory.md). The re-plan coverage loss is now caught by `gate-final` (every planned test must be on main). Run 8, **second intent** (`INTENT=wordfreq`, two dependent stories): 12/12, 18m05s, $1.53, 54 blocks; found scope globs expanded against the worktree (29 GREEN blocks) and `mod common;` across test crates reported as a HIGH duplicate (57 review halts), both fixed. Run 9 (wordfreq again): 12/12, 19m42s, $1.56, 29 blocks; the ctx-symbols fix had not reached the run (`ensure-tools` rebuilt only on a version bump, fixed) and the planner named story dirs `S1-01/` (block message now names `s1-01-<slug>/`). Run 10 (rebuilt tools): 12/12, $1.20, but 44m48s and 45 blocks: the duplicate false positive is gone (0 HIGH); a RED worker's Write/Edit to the story's output.md was refused by Claude Code's worktree isolation and it stopped 29 times without trying a shell append (fixed: the block and the personas say how), plus 3 stale-`ATTEMPT` tamper blocks (fixed). Fixed after run 10: a worker that repeats an identical block 3 times is released unverified with a `STUCK` marker the supervisor acts on (reason: 29 minutes lost in run 10). Run 11 (all run-10 fixes): 12/12, **12m56s, $0.86, 6 blocks** (from 44m48s, $1.20, 45): no repeated block, one RED report-append block fixed on the first retry, one scaffold fmt block, two structural-reviewer comms blocks (STORY_DIR unset for a sprint-wide review, then a stale block) and 12 `gate-validate-artifact` write blocks on the planner's artifacts. Fixed after run 11: the artifact-write block carries md-db's diagnostics (the planner guessed twice per artifact: 12 of 25 verdicts), `gate-dispatch` requires an existing `STORY_DIR` and names the sprint dir for a whole-wave structural review (2 blocks), and OpenCode reports a stuck release as unverified. Run 12 (those fixes): 12/12, 15m22s, $1.10, 21 blocks: the planner fixed each artifact on the first retry (8 write blocks, from 12) and the reviewer's contract had `STORY_DIR`, but a stale worktree checkout under `.claude/worktrees/` made `ctx-symbols conflicts` report every symbol of main as a duplicate and the reviewer was blocked 10 times on a HIGH finding it could not fix (fixed: the walk skips host checkouts, `.agentic/`, `.transcripts/`). The stuck release now covers the structural reviewer and the final gate too (reason: 10 identical blocks, run 12). Gap: `targets check` blocked the planner twice because a context target named a planned symbol not yet in `src/lib.rs` (reason: planned symbols do not exist before SCAFFOLD; §4.6 owner to decide). Run 13 (those fixes): 12/12, 16m31s, $1.33, 16 blocks; the first build of the walk exclusion also excluded a worktree that IS a host checkout, so the scaffold gate counted every symbol 0 times (9 blocks; the worker was then released by the stuck release and the run still finished); fixed before the run ended (the exclusion applies below the walk root) and covered by a conformance case. Open: `targets check` blocks on the planner's first `targets.json` (2 per run, fixed on retry; §4.6 owner), and sub-agents reading the plugin's own source rose to 17 (from 2–10). Run 14 (all fixes): 12/12, **13m47s, $0.88, 11 blocks**, one host-run block in all (RED touched `src/lib.rs`, fixed on retry), no stuck worker; the rest are planning write blocks fixed on the first retry and workers' own self-checks. The supervisor's GREEN mandates in runs 11–14 named one symbol and `src/lib.rs only`, and the diff scope (from `tasks.md` or `targets scope`) bounds them anyway, so the "GREEN instructions overreaching" item is closed as not reproduced. The planner persona now carries the schema's artifact shapes and the `targets check` rules (to measure). Run 15 (planner shapes + rules, `task-worktree merge`): **13/13**, 13m13s, $1.42, 6 blocks: **0 planning write blocks** (from 8–12: every artifact and `targets.json` passed first time), no host-run block, and §4.7 is live on Claude Code for the first time: 8 commits, 8 `commit.created`, 6 `worktree.merged`, every merged commit answers `events trace` (new e2e check). Plugin-source reads, counted precisely (tool calls naming a plugin path; `--schema` uses excluded): 9, 8, 6 in runs 13–15, all by the supervisor reading `init.template.md` and `log-execution` to learn shapes SKILL.md now states (`log-execution --help` added). Next: measure those reads at 0–2; the remaining blocks are workers' own first-try slips (scaffold fmt, RED touching `src/`), so the next cost lever is the supervisor's turns and the per-task worker prompts |
    | **M0.6 — Gates fail closed** | v0.11.x | A gate that crashes (exit 1, e.g. an unbound variable) is a non-blocking hook error on Claude Code, so the action goes through. Make every gate trap unexpected errors into exit 2 with the reason, and add a conformance case per gate. Reason: the smoke run showed one real instance | done: `_failclosed.sh` on all 10 sub-agent gates + supervisor-scope inside a sprint; `scripts/test/fail-closed.test.py` (5 checks). Left fail-open on purpose: `gate-commit-author` and `gate-validate-artifact` outside a sprint, which see every session on the machine |
    | **M0.7 — Supervisor-owned task contract** | v0.11.x | The contract lives in the main tree, `.agentic/state/tasks/<TASK_ID>.env`, and the dispatch prompt's first line names the task; gates and the time box find it from the host's transcript, a worktree `task.env` that disagrees blocks, and sub-agents may not Write/Edit `.agentic/state/` or `task.env`. Settled by a live probe: Claude Code's `isolation: "worktree"` makes `.claude/worktrees/agent-<id>` at dispatch, so a pre-written in-worktree contract was impossible there. Bash writes are guarded too (a command naming the contract with a write verb is blocked; a heuristic, so an obfuscated write still gets through). On hosts with no sub-agent transcript the worktree's `TASK_ID` (or its `.agentic/worktrees/<id>` path) picks the contract | done: `make conformance` (58 today) + `scripts/test/{budget-worktree,supervisor-scope}.test.py` |
    | **M0.5 — Rename to aloop** | v0.11.0 | Plugin, marketplace, skill, CLI (`aloop`, `aloop init`), OpenCode plugin file and docs renamed; `agentic`, `agentic init`, `agentic-init` and `agentic bind` kept as aliases; OpenCode/Codex installs clean up pre-rename files. Data layout and settings keys unchanged (moved to M1). GitHub repo rename is the owner's call | done: `scripts/test/rename-compat.test.py` |
    Expand Down
    2 changes: 1 addition & 1 deletion plugin/bin/README.md
    Original file line number Diff line number Diff line change
    Expand Up @@ -43,7 +43,7 @@ ctx-symbols) → WARN + grep fallback; never a silent pass.
    | `worktree-hygiene` | SubagentStart · exec roles — sparse-checks `docs/agents/` out of the new worktree (workers reach it via the absolute `STORY_DIR` back into the main tree, never their own copy) + bootstraps the main tree's `.gitignore` with `.agentic/`/`.transcripts/` if missing. Runs regardless of whether the harness's built-in isolation or `worktree-create` made the worktree. |
    | `worktree-create` | OPTIONAL `git worktree add` helper — wire into **user `settings.json`** as a `WorktreeCreate` hook for a non-git VCS (NOT a valid plugin-manifest event). Built-in `isolation: "worktree"` covers plain git. Also applies the same sparse-checkout + `.gitignore` bootstrap as `worktree-hygiene`. |
    | `worktree-remove` | OPTIONAL `git worktree remove` cleanup companion for the above (user `settings.json` `WorktreeRemove`) |
    | `log-execution` | append a transition line to `execution.log` |
    | `log-execution` | `wave-K activity SN-NN transition [detail…] [execution.log]` — append one transition line (default ledger: the latest sprint's); `--help` |
    | `ensure-tools` | SessionStart — builds md-db + ctx-symbols into `$CLAUDE_PLUGIN_DATA/bin` on first run / version change (background), tells the model where the tools are. `--sync` build now · `--wait` · `--resolve T` real binary path |
    | `md-db`, `ctx-symbols` | PATH shims (this `bin/` is on PATH in every session): exec the real binary, building it once if needed. Gates test presence with `ensure-tools --resolve`, never `command -v` |
    | `aloop` | the one entrypoint: `aloop <tool> …` (`aloop --list`); `aloop init` = aloop-init, `aloop install` = install. Linked into `~/.local/bin` for Codex/OpenCode |
    Expand Down
    Loading
    Loading
    0