8000 orchestrator: one-task waves run in the foreground; the supervisor never commits on main; live run 16 by adeelahmad · Pull Request #20 · adeelahmad/aloop · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,14 @@ tag in lockstep. Tag releases as `vMAJOR.MINOR.PATCH`.
## [Unreleased]

### Changed
- **The supervisor never commits on main** (SKILL.md): every commit on main arrives through
`task-worktree merge`. Reason: live run 16's one untraceable commit was the supervisor's own
`Track Cargo.lock`; the e2e fixture now commits the lockfile in its seed.
- **A wave with one dispatchable task runs in the foreground** (SKILL.md orchestrator loop).
Reason: live run 15 dispatched every single-task wave in the background and woke 16 times,
each wake re-reading the supervisor's context (5.5M cached tokens, $1.26 of $1.42); run 14,
foreground, woke twice for $0.75. Background dispatch + `budget watch` stays the rule for
parallel waves. The host table's "merge a passed task" row names `task-worktree merge`.
- **The soft token limit is telemetry only** (ROADMAP §3.1). Crossing it no longer tells
the worker to wrap up; `budget` records `task.token_soft_limit` once per attempt in
`.agentic/logs/events.jsonl`, `stats.md` counts it, and `NEXT.md` lists those tasks as
Expand Down
4 changes: 2 additions & 2 deletions docs/OVERVIEW.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,8 +103,8 @@ The model can ignore a skill; it cannot ignore an exit code.
- The north-star run, `make e2e-live YES=1 [INTENT=wordfreq]`: a one-line intent becomes
planned, test-first, gate-verified, merged code in a real Claude Code session with the
approval given up front. Two intents pass every check (semver parser: 11/11, ~$0.65–1.15,
11–19 min; word frequencies with two dependent stories: 13/13, $0.86–1.56, 13–45 min; the last run 13m13s, $1.42, no planning write
block, every merged commit traced to its task by `events trace`). Every run so far has found a gate false positive or a
11–19 min; word frequencies with two dependent stories: 13/13, $0.86–1.56, 13–45 min; the last run 13m04s, $0.79, 4 supervisor wakes,
every task commit traced to its task by `events trace`). Every run so far has found a gate false positive or a
dead path that unit tests missed; ROADMAP M0.8 lists them.
- One measured real run (*TheNoteBook*, a Go backend): ~17K LOC, 322 tests, 74 commits in 6
sprints for ~1.55M output tokens, 76% of it on Sonnet. See the
Expand Down
2 changes: 1 addition & 1 deletion docs/ROADMAP.md
BD20
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ spec; this table is the plan. Versions are targets, not promises.
| Milestone | Target | Contents | Status |
|---|---|---|---|
| **M0 — Hygiene** | v0.10.x | Mark `DESIGN` / `ARCHITECTURE` / `KICKSTART` historical; fix stale counts (8 → 9 roles) and the README CI section; add the `CONTRIBUTING.md` the README badge links to; one live-session smoke test of the hook wiring (open since `DEVLOG` v0.1) | done: live smoke test `make smoke-live YES=1` (6 checks, ~$0.004 on haiku); its first run found the Linux fail-open in `gate-supervisor-scope` (fixed, regression test added) |
| **M0.8 — North-star run** | v0.11.x | `make e2e-live YES=1`: a one-line intent (semver parser) goes through retrospective, intake, standards, planner, RED/SCAFFOLD/GREEN per task, structural review and the final gate to merged, tested code in a real Claude Code session, with the approval given up front. Reason: nothing had ever run the whole loop live; the first run found that no role gate fired on plugin installs | run 1: done end to end, 19 min, $1.15 (sonnet supervisor, haiku workers), 22 tests on main, 9/11 checks; found and fixed the `aloop:<role>` matcher bug, a selfcheck crash and copy-merging. Run 2: **11/11**, 12m46s, $0.92, host gates on every role, worktree branches merged with `--no-ff`; found gates looping on the supervisor's contract slips (fixed). Run 3: 11/11, 12m00s, **$0.79**; found GREEN looping on RED's unformatted tests (fixed: RED/SCAFFOLD run the format check) and selfcheck missing the contract in Claude Code worktrees (fixed: the contract is copied in). Run 4 (all fixes): 11/11, 17m56s, $1.10, 12 gate blocks and no loops (28 → 17 → 12). Cost varies $0.79–1.15 per run; next: cut it (scaffold symbol parsing slips such as `(enum,`, supervisor turns), then a second intent beyond the semver fixture. Run 5: 11/11 but 29 min, $1.83, 44 blocks (the planner made 5 tasks; 27 false isolation blocks after workers `cd`'d to the main tree, fixed; subagents also read the plugin's own source, which costs tokens). Run 6: 11/11, **11m10s, $0.65**, plugin-source reads 30 → 2 after the agent prompts said not to; found a dispatch-before-contract loop (25 blocks, fixed by `gate-dispatch`) and annotated scaffold-symbol lines (fixed). Run 7 (after #9): 11/11, 13m46s, $0.98, 20 blocks, no dispatch loop; found RED blocked for listing an earlier task's passing tests, scaffold-symbols tables with a header row and trait-impl lines, and a guessed memory.md frontmatter (all fixed; `aloop init` now seeds memory.md). The re-plan coverage loss is now caught by `gate-final` (every planned test must be on main). Run 8, **second intent** (`INTENT=wordfreq`, two dependent stories): 12/12, 18m05s, $1.53, 54 blocks; found scope globs expanded against the worktree (29 GREEN blocks) and `mod common;` across test crates reported as a HIGH duplicate (57 review halts), both fixed. Run 9 (wordfreq again): 12/12, 19m42s, $1.56, 29 blocks; the ctx-symbols fix had not reached the run (`ensure-tools` rebuilt only on a version bump, fixed) and the planner named story dirs `S1-01/` (block message now names `s1-01-<slug>/`). Run 10 (rebuilt tools): 12/12, $1.20, but 44m48s and 45 blocks: the duplicate false positive is gone (0 HIGH); a RED worker's Write/Edit to the story's output.md was refused by Claude Code's worktree isolation and it stopped 29 times without trying a shell append (fixed: the block and the personas say how), plus 3 stale-`ATTEMPT` tamper blocks (fixed). Fixed after run 10: a worker that repeats an identical block 3 times is released unverified with a `STUCK` marker the supervisor acts on (reason: 29 minutes lost in run 10). Run 11 (all run-10 fixes): 12/12, **12m56s, $0.86, 6 blocks** (from 44m48s, $1.20, 45): no repeated block, one RED report-append block fixed on the first retry, one scaffold fmt block, two structural-reviewer comms blocks (STORY_DIR unset for a sprint-wide review, then a stale block) and 12 `gate-validate-artifact` write blocks on the planner's artifacts. Fixed after run 11: the artifact-write block carries md-db's diagnostics (the planner guessed twice per artifact: 12 of 25 verdicts), `gate-dispatch` requires an existing `STORY_DIR` and names the sprint dir for a whole-wave structural review (2 blocks), and OpenCode reports a stuck release as unverified. Run 12 (those fixes): 12/12, 15m22s, $1.10, 21 blocks: the planner fixed each artifact on the first retry (8 write blocks, from 12) and the reviewer's contract had `STORY_DIR`, but a stale worktree checkout under `.claude/worktrees/` made `ctx-symbols conflicts` report every symbol of main as a duplicate and the reviewer was blocked 10 times on a HIGH finding it could not fix (fixed: the walk skips host checkouts, `.agentic/`, `.transcripts/`). The stuck release now covers the structural reviewer and the final gate too (reason: 10 identical blocks, run 12). Gap: `targets check` blocked the planner twice because a context target named a planned symbol not yet in `src/lib.rs` (reason: planned symbols do not exist before SCAFFOLD; §4.6 owner to decide). Run 13 (those fixes): 12/12, 16m31s, $1.33, 16 blocks; the first build of the walk exclusion also excluded a worktree that IS a host checkout, so the scaffold gate counted every symbol 0 times (9 blocks; the worker was then released by the stuck release and the run still finished); fixed before the run ended (the exclusion applies below the walk root) and covered by a conformance case. Open: `targets check` blocks on the planner's first `targets.json` (2 per run, fixed on retry; §4.6 owner), and sub-agents reading the plugin's own source rose to 17 (from 2–10). Run 14 (all fixes): 12/12, **13m47s, $0.88, 11 blocks**, one host-run block in all (RED touched `src/lib.rs`, fixed on retry), no stuck worker; the rest are planning write blocks fixed on the first retry and workers' own self-checks. The supervisor's GREEN mandates in runs 11–14 named one symbol and `src/lib.rs only`, and the diff scope (from `tasks.md` or `targets scope`) bounds them anyway, so the "GREEN instructions overreaching" item is closed as not reproduced. The planner persona now carries the schema's artifact shapes and the `targets check` rules (to measure). Run 15 (planner shapes + rules, `task-worktree merge`): **13/13**, 13m13s, $1.42, 6 blocks: **0 planning write blocks** (from 8–12: every artifact and `targets.json` passed first time), no host-run block, and §4.7 is live on Claude Code for the first time: 8 commits, 8 `commit.created`, 6 `worktree.merged`, every merged commit answers `events trace` (new e2e check). Plugin-source reads, counted precisely (tool calls naming a plugin path; `--schema` uses excluded): 9, 8, 6 in runs 13–15, all by the supervisor reading `init.template.md` and `log-execution` to learn shapes SKILL.md now states (`log-execution --help` added). Next: measure those reads at 0–2; the remaining blocks are workers' own first-try slips (scaffold fmt, RED touching `src/`), so the next cost lever is the supervisor's turns and the per-task worker prompts |
| **M0.8 — North-star run** | v0.11.x | `make e2e-live YES=1`: a one-line intent (semver parser) goes through retrospective, intake, standards, planner, RED/SCAFFOLD/GREEN per task, structural review and the final gate to merged, tested code in a real Claude Code session, with the approval given up front. Reason: nothing had ever run the whole loop live; the first run found that no role gate fired on plugin installs | run 1: done end to end, 19 min, $1.15 (sonnet supervisor, haiku workers), 22 tests on main, 9/11 checks; found and fixed the `aloop:<role>` matcher bug, a selfcheck crash and copy-merging. Run 2: **11/11**, 12m46s, $0.92, host gates on every role, worktree branches merged with `--no-ff`; found gates looping on the supervisor's contract slips (fixed). Run 3: 11/11, 12m00s, **$0.79**; found GREEN looping on RED's unformatted tests (fixed: RED/SCAFFOLD run the format check) and selfcheck missing the contract in Claude Code worktrees (fixed: the contract is copied in). Run 4 (all fixes): 11/11, 17m56s, $1.10, 12 gate blocks and no loops (28 → 17 → 12). Cost varies $0.79–1.15 per run; next: cut it (scaffold symbol parsing slips such as `(enum,`, supervisor turns), then a second intent beyond the semver fixture. Run 5: 11/11 but 29 min, $1.83, 44 blocks (the planner made 5 tasks; 27 false isolation blocks after workers `cd`'d to the main tree, fixed; subagents also read the plugin's own source, which costs tokens). Run 6: 11/11, **11m10s, $0.65**, plugin-source reads 30 → 2 after the agent prompts said not to; found a dispatch-before-contract loop (25 blocks, fixed by `gate-dispatch`) and annotated scaffold-symbol lines (fixed). Run 7 (after #9): 11/11, 13m46s, $0.98, 20 blocks, no dispatch loop; found RED blocked for listing an earlier task's passing tests, scaffold-symbols tables with a header row and trait-impl lines, and a guessed memory.md frontmatter (all fixed; `aloop init` now seeds memory.md). The re-plan coverage loss is now caught by `gate-final` (every planned test must be on main). Run 8, **second intent** (`INTENT=wordfreq`, two dependent stories): 12/12, 18m05s, $1.53, 54 blocks; found scope globs expanded against the worktree (29 GREEN blocks) and `mod common;` across test crates reported as a HIGH duplicate (57 review halts), both fixed. Run 9 (wordfreq again): 12/12, 19m42s, $1.56, 29 blocks; the ctx-symbols fix had not reached the run (`ensure-tools` rebuilt only on a version bump, fixed) and the planner named story dirs `S1-01/` (block message now names `s1-01-<slug>/`). Run 10 (rebuilt tools): 12/12, $1.20, but 44m48s and 45 blocks: the duplicate false positive is gone (0 HIGH); a RED worker's Write/Edit to the story's output.md was refused by Claude Code's worktree isolation and it stopped 29 times without trying a shell append (fixed: the block and the personas say how), plus 3 stale-`ATTEMPT` tamper blocks (fixed). Fixed after run 10: a worker that repeats an identical block 3 times is released unverified with a `STUCK` marker the supervisor acts on (reason: 29 minutes lost in run 10). Run 11 (all run-10 fixes): 12/12, **12m56s, $0.86, 6 blocks** (from 44m48s, $1.20, 45): no repeated block, one RED report-append block fixed on the first retry, one scaffold fmt block, two structural-reviewer comms blocks (STORY_DIR unset for a sprint-wide review, then a stale block) and 12 `gate-validate-artifact` write blocks on the planner's artifacts. Fixed after run 11: the artifact-write block carries md-db's diagnostics (the planner guessed twice per artifact: 12 of 25 verdicts), `gate-dispatch` requires an existing `STORY_DIR` and names the sprint dir for a whole-wave structural review (2 blocks), and OpenCode reports a stuck release as unverified. Run 12 (those fixes): 12/12, 15m22s, $1.10, 21 blocks: the planner fixed each artifact on the first retry (8 write blocks, from 12) and the reviewer's contract had `STORY_DIR`, but a stale worktree checkout under `.claude/worktrees/` made `ctx-symbols conflicts` report every symbol of main as a duplicate and the reviewer was blocked 10 times on a HIGH finding it could not fix (fixed: the walk skips host checkouts, `.agentic/`, `.transcripts/`). The stuck release now covers the structural reviewer and the final gate too (reason: 10 identical blocks, run 12). Gap: `targets check` blocked the planner twice because a context target named a planned symbol not yet in `src/lib.rs` (reason: planned symbols do not exist before SCAFFOLD; §4.6 owner to decide). Run 13 (those fixes): 12/12, 16m31s, $1.33, 16 blocks; the first build of the walk exclusion also excluded a worktree that IS a host checkout, so the scaffold gate counted every symbol 0 times (9 blocks; the worker was then released by the stuck release and the run still finished); fixed before the run ended (the exclusion applies below the walk root) and covered by a conformance case. Open: `targets check` blocks on the planner's first `targets.json` (2 per run, fixed on retry; §4.6 owner), and sub-agents reading the plugin's own source rose to 17 (from 2–10). Run 14 (all fixes): 12/12, **13m47s, $0.88, 11 blocks**, one host-run block in all (RED touched `src/lib.rs`, fixed on retry), no stuck worker; the rest are planning write blocks fixed on the first retry and workers' own self-checks. The supervisor's GREEN mandates in runs 11–14 named one symbol and `src/lib.rs only`, and the diff scope (from `tasks.md` or `targets scope`) bounds them anyway, so the "GREEN instructions overreaching" item is closed as not reproduced. The planner persona now carries the schema's artifact shapes and the `targets check` rules (to measure). Run 15 (planner shapes + rules, `task-worktree merge`): **13/13**, 13m13s, $1.42, 6 blocks: **0 planning write blocks** (from 8–12: every artifact and `targets.json` passed first time), no host-run block, and §4.7 is live on Claude Code for the first time: 8 commits, 8 `commit.created`, 6 `worktree.merged`, every merged commit answers `events trace` (new e2e check). Plugin-source reads, counted precisely (tool calls naming a plugin path; `--schema` uses excluded): 9, 8, 6 in runs 13–15, all by the supervisor reading `init.template.md` and `log-execution` to learn shapes SKILL.md now states (`log-execution --help` added). Run 15 cost more than run 14 ($1.42 vs $0.88) because the supervisor dispatched every single-task wave in the background and woke 16 times (run 14: 2), each wake re-reading its context: 5.5M cached Sonnet tokens vs 2.8M. SKILL.md now runs a one-task wave in the foreground. Run 16 (that rule): 12/13, **13m04s, $0.79** (from $1.42), **4 wakes** (from 16), 2.3M cached Sonnet tokens (from 5.5M), 9 blocks, 4 plugin-source reads (from 6); the one failed check was the trace: the supervisor committed `Track Cargo.lock` on main itself (the fixture left the lockfile untracked), a commit no task owns. SKILL.md: the supervisor never commits on main; the fixture seeds the lockfile. Next: a deterministic gate for that rule if a run breaks it again (a `git commit` in the main checkout by the supervisor while a sprint executes), and the workers' first-try slips (scaffold fmt, RED touching `src/`) |
| **M0.6 — Gates fail closed** | v0.11.x | A gate that crashes (exit 1, e.g. an unbound variable) is a non-blocking hook error on Claude Code, so the action goes through. Make every gate trap unexpected errors into exit 2 with the reason, and add a conformance case per gate. Reason: the smoke run showed one real instance | done: `_failclosed.sh` on all 10 sub-agent gates + supervisor-scope inside a sprint; `scripts/test/fail-closed.test.py` (5 checks). Left fail-open on purpose: `gate-commit-author` and `gate-validate-artifact` outside a sprint, which see every session on the machine |
| **M0.7 — Supervisor-owned task contract** | v0.11.x | The contract lives in the main tree, `.agentic/state/tasks/<TASK_ID>.env`, and the dispatch prompt's first line names the task; gates and the time box find it from the host's transcript, a worktree `task.env` that disagrees blocks, and sub-agents may not Write/Edit `.agentic/state/` or `task.env`. Settled by a live probe: Claude Code's `isolation: "worktree"` makes `.claude/worktrees/agent-<id>` at dispatch, so a pre-written in-worktree contract was impossible there. Bash writes are guarded too (a command naming the contract with a write verb is blocked; a heuristic, so an obfuscated write still gets through). On hosts with no sub-agent transcript the worktree's `TASK_ID` (or its `.agentic/worktrees/<id>` path) picks the contract | done: `make conformance` (58 today) + `scripts/test/{budget-worktree,supervisor-scope}.test.py` |
| **M0.5 — Rename to aloop** | v0.11.0 | Plugin, marketplace, skill, CLI (`aloop`, `aloop init`), OpenCode plugin file and docs renamed; `agentic`, `agentic init`, `agentic-init` and `agentic bind` kept as aliases; OpenCode/Codex installs clean up pre-rename files. Data layout and settings keys unchanged (moved to M1). GitHub repo rename is the owner's call | done: `scripts/test/rename-compat.test.py` |
Expand Down
Loading
Loading
0