8000 Run 47: cli on #81, 13/13, 2 blocks by adeelahmad · Pull Request #89 · adeelahmad/aloop · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/ROADMAP.md
8000
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ original plan.
| **M5 — Narrow context** | v0.16 | Session 6: semantic execution map, shadow workspace, `target` / `context` CLI (§4.6) | done: targets, shadow workspace, `target`/`context`, LSP seam, GREEN scope from targets, and tasks.md/validate.md/plan.md rendered from the semantic plan (`targets render`; tests: `semantic-targets`, `lsp-adapter`, `targets-scope`, `plan-render`) |
| **M6 — Audit** | v0.17 | Sessions 7–8: event stream, rationale, authority, lineage query (§4.7); provider seam (§4.8) | done: `aladdin.event/1` stream, decisions, authority, lineage, `events trace SHA`, commit messages and graph export (§4.7); impact provider seam (§4.8) |
| **M7 — Any host** | v0.18 | Session 9: `HostAdapter`, capability matrix, `.aladdin/config.yaml`, host swap, metrics (§4.9–4.10) | done: `HostAdapter` + `host check` in CI, capability matrix, usage accuracy, `.aladdin/config.yaml` with migration, `host swap`, the full metrics table incl. context-window share (tests: `host-adapter`, `portable-config`, `host-swap`, `metrics`) |
| **M8 — Aladdin 1.0** | v1.0 | Session 10: `/aladdin` + `aladdin` CLI with aliases, site, measured public runs (§4.11). The rename to aloop shipped early in v0.11.0 | in progress: `bin/aladdin` + `/aladdin` with `aloop`/`agentic` aliases (test: `aladdin-cli.test.py`); natural-language auto-trigger on every host (`intent-trigger`, test: `intent-trigger.test.py`); README reordered product-first; site at `docs/index.html` (Pages switch is the owner's). evidence ledger `docs/evidence/README.md` + `run-summary` (test: `evidence.test.py`). `e2e-live.sh --host codex|opencode` runs the same loop on those hosts (`aloop install <host>` into the run repo, then `codex exec --json` / `opencode run --format json`; `--dry-run` prints the command; test: `e2e-live-hosts.test.py`). Open: the measured public runs themselves (this environment has no `codex`/`opencode` CLI: run `make e2e-live YES=1 HOST=codex` and `HOST=opencode` on a machine with both and add the rows; reason: a number we cannot measure is not published) Mac release run (2026-10-19) found two bugs Linux CI never sees, both fixed with tests: `stats sprint-close` counted 0 attempts when the project sat behind a symlink (macOS `/var` → `/private/var`; paths are now compared by realpath), and `make conformance` hung when another aloop checkout's `plugin/bin` was ahead on PATH (`ensure-tools --resolve` returned that checkout's shim; any aloop bin dir is now skipped). RED is now verified on any language: a non-cargo repo runs its `standards.md` matrix's test command, and a RED whose tests pass or do not collect is blocked (pytest, `go test`; test: `any-language.test.py`). Reason: the north star is any intent, and RED was dynamically checked only for Rust. GREEN and the final gate now refuse Python/Go/JS skips, and the final gate finds a planned `def`/`func` test (before, it found only `fn`, so every planned Python test blocked). `INTENT=slugify` is the Python fixture. Run 36 (slugify, the first Python run): 11/12, 11m37s, $0.83, 5 wakes, 0 plugin-source reads, 15 blocks (9 caught by self-checks). The sprint merged a correct `slugify` with 13 passing tests and the final gate passed. The one failed check was the fixture's own: bare `pytest` leaves the repo root off `sys.path` (now `python3 -m pytest`). It found two gate bugs that Rust runs had hidden. First, a worker's worktree has no `docs/agents/` (git-ignored), so the gates never found `standards.md` and ran the default Rust matrix: RED blocked 4 times on `cargo fmt`, and in a Rust project a custom matrix was silently swapped for the default. Second, `ctx-symbols conflicts` walked the merged task checkout under `.aladdin/worktrees/` and reported `slugify` as a HIGH duplicate, so the structural reviewer was rejected 4 times. Fixed with tests: the gates read the story's sprint `standards.md` (else this tree, else the main tree), and a non-cargo project with no matrix blocks with a message naming `standards.md`; ctx-symbols skips `.aladdin/`. Run 37 on these fixes: 12/12, 9m42s, $0.69, 1 wake, 11 blocks (7 caught by self-checks). It found that a `standards.md` whose matrix is a table parses empty, so RED blocked 3 times; `gate-standards-cited` now refuses a standards.md with no runnable `## Gate matrix` block (test: `any-language.test.py`). Run 38 (`INTENT=roman`, the first Go run): 12/12, 14m15s, $1.02, 3 wakes, 9 blocks (7 caught by self-checks). It found that the sprint dir's `init.md` had no `type: init` header (the supervisor began it with a dispatch block), so the structural reviewer was blocked 7 times; `aloop budget contract` now writes that header (test: `budget-contract.test.py`). Run 39 on the fix: 12/12, 7m52s, $0.56, 0 init.md blocks. It found that GREEN's default scope let it touch Rust test files only, so a Go GREEN removing RED's `_test.go` shim was blocked 10 times; GREEN now shares RED's any-language test globs (test: `any-language.test.py`). Run 40 on it: 12/12, 13m03s, $0.75, 1 wake, 0 GREEN scope blocks (5 blocks, all caught by self-checks). The structural reviewer's HIGH foundation-poisoning finding in runs 38 and 40 was a false positive: a worker's `selfcheck` resolved a stale 0.11.0 `ctx-symbols` (before the `.aladdin/` exclusion) ahead of the session's current build; `ensure-tools --resolve` now skips a stale own build (test: `ensure-tools.test.py`). RED is verified on JS too (a test file that cannot load what it imports is blocked; test: `any-language.test.py`), and `INTENT=camel` is the JS fixture. Run 41 (camel, the first JS run): 12/12, 6m45s, $0.56, 1 wake, 2 blocks (both RED self-checks refusing a source edit). The loop is now measured on Rust, Python, Go and JS. Run 42 (`INTENT=extend`, the first brownfield intent: add `compare` to a crate whose `parse` already works and is tested): 13/13, 7m11s, $0.67, 2 blocks (both self-check catches); the seed's code and tests were left unchanged and still pass (reason: every earlier fixture started from an empty library, and most real intents land on existing code). Run 43 (`INTENT=modify`: change that existing `parse` to accept a leading `v` and trim whitespace): 13/13, 9m10s, $0.67, 6 blocks, all self-check catches; the change was 3 lines in `parse`, with its signature and seed tests kept. `INTENT=cli` (an application: a wc-like binary tested by running it, checked by running the merged binary) is ready; run 44 (the retry after the usage limit lifted): 13/13, 24m55s, $1.54, 6 blocks (all self-check catches), and the merged binary works. Its cost came from planning: 2 stories and 7 tasks for a small tool, so 7 RED/SCAFFOLD/GREEN cycles and 56 supervisor turns (runs 42–43: 1–2 tasks, 16–30 turns). The planner now sizes tasks up to the budget instead of one per behaviour (reason: per-task cost is a fixed ~8 supervisor turns). Run 45 on it: 3 tasks (from 7), 13/13, 16m22s, $1.07. Its HIGH foundation-poisoning halts were `ctx-symbols` reading a test's `const USAGE` as a duplicate of the binary's; a test constant no longer counts (test: `conflicts.rs`). Run 46 on both fixes: 3 tasks, 13/13, 15m19s, $0.99, no review halt, 6/6 blocks self-check catches. Its 4 `targets` "symbol is not defined" blocks were T3 modifying symbols T2 plans in an existing file; a symbol another target plans now counts (test: `semantic-targets.test.py` case 8). Planner friction next: across runs 36–41 the most frequent block was `targets` "create needs planned_symbol" on a new test file (8); a test-file create now takes it from the task's `tests` (test: `semantic-targets.test.py`). The `gate-validate-artifact` plan.md refusals (4) were the persona's bare `stage: 1`, now quoted and checked (test: `persona-shapes.test.py`). Next: the final-gate block order (3). Running every suite under a locally built bash 3.2 found that the `standards.md` gate matrix was silently replaced by the default Rust matrix (`mapfile` does not exist in bash 3.2). It is fixed and guarded by `bash-portable.test.py`. A `macos` CI job now runs the host tests and the conformance fixtures on macOS (reason: both were found only by hand on the owner's Mac). |
| **M8 — Aladdin 1.0** | v1.0 | Session 10: `/aladdin` + `aladdin` CLI with aliases, site, measured public runs (§4.11). The rename to aloop shipped early in v0.11.0 | in progress: `bin/aladdin` + `/aladdin` with `aloop`/`agentic` aliases (test: `aladdin-cli.test.py`); natural-language auto-trigger on every host (`intent-trigger`, test: `intent-trigger.test.py`); README reordered product-first; site at `docs/index.html` (Pages switch is the owner's). evidence ledger `docs/evidence/README.md` + `run-summary` (test: `evidence.test.py`). `e2e-live.sh --host codex|opencode` runs the same loop on those hosts (`aloop install <host>` into the run repo, then `codex exec --json` / `opencode run --format json`; `--dry-run` prints the command; test: `e2e-live-hosts.test.py`). Open: the measured public runs themselves (this environment has no `codex`/`opencode` CLI: run `make e2e-live YES=1 HOST=codex` and `HOST=opencode` on a machine with both and add the rows; reason: a number we cannot measure is not published) Mac release run (2026-10-19) found two bugs Linux CI never sees, both fixed with tests: `stats sprint-close` counted 0 attempts when the project sat behind a symlink (macOS `/var` → `/private/var`; paths are now compared by realpath), and `make conformance` hung when another aloop checkout's `plugin/bin` was ahead on PATH (`ensure-tools --resolve` returned that checkout's shim; any aloop bin dir is now skipped). RED is now verified on any language: a non-cargo repo runs its `standards.md` matrix's test command, and a RED whose tests pass or do not collect is blocked (pytest, `go test`; test: `any-language.test.py`). Reason: the north star is any intent, and RED was dynamically checked only for Rust. GREEN and the final gate now refuse Python/Go/JS skips, and the final gate finds a planned `def`/`func` test (before, it found only `fn`, so every planned Python test blocked). `INTENT=slugify` is the Python fixture. Run 36 (slugify, the first Python run): 11/12, 11m37s, $0.83, 5 wakes, 0 plugin-source reads, 15 blocks (9 caught by self-checks). The sprint merged a correct `slugify` with 13 passing tests and the final gate passed. The one failed check was the fixture's own: bare `pytest` leaves the repo root off `sys.path` (now `python3 -m pytest`). It found two gate bugs that Rust runs had hidden. First, a worker's worktree has no `docs/agents/` (git-ignored), so the gates never found `standards.md` and ran the default Rust matrix: RED blocked 4 times on `cargo fmt`, and in a Rust project a custom matrix was silently swapped for the default. Second, `ctx-symbols conflicts` walked the merged task checkout under `.aladdin/worktrees/` and reported `slugify` as a HIGH duplicate, so the structural reviewer was rejected 4 times. Fixed with tests: the gates read the story's sprint `standards.md` (else this tree, else the main tree), and a non-cargo project with no matrix blocks with a message naming `standards.md`; ctx-symbols skips `.aladdin/`. Run 37 on these fixes: 12/12, 9m42s, $0.69, 1 wake, 11 blocks (7 caught by self-checks). It found that a `standards.md` whose matrix is a table parses empty, so RED blocked 3 times; `gate-standards-cited` now refuses a standards.md with no runnable `## Gate matrix` block (test: `any-language.test.py`). Run 38 (`INTENT=roman`, the first Go run): 12/12, 14m15s, $1.02, 3 wakes, 9 blocks (7 caught by self-checks). It found that the sprint dir's `init.md` had no `type: init` header (the supervisor began it with a dispatch block), so the structural reviewer was blocked 7 times; `aloop budget contract` now writes that header (test: `budget-contract.test.py`). Run 39 on the fix: 12/12, 7m52s, $0.56, 0 init.md blocks. It found that GREEN's default scope let it touch Rust test files only, so a Go GREEN removing RED's `_test.go` shim was blocked 10 times; GREEN now shares RED's any-language test globs (test: `any-language.test.py`). Run 40 on it: 12/12, 13m03s, $0.75, 1 wake, 0 GREEN scope blocks (5 blocks, all caught by self-checks). The structural reviewer's HIGH foundation-poisoning finding in runs 38 and 40 was a false positive: a worker's `selfcheck` resolved a stale 0.11.0 `ctx-symbols` (before the `.aladdin/` exclusion) ahead of the session's current build; `ensure-tools --resolve` now skips a stale own build (test: `ensure-tools.test.py`). RED is verified on JS too (a test file that cannot load what it imports is blocked; test: `any-language.test.py`), and `INTENT=camel` is the JS fixture. Run 41 (camel, the first JS run): 12/12, 6m45s, $0.56, 1 wake, 2 blocks (both RED self-checks refusing a source edit). The loop is now measured on Rust, Python, Go and JS. Run 42 (`INTENT=extend`, the first brownfield intent: add `compare` to a crate whose `parse` already works and is tested): 13/13, 7m11s, $0.67, 2 blocks (both self-check catches); the seed's code and tests were left unchanged and still pass (reason: every earlier fixture started from an empty library, and most real intents land on existing code). Run 43 (`INTENT=modify`: change that existing `parse` to accept a leading `v` and trim whitespace): 13/13, 9m10s, $0.67, 6 blocks, all self-check catches; the change was 3 lines in `parse`, with its signature and seed tests kept. `INTENT=cli` (an application: a wc-like binary tested by running it, checked by running the merged binary) is ready; run 44 (the retry after the usage limit lifted): 13/13, 24m55s, $1.54, 6 blocks (all self-check catches), and the merged binary works. Its cost came from planning: 2 stories and 7 tasks for a small tool, so 7 RED/SCAFFOLD/GREEN cycles and 56 supervisor turns (runs 42–43: 1–2 tasks, 16–30 turns). The planner now sizes tasks up to the budget instead of one per behaviour (reason: per-task cost is a fixed ~8 supervisor turns). Run 45 on it: 3 tasks (from 7), 13/13, 16m22s, $1.07. Its HIGH foundation-poisoning halts were `ctx-symbols` reading a test's `const USAGE` as a duplicate of the binary's; a test constant no longer counts (test: `conflicts.rs`). Run 46 on both fixes: 3 tasks, 13/13, 15m19s, $0.99, no review halt, 6/6 blocks self-check catches. Its 4 `targets` "symbol is not defined" blocks were T3 modifying symbols T2 plans in an existing file; a symbol another target plans now counts (test: `semantic-targets.test.py` case 8). Run 47 on it: 13/13, $1.04, 2 blocks (from 6), no "not defined"; but 5 tasks in 2 stories, so planner task count still varies (next: measure the spread before tightening the rule). Planner friction next: across runs 36–41 the most frequent block was `targets` "create needs planned_symbol" on a new test file (8); a test-file create now takes it from the task's `tests` (test: `semantic-targets.test.py`). The `gate-validate-artifact` plan.md refusals (4) were the persona's bare `stage: 1`, now quoted and checked (test: `persona-shapes.test.py`). Next: the final-gate block order (3). Running every suite under a locally built bash 3.2 found that the `standards.md` gate matrix was silently replaced by the default Rust matrix (`mapfile` does not exist in bash 3.2). It is fixed and guarded by `bash-portable.test.py`. A `macos` CI job now runs the host tests and the conformance fixtures on macOS (reason: both were found only by hand on the owner's Mac). |
| **M9 — Mission layer** | v0.13 | The "Missionary Aladdin Genie" spec (2026-10-17): a human-set mission above goals; fixed roles (research, product testing, state, evaluation, planning, ethics) build a confirmed baseline and a reproducible evaluation layer from user journeys; goals derived from the gap list become Aladdin intents; re-evaluate on a cadence and after each sprint; GA4 usage informs goals and is never a gate; every element has a deterministic check (§4.12). Reason: the owner's mission-driven loop, stacked on Aladdin | in progress: M9.0 core (`aloop mission init / check / approve / status`, sealed mission + approval events, area discovery, test glob; test: `mission-core.test.py`); M9.2 baseline, M9.3 journeys and M9.10 assess/evaluate (`mission-{baseline,journeys,evaluate}.test.py`). Open tracks: M9.1, M9.4–M9.9; the task graph is in §4.12, track notes in `plugin/mission/README.md`; hardened after an adversarial review (ledger records over status fields) |

**Order rationale:** identity and conformance fixtures first, so every later change is
Expand Down
4 changes: 4 additions & 0 deletions docs/evidence/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ the cost cell reads — and the intent cell names the host. What each run found
| run 44 | cli (Rust binary), first run | v0.12.0 + #68 (216dd4c) | 13/13 | 24m55s | $1.54 | 3 | 0 | 6 | `make e2e-live YES=1 INTENT=cli` |
| run 45 | cli (Rust binary), on the task-sizing rule | v0.12.0 + #76 (eb00a74) | 13/13 | 16m22s | $1.07 | 3 | 0 | 8 | `make e2e-live YES=1 INTENT=cli` |
| run 46 | cli (Rust binary), on both #76 fixes | v0.12.0 + #76 (286bf45) | 13/13 | 15m19s | $0.99 | 2 | 2 | 6 | `make e2e-live YES=1 INTENT=cli` |
| run 47 | cli (Rust binary), on #81 | v0.12.0 + #81 (a653fd4) | 13/13 | 15m19s | $1.04 | 2 | 1 | 2 | `make e2e-live YES=1 INTENT=cli` |

Columns: *checks* are the e2e script's own assertions (12 for `semver`, 13 for `wordfreq` since
the commit-trace check); *wakes* are supervisor turns that ended and resumed (each re-reads
Expand Down Expand Up @@ -88,3 +89,6 @@ no longer counts as a duplicate definition; a test function redefining a source
Run 46 is the same tool on both fixes: 3 tasks, $0.99 and 15m19s, no structural-review halt, and
all 6 blocks were the workers' own self-checks (targets naming symbols not yet written, RED's
`cargo fmt`). The `cli` loop now costs about what one-task brownfield runs do.
Run 47 (on #81, which lets a later task modify a symbol an earlier one plans) had 2 blocks (from
6), both the workers' own self-checks, and no `targets` "not defined" refusal. The planner made 2
stories and 5 tasks this time, so cost stayed at $1.04: task count still varies run to run.
Loading
0