8000 Trim the supervisor's playbook; RED's compile block names the missing symbol (run 26) by adeelahmad · Pull Request #43 · adeelahmad/aloop · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ tag in lockstep. Tag releases as `vMAJOR.MINOR.PATCH`.
- The planner's stop gate (`gate-stage2-complete` on SubagentStop) now logs its verdicts to `gates.jsonl` and the event stream like every other gate, and the stuck release covers the planner: the same block three times in a row releases its stop unverified with a `state/stuck/plan-sprintN.<agent>` marker the supervisor acts on (a re-dispatch is a new agent id and starts over). A direct call (the supervisor's own check, `selfcheck`) is logged as a self-check and never releases. Reason: live e2e run 19's re-planning planner was blocked 9 times on one unchanged reason, none counted. Test: `scripts/test/planner-stuck.test.py`.

### Fixed
- `gate-red-verify`: when RED's tests do not compile, the block names each symbol they cannot find and the shim to add (`pub fn <name>(…) -> … { todo!() }` or `pub struct <Name>;` in `tests/common/mod.rs`, marked `// agentic:shim`, called through `mod common;`) instead of a bare "add mod-common shims". Reason: live run 26, a RED worker blocked 5 times and hit the stuck release. Test: conformance §6.3.
- The comms gate's "latest output.md block is not from '<role>'" block now says how to append the
report from a worktree (`cat >> … <<'EOF'`), like the missing-block one (live run 25: a GREEN worker
refused by the Write tool stopped three times on it and hit the stuck release). Conformance case.
Expand Down Expand Up @@ -77,6 +78,7 @@ tag in lockstep. Tag releases as `vMAJOR.MINOR.PATCH`.
`0`, so the scaffold gate's block text broke mid-sentence on CI-like hosts; it prints one number.

### Changed
- The supervisor's playbook is 16% shorter: the sections a normal run does not need (Codex/OpenCode notes, the budget re-plan route and breach classes, review blind spots, the script list, transcripts, worktree hygiene) moved to `skills/aloop/reference.md`, read on demand from one-line pointers; OpenCode installs copy it beside the skill. Reason: SKILL.md is re-read on every supervisor turn (run 26: 6k fewer tokens per turn). Test: `aladdin-cli.test.py`.
- SKILL.md: a RED mandate names `tests/common/mod.rs` as the shims' home, never a `src/` path
(one `gate-red-verify` block per live run 14–18 came from a mandate saying "shim in `src/lib.rs`").

Expand Down
2 changes: 1 addition & 1 deletion docs/ROADMAP.md

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions docs/TRACKING.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,3 +54,4 @@ Status of every requirement in [ROADMAP.md](ROADMAP.md), in the four states §8
| 7 | The final-gate persona's suppression check is the gate's (`assert_no_suppression`), never its own list | IMPLEMENTED_TESTED | `scripts/test/suppression-contract.test.py` |
| 4.11 | The north-star run reproducible on every host (`e2e-live.sh --host codex|opencode`: host install + headless CLI, same checks) | IMPLEMENTED_TESTED | `scripts/test/e2e-live-hosts.test.py` (measured Codex/OpenCode rows still pending: no CLIs here) |
| 7 | The sprint and every STORY_DIR live in `docs/agents/sprintN/`, never in the git-ignored run-data dir (`gate-stage2-complete`, `gate-dispatch`) | IMPLEMENTED_TESTED | `scripts/test/planner-stuck.test.py`, `scripts/test/dispatch.test.py` |
| 4.11 | The supervisor's playbook keeps on-demand sections in `skills/aloop/reference.md`, installed beside it on OpenCode | IMPLEMENTED_TESTED | `scripts/test/aladdin-cli.test.py` |
1 change: 1 addition & 0 deletions docs/evidence/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ the cost cell reads — and the intent cell names the host. What each run found
| run 23 | wordfreq | v0.12.0 + #33 (7417bb9) | 13/13 | 10m58s | $0.76 | 4 | 1 | 0 | `make e2e-live YES=1 INTENT=wordfreq` |
| run 24 | wordfreq, haiku supervisor | v0.12.0 + #39 (9826566) | 10/13 | 19m09s | $0.66 | 4 | 1 | 6 | `make e2e-live YES=1 INTENT=wordfreq MODEL=haiku` |
| run 25 | wordfreq | v0.12.0 + #40 (9c6a16c) | 13/13 | 16m13s | $0.98 | 3 | 0 | 5 | `make e2e-live YES=1 INTENT=wordfreq` |
| run 26 | wordfreq | v0.12.0 + #41 (685a48b), trimmed playbook | 13/13 | 16m01s | $0.95 | 2 | 1 | 7 | `make e2e-live YES=1 INTENT=wordfreq` |

Columns: *checks* are the e2e script's own assertions (12 for `semver`, 13 for `wordfreq` since
the commit-trace check); *wakes* are supervisor turns that ended and resumed (each re-reads
Expand Down
2 changes: 1 addition & 1 deletion plugin/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,7 @@ pipeline/ design source of truth (persona/init/artifacts/gate
one-story sprint. The hook *wiring* should be smoke-tested in a live Claude Code
session before relying on it in anger.
- Suppression detection is syntactic; a test weakened by deleting an assertion is
caught by the RED-invariant + diff-scope, not by grep. See SKILL.md
caught by the RED-invariant + diff-scope, not by grep. See skills/aloop/reference.md
"Known gate limitations."

## License
Expand Down
2 changes: 1 addition & 1 deletion plugin/agents/planner.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ one or more tasks hit their hard budget. Your init block lists which tasks are D
- Each killed task carries a **breach class** (its output.md block's `| breach class |`
row, with evidence). Confirm or override it from the transcript, write the class you act
on under the task as `breach: <CLASS> — <one-line reason>`, and act on THAT class
(SKILL.md "Breach classes"): only `TASK_TOO_LARGE` / `CONTEXT_PATHOLOGY` / `UNKNOWN`
(skills/aloop/reference.md "Breach classes"): only `TASK_TOO_LARGE` / `CONTEXT_PATHOLOGY` / `UNKNOWN`
default to a split; `AGENT_LOOPING` reframes, `PLAN_AMBIGUOUS` repairs the contract,
`BLOCKED_DEPENDENCY` reorders, the slow classes adjust commands or the `budget:` time
line, `EXTERNAL_SERVICE` isolates or mocks it.
Expand Down
6 changes: 3 additions & 3 deletions plugin/bin/budget
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ each with a SOFT and a HARD limit:
spent after the stop (ROADMAP §3.5). A `<worktree>/.aladdin/budget-exceeded`
marker lets its SubagentStop gate release it, and `budget watch` tells
the supervisor to TaskStop it if the hook could not. The task is NOT retried — it goes back to the planner
to be split (SKILL.md "Budget overrun -> mid-sprint re-plan").
to be split (skills/aloop/reference.md "Budget overrun -> mid-sprint re-plan").

Resolution order (first set wins): per-task `budget:` line in tasks.md >
BUDGET_* in the worktree's task.env copy > docs/agents/defaults.md > built-ins.
Expand Down Expand Up @@ -474,7 +474,7 @@ def harness_report(st, left, cls):
"the worker was not asked to report.\n\n### Result\n| Check | Status | Detail |\n|---|---|---|\n"
"| budget | HARD (%s) | %s |\n| worktree diff | discarded | %s |\n"
"| breach class | %s | %s |\n\n### Next\n"
"re-plan: confirm or override the breach class, then act on it (SKILL.md \"Breach classes\"). "
"re-plan: confirm or override the breach class, then act on it (skills/aloop/reference.md \"Breach classes\"). "
"Transcript: %s\n") % (
st.get("task") or "?", st.get("attempt") or "?", st["role"], ts, cls, left, cls, left, diff,
breach, why.replace("|", "/"), st.get("transcript") or "?")
Expand Down Expand Up @@ -824,7 +824,7 @@ def watch(cwd, interval, grace):
if kill:
print("-> KILL lines: the hook could not stop these — TaskStop each, then `budget mark-killed <agent_id>`.")
print("-> Budget overrun: discard these attempts (never merge), PAUSE the sprint and dispatch the "
"planner in replan mode (SKILL.md 'Budget overrun'). Do NOT retry these tasks as-is.")
"planner in replan mode (skills/aloop/reference.md 'Budget overrun'). Do NOT retry these tasks as-is.")
return 3
if not running:
print("IDLE: no worker running.")
Expand Down
23 changes: 21 additions & 2 deletions plugin/bin/gate-red-verify
Original file line number Diff line number Diff line change
Expand Up @@ -53,8 +53,27 @@ run_format_checks
if [ "${GATE_RUN_TESTS:-auto}" != "0" ] && is_cargo_repo; then
root="$(repo_dir)"
note "compiling tests (must succeed against mod-common shims)"
( cd "$root" && cargo test --no-run --workspace ) \
|| fail "RED tests do NOT compile — they fail by missing symbol, not by assertion (add mod-common shims)"
_build="$(mktemp)"
if ! ( cd "$root" && cargo test --no-run --workspace ) >"$_build" 2>&1; then
grep -E '^(error|warning: unused)|^ *--> ' "$_build" | head -20 >&2
# Name what the tests cannot find and where its shim goes (live e2e run 25: a RED worker
# blocked 5 times on a bare "add mod-common shims" after reverting a src/ skeleton).
_missing="$(sed -nE \
-e 's/.*error\[E04(25|12|22|23)\]: cannot find [a-z ]+ `([A-Za-z_][A-Za-z0-9_]*)`.*/\2/p' \
-e 's/.*error\[E0432\]: unresolved import `([A-Za-z0-9_:]*::)?([A-Za-z_][A-Za-z0-9_]*)`.*/\2/p' \
-e 's/.*error\[E0599\]: no (function or associated item|method) named `([A-Za-z_][A-Za-z0-9_]*)`.*/\2/p' \
"$_build" | awk '!seen[$0]++' | head -6)"
rm -f "$_build"
if [ -n "$_missing" ]; then
_names="$(printf '`%s` ' $_missing)"; _shims=""
for _m in $_missing; do
case "$_m" in [A-Z]*) _shims="$_shims pub struct $_m; //" ;; *) _shims="$_shims pub fn $_m(…) -> … { todo!() } //" ;; esac
done
fail "RED tests do NOT compile: they cannot find ${_names% }. RED's tests must compile and fail by assertion, so each missing item gets a shim in tests/common/mod.rs, marked // agentic:shim:${_shims%//} — and the test files call it through \`mod common;\` (common::$(printf '%s' "$_missing" | head -1)), not the crate; src/ is SCAFFOLD's and GREEN's."
fi
fail "RED tests do NOT compile — they fail by missing symbol, not by assertion: add the missing items as shims in tests/common/mod.rs (marked // agentic:shim) and call them through \`mod common;\`"
fi
rm -f "$_build"
note "running tests (must fail by assertion)"
if ( cd "$root" && cargo test --workspace >/dev/null 2>&1 ); then
fail "RED tests PASS with no production code — they exercise nothing"
Expand Down
4 changes: 4 additions & 0 deletions plugin/bin/install
Original file line number Diff line number Diff line change
Expand Up @@ -194,6 +194,10 @@ def install_opencode(opts, st):
desc = re.sub(r"\s+", " ", fm.get("description", "")).strip(" >-")[:1024]
text = "---\nname: %s\ndescription: %s\n---\n<!-- %s -->\n%s" % (new_name, json.dumps(desc, ensure_ascii=False), MARK, host_body(body, "opencode"))
write(os.path.join(base, "skills", new_name, "SKILL.md"), text, written)
ref = os.path.join(ROOT, "skills", skill, "reference.md") # the playbook's on-demand sections
if os.path.isfile(ref):
write(os.path.join(base, "skills", new_name, "reference.md"),
"<!-- %s -->\n%s" % (MARK, host_body(open(ref).read(), "opencode")), written)
write(os.path.join(base, "commands", "aloop-init.md"),
"---\ndescription: Set up aloop for this project (limits, models, .gitignore, commit author), then start sprint planning\n---\n"
"<!-- %s -->\nLoad the `aloop-init` skill with the skill tool and follow it exactly. $ARGUMENTS\n" % MARK, written)
Expand Down
Loading
Loading
0