8000 fix: refuse execution dispatches before the task contract exists by adeelahmad · Pull Request #9 · adeelahmad/aloop · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,11 @@ tag in lockstep. Tag releases as `vMAJOR.MINOR.PATCH`.
## [Unreleased]

### Added
- **`gate-dispatch`** (PreToolUse on Agent/Task): the supervisor's dispatch of an execution
worker is refused until its prompt opens with `TASK_ID: <id>` and the contract exists.
e2e run 6 dispatched the structural reviewer before writing its contract, and its gate
blocked 25 times on a `STORY_DIR` the worker could not supply. `scripts/test/dispatch.test.py`.

- **End-to-end live run (north star).** `make e2e-live YES=1` turns a one-line intent
(a semver parser) into planned, tested, merged code in a real Claude Code session with
the plugin, then checks the artifacts, gate verdicts, merge, tests and cost. First run:
Expand Down Expand Up @@ -78,6 +83,11 @@ tag in lockstep. Tag releases as `vMAJOR.MINOR.PATCH`.
reading it stays allowed. 2 checks failed before.

### Fixed
- **Annotated `scaffold-symbols` lines were split into fake symbols** (`(enum)`, `@`, …), each
"defined 0 times". The first token of a line is the symbol; non-identifiers are dropped.
- **Sub-agents read the plugin's own source** (19–30 commands per e2e run) to work out
formats `selfcheck` already reports. The agent prompts now say not to; run 6 had 2, and
cost its lowest yet: $0.65 in 11 min.
- **A Claude Code worker could get another task's budget.** With no `task.env` in its own
worktree, the time box guessed "the only green-worker worktree" and applied that task's
limits. It now takes the worker's own worktree from its cwd first.
Expand Down
1 change: 1 addition & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,7 @@ test-hosts: ## Budget/worktree regression + Codex hooks emulation + OpenCode plu
python3 scripts/test/rename-compat.test.py
python3 scripts/test/supervisor-scope.test.py
python3 scripts/test/hook-matchers.test.py
python3 scripts/test/dispatch.test.py
python3 scripts/test/fail-closed.test.py
@command -v node >/dev/null && node scripts/test/opencode-plugin.test.mjs || { echo "node not found — skipped OpenCode test (CI runs it)"; }

Expand Down
2 changes: 1 addition & 1 deletion docs/ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ spec; this table is the plan. Versions are targets, not promises.
| Milestone | Target | Contents | Status |
|---|---|---|---|
| **M0 — Hygiene** | v0.10.x | Mark `DESIGN` / `ARCHITECTURE` / `KICKSTART` historical; fix stale counts (8 → 9 roles) and the README CI section; add the `CONTRIBUTING.md` the README badge links to; one live-session smoke test of the hook wiring (open since `DEVLOG` v0.1) | done: live smoke test `make smoke-live YES=1` (6 checks, ~$0.004 on haiku); its first run found the Linux fail-open in `gate-supervisor-scope` (fixed, regression test added) |
| **M0.8 — North-star run** | v0.11.x | `make e2e-live YES=1`: a one-line intent (semver parser) goes through retrospective, intake, standards, planner, RED/SCAFFOLD/GREEN per task, structural review and the final gate to merged, tested code in a real Claude Code session, with the approval given up front. Reason: nothing had ever run the whole loop live; the first run found that no role gate fired on plugin installs | run 1: done end to end, 19 min, $1.15 (sonnet supervisor, haiku workers), 22 tests on main, 9/11 checks; found and fixed the `aloop:<role>` matcher bug, a selfcheck crash and copy-merging. Run 2: **11/11**, 12m46s, $0.92, host gates on every role, worktree branches merged with `--no-ff`; found gates looping on the supervisor's contract slips (fixed). Run 3: 11/11, 12m00s, **$0.79**; found GRE 10BC8 EN looping on RED's unformatted tests (fixed: RED/SCAFFOLD run the format check) and selfcheck missing the contract in Claude Code worktrees (fixed: the contract is copied in). Run 4 (all fixes): 11/11, 17m56s, $1.10, 12 gate blocks and no loops (28 → 17 → 12). Cost varies $0.79–1.15 per run; next: cut it (scaffold symbol parsing slips such as `(enum,`, supervisor turns), then a second intent beyond the semver fixture. Run 5: 11/11 but 29 min, $1.83, 44 blocks (the planner made 5 tasks; 27 false isolation blocks after workers `cd`'d to the main tree, fixed; subagents also read the plugin's own source, which costs tokens). Next: rerun green, then the gaps it showed (a test-coverage loss when a RED re-plan drops tests; supervisor-written GREEN instructions overreaching the next task) |
| **M0.8 — North-star run** | v0.11.x | `make e2e-live YES=1`: a one-line intent (semver parser) goes through retrospective, intake, standards, planner, RED/SCAFFOLD/GREEN per task, structural review and the final gate to merged, tested code in a real Claude Code session, with the approval given up front. Reason: nothing had ever run the whole loop live; the first run found that no role gate fired on plugin installs | run 1: done end to end, 19 min, $1.15 (sonnet supervisor, haiku workers), 22 tests on main, 9/11 checks; found and fixed the `aloop:<role>` matcher bug, a selfcheck crash and copy-merging. Run 2: **11/11**, 12m46s, $0.92, host gates on every role, worktree branches merged with `--no-ff`; found gates looping on the supervisor's contract slips (fixed). Run 3: 11/11, 12m00s, **$0.79**; found GREEN looping on RED's unformatted tests (fixed: RED/SCAFFOLD run the format check) and selfcheck missing the contract in Claude Code worktrees (fixed: the contract is copied in). Run 4 (all fixes): 11/11, 17m56s, $1.10, 12 gate blocks and no loops (28 → 17 → 12). Cost varies $0.79–1.15 per run; next: cut it (scaffold symbol parsing slips such as `(enum,`, supervisor turns), then a second intent beyond the semver fixture. Run 5: 11/11 but 29 min, $1.83, 44 blocks (the planner made 5 tasks; 27 false isolation blocks after workers `cd`'d to the main tree, fixed; subagents also read the plugin's own source, which costs tokens). Run 6: 11/11, **11m10s, $0.65**, plugin-source reads 30 → 2 after the agent prompts said not to; found a dispatch-before-contract loop (25 blocks, fixed by `gate-dispatch`) and annotated scaffold-symbol lines (fixed). Next: rerun green, then the gaps it showed (a test-coverage loss when a RED re-plan drops tests; supervisor-written GREEN instructions overreaching the next task) |
| **M0.6 — Gates fail closed** | v0.11.x | A gate that crashes (exit 1, e.g. an unbound variable) is a non-blocking hook error on Claude Code, so the action goes through. Make every gate trap unexpected errors into exit 2 with the reason, and add a conformance case per gate. Reason: the smoke run showed one real instance | done: `_failclosed.sh` on all 10 sub-agent gates + supervisor-scope inside a sprint; `scripts/test/fail-closed.test.py` (5 checks). Left fail-open on purpose: `gate-commit-author` and `gate-validate-artifact` outside a sprint, which see every session on the machine |
| **M0.7 — Supervisor-owned task contract** | v0.11.x | The contract lives in the main tree, `.agentic/state/tasks/<TASK_ID>.env`, and the dispatch prompt's first line names the task; gates and the time box find it from the host's transcript, a worktree `task.env` that disagrees blocks, and sub-agents may not Write/Edit `.agentic/state/` or `task.env`. Settled by a live probe: Claude Code's `isolation: "worktree"` makes `.claude/worktrees/agent-<id>` at dispatch, so a pre-written in-worktree contract was impossible there. Bash writes are guarded too (a command naming the contract with a write verb is blocked; a heuristic, so an obfuscated write still gets through). On hosts with no sub-agent transcript the worktree's `TASK_ID` (or its `.agentic/worktrees/<id>` path) picks the contract | done: `make conformance` (30) + `scripts/test/{budget-worktree,supervisor-scope}.test.py` |
| **M0.5 — Rename to aloop** | v0.11.0 | Plugin, marketplace, skill, CLI (`aloop`, `aloop init`), OpenCode plugin file and docs renamed; `agentic`, `agentic init`, `agentic-init` and `agentic bind` kept as aliases; OpenCode/Codex installs clean up pre-rename files. Data layout and settings keys unchanged (moved to M1). GitHub repo rename is the owner's call | done: `scripts/test/rename-compat.test.py` |
Expand Down
3 changes: 2 additions & 1 deletion plugin/bin/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,8 @@ ctx-symbols) → WARN + grep fallback; never a silent pass.
## Gates (which hook fires each)
| script | event · matcher | checks | exit 2 when |
|--------|-----------------|--------|-------------|
| gate-supervisor-scope | PreToolUse · Write\|Edit\|MultiEdit | MAIN agent (no `agent_type`) may not write production source mid-sprint, nor hand-author planning artifacts (stories/tasks/validate/plan/intake/standards.md — dispatch the activity); sub-agents (`agent_type` present) exempt | supervisor writes source / hand-writes a planner-owned artifact |
| gate-supervisor-scope | PreToolUse · Write\|Edit\|MultiEdit\|Bash | MAIN agent (no `agent_type`) may not write production source mid-sprint, nor hand-author planning artifacts (stories/tasks/validate/plan/intake/standards.md — dispatch the activity); sub-agents (`agent_type` present) exempt | supervisor writes source / hand-writes a planner-owned artifact |
| gate-dispatch | PreToolUse · Agent\|Task | the MAIN agent's dispatch of an execution role opens with `TASK_ID: <id>` and that task's contract `.agentic/state/tasks/<id>.env` exists (repos with `.agentic/` only) | no TASK_ID line / contract not written yet |
| gate-tooling | SubagentStart · exec roles + supervisor self-check | md-db + ctx-symbols installed (execution may not run grep-degraded); runs `ensure-tools --sync` first | a backend is missing and can't be built |
| gate-validate-artifact | PostToolUse · Write | md-db-validate the written artifact | artifact malformed |
| gate-red-verify | SubagentStop · red-worker | worktree-isolated; appended output.md block (validate_comms); every new test FAILS BY ASSERTION; no regression; diff = tests+shims | no/stale output.md block / shared-tree run / a new test passes / prod code / regression |
Expand Down
42 changes: 42 additions & 0 deletions plugin/bin/gate-dispatch
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
#!/usr/bin/env bash
# gate-dispatch — PreToolUse(Agent|Task) on the SUPERVISOR's dispatch of an execution worker.
#
# A worker's gates read its task contract (.agentic/state/tasks/<TASK_ID>.env), found through the
# `TASK_ID: <id>` line the dispatch prompt opens with. Dispatched before that contract exists, the
# worker's SubagentStop gate cannot find STORY_DIR and blocks on every stop — a block the worker
# cannot fix (live e2e run 6: 25 blocks). So the dispatch itself is refused until it is right.
#
# Only aloop execution roles, only from the main session (a sub-agent's own Agent calls carry
# agent_type), only in a repo with aloop state (.agentic/). Exit: 0 allow · 2 deny. SKIP_HOOKS=1.
set -uo pipefail
GATE_NAME=dispatch
[ "${SKIP_HOOKS:-}" = "1" ] && exit 0
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
payload="$(cat 2>/dev/null || true)"

command -v python3 >/dev/null 2>&1 || exit 0
read -r role tid < <(printf '%s' "$payload" | python3 -c '
import json, re, sys
p = json.load(sys.stdin)
if p.get("agent_type"):
print("- -"); sys.exit()
ti = p.get("tool_input") or {}
role = (ti.get("subagent_type") or "").split(":")[-1]
m = re.match(r"\s*TASK_ID:\s*([A-Za-z0-9._-]+)", ti.get("prompt") or "")
print(role or "-", m.group(1) if m else "-")' 2>/dev/null) || exit 0

case "$role" in red-worker|scaffolder|green-worker|structural-reviewer|final-gate) ;; *) exit 0 ;; esac
# shellcheck source=/dev/null
. "$HERE/_paths.sh"
[ -d "$AGENTIC_HOME" ] || exit 0

if [ "$tid" = "-" ]; then
echo "BLOCK[$GATE_NAME]: start the $role dispatch prompt with the line \`TASK_ID: <id>\` — its gates find the task contract through it." >&2
exit 2
fi
if [ ! -f "$AGENTIC_STATE/tasks/$tid.env" ]; then
echo "BLOCK[$GATE_NAME]: write the task contract before dispatching: $AGENTIC_STATE/tasks/$tid.env" >&2
echo " (TASK_ID, ATTEMPT, AGENT_ROLE=$role, STORY_DIR, SCOPE_GLOBS, BASE_REF, BUDGET_* — see the skill's \"The task contract\")" >&2
exit 2
fi
exit 0
5 changes: 4 additions & 1 deletion plugin/bin/gate-scaffold-verify
Original file line number Diff line number Diff line change
Expand Up @@ -29,9 +29,12 @@ symbols="${SCAFFOLD_SYMBOLS:-}"
if [ -z "$symbols" ]; then
sf="$root/.agentic/scaffold-symbols"
if [ -f "$sf" ]; then
symbols="$(grep -vE '^[[:space:]]*(#|$)' "$sf" | tr -s ' \t' '\n' | sort -u)"
# one symbol per line; anything after it on the line is a note ("Foo (enum) @ src/lib.rs")
symbols="$(grep -vE '^[[:space:]]*(#|$)' "$sf" | awk '{print $1}' | sort -u)"
fi
fi
# keep only identifiers / paths (`Foo`, `crate::Foo`): punctuation is never a symbol
symbols="$(printf '%s\n' $symbols | tr -d '`,' | grep -E '^[A-Za-z_][A-Za-z0-9_]*(::[A-Za-z_][A-Za-z0-9_]*)*$' | sort -u)"
[ -z "$symbols" ] && warn "no scaffold symbols resolved (set SCAFFOLD_SYMBOLS or write .agentic/scaffold-symbols); uniqueness check skipped"

# 2) Each symbol defined exactly once across the tree.
Expand Down
2 changes: 2 additions & 0 deletions plugin/hooks/hooks.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@
{ "type": "command", "command": "\"${CLAUDE_PLUGIN_ROOT}/bin/gate-supervisor-scope\"" } ] },
{ "matcher": "Bash", "hooks": [
{ "type": "command", "command": "\"${CLAUDE_PLUGIN_ROOT}/bin/gate-commit-author\"", "timeout": 15 } ] },
{ "matcher": "Agent|Task", "hooks": [
{ "type": "command", "command": "\"${CLAUDE_PLUGIN_ROOT}/bin/gate-dispatch\"", "timeout": 15 } ] },
{ "matcher": "*", "hooks": [
{ "type": "command", "command": "\"${CLAUDE_PLUGIN_ROOT}/bin/host-adapter\" hook", "timeout": 15 },
{ "type": "command", "command": "\"${CLAUDE_PLUGIN_ROOT}/bin/budget\" hook pre", "timeout": 15 } ] }
Expand Down
2 changes: 1 addition & 1 deletion plugin/skills/aloop/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -746,7 +746,7 @@ Before each dispatch, write the contract in the MAIN tree at
`SCOPE_GLOBS` (GREEN diff-scope), `SCAFFOLD_SYMBOLS` (or write
`.agentic/scaffold-symbols`), `BASE_REF` (diff base), and the four `BUDGET_*` lines printed by
`budget resolve --tasks <STORY_DIR>/tasks.md --task Tn`. Then make the dispatch prompt's
FIRST line `TASK_ID: <TASK_ID>`. Write plain `KEY=value` lines (quotes allowed, lists space- or comma-separated, no `$VAR`
FIRST line `TASK_ID: <TASK_ID>`. (`gate-dispatch` refuses an execution dispatch without that line or before the contract exists.) Write plain `KEY=value` lines (quotes allowed, lists space- or comma-separated, no `$VAR`
expansion, no commands): the gate library PARSES the contract keys and ignores everything else.

Why the main tree: on Claude Code `isolation: "worktree"` creates the worktree
Expand Down
10 changes: 10 additions & 0 deletions scripts/conformance/claude-code.test.py
Original file line number Diff line number Diff line change
Expand Up @@ -154,6 +154,16 @@ def red_ok():
case(3, "a formatted RED passes the same format check", "gate-red-verify", red_ok_wt, 0,
extra={"STANDARDS_FILE": FMT_STANDARDS})

# SCAFFOLD: `.agentic/scaffold-symbols` lines may carry notes after the symbol (live e2e run 6:
# "ParseError (enum) @ src/lib.rs" was split into `(enum)`, `@`, … and each "defined 0 times").
wt = worktree("scaffolder")
write(os.path.join(wt, "src", "lib.rs"), LIB + "\npub fn sub(_a: i32, _b: i32) -> i32 {\n panic!(\"SUB-AGENT-TODO\")\n}\n")
write(os.path.join(wt, "tests", "sub.rs"), GREEN_TEST)
write(os.path.join(wt, ".agentic", "scaffold-symbols"), "# symbols\nsub (fn) @ src/lib.rs\n")
commit(wt, "scaffold")
report("scaffolder", [("src/lib.rs::sub", "STUB")])
case(3, "SCAFFOLD with annotated scaffold-symbols lines is accepted", "gate-scaffold-verify", wt, 0)

# §6.4 RED cannot write production code
wt = red_ok()
write(os.path.join(wt, "src", "lib.rs"), GREEN_LIB)
Expand Down
66 changes: 66 additions & 0 deletions scripts/test/dispatch.test.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
#!/usr/bin/env python3
"""gate-dispatch: the supervisor may not dispatch an execution worker before its task contract
exists. Regression (live e2e run 6): the contract was written after the structural reviewer was
dispatched, its gate could not find STORY_DIR and blocked 25 times — a block the worker cannot fix.

python3 scripts/test/dispatch.test.py
"""
import json
import os
import subprocess
import tempfile

GATE = os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", "..", "plugin", "bin", "gate-dispatch")
T = tempfile.mkdtemp(prefix="aloop-dispatch-")
ENV = {k: v for k, v in os.environ.items() if k != "SKIP_HOOKS"}
n = 0


def ok(msg):
global n
n += 1
print("ok - " + msg)


def run(subagent_type, prompt, agent_type=None):
p = {"session_id": "s", "cwd": T, "tool_name": "Agent",
"tool_input": {"subagent_type": subagent_type, "prompt": prompt, "isolation": "worktree"}}
if agent_type:
p["agent_type"] = agent_type
return subprocess.run([GATE], input=json.dumps(p), cwd=T, env=ENV, capture_output=True, text=True)


subprocess.run(["git", "init", "-q"], cwd=T, check=True)
os.makedirs(os.path.join(T, ".agentic", "state", "tasks"))

r = run("aloop:green-worker", "TASK_ID: S1-01-T1\nDo GREEN.")
assert r.returncode == 2 and "S1-01-T1.env" in r.stderr, r
ok("a worker dispatched before its contract exists is denied")

r = run("aloop:structural-reviewer", "Review the story.")
assert r.returncode == 2 and "TASK_ID:" in r.stderr, r
ok("a worker dispatch whose prompt names no TASK_ID is denied")

open(os.path.join(T, ".agentic", "state", "tasks", "S1-01-T1.env"), "w").write("TASK_ID=S1-01-T1\n")
r = run("aloop:green-worker", "TASK_ID: S1-01-T1\nDo GREEN.")
assert r.returncode == 0, r
ok("a worker dispatched after its contract is written is allowed")

for st in ("aloop:planner", "aloop:intake", "general-purpose", "Explore"):
r = run(st, "Plan it.")
assert r.returncode == 0, (st, r)
ok("planning roles and non-aloop agents are not affected")

r = run("aloop:green-worker", "TASK_ID: S1-09-T9\nDo GREEN.", agent_type="aloop:planner")
assert r.returncode == 0, r
ok("a sub-agent's own Agent calls are not this gate's business")

empty = tempfile.mkdtemp(prefix="aloop-dispatch-none-")
subprocess.run(["git", "init", "-q"], cwd=empty, check=True)
p = {"session_id": "s", "cwd": empty, "tool_name": "Agent",
"tool_input": {"subagent_type": "aloop:green-worker", "prompt": "TASK_ID: X\n"}}
r = subprocess.run([GATE], input=json.dumps(p), cwd=empty, env=ENV, capture_output=True, text=True)
assert r.returncode == 0, r
ok("a repo with no aloop state is not affected")

print("\n%d checks passed" % n)
Loading
0