8000 Record live run 23: wordfreq under the model gate, 13/13, 10m58s, $0.76, 0 blocks by adeelahmad · Pull Request #39 · adeelahmad/aloop · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ tag in lockstep. Tag releases as `vMAJOR.MINOR.PATCH`.
## [Unreleased]

### Added
- `gate-dispatch` now refuses a dispatch of any aloop role whose `model` is not the project's confirmed one (`aloop models resolve <role>`, matched by model family so `haiku` and `claude-haiku-5-5` agree), or that passes no model (which would fall back to the agent file's default, opus for the planning roles). Reason: live e2e run 20 dispatched planners on opus and workers on sonnet against a project whose defaults.md said haiku for both, and cost $2.79 instead of $0.99 for the same 12/12 result. Test: `scripts/test/dispatch.test.py`. Live run 21 with the gate: 12/12, 8m17s, $0.66, every dispatch on haiku (ledger rows for runs 20 and 21 in `docs/evidence/README.md`).
- `gate-dispatch` now refuses a dispatch of any aloop role whose `model` is not the project's confirmed one (`aloop models resolve <role>`, matched by model family so `haiku` and `claude-haiku-5-5` agree), or that passes no model (which would fall back to the agent file's default, opus for the planning roles). Reason: live e2e run 20 dispatched planners on opus and workers on sonnet against a project whose defaults.md said haiku for both, and cost $2.79 instead of $0.99 for the same 12/12 result. Test: `scripts/test/dispatch.test.py`. Live run 21 with the gate: 12/12, 8m17s, $0.66, every dispatch on haiku Run 23 on the two-story `wordfreq` intent: 13/13, 10m58s, $0.76, 0 gate blocks (ledger rows for runs 20–23 in `docs/evidence/README.md`).
- Evidence ledger (ROADMAP §4.11): `docs/evidence/README.md` lists every live run behind a public
number (intent, plugin version, checks, wall, cost, wakes, plugin reads, gate blocks, reproduce
command); `bin/run-summary <run-dir>` writes a run's `summary.json`, `summary.md` and ledger row from
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ given up front — reproducible with `make e2e-live YES=1` (spends about a dolla
| a semver parser ([run 6](docs/evidence/README.md)) | 11/11 checks, tests on main, worktree branches merged `--no-ff` | 11m10s | $0.65 |
| a word-frequency tool, two dependent stories ([run 9](docs/evidence/README.md)) | 12/12 checks | 19m42s | $1.56 |
| the same tool on v0.12.0 ([run 19](docs/evidence/README.md)) | 13/13 checks, 1 supervisor wake, every merged commit traces to its task | 15m24s | $1.01 |
| the same tool with the dispatch model gate ([run 23](docs/evidence/README.md)) | 13/13 checks, 0 gate blocks, every dispatch on the project's model | 10m58s | $0.76 |

Every public number is a row in [docs/evidence/README.md](docs/evidence/README.md), with the command
that reproduces it; what each run found and fixed is logged in [ROADMAP §0](docs/ROADMAP.md#0-at-a-glance)
Expand Down
2 changes: 1 addition & 1 deletion docs/ROADMAP.md

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions docs/evidence/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ the cost cell reads — and the intent cell names the host. What each run found
| run 20 | semver | v0.12.0 + #31 (915810c) | 12/12 | 10m47s | $2.79 | 1 | 2 | 1 | `make e2e-live YES=1 INTENT=semver` |
| run 21 | semver | v0.12.0 + #33 (1ad94e2) | 12/12 | 8m17s | $0.66 | 1 | 0 | 3 | `make e2e-live YES=1 INTENT=semver` |
| run 22 | wordfreq | v0.12.0 + #37 (361f8d7) | 13/13 | 16m18s | $1.14 | 2 | 0 | 3 | `make e2e-live YES=1 INTENT=wordfreq` |
| run 23 | wordfreq | v0.12.0 + #33 (7417bb9) | 13/13 | 10m58s | $0.76 | 4 | 1 | 0 | `make e2e-live YES=1 INTENT=wordfreq` |

Columns: *checks* are the e2e script's own assertions (12 for `semver`, 13 for `wordfreq` since
the commit-trace check); *wakes* are supervisor turns that ended and resumed (each re-reads
Expand Down
2 changes: 1 addition & 1 deletion docs/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ <h2 id="audit">The audit tree</h2>
<p>Decisions carry a one-line reason, evidence refs and an authority ref. Usage is reported by class (fresh input, cache read, cache write, output, context) and marked <em>estimated</em> where a host does not report it.</p>

<h2 id="evidence">Evidence, not claims</h2>
<p>Every public number comes from the harness's own <code>stats.md</code> and is reproducible with <code>make e2e-live YES=1</code>: a one-line intent through intake, standards, planning, RED/SCAFFOLD/GREEN per task, structural review and the final gate, to merged and tested code in a real session. The semver intent ran end to end in 11m10s for $0.65 (run 6); a two-story word-frequency tool in 19m42s for $1.56 (run 9). Every number here is a row in <a href="https://github.com/adeelahmad/agentic-agile/blob/master/docs/evidence/README.md">docs/evidence/README.md</a> with the command that reproduces it, and every requirement on the roadmap is tracked with the test that proves it in <a href="https://github.com/adeelahmad/agentic-agile/blob/master/docs/TRACKING.md">docs/TRACKING.md</a>.</p>
<p>Every public number comes from the harness's own <code>stats.md</code> and is reproducible with <code>make e2e-live YES=1</code>: a one-line intent through intake, standards, planning, RED/SCAFFOLD/GREEN per task, structural review and the final gate, to merged and tested code in a real session. The semver intent ran end to end in 11m10s for $0.65 (run 6); a two-story word-frequency tool in 19m42s for $1.56 (run 9), and in 10m58s for $0.76 with no gate block once every dispatch ran on the project's model (run 23). Every number here is a row in <a href="https://github.com/adeelahmad/agentic-agile/blob/master/docs/evidence/README.md">docs/evidence/README.md</a> with the command that reproduces it, and every requirement on the roadmap is tracked with the test that proves it in <a href="https://github.com/adeelahmad/agentic-agile/blob/master/docs/TRACKING.md">docs/TRACKING.md</a>.</p>

<footer>MIT. <code>md-db</code> is vendored under AGPL-3.0. Formerly <em>agentic-agile</em>; the old names keep working. The lamp is an original drawing: a lantern whose flame is a red-to-green loop.</footer>
</main>
Expand Down
Loading
0