8000 dsh cost includes compaction and title calls; macOS CI runs the real dsh test; host tests pass with md-db installed by adeelahmad · Pull Request #118 · adeelahmad/aloop · GitHub
[go: up one dir, main page]
Skip to content

dsh cost includes compaction and title calls; macOS CI runs the real dsh test; host tests pass with md-db installed - #118

Draft
adeelahmad wants to merge 7 commits into
masterfrom
claude/deepseek-host-f3orp6
Draft

dsh cost includes compaction and title calls; macOS CI runs the real dsh test; host tests pass with md-db installed#118
adeelahmad wants to merge 7 commits into
masterfrom
claude/deepseek-host-f3orp6

Conversation

@adeelahmad
@adeelahmad adeelahmad commented Oct 20, 2026
Copy link
Copy Markdown
Owner

Requested by Adeel · project thread

Before: #104's review left these gaps (4 of its 11 deferred findings), and one more turned up while combining them with master:

  • A dsh run's usage and cost left out dsh's own compaction and session-title calls, which are billed.
  • A short dsh run measured at $0.0014 was stored as cost_usd: 0.0 and shown as $0.00, the cell for nothing measured.
  • After a compaction, the handoff reached the model one step late.
  • The macos CI job never ran dsh-runtime.test.mjs: with no dsh installed it printed "skip" and exited 0.
  • make conformance took a ctx-symbols from the caller's CLAUDE_PLUGIN_DATA / XDG_DATA_HOME.
  • make test-hosts failed on any machine with md-db installed. supply-chain.test.py (from GREEN's dependency check reads only the plan text the human's grant pins (M0.9, M10.2) #113) ran gate-stage2-complete on planning docs md-db's schema rejects, so it passed only on CI's fallback.

After: each is fixed, each with a test that fails on the old code. The other 7 deferred findings (payload robustness and gate-final) are still in re-review and will come in their own PR.

How:

  • dsh usage (hosts/dsh/aloop.mjs, _usage.py): every call dsh makes outside the agent loop (llm/stream with a purpose) is logged Claude-shaped and marked aux. An aux call is charged like any call, but it never counts as the context in use.
  • sub-cent cost (run-summary): for --host dsh, the total keeps 6 decimals and the row cell keeps 4. A cost below that prints <$0.0001, judged on the unrounded total.
    • evidence.test.py now accepts exactly the cells run-summary prints for a measured cost on the row's HOST=, read as a whole token.
    • It refuses an unmeasured cost: —, or a price on a Codex or OpenCode row.
    • A README or site figure written <$0.0001 must also be a ledger row.
  • compaction handoff: the SessionStart(compact) context now joins the first request after the compaction. It is appended in agent/request before dsh builds the request, and only once.
  • macos CI: the macos job installs the pinned dsh and sets ALOOP_REQUIRE_DSH=1, as the hosts job does. readme.test.py check 7 fails on the plain edits that skip, forgive or off-pin that step or the host tests: if:, continue-on-error, defaults:, shell:, || true, quoted keys, an env other than BASH32, a cleared DSH_BIN, or rc.20 for rc.2. Check 8 holds the ci.yml header's job count and its README section.
  • conformance: the harness drops the caller's tools dir. PATH is still the caller's, and that is an open ROADMAP §4.9 item.
  • md-db: the supply-chain fixture's planning docs now pass planning-artifacts.kdl. New ROADMAP item: no CI job runs the host suites with md-db on PATH.

Review

Two adversarial verifiers per fix, two rounds. Round 1 found real problems, which were fixed with tests. In round 2 both verifiers judged both fixes sound. Their remaining nits are fixed here ( 8000 553845d). Each new check in that commit fails a mutation that passed before:

  • 7 ci.yml or README edits that passed the old check 7 or 8 and now fail it;
  • HOST=dsh-x and HOST=dsh.x rows;
  • <$0.0001 README and site figures.

One known misstatement: a9992fa's message says "22 mutations (21 of 22)". The real figures are 21 edits, 20 of which passed before and all 21 fail now. It stays as is to avoid a rewrite.

Evidence

Local on 9ac526e, with DSH_BIN and ALOOP_REQUIRE_DSH=1:

Suite counts:

  • dsh-runtime 53 (was 50);
  • dsh-plugin 48 (was 46);
  • evidence 5 (was 4);
  • e2e-live-hosts 17;
  • readme 8 (was 7);
  • supply-chain 8;
  • opencode-plugin 40.

Not measured: a live DeepSeek run still needs a DEEPSEEK_API_KEY.

Generated by Claude Code

claude added 7 commits October 20, 2026 18:40
@claude
dsh: price compaction and title calls, keep sub-cent cost, hand off o…
7e99394 
…n the first request after a compaction

- The dsh plugin now logs the usage of every call dsh makes outside the agent
  loop (llm/stream with a `purpose`: the compaction summary, the session
  title) from the call's own stream, Claude-shaped and marked `aux`. Before,
  only assistant/message usage was logged, so the ledger, the time box and
  run-summary --host dsh left those billed calls out and reported the rest as
  the run's total (runtime: 42 priced of 44 served). _usage keeps an aux call
  out of the context in use.
- run-summary keeps a dsh total to 6 decimals and its row cell to 4
  (<$0.0001 below), so a measured $0.0014 run no longer reads cost_usd 0.0 /
  $0.00.
- The SessionStart(compact) context now joins the first request after a
  compaction (appended in agent/request before dsh builds it), the overflow
  retry included; an inbox inject reached the model one step late.

Tests (each failing before): dsh-runtime (priced calls = served replies,
cost_usd > 0 and the sum of cost_by_model, handoff on the retried request),
dsh-plugin, budget-meter, e2e-live-hosts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AiLhAWTRtwbgMCDQNJUbea
@claude
dsh cost cell passes the evidence ledger check; handoff fallbacks and…
609a4cb 
… principles asserted; suite counts

- evidence.test.py required `$N.NN` on every ledger row, so a dsh row as
  run-summary prints it (`$0.0231`, `<$0.0001`) could not be added without
  failing CI. The cost cell is now checked against exactly what run-summary
  prints for a measured cost on the row's host (the HOST= of its reproduce
  command): `$N.NN` for Claude Code, `$N.NNNN` or `<$0.0001` for dsh. An
  unmeasured cost is still refused (`—`), and so is a price on a Codex or
  OpenCode row, which `$N.NN` accepted before. New check: run-summary's own
  Claude and dsh rows pass, other formats do not.
- run-summary chose `<$0.0001` from the rounded total, so a dsh run under half
  a micro-dollar (cost_usd 0.0) printed `$0.0000`; it reads the unrounded
  total now. Test: e2e-live-hosts (a $0.00000015 run).
- dsh-plugin: the compaction handoff carries the core principles; a role's
  own request after its compaction gets nothing (the request is now sent);
  an aborted request and a session log that refuses the append both fall
  back to inject, once. dsh-runtime: the overflow retry carries the core
  principles, once.
- Suite counts: dsh-plugin 48, dsh-runtime 53 (CHANGELOG, ROADMAP, OVERVIEW);
  evidence README, ROADMAP and the plugin READMEs name the dsh cost cell;
  ROADMAP §4.9 gets the open item that a Codex/OpenCode run cannot be a
  ledger row (no measured cost).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AiLhAWTRtwbgMCDQNJUbea
@claude
CI: macos runs the real dsh runtime test; conformance ignores the cal…
bf1d834 
…ler's tools dir

The macos job ran `make test-hosts` with no dsh installed, so
dsh-runtime.test.mjs (50 checks) printed "skip" and exited 0 while the
README said macOS ran the same host tests. It now installs the pinned dsh
exactly as the hosts job does and sets DSH_BIN and ALOOP_REQUIRE_DSH=1.
readme.test.py checks that every job running the host tests has that same
install step, with the pin the test names, before `make test-hosts` (on
the old ci.yml: "CI job 'macos' runs the host tests without installing dsh
first").

The conformance harness passed CLAUDE_PLUGIN_DATA and XDG_DATA_HOME
through, and ensure-tools looks there before PATH. With a passing
ctx-symbols under XDG_DATA_HOME, HEAD scored 72/82. The harness now drops
both, for the gates and for the focused suites it runs: 82/82 with the
same decoy as XDG_DATA_HOME and as CLAUDE_PLUGIN_DATA.

The 6.6 long-report case now has long notes after `### Next`. With only
the Next check back in its old `printf | grep -q` form, 6.6 used to pass
(81/82; only the 6.3 regressed case caught it) and now fails (80/82). The
CHANGELOG, ROADMAP and the 6.3 comment now say what the old gates did.
With e2ed5aa's gate hunks reverted, every report longer than a pipe holds
was blocked at the role line's `head -1` ("not from '<role>'"), never
passed. A regressed RED got through only now and then, on a report short
enough to fit in a pipe.

Three focused suites (lsp-adapter, plan-render, semantic-targets) still
read XDG_DATA_HOME when run directly. This is now a ROADMAP 4.9 open item.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AiLhAWTRtwbgMCDQNJUbea