Repository navigation
Add Pi as a CLI Base Agent (acting agent and pi Evolver) - #89
Conversation
…-Lab#77 Infini-AI-Lab#77 (Claude subscription) reworked CliAgent's result: a CLI's `outcome()` reports `failed` for a session that exited 0 without a result, and every caller reads it. Pi now implements `outcome()` (a last request that failed or was aborted is `failed`) instead of the shared `exit_code` hook this branch had added, and takes its home from `last_file`'s directory as Claude Code does, so the shared runner's `command()` keeps main's signature. Pi takes no `login` (acting agent, isolate check, the Evolver's sign-in check); `isolation: container`, now Codex's and Claude Code's, stays refused for Pi. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…sume, RRSI, docs) - A bound model's cost follows its price: Pi's own figure counted a model with no cost in models.json as free (an Anthropic reference recorded $0) and ignored a given `price`. - A session that did not settle, aborted, or whose reply was cut at its output limit is `failed`, as Pi's text mode exits 1 for each. - The MCP servers set Pi's per-request `timeout` (1800 s for the call's tools, 120 s for deployed ones, as Codex's tool_timeout_sec) instead of Pi's 60 s, which cut off Trainer tools that run rollouts. - A compaction's summary request (`compaction_end.result.usage`) counts as a call. - An inherited PI_CODING_AGENT_SESSION_DIR is emptied, so sessions stay in the home where a resume finds them. - RRSI reads a log found below a trajectory with that host's reader (a nested agent-pi.jsonl gave no steps). - Docs: the recorded cost, `evolver.base: pi` beside `evolver.model`, `pi` in config-fields, and that Pi speaks chat completions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Inside a run (HA_ADAPTER_SANDBOX=1, every task-sandboxed acting run) the stdio tool server a deployed workspace starts (`.codex/config.toml`, `.mcp.json`, `.pi/mcp.json`) builds a sandboxed Session, which loads no adapter in the server's process; the server read `session.adapter.manifest` and crashed before MCP started, so the adapter's tools never reached Codex, Claude Code or Pi. It now reads `session.manifest`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pi offers a model it does not know text only and drops a tool's image for it (a screenshot, a page read as an image). `image_input: true` (agent.pi, evolver.pi) writes the bound model's `input: ["text", "image"]` into models.json; the default stays text only, since a text-only model refuses images. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Thanks for adding Pi. We reviewed the PR (two independent reviews per round, each finding checked against the code
and, for Pi's behaviour, against Pi 1.5.0's source) and pushed four commits to this branch. All four are in the PR now.
2361862: merge main (#77) into the branch
#77 (Claude subscription) reworked how CliAgent reports a session: each CLI now implements outcome(), and
callers treat failed (exit 0 without a result) as a failure. So the merge:
- Replaces the shared
exit_codehook withPi.outcome(). A failed Pi session now reads as exit code 0 with
extra["failed"], not exit code 1 withcli_exit_code. - Has
Pi.command()take its home fromlast_file's directory, as Claude Code does. The sharedcommand()signature
stays as on main, without thehomekeyword. - Makes Pi refuse
loginin the acting agent, theisolate.pycheck and the Evolver's sign-in check.
isolation: container, which now runs Codex and Claude Code, stays refused for Pi.
47fa33c: review fixes
Each fix has a regression test that fails without it.
- Cost. For a bound model, Pi's own cost figure was kept. Pi prices a model with no
costinmodels.jsonat
0, so a model bound to Anthropic recorded $0, and apricegiven in the config was ignored. Now the cost of a
bound model is computed from its price (Pi.adjust_usage). - Failed sessions. A session that did not settle, aborted, or whose reply was cut at its output limit
(stopReason: length) was reported as done. It is nowfailed, matching Pi's text mode, which exits 1 for each
of these. - Tool timeout. Pi's MCP servers default to a 60 s per-request timeout, which cut off Trainer tools that run
rollouts. Both servers now settimeout, matching Codex'stool_timeout_sec: 1800 s for the call's tools and
120 s for tools deployed withhermit deploy --host pi. - Compaction. The summary request's usage (
compaction_end.result.usage) is now counted as a call. - Resume. An inherited
PI_CODING_AGENT_SESSION_DIRmoved sessions out of the private home, so a resume
silently started over. It is now emptied. - RRSI. A nested
agent-pi.jsonlwas read with the Codex parser and gave 0 steps. It is now read by its host's
reader. - Docs. Four corrections: how the recorded cost is computed;
evolver.base: pialongsideevolver.model(without
it the config is refused);piinconfig-fields.md; and the backend guide's "the CLI agents do not [speak chat
completions]".
68b42b2: tool server in a sandboxed run (also affects Codex and Claude Code)
In a task-sandboxed acting run, the stdio tool server a deployed workspace starts crashed at startup:
mcp_server.py read session.adapter.manifest, but a sandboxed Session has adapter = None. As a result the
adapter's tools never loaded. This predates the PR and hit Codex and Claude Code as well, but it also broke this
PR's Pi acting path, so it is fixed here (session.manifest), with an end-to-end test that starts the server
sandboxed and calls a tool.
1f207fd: image_input
Pi treats a model it does not know as text-only, so a bound vision model never received images that tools returned.
image_input: true (in agent.pi or evolver.pi) declares the model's input: ["text", "image"]. The default
stays text-only, because a text-only model rejects images.
We also updated the PR description to match: it no longer describes the exit_code hook and the home keyword,
and it has a short "After review" section.
CI passes on 1f207fd, the head with all four commits.
Noted, no change in this PR
- Lost usage. A compaction that fails loses its usage, because Pi reports none for it.
- Same on main. These also happen with Codex and Claude, so they are not specific to this PR:
- the acting agent's retry loop keeps only the last attempt's usage;
extra_bodyis not passed to the CLI.
- Rare cases. These need unusual inputs or setups:
- a prompt starting with
@; - system instructions longer than 120 KB in a sandbox;
- session ids containing
_or.; - a workspace
.pi/settings.jsonthat setssessionDir.
- a prompt starting with
🤖 Generated with Claude Code
…Evolver) DeepSeek Harness (`dsh`, npm @deepseek-ai/dsh 0.2.2-rc.1) beside codex, claude and pi, in the same three layers as Pi (#89), on the runner's outcome() of #77. - Layer 1: a <backend>/<model> reference becomes a route of dsh's pi-ai adapter (Binding.dsh_provider): the backend's URL in the first dialect it serves, the key by its variable's name (apiKeyEnv), the context window. dsh reports no cost: the backend's price applies. - Layer 2: Dsh(CliAgent) runs `dsh --profile headless --json` in a private DSH_HOME, HOME and DSH_AGENTS_HOME, telemetry off, the caller's own DSH_* variables dropped. The session's settings are a patch layer of dsh's own rows (--patch): the bound model as the default model (effort as its reasoningEffort, image_input as Pi's), the call's tools as an MCP server offered directly (1800 s per call), web search and fetch off unless asked for, no title request, session log, plugin list, sub-agents or workflows. A server that needs no key gets the placeholder key through HERMIT_DSH_NO_KEY (a route that names none is refused by dsh). Approvals are never asked (--approval-policy never); permission_mode picks dsh's own sandbox. Resume by --session-id when the home keeps the session; the workspace's .dsh/skills are also the session's own (dsh reads project skills only at the nearest folder with a .git). outcome(): a run without a completed `final` has failed, whatever dsh's exit code (SIGTERM exits 0). Usage from step_end events; the trajectory from the stream, a command's exit code from dsh's end markers (a timeout or a stop: unknown; "(no output)": empty); every unfinished end is an error step. dsh's outage codes (RATE_LIMIT, SERVER, TIMEOUT, TRANSPORT, EMPTY_RESPONSE, STREAM_CLOSED) join UNAVAILABLE. Roles, a login and container isolation are refused. - Layer 3: hermit/hosts/dsh.py (`hermit deploy --host dsh`): AGENTS.md, .dsh/skills and the adapter's tool server (120 s per call) as an insert row of .dsh/hermit.patch.yml, which the acting agent passes with --patch. The AgentEvolver, the loader and the config accept dsh; methods/dsh/ is the new Evolver (dsh's own sandbox off in the evolver sandbox, as Codex's is). - Tests: tests/core/test_dsh_agent.py over streams recorded from dsh 0.2.2-rc.1 (tests/fixtures/dsh) and, marked integration, the installed dsh: a session unsandboxed and in bubblewrap, the acting agent inside a git repository (the deployed skill offered, the adapter's own tool server answering), a keyless server, and --dump-config holding every row the session patches; the deploy host; selection entries. Checked against dsh's source at the tag dsh-v0.2.2-rc.1 and with the installed dsh. First built on #89's earlier head; rebuilt on main after #89 merged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…Evolver) (#90) * Add DeepSeek Harness (dsh) as a CLI Base Agent (acting agent and dsh Evolver) DeepSeek Harness (`dsh`, npm @deepseek-ai/dsh 0.2.2-rc.1) beside codex, claude and pi, in the same three layers as Pi (#89), on the runner's outcome() of #77. - Layer 1: a <backend>/<model> reference becomes a route of dsh's pi-ai adapter (Binding.dsh_provider): the backend's URL in the first dialect it serves, the key by its variable's name (apiKeyEnv), the context window. dsh reports no cost: the backend's price applies. - Layer 2: Dsh(CliAgent) runs `dsh --profile headless --json` in a private DSH_HOME, HOME and DSH_AGENTS_HOME, telemetry off, the caller's own DSH_* variables dropped. The session's settings are a patch layer of dsh's own rows (--patch): the bound model as the default model (effort as its reasoningEffort, image_input as Pi's), the call's tools as an MCP server offered directly (1800 s per call), web search and fetch off unless asked for, no title request, session log, plugin list, sub-agents or workflows. A server that needs no key gets the placeholder key through HERMIT_DSH_NO_KEY (a route that names none is refused by dsh). Approvals are never asked (--approval-policy never); permission_mode picks dsh's own sandbox. Resume by --session-id when the home keeps the session; the workspace's .dsh/skills are also the session's own (dsh reads project skills only at the nearest folder with a .git). outcome(): a run without a completed `final` has failed, whatever dsh's exit code (SIGTERM exits 0). Usage from step_end events; the trajectory from the stream, a command's exit code from dsh's end markers (a timeout or a stop: unknown; "(no output)": empty); every unfinished end is an error step. dsh's outage codes (RATE_LIMIT, SERVER, TIMEOUT, TRANSPORT, EMPTY_RESPONSE, STREAM_CLOSED) join UNAVAILABLE. Roles, a login and container isolation are refused. - Layer 3: hermit/hosts/dsh.py (`hermit deploy --host dsh`): AGENTS.md, .dsh/skills and the adapter's tool server (120 s per call) as an insert row of .dsh/hermit.patch.yml, which the acting agent passes with --patch. The AgentEvolver, the loader and the config accept dsh; methods/dsh/ is the new Evolver (dsh's own sandbox off in the evolver sandbox, as Codex's is). - Tests: tests/core/test_dsh_agent.py over streams recorded from dsh 0.2.2-rc.1 (tests/fixtures/dsh) and, marked integration, the installed dsh: a session unsandboxed and in bubblewrap, the acting agent inside a git repository (the deployed skill offered, the adapter's own tool server answering), a keyless server, and --dump-config holding every row the session patches; the deploy host; selection entries. Checked against dsh's source at the tag dsh-v0.2.2-rc.1 and with the installed dsh. First built on #89's earlier head; rebuilt on main after #89 merged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * dsh: fixes from review round 1 (two independent reviewers on 03fafff) Each fix was checked against dsh's source at dsh-v0.2.2-rc.1 and, where it shows, against the installed dsh (new recorded streams in tests/fixtures/dsh); each has a test that fails without it. - Stream reading: a bash result dsh's --json cut at 8 KiB (`truncated`) lost its exit marker and read as exit 0; now unknown, and the text says it was cut. A command started in the background has no status yet (unknown). The end markers are anchored at the very end (\Z), as dsh's JavaScript reads them, so a command that printed a marker of its own and exited 0 is a clean exit. - Usage: a step that answered counts as a call even when the provider reported no usage (dsh reports zeros), and so does a step dsh reports no usage for (an attempt without any); what dsh never reports (compaction, a step's partial usage) is documented. parse_log reads Pi's and dsh's streams with their own readers. - outcome(): a session whose tools did not start (dsh writes the MCP row's error on stderr and runs on without them) has failed. A run that wrote only an `error` event (a usage error) or nothing (a working directory's .env that sets PATH, PYTHONPATH, ...) started no session, so the acting agent reports it instead of recording each task as a wrong answer: outcome() may now say `started` (shared runner; absent, as before). - The workspace is the project wherever it lies: the session's layer sets the skill-filesystem row to the workspace's .dsh/skills alone and the agent-instructions row to the deployment's marker (.hermit-deploy.json) as the project root, so a workspace inside a git repository gets its own skills and none of the repository's skills or AGENTS.md (they outranked the adapter's). This replaces copying the skills to the home. - `--add-dir` takes the real path of an existing directory, as dsh requires (a link failed every command and write); an effort without a model is refused; user_home() follows DSH_HOME. - The acting agent passes agent.dsh.permission_mode on (it was dropped) and refuses a mode dsh has not. - hermit deploy --host dsh: the patch layer also names the workspace's .dsh/skills (a workspace below a repository's root kept none), and a context past the 65,536 bytes dsh reads of a project's instructions is refused. - The AHE template has a `dsh` wire (as Pi's: hooks run by hand hand the memory over); mwm and RRSI refuse a backend other than Codex or Claude Code at setup (pi and dsh were accepted and ran without their model). - Docstrings say what dsh reads (.env files, the instruction chain) and sends (its own provider's ids). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * dsh: fixes from review round 2 (two independent reviewers on 7a076fb) Each fix was checked against dsh's source at dsh-v0.2.2-rc.1 and, where it shows, against the installed dsh; each has a test that fails without it. - A task's data is bound in the workspace, not linked (binds_mounts, as for Claude Code): dsh's glob and grep run ripgrep without --follow, so a linked folder was never listed or matched. dsh's own sandbox still runs nested in that mount namespace (integration test). - Web search with a model of another backend: dsh's search is DeepSeek's own service (its Messages API on DEEPSEEK_API_KEY), so the session searched on a provider and key the run did not bind, outside its usage. Such a session gets web fetch only, as Codex's hosted search is off for a backend other than OpenAI's; a DeepSeek model or dsh's own provider keeps search. - Usage: a step dsh reports no usage for counts as a call only if it produced something (a step whose requests all failed is none), and price_usage leaves calls of which no token was reported unpriced: their cost is unknown, not $0.00 (shared: a usage with calls and no tokens is no longer priced at 0 for any CLI). - The instructions' project root: no root marker, so it is the working directory, the workspace. The deployment marker (.hermit-deploy.json, which every host's deployment writes) made a session without a deployment take a deployed folder above it for its root, and read its AGENTS.md. - hermit deploy --host dsh: the tool server's row fails at a session's start when dsh cannot start it (failOnStartupError, as Codex's `required`), which the acting agent reads as a failed session; it was retried in the background while the session ran without the adapter's tools. The 65,536-byte check counts dsh's frame around AGENTS.md (304 bytes, a </system-reminder> one byte more): an AGENTS.md just under the cap was deployed and cut. Docstring: the command names its profile; the skills row replaces a skill-filesystem config of the user's own. - dsh's effort levels and permission modes are checked as the run config loads (an unknown one failed every session, one task at a time), and an effort level dsh has not is refused in the session. - The AHE learners' texts name a Pi or dsh acting agent's stream (agent-pi.jsonl, agent-dsh.jsonl) instead of agent-codex.jsonl (Pi's entry was missing on main too). - Tests: a recorded stream of a session stopped by SIGTERM (stopped.jsonl: exit 0, no `final`, an `error` event) replaces a cut success stream, and is run through the acting agent and an AgentEvolver step; the deploy refuses at the exact limit and an MCP server name dsh refuses. Correction to round 1's message: on 03fafff a dsh usage error was already an error record per task (failed), not a wrong answer; `started` now makes the acting agent stop the run and say so. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * dsh: fixes from review round 3 (two independent reviewers on 9d87553) Each fix was checked against dsh's source at dsh-v0.2.2-rc.1 and, where it shows, against the installed dsh; each has a test that fails without it. - Usage: round 2's shared rule (a usage with calls and no tokens is not priced) is gone. It made Claude Code's results of refused requests (a 529, a 401: one turn, no tokens) cost unknown instead of $0.00, and the AHE and RRSI evolvers' merged usage then dropped the step's known spend from the budget. Usage gains `unreported_calls`, the calls whose tokens a CLI did not report; only dsh's reader sets it (a step that produced something without usage: the provider reported none, or dsh retried the step and so left its usage out), and price_usage leaves such a usage's cost unknown rather than pricing the part reported, which under rate limits may be far less. - dsh's transcripts are plain JSON lines (the session layer sets session-persistence-jsonl to compression none, its root as dsh's own `!!js dshHomePath('sessions')`): they were zstd, which hermit.redact cannot scrub a key from. A scrubbed transcript still resumes. - The dsh Evolver's sessions take no instruction file (instruction_files false: the agent-instructions budget 0): dsh adds a folder's AGENTS.md when its file tools touch it, so the learner was given the adapter's AGENTS.md, the acting agent's policy, as instructions. Codex's learner is given none. - dsh's settings are checked as the run loads for the dsh Evolver too (an effort it has not failed every learner step, after its rollouts); evolver.dsh.permission_mode is refused (the Evolver runs dsh's own sandbox off), and YAML's bare `off`, a boolean, is `off`. - Web search is kept only for a model reached at api.deepseek.com on DEEPSEEK_API_KEY, where dsh's search goes (a `deepseek` backend moved by DEEPSEEK_BASE_URL searched elsewhere). - A run that ends CONTEXT_WINDOW_EXCEEDED with a backend that declares no context_window says to declare it (recorded: context_window.jsonl, vLLM's refusal of dsh's 128,000 output tokens; with the window declared dsh sizes the request and completes). - A unit test with the fake dsh for the task data's binding, which CI runs where bubblewrap can bind (the only test needed the dsh CLI). - Wording: the deployed tool server's row makes dsh report a server it cannot start and the acting agent fail the session (dsh itself goes on); a run dsh never started is each task's error record, not a stopped run (round 2's message said so); the deploy's size check is exact for a session at the workspace's root; the linked-data warning names dsh's glob and grep. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * dsh: fixes from review round 4 (two independent reviewers on 2021944) Each fix was checked against dsh's source at dsh-v0.2.2-rc.1 and against the installed dsh; each has a test that fails without it. - A step dsh retried after the provider refused a request lost its cache counts: the refused attempt reports zeros, pi-ai reports a count of 0 as none, and dsh sums a count into the step's only when every attempt reported it, while the step's totalTokens still holds them. The reader counts what the total holds beyond the counts reported as input, and the call as unreported (its cost unknown: those tokens were read from the cache or written to it, which are priced apart). Recorded: retried.jsonl (a 503, then 1000 prompt tokens, 900 cached: dsh reports inputTokens 100, totalTokens 1010). Round 3 said such a step's usage is left out; dsh does that only for an attempt that reports no usage at all (a stream that ended before its last event, an attempt a stop cut short). - The dsh Evolver's settings are checked from its params as the Trainer has them (its method's own under the run's, the model bound), in login.evolver_problems, which train runs before its first step and hermit check now reports for an Evolver without a login. Round 3's check, as the run loaded, missed the dsh method named with its params (`backend: dsh` comes from the method's manifest) and refused an effort given with the run's own model. The acting agent's are still checked as the run loads; one function (cli.dsh_problems) holds both, and isolate.py's change is that call. - dsh's own provider (a model of no binding, DeepSeek's API) takes the efforts off, low, high and max; another level failed every session (UNSUPPORTED_REASONING_EFFORT). - evolver.dsh.instruction_files is refused, as evolver.dsh.permission_mode is: the Evolver dropped it. - The caller's DEEPSEEK_SEARCH_BASE_URL is dropped with its DSH_* variables: without the sandbox it moved dsh's web search off DeepSeek's API. - The context_window hint follows dsh's error whole (once), not its first 300 characters. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * dsh: fixes from review round 5 (two independent reviewers on f4b7b09) Each fix was checked against dsh's source at dsh-v0.2.2-rc.1 and against the installed dsh; each has a test that fails without it. - A step dsh retried after a refused request is priced again, its dropped cache tokens counted as input. Round 4 made such a session's cost unknown, and a USD budget then counted none of its spend (rollouts and learner steps add no dollars for an unknown cost), under the very load that causes retries. Its tokens are known, and hermit prices a cache write as input: the price is the step's with the counts reported, or more if those tokens were read from the cache, never less. A call stays unreported, and its session's cost unknown, only when dsh reported none of its tokens. - The caller's DEEPSEEK_BASE_URL is dropped as DEEPSEEK_SEARCH_BASE_URL is: without the sandbox it sent the requests of dsh's own provider, with DEEPSEEK_API_KEY, elsewhere. A variable the session is given (`env`) still passes. - dsh's settings check takes what each agent uses: the Evolver's danger-full-access and instruction_files false are accepted (round 4 refused them); the acting agent's instruction_files false is refused (it was dropped; it would cut the adapter's context, which reaches dsh as its AGENTS.md). - Tests pin the session's tool row (failOnStartupError, the 30-minute call timeout), every row DSH_OFF turns off, all six outage codes, and dsh's own provider's levels for the Evolver; the outage test is named for what it checks. - Wording: effort off asks a bound model for no reasoning level, which leaves its server's default (dsh's own provider turns thinking off); the context-window hint keeps dsh's error line as far as it fits. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * dsh: fixes from review round 6 (two independent reviewers on f1a92ea) Each finding was checked against dsh's source at dsh-v0.2.2-rc.1 and the installed dsh; each test added fails without what it checks. No behaviour changes: the fixes are tests and claims. - Tests: a session's own `env` reaches dsh while the caller's DEEPSEEK_BASE_URL is dropped; the integration test of dsh's own sandbox shows a command's write outside the workspace refused (it passed with the sandbox off); a bash call that failed to run (a result dsh marks `error`) has no exit code; a linked session folder, or a session in a linked folder, is not resumed; a stopped session's usage is pinned (none). - The reader's account: the last step of a session a stop cut short is not counted (dsh writes a step's usage at its end, after its tool calls ran, and a stopped session writes nothing more); the docstring said such a step was a call of unknown tokens. dsh's own adapter, not pi-ai, drops a cache count of 0. A retried step's price is never less than with its counts for a price whose cached input is no dearer than its input; on the Responses API the total is the server's own `total_tokens`, taken as given. - dropped_env says why a sandbox's grant of DEEPSEEK_BASE_URL does not reach dsh (hermit's own is its deepseek backend's chat completions root, which dsh reads as its own provider's Messages root) and that `env` gives dsh its own; the Settings paragraph says so too. - A stale test docstring: the Evolver's own permission mode and instruction_files are taken. Not changed, disclosed: the acting agent's re-run after an outage keeps only its last attempt's usage, for every CLI (shared code, a separate change). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * dsh: leave the Pi fixes to their own PRs The Pi fixes this branch carried change Pi's behaviour on main, not dsh's: parse_log's Pi reader, the AHE Evolver's texts for a Pi acting agent's stream, and mwm and RRSI refusing a Pi learner. They go to their own PRs (fix/pi-stream-readers, fix/mwm-rrsi-backend-check). This branch keeps dsh's entries only: parse_log reads a dsh stream as dsh's, the AHE texts name a dsh acting agent's stream, and mwm and RRSI refuse dsh (the other PR's general check covers it too, whichever merges first). The integration test of dsh's own sandbox keeps the session's home under ~/.cache/hermit-tests, as the test checklist asks of a check under bubblewrap (it was under /var/tmp). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * dsh: test which steps of dsh's stream count as calls Review round 7 (five reviewers on 37aee8c, one lens each, every finding then reproduced by a separate agent told to refute it) found 34 issues, all confirmed: one of medium severity, the rest low. The medium one: no test guarded the reader's three rules for a step whose tokens dsh did not report (a step that only called a tool is a call, so is one that only thought, and what a step produced does not carry over to the next); each, undone, left every test passing while it misstated a session's cost. The new test fails under each. The low ones are listed in the PR description, not changed. Also: a round-6 test docstring rewrapped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * dsh: test a command dsh moves to the background at its timeout In a hermit dsh session a command still running at its timeout (dsh's default is 60 s) is moved to a background job, unless dsh's job limit is reached: the session's layer leaves tool-bash's promoteOnTimeout at its default, true. Its result ends `[still running after <ms>ms; moved to background job <id>]` with no exit marker, and only the DSH_BACKGROUND check keeps it from reading as exit 0. No test held that case: bash.jsonl was recorded with promotion off. Review round 7 found this (rated medium by its reviewer, low by the agent that reproduced it, as the code was right); the author asked for it fixed. promoted.jsonl is recorded through make("dsh").run, as the other Base Agent streams are, and the test now reads it, and also a command stopped from outside the call (`[stopped: <reason>]`, which no headless session writes, so written as dsh's tool-bash renders it), with and without a marker after it. Each check undone (the background check, the `stopped:` alternative, the markers after a stop) fails the test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * dsh: read a timeout marker at any timeout dsh prints (review round 8) Review round 8 (two reviewers on f4792e0, one on the test and its fixture, one on the claims, each finding then reproduced by a separate agent told to refute it) found: - dsh prints a command's timeout as the model gave it, a JavaScript number (`1000.5`, `0.5`), and both timeout markers were matched only at a whole number of milliseconds: a command still running at such a timeout, moved to a background job, read as exit 0, and one killed at it that trapped the signal read as its trapped exit (rated medium). Both markers now match any number dsh prints. Recorded: promoted.jsonl's third call (1000.5 ms, moved to the background) and bash.jsonl's ninth (1000.5 ms, killed, exit 0 trapped). - The test of a promoted command held only one with output before its marker; the marker alone, what a command that printed nothing yet gets, rests on DSH_BACKGROUND matching without a newline before it, which nothing held (rated medium). promoted.jsonl's second call is `sleep 5`. - Wording (low, in f4792e0's own docstring): the stop marker comes before the signal or exit marker, not before every other marker (a sandbox denial's come first). Each fix undone fails its test: the marker with a newline required before it, and either marker at whole milliseconds only. bash.jsonl is recorded again by the same script, its first eight results unchanged; its steps' count in the usage test follows (10). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * dsh: wording from review round 9 Review round 9 (two reviewers on d25958a, one on the code and its tests, one on the claims, each finding then reproduced by a separate agent told to refute it) found 6 issues (4 distinct), all low. Three are wording this change's own text got wrong, corrected here (comments, a docstring and the fixture README; no code or test changes): - dsh prints the timeout it applied, not always the model's: its default (60 s) when the model gives none, and at most its cap (600 s). - DSH_NO_OUTPUT's comment said `(no output)` comes before the marker of a command that printed nothing, which a command moved to the background does not get (its marker stands alone). The fourth, the exponent form both timeout patterns match (`1e-7`, a timeout under 0.000001 ms) held by no test, is listed in the PR description, not changed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * dsh: an outage only by the run's own end; the web-search test on the built-ins alone (post-PR audit) - Dsh.outcome read `unavailable` from every line of the stream that is no event, and dsh starts a deployed stdio tool server on its own stderr, which the stream keeps: outside the task sandbox, a tool's line with an outage's words (`Service Unavailable`) made a session that failed for another reason (`max-tokens`) an outage, run `retries` times and then an error of unknown cost. It is now read from the `final` event's error code (DSH_OUTAGES, Dsh.outage), and UNAVAILABLE is main's again. - test_dsh_refuses_roles_and_a_container_and_searches_only_on_deepseeks_own_provider bound its DeepSeek models through the developer's own backends (HA_BACKENDS or ~/.config/hermit/backends.yaml, DEEPSEEK_BASE_URL, DS_BASE_URL), so it failed wherever those move `deepseek`: it binds them on the built-ins alone. - The fixture README: tools_down.jsonl's port is one Node's fetch refuses, and the Base Agent's stream has that run's stderr lines first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * dsh: an overload dsh leaves unclassified is an outage by its words (review round 10) - cacb4ae read an outage from the final event's code only, which narrowed a43698d's rule: a provider overloaded inside a 200 stream (the Messages API's `overloaded_error` event, a chat server's error chunk) has no status, and dsh ends the run with pi-ai's unclassified PI_AI_ERROR, which it does not retry; the task was then not run again. Such an end is now read by its words (UNAVAILABLE, as Pi's are), and only dsh's own end is read so, never a tool's stderr. Recorded through the Base Agent: overloaded.jsonl. - A test docstring described parse_log as main had it before #49. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * dsh: guards and wording from review round 11 (post-PR audit of ace8fab) - A kept home's patch layer is written as a file of its own, never through a link or into a pipe the last session left (the dsh Evolver's agent_home): the guard had no test. - An end dsh leaves unclassified (PI_AI_ERROR) that is no outage, with a tool's outage words on dsh's stderr, runs once: ace8fab's new path had no test that its words come from the end alone. - PI_AI_ERROR is the code of dsh's pi-ai adapter, not pi-ai's; outcome() and errors() no longer say the end's code alone tells an outage. - test_deploy.py: the dsh test is named for what it holds, and its comments say what dsh does (a server that does not start is reported; the dsh acting agent fails the session; a project's root may be the working directory itself), as hermit/hosts/dsh.py now does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * dsh: documentation (the pages, the changelog, the AHE template's notes) DeepSeek Harness joins the pages that name the CLI Base Agents, their settings, deployment, models and installation, in the wording of Pi's and mini-swe-agent's entries; what is its own: its inner sandbox (permission_mode, and what it does to environment.write and /tmp), its web search, its patch layer and how a session loads it, a tool server that does not start, and the usage it leaves out. No code changed. convert's lists are untouched: dsh has no plugin format. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * dsh: documentation, review round 13's corrections Round 13 read f0d2091: 16 findings, all low. dsh's commands each have a /tmp of their own and its default sandbox refuses a declared writable path (the sandbox guide); how to write a provider of one's own (model-backends); what a web fetch reaches; which price the cost follows; what Landlock, a patch layer and the request left out of the usage are. No code changed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * dsh: documentation, two corrections from the read-back The sandbox guide's two ways to reach a path under /tmp are for reading it (a mount is read-only); the requests that summarize a session's history are uncounted each time, not once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * dsh: deploy retires stale skills as every host does since #97 hermit deploy --host dsh now calls check_skills before it writes anything (a redeployment refuses to replace a skill folder no deployment wrote, or to write through a linked skills folder) and retire_skills after it copies the skills (a skill the last deployment exported and this one does not moves to .retired-skills/). dsh copies its skills before it writes its patch layer, whose skill-filesystem row names them, so retire_skills follows copy_skills there. #97's redeployment test gains the dsh case, which fails without the retire_skills call; no test covers the check_skills call, as on main for every host but Codex. #97's CHANGELOG entry names .dsh/skills. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Change
Adds Pi (
pi, v1.5.0) as a third CLI Base Agent besidecodexandclaude, as oneCliAgentsubclass. It can act (agent.base: pi) and learn (the newpiEvolver).<backend>/<model>reference becomes Pi'smodels.jsonprovider entry (Binding.pi_provider): the key written as$VARIABLE(never the value), the backend's price, and the first dialect the backend serves (chat completions, then Messages, then Responses).Pi(CliAgent)runspi --mode jsonin a private home (PI_CODING_AGENT_DIR,HOME), offline.exposure: direct; Pi's default would hide them behind its Code Mode.records.parse_pi,trajectory.pi_steps).Unsupported.Pi.outcome()reportsfailed(main's mechanism since Claude Code subscriptions as a shared model backend for acting and learning (#70) #77: exit 0 without a result) for a session that did not settle, aborted, or whose last request failed, was aborted or was cut at its output limit. No change to the shared runner's interface.hosts/pi.pyand passes--approve, without which Pi skips the workspace's.pi/tools and skills.AgentEvolver, the loader and the config acceptpi. The newmethods/pi/Evolver copies no login (home: []).hermit deploy --host pi: the adapter's MCP server is now declaredexposure: direct. This is a visible change for existing deploy users.~/.pi/agent/auth.json) are left for later.hermit/isolate.py, one word:agent_home.mkdir(parents=True). The task sandbox could not create Pi's two-level user folder (.pi/agent), so every Pi acting call crashed. Under the sandbox checklist this needs core-team agreement (Please confirm here @haizhongzheng )agent-layers.md,model-backends.md,config-fields.md,task.md,evolver.md,methods/agent.md,custom-evolver.md,compatibility.md,installation.md, CHANGELOG.After review
Four commits were added on review (details in the review comment):
outcome()instead of its ownexit_codehook, and takes its home fromlast_file's directory instead of ahomekeyword oncommand(). Pi refusesloginandisolation: container.failed;timeoutis 1800 s / 120 s instead of Pi's 60 s;PI_CODING_AGENT_SESSION_DIRis emptied;session.adapteris None in a sandboxed Session), so adapter tools never loaded. It now readssession.manifest.image_input:image_input: true(inagent.piorevolver.pi) declares a bound model that takes images (Pi's default for an unknown model is text only).The table below is from before these commits.
Validation
Run on Linux with bubblewrap 0.9.0, Python 3.12 and Pi 1.5.0 (release binary, checked against
SHA256SUMS). No real provider was used: a local mock chat-completions server stands in for the model.python -m pytest --integration, pristineorigin/main(6b258fb)python -m pytest --integration, this branch (with6b258fbmerged)tests/core/test_pi_agent.py(11 tests, about 5 s)pi, unsandboxed and in bubblewrapCLI_HOMES,STREAMSand the unknown-agent messagehermit train+hermit eval, sandbox enforced, Pi acting and learning (docs' toy task)examples/clbench_database(golden parity), which already fail onmain.test_an_installed_cli_starts_with_only_its_own_paths.tests/fixtures/pi/) are real Pi output; only two paths are changed, as their README says.HTTPS_PROXYis confirmed from its source (anEnvHttpProxyAgentset up inmain.ts), not from a run.Review
agent.base: pi,evolver.base: pi, api:settings section, theprovidersetting. One visible change:hermit deploy --host pinow writesexposure: direct.models.jsonholds the key's variable name, never its value.UNAVAILABLE),isolate.py(mkdir(parents=True); Pi refuseslogin) andlogin.evolver_problems(the same for thepiEvolver).unavailable, so the acting agent retries.make("pi").run, the acting agent'srun,hermit trainandhermit eval. The stand-ins are only at the edges (the CLI, the model).tests/core/; the fixtures are registered intests/selection.py(CORE) andtests/README.md.integrationtests.test_selection.py'sBUILT_INgainspi.CliAgentinside the task and evolver sandboxes. All of its variables come fromhome_env(). Run records:rollout-1.json / learner-1.json (excerpt)
isolate.pychanges. Confirmed by post-pr.