Repository navigation
Add official SHE evolution on AgentDyn with Lightning - #104
Merged
haizhongzheng merged 4 commits intoNov 13, 2026
Merged
Conversation
4 of 5 tasks
- evolver.base: pi works: call_model takes pi, as Infini-AI-Lab#92 documents for every method. - setup() refuses a bad proposer, debugger, debugger_batch_size or gate_mode, or a missing learner model, before any TRAIN episode. - Each round must measure the whole TRAIN set (is_epoch_end) before the official selector compares it; an error before the round's records still persists the stop flag. - Case IDs follow upstream: `<id>::<condition>`, and a user task's clean case `task_<suite>-<user_task>::clean`, so upstream's clean-failure guidance and regression risks reach the learner. - Hook staging makes every file readable, so SHE's hooks start under umask 077. - CI installs --group she; run.yaml reads full-test-v1, names its llm and enables the semantic judge. - Docs: SHE in the Pi tables; the method index, shell-tool and launch statements corrected; results marked for reference only (the learner ran without Hermit's sandbox), as for Infini-AI-Lab#103. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ightning # Conflicts: # docs/recipes/agentdyn.md # examples/agentdyn/evosafe/README.md # examples/agentdyn/evosafe/REPRODUCTION.md # tests/methods/evosafe/test_agentdyn_webpages.py
haizhongzheng
approved these changes
Nov 13, 2026
Member
There was a problem hiding this comment.
Thanks for this, and for the careful write-up of the protocol and its limits. We reviewed ceb8d62 with two independent reviewers over five rounds, verified every finding against the code and the pinned SHE source, and pushed the fixes to your branch as 6011119, then merged main (#103) as de2ac7e.
What we changed (each code fix has a test that fails without it)
- Pi.
evolver.base: pifailed at the first learner call; #92 documents every method on Pi, so SHE now takes it. - Settings checked at setup. A wrong proposer, debugger, batch size or gate mode, or a missing learner model, was found only after a full TRAIN round.
setup()now refuses them before any episode. The committedrun.yamlhad no learner model. - Whole TRAIN each round. With
data.step_sizebelow the TRAIN size, the official selector compared different case subsets. A round that does not end the epoch now stops before selection, and the stop holds even withoutallow_early_stop. - Case IDs as upstream reads them. Without
::<condition>, and thetask_<task>::cleanform for a user task's clean case, upstream's clean-failure guidance never fired, and regression risks named no observed case. - Hook staging. Under umask 077, the staged artifacts were unreadable in the hook container.
- CI. CI did not install
--group she, so the SHE tests always skipped there. - Docs and recipes. SHE is in the Pi tables; the method index no longer asks for a validation split; the shell-tool and launch statements are corrected;
run.yamlreadsfull-test-v1and enables the semantic judge. - Results are for reference only. The learner ran with
--no-sandbox, so, as for #103's EvoSafe comparison, the numbers are marked as not a reported result.
Noted, not changed
- Upstream
update_jsonldiscards its_deep_mergeresult, so such an edit is lost. Upstream's prompt does not offer this mode, and we keep the official code. - The learner's regexes in upstream's patcher run in the Trainer process.
Tests. With the pinned SHE dependency: 326 passed, 51 skipped (Docker and native AgentDyn tests). CI passed on de2ac7e.
🤖 Generated with Claude Code
haizhongzheng
marked this pull request as ready for review
November 13, 2026 02:07
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Change
This PR adds official SHE evolution to AgentDyn. Nemotron 3.5 Lightning is the frozen actor and semantic judge, and GPT-6 Sol via Codex is the learner. SHE evolves four artifacts: the system prompt, rule bank, safety memory, and tool policy.
Each round evaluates the current adapter on TRAIN, applies the official score and initial safe-utility floor, and restores measured best before proposing the next edit. The next round evaluates that proposal. Finalization exports the measured best adapter. The reserved validation split is unused.
The actor and semantic judge use Hermit's shared direct model client. Learner roles use the shared Base Agent (the run's
llm, Codex, Claude Code or Pi) for CLI execution, authentication, structured output, cancellation, and usage. Framework changes register the optional SHE dependency, its test group and its CI install. The AgentDyn runner handles method integration and keeps learned hooks inside the existing networkless Docker boundary.See the method and upstream adaptations, AgentDyn recipe, and results and reproduction details.
Validation
python -m pytest tests/methods/she tests/methods/evosafe tests/core/test_selection.py tests/core/test_pi_agent.py tests/core/test_spec.py tests/core/test_packaging.pywith the pinned SHE dependency: 326 passed, 51 skipped (Docker and native AgentDyn tests).pytest --changed --base origin/mainbefore the merge ofmain: 1389 passed, 168 skipped; its 2 failures were Harness² tests whose dependency group was not installed.For reference only, not a reported result: the learner ran without Hermit's sandbox (
--no-sandbox, the host lacked usable bubblewrap), which a reported result requires. This is the same status as the EvoSafe comparison in #103. The run also predates the review fixes belo 8000 w.All execution errors reached the 25-step limit without a terminal response. The breakdown was 0 clean and 14 injected with no adapter, 1 clean and 13 injected with initial SHE, and 2 clean and 15 injected with final SHE. Success and utility rates use completed episodes, and attack success rates exclude these execution errors. In this single run, the final adapter reduced observed attack success and improved safe task utility, while clean task success declined.
Review
gpt-6.1-solsession each round, every finding verified), fixed in6011119, each code fix with a test that fails without it:evolver.base: pifailed at the first learner call; SHE now takes Pi, as Run every method on Pi (evolver.base: pi) #92 documents for every method.setup()now refuses them before any episode. The committedrun.yamlhad no learner model.data.step_sizebelow the TRAIN size, the official selector compared different case subsets. Each round must now measure the whole TRAIN set, and the stop persists even withoutallow_early_stop.::<condition>suffix and thetask_<task>::cleanform upstream reads, so upstream's clean-failure guidance never fired and regression risks named no observed case.--group she, so the SHE tests always skipped there.run.yamlreadsfull-test-v1and enables the semantic judge.update_jsonldiscards its merge (upstream bug; the official code is kept), and learner regexes run in the Trainer process.main(Expand AgentDyn TEST and use Lightning in both EvoSafe recipes #103 squash-merged) is merged in:de2ac7e.🤖 Generated with Claude Code