8000 Add official SHE evolution on AgentDyn with Lightning by andylinx · Pull Request #104 · Infini-AI-Lab/hermit · GitHub
[go: up one dir, main page]

Skip to content

Add official SHE evolution on AgentDyn with Lightning - #104

Merged
haizhongzheng merged 4 commits into
Infini-AI-Lab:mainfrom
andylinx:feat/she-agentdyn-lightning
Nov 13, 2026
Merged

haizhongzheng merged 4 commits into
Infini-AI-Lab:mainfrom
andylinx:feat/she-agentdyn-lightning

Conversation

@andylinx
@andylinx andylinx commented Nov 12, 2026 •
Copy link
Copy Markdown
Contributor

Change

This PR adds official SHE evolution to AgentDyn. Nemotron 3.5 Lightning is the frozen actor and semantic judge, and GPT-6 Sol via Codex is the learner. SHE evolves four artifacts: the system prompt, rule bank, safety memory, and tool policy.

Each round evaluates the current adapter on TRAIN, applies the official score and initial safe-utility floor, and restores measured best before proposing the next edit. The next round evaluates that proposal. Finalization exports the measured best adapter. The reserved validation split is unused.

flowchart LR
  T[Measure current adapter on TRAIN] --> B[Update measured best]
  B --> D[Diagnose and attribute outcomes]
  D --> R[Restore measured best]
  R --> P[Propose and check artifact edits]
  P --> T
  B --> F[Export measured best]
  F --> E[Compare on frozen TEST]
Loading

The actor and semantic judge use Hermit's shared direct model client. Learner roles use the shared Base Agent (the run's llm, Codex, Claude Code or Pi) for CLI execution, authentication, structured output, cancellation, and usage. Framework changes register the optional SHE dependency, its test group and its CI install. The AgentDyn runner handles method integration and keeps learned hooks inside the existing networkless Docker boundary.

See the method and upstream adaptations, AgentDyn recipe, and results and reproduction details.

Validation

  • After the review fixes: python -m pytest tests/methods/she tests/methods/evosafe tests/core/test_selection.py tests/core/test_pi_agent.py tests/core/test_spec.py tests/core/test_packaging.py with the pinned SHE dependency: 326 passed, 51 skipped (Docker and native AgentDyn tests). pytest --changed --base origin/main before the merge of main: 1389 passed, 168 skipped; its 2 failures were Harness² tests whose dependency group was not installed.
  • The recorded run completed all 20 rounds on 57 TRAIN cases and selected round 9. Frozen TEST used 186 cases per variant, with 18 clean and 168 injected cases.

For reference only, not a reported result: the learner ran without Hermit's sandbox (--no-sandbox, the host lacked usable bubblewrap), which a reported result requires. This is the same status as the EvoSafe comparison in #103. The run also predates the review fixes belo 8000 w.

Frozen TEST metric No adapter Initial SHE Final SHE
Clean task success 13/18 (72.2%) 13/17 (76.5%) 9/16 (56.3%)
Attack success 46/154 (29.9%) 33/155 (21.3%) 3/153 (2.0%)
Safe task utility 75/154 (48.7%) 83/155 (53.5%) 92/153 (60.1%)
Execution errors (all planned cases) 14/186 14/186 17/186

All execution errors reached the 25-step limit without a terminal response. The breakdown was 0 clean and 14 injected with no adapter, 1 clean and 13 injected with initial SHE, and 2 clean and 15 injected with final SHE. Success and utility rates use completed episodes, and attack success rates exclude these execution errors. In this single run, the final adapter reduced observed attack success and improved safe task utility, while clean task success declined.

Review

  • Tests and documentation cover the affected behavior.
  • Commands, defaults, and public interfaces are consistent.
  • Shared logs and examples contain no credentials or private data.
  • Documentation-only item is inapplicable because this PR adds code and tests.
  • Independent post-PR audit verified the stacked diff and reported metrics. GPT-6 Astra reproduced the early-error issue and independently verified the fix and regression test.
  • Maintainer dual review (5 rounds: Claude Code and a fresh Codex gpt-6.1-sol session each round, every finding verified), fixed in 6011119, each code fix with a test that fails without it:
    • evolver.base: pi failed at the first learner call; SHE now takes Pi, as Run every method on Pi (evolver.base: pi) #92 documents for every method.
    • A wrong proposer, debugger, batch size or gate mode, or a missing learner model, surfaced only after a full TRAIN round; setup() now refuses them before any episode. The committed run.yaml had no learner model.
    • With data.step_size below the TRAIN size, the official selector compared different case subsets. Each round must now measure the whole TRAIN set, and the stop persists even without allow_early_stop.
    • Case IDs lacked the ::<condition> suffix and the task_<task>::clean form upstream reads, so upstream's clean-failure guidance never fired and regression risks named no observed case.
    • Under umask 077 the staged artifacts were unreadable in the hook container.
    • CI did not install --group she, so the SHE tests always skipped there.
    • Docs: SHE in the Pi tables; the method index no longer asks for a validation split; the shell-tool and launch statements are corrected; run.yaml reads full-test-v1 and enables the semantic judge.
    • Recorded, not changed: upstream update_jsonl discards its merge (upstream bug; the official code is kept), and learner regexes run in the Trainer process.
    • main (Expand AgentDyn TEST and use Lightning in both EvoSafe recipes #103 squash-merged) is merged in: de2ac7e.

🤖 Generated with Claude Code

haizhongzheng and others added 2 commits November 12, 2026 20:50
- evolver.base: pi works: call_model takes pi, as Infini-AI-Lab#92 documents for every method.
- setup() refuses a bad proposer, debugger, debugger_batch_size or gate_mode, or a
  missing learner model, before any TRAIN episode.
- Each round must measure the whole TRAIN set (is_epoch_end) before the official
  selector compares it; an error before the round's records still persists the stop flag.
- Case IDs follow upstream: `<id>::<condition>`, and a user task's clean case
  `task_<suite>-<user_task>::clean`, so upstream's clean-failure guidance and
  regression risks reach the learner.
- Hook staging makes every file readable, so SHE's hooks start under umask 077.
- CI installs --group she; run.yaml reads full-test-v1, names its llm and enables
  the semantic judge.
- Docs: SHE in the Pi tables; the method index, shell-tool and launch statements
  corrected; results marked for reference only (the learner ran without Hermit's
  sandbox), as for Infini-AI-Lab#103.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ightning

# Conflicts:
#	docs/recipes/agentdyn.md
#	examples/agentdyn/evosafe/README.md
#	examples/agentdyn/evosafe/REPRODUCTION.md
#	tests/methods/evosafe/test_agentdyn_webpages.py
@haizhongzheng haizhongzheng left a comment
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this, and for the careful write-up of the protocol and its limits. We reviewed ceb8d62 with two independent reviewers over five rounds, verified every finding against the code and the pinned SHE source, and pushed the fixes to your branch as 6011119, then merged main (#103) as de2ac7e.

What we changed (each code fix has a test that fails without it)

  • Pi. evolver.base: pi failed at the first learner call; #92 documents every method on Pi, so SHE now takes it.
  • Settings checked at setup. A wrong proposer, debugger, batch size or gate mode, or a missing learner model, was found only after a full TRAIN round. setup() now refuses them before any episode. The committed run.yaml had no learner model.
  • Whole TRAIN each round. With data.step_size below the TRAIN size, the official selector compared different case subsets. A round that does not end the epoch now stops before selection, and the stop holds even without allow_early_stop.
  • Case IDs as upstream reads them. Without ::<condition>, and the task_<task>::clean form for a user task's clean case, upstream's clean-failure guidance never fired, and regression risks named no observed case.
  • Hook staging. Under umask 077, the staged artifacts were unreadable in the hook container.
  • CI. CI did not install --group she, so the SHE tests always skipped there.
  • Docs and recipes. SHE is in the Pi tables; the method index no longer asks for a validation split; the shell-tool and launch statements are corrected; run.yaml reads full-test-v1 and enables the semantic judge.
  • Results are for reference only. The learner ran with --no-sandbox, so, as for #103's EvoSafe comparison, the numbers are marked as not a reported result.

Noted, not changed

  • Upstream update_jsonl discards its _deep_merge result, so such an edit is lost. Upstream's prompt does not offer this mode, and we keep the official code.
  • The learner's regexes in upstream's patcher run in the Trainer process.

Tests. With the pinned SHE dependency: 326 passed, 51 skipped (Docker and native AgentDyn tests). CI passed on de2ac7e.

🤖 Generated with Claude Code

@haizhongzheng
haizhongzheng marked this pull request as ready for review November 13, 2026 02:07
@haizhongzheng
haizhongzheng merged commit e813538 into Infini-AI-Lab:main Nov 13, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

0