8000 Highlight coding-plan evolution and simplify README by haizhongzheng · Pull Request #80 · Infini-AI-Lab/hermit · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 16 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
[![WeChat](https://img.shields.io/badge/WeChat-community-07C160?logo=wechat&logoColor=white)](assets/community/wechat.md)
[![Discord](https://img.shields.io/badge/Discord-community-5865F2?logo=discord&logoColor=white)](https://discord.gg/CKjBjgVQDV)

**[Get Started](#get-started)** · **[Harness Adapters](#what-is-a-harness-adapter)** · **[Methods](#supported-harness-evolution-methods)** · **[Examples and Recipes](#examples-and-recipes)** · **[FAQ](#faq)** · **[Docs](https://infini-ai-lab.github.io/hermit/docs/)**
**[Get Started](#get-started)** · **[Harness Adapters](#what-is-a-harness-adapter)** · **[Methods](#supported-harness-evolution-methods-more-to-come)** · **[Examples and Recipes](#examples-and-recipes)** · **[FAQ](#faq)** · **[Docs](https://infini-ai-lab.github.io/hermit/docs/)**

</div>

Expand All @@ -25,6 +25,7 @@ Hermit is a framework for training, evaluating, and deploying harness adapters,

## News

- **[2026/11]** **Harness evolving with your coding plan. No API key needed.** Check out the [tutorial](docs/tutorials/codex-plan.md).
- **[2026/11]** **Hermit v0.2.0** released — three-layer agents, Codex plan-backed acting and learning, and six new harness evolution methods. See the [release notes](https://github.com/Infini-AI-Lab/hermit/releases/tag/v0.2.0).
- **[2026/10]** New harness evolution supported: **[Meta-Harness Adapter](https://infini-ai-lab.github.io/hermit/docs/methods/meta_harness_adapter.html)**, **[ReHarness](https://infini-ai-lab.github.io/hermit/docs/methods/reharness.html)**, and **[Harness²](https://infini-ai-lab.github.io/hermit/docs/methods/harness2.html)**.
- **[2026/10]** **Hermit v0.1.0** released — the first public release for training, evaluating, and deploying harness adapters. See the [release post](https://x.com/haizhong_zheng/status/2114768121705775463).
Expand All @@ -43,22 +44,19 @@ Many harnesses are black boxes; even open-source ones have carefully designed co

## A Unified Framework for Harness Evolution

### Supported Harness Evolution Methods
### Supported Harness Evolution Methods (More to Come)

<table align="center">
<thead>
<tr>
<th>Status</th>
<th>Methods</th>
</tr>
</thead>
<tbody>
<tr>
<td>✅ <strong>Supported</strong></td>
<td><a href="https://infini-ai-lab.github.io/hermit/docs/methods/gepa.html">GEPA</a> · <a href="https://infini-ai-lab.github.io/hermit/docs/methods/skillopt.html">SkillOpt</a> · <a href="https://infini-ai-lab.github.io/hermit/docs/methods/mwm.html">MetaWorld Model</a> · <a href="https://infini-ai-lab.github.io/hermit/docs/methods/harness2.html">Harness²</a> · <a href="https://infini-ai-lab.github.io/hermit/docs/methods/continual_harness.html">Continual Harness</a> · <a href="https://infini-ai-lab.github.io/hermit/docs/methods/reharness.html">ReHarness</a> · <a href="https://infini-ai-lab.github.io/hermit/docs/methods/meta_harness_adapter.html">Meta-Harness Adapter</a> · <a href="https://infini-ai-lab.github.io/hermit/docs/methods/ace.html">ACE</a> · <a href="https://infini-ai-lab.github.io/hermit/docs/methods/ahe_adapter.html">AHE Adapter</a> · <a href="https://infini-ai-lab.github.io/hermit/docs/methods/rrsi_adapter.html">RRSI Adapter</a> · <a href="https://infini-ai-lab.github.io/hermit/docs/methods/evosafe.html">EvoSafeHarness</a></td>
</tr>
</tbody>
</table>
- [GEPA](https://infini-ai-lab.github.io/hermit/docs/methods/gepa.html)
- [SkillOpt](https://infini-ai-lab.github.io/hermit/docs/methods/skillopt.html)
- [MetaWorld Model](https://infini-ai-lab.github.io/hermit/docs/methods/mwm.html)
- [Harness²](https://infini-ai-lab.github.io/hermit/docs/methods/harness2.html)
- [Continual Harness](https://infini-ai-lab.github.io/hermit/docs/methods/continual_harness.html)
- [ReHarness](https://infini-ai-lab.github.io/hermit/docs/methods/reharness.html)
- [Meta-Harness Adapter](https://infini-ai-lab.github.io/hermit/docs/methods/meta_harness_adapter.html)
- [ACE](https://infini-ai-lab.github.io/hermit/docs/methods/ace.html)
- [AHE Adapter](https://infini-ai-lab.github.io/hermit/docs/methods/ahe_adapter.html)
- [RRSI Adapter](https://infini-ai-lab.github.io/hermit/docs/methods/rrsi_adapter.html)
- [EvoSafeHarness](https://infini-ai-lab.github.io/hermit/docs/methods/evosafe.html)

Hermit provides an RL-style training loop for evolving harness adapters:

Expand All @@ -77,12 +75,6 @@ Hermit provides an RL-style training loop for evolving harness adapters:
<img src="assets/training-loop.gif" alt="Training pseudocode: initialize the adapter, run and judge tasks, call evolver.step, check and commit the proposal, then finalize." width="100%">
</p>

#### Run from the Command Line:

<p align="center">
<img src="assets/install-and-use.gif" alt="An animated command overview of installing Hermit, training an adapter, deploying it, and converting it into a plugin." width="100%">
</p>

## Get Started

### Install
Expand Down Expand Up @@ -163,6 +155,8 @@ The generated `AGENTS.md` contains the learned rule. Start Codex in that workspa

[Full walkthrough, expected outputs, and repeat-run instructions →](https://infini-ai-lab.github.io/hermit/docs/get-started/quickstart.html)

**Run harness evolution with your coding plan.** Use your Codex/ChatGPT subscription for both acting and learning, with no separate model API key required. Plan-backed recipes are available for GEPA, SkillOpt, Harness², Continual Harness, ACE, AHE Adapter, and RRSI Adapter. See the [setup guide and recipes](https://infini-ai-lab.github.io/hermit/docs/recipes/officeqa.html#a-model-on-a-chatgpt-login) to get started; model access and usage follow your plan.

**Ready for a real agent?** [Train with an API model](https://infini-ai-lab.github.io/hermit/docs/get-started/first-model.html) · [Run OfficeQA](https://infini-ai-lab.github.io/hermit/docs/recipes/officeqa.html) · [Bring your own task](https://infini-ai-lab.github.io/hermit/docs/tutorials/your-own-task.html)

### Examples and Recipes
Expand Down
2 changes: 2 additions & 0 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ These docs describe the **0.2.0** code in this checkout.
|---|---|---|
| Run a complete training cycle without model access | [Quick Start](get-started/quickstart.md) | A changed adapter, held-out scores, and a host workspace |
| Run with your model API | [First real-model training](get-started/first-model.md) | Acting calls, learner edits, and held-out evaluation on included data |
| Run with your coding plan | [Harness evolution with your coding plan](tutorials/codex-plan.md) | A small benchmark run with Codex acting and learning |
| Reproduce a benchmark experiment | [Recipe catalog](recipes/index.md) | A run using the recipe's data and model setup |
| Choose a learning method | [Choose a method](methods/index.md) | A suitable template, Evolver, and split plan |
| Improve your own agent | [Bring your task](tutorials/your-own-task.md) | A pipeline, judge, task description, and run config |
Expand All @@ -36,6 +37,7 @@ Passing file checks establishes structural validity. Performance improvement req
get-started/installation
get-started/quickstart
get-started/first-model
tutorials/codex-plan
framework/what-is-a-harness-adapter
faq
```
Expand Down
191 changes: 191 additions & 0 deletions docs/tutorials/codex-plan.md 10000
8BD0
Original file line number Diff line number Diff line change
@@ -0,0 +1,191 @@
# Run harness evolution with your coding plan

Use your Codex / ChatGPT plan to run both the acting agent and the learner in Hermit. You can try methods such as
GEPA, SkillOpt, ACE and RRSI Adapter without configuring a separate model API key or API billing account. Hermit
runs the method's evaluation and selection loop; Codex supplies the model sessions that act and learn.

This tutorial uses the existing **RRSI Adapter smoke run on OfficeQA**: four training questions, two trials per
question, one round of two candidate adapters, then four held-out questions. It checks the integration rather than
establishing a performance gain. It still needs the benchmark data and execution environment described below.

## Start with a coding-agent prompt

Open the Hermit checkout in your coding agent and give it this prompt. Replace the model placeholder with a model
available to your Codex login.

```text
Help me run a small harness-evolution experiment using my Codex / ChatGPT plan.
Use <my Codex model name> for both acting and learning, without a model API key.

Read AGENTS.md, docs/tutorials/codex-plan.md and the RRSI Adapter example.
Check the prerequisites, then use examples/officeqa/rrsi_adapter/smoke_luna.yaml.
Keep its one-round protocol. Set acting concurrency to 2 and the training time
budget to 2 hours. Use the same model settings for check, train and evaluation.

Use the existing data-preparation scripts, container and egress-proxy setup.
If the OfficeQA dataset or another prerequisite is missing, tell me exactly what
I need to provide. Do not substitute synthetic data for the benchmark.

Run the checks, train, and evaluate both the seed and final adapter. Inspect
whether the round completed, which candidate was selected, and any errors.
Report the commands, output paths and results. Keep generated artifacts local;
do not commit or push anything.
```

The following sections explain the setup and commands behind that prompt.

## 1. Prepare Codex and the benchmark

Use a **Linux machine with Python 3.12+, Git, bubblewrap and Docker** for this example. RRSI's proposer and search
roles use Hermit's Linux sandbox; acting sessions run in Docker. Complete [Installation](../get-started/installation.md)
and install the two dependency groups from the repository root:

```bash
python -m pip install --upgrade "pip>=25.1"
python -m pip install -e . --group rrsi --group skillopt
```

`rrsi` supplies the search method; `skillopt` supplies OfficeQA's shared data parsing and task utilities.

Install [Codex CLI](https://learn.chatgpt.com/docs/codex/cli), then sign in with ChatGPT on the machine that will run
Hermit. Hermit currently needs Codex's file-based login cache:

```bash
codex -c 'cli_auth_credentials_store="file"' login
codex login status
```

Confirm that the authentication method is ChatGPT. Hermit reads `auth.json` under `CODEX_HOME`, or `~/.codex` when
it is unset; a login held only in the OS credential store is not enough. Hermit manages per-session copies and
refreshes through its shared login directory. See [OpenAI's authentication guide](https://learn.chatgpt.com/docs/auth)
for sign-in and credential-storage options.

Complete these existing OfficeQA setup steps once:

- [Prepare the data](../recipes/officeqa.md#prepare-the-data): obtain the gated dataset, then run `prepare.py` with
the CSV and parsed corpus paths. This writes the task splits and initial skill used below. Dataset access is
separate from model authentication.
- [Run in the Codex harness](../recipes/officeqa.md#run-in-the-codex-harness) and
[A model on a ChatGPT login](../recipes/officeqa.md#a-model-on-a-chatgpt-login): build the Linux Codex image,
create the internal Docker network, and start the login egress proxy. The example names
`officeqa-codex:0.164.0`, `oq-sandbox` and `oq-egress-login`. The image needs a Linux Codex distribution matching
Docker's architecture, even if you normally use Codex on another operating system.

Then prepare RRSI's seed and trial rows:

```bash
python examples/officeqa/rrsi_adapter/make_seed.py
python examples/officeqa/rrsi_adapter/make_rows.py
```

## 2. Check, train and evaluate

Run from the repository root. Choose a model your login can actually use; the checked-in recipe's
`gpt-5.6-luna` is the model used for its reported experiment, not a requirement to use your plan.

```bash
HERMIT_CODEX_MODEL='your-codex-model-name'

hermit check --config examples/officeqa/rrsi_adapter/smoke_luna.yaml --no-probe \
agent.model.name="$HERMIT_CODEX_MODEL" evolver.model.name="$HERMIT_CODEX_MODEL" \
agent.concurrency=2 trainer.budget.hours=2

hermit train --config examples/officeqa/rrsi_adapter/smoke_luna.yaml \
agent.model.name="$HERMIT_CODEX_MODEL" evolver.model.name="$HERMIT_CODEX_MODEL" \
agent.concurrency=2 trainer.budget.hours=2

hermit eval --config examples/officeqa/rrsi_adapter/smoke_luna.yaml --adapter seed_adapter \
agent.model.name="$HERMIT_CODEX_MODEL" evolver.model.name="$HERMIT_CODEX_MODEL" \
agent.concurrency=2

hermit eval --config examples/officeqa/rrsi_adapter/smoke_luna.yaml --adapter final \
agent.model.name="$HERMIT_CODEX_MODEL" evolver.model.name="$HERMIT_CODEX_MODEL" \
agent.concurrency=2
```

`check --no-probe` checks prerequisites without a task rollout; it does not prove model access or learner execution.
Training normally evaluates H_0 and two candidates: **24 acting rollouts**, plus candidate liveness smokes and the
proposer, analyst, digester and critic sessions. Each evaluation command adds four acting rollouts. A small task
split can still consume substantial plan usage. The two-hour training budget can end the run before its round
finishes; probes and separate evaluations are outside that training budget.

Outputs are under `examples/officeqa/rrsi_adapter/runs/smoke-luna-rrsi/`. Inspect:

- `checkout/ADAPTER.md` and any additional learned files;
- `step_NNNN/output/report.md` for candidate screening, round decisions and the selected incumbent;
- `workspace/state/rrsi/frontier.json` for the incumbent and round trajectory;
- `eval/seed_adapter/summary.json` and `eval/final/summary.json` for held-out scores and execution errors.

Check that the round completed before treating the output as a completed RRSI smoke. `final` is the incumbent H_T;
an interrupted run can return its last incumbent. A run that completes with no accepted candidate is also valid.
Keep model and task settings fixed when comparing seed and final scores.

For a separate experiment, change both `trainer.run_name` and `trainer.tree_dir`; apply both overrides consistently
to its training and evaluation commands. An unfinished run resumes automatically. Keep generated records and adapter snapshots local,
as described in [Contributing](https://github.com/Infini-AI-Lab/hermit/blob/main/CONTRIBUTING.md).

## How this works for other methods

There are two learner configurations, depending on how the method calls its model:

| Method's learner | Configuration | Examples |
|---|---|---|
| Requests text replies for reflection or revision | `llm.cli: codex` with `login: chatgpt`; each request becomes a Codex session in its own container | GEPA, SkillOpt, ACE, Continual Harness |
| Edits files as an agent | `evolver.base: codex` and `evolver.model.login: chatgpt`; sessions run in the method's workspace | Harness², Meta-Harness Adapter, AHE Adapter, RRSI Adapter |

For the first form, the model settings look like this; use the method's existing recipe for its remaining fields:

```yaml
llm:
cli: codex
model: your-codex-model-name
login: chatgpt
effort: medium
web_search: false
isolation: container
container:
image: officeqa-codex:0.164.0
network: oq-sandbox
proxy: http://oq-egress-login:3128
timeout: 3600
```

For the second form, keep the method's `evolver.name` and parameters, and configure its Base Agent and model:

```yaml
evolver:
name: rrsi_adapter
# Keep the recipe's method-specific params here.
base: codex
model: {name: your-codex-model-name, login: chatgpt, effort: medium}
```

The acting agent also needs `agent.base: codex` and `agent.model.login: chatgpt`, as in the smoke configuration.
Changing only the learner does not change an API-based actor. Use a complete recipe for the task's preparation,
scoring, container and sandbox settings. For example, [ACE](../methods/ace.md) already provides
`examples/officeqa/ace/smoke_codex_luna.yaml`; GEPA and SkillOpt have `run_codex_luna.yaml` recipes.

Plan-backed sessions use the account's available models and usage limits. Hermit reports unknown dollar cost when
there is no per-token price; use time/token budgets and inspect the shared plan status described in
[the login setup](../recipes/officeqa.md#a-model-on-a-chatgpt-login). Extra services still need their own setup:
ReHarness's embedding model, for example, is not provided by this Codex text-model route.

## What about a Claude Code plan?

Claude Code also supports subscription sign-in, including Pro and Max; see
[Claude's subscription setup](https://support.claude.com/en/articles/11145838-use-claude-code-with-your-pro-max-or-ultra-plan).
Hermit already has a Claude Code Base Agent, but its current coverage differs from the Codex walkthrough above:

| Path | Current support |
|---|---|
| Acting agent | `agent.base: claude` uses Claude Code with the task's preparation and scoring functions. The current container runner is Codex-only, so it needs its own execution setup. |
| Agent-style learner | The built-in `claude` Evolver and Meta-Harness Adapter support Claude Code sessions. RRSI Adapter supports a Claude proposer, but its analyst, digesters and critic still require Codex. AHE Adapter's learners are Codex-only. |
| Text-reply learner (`llm.cli`) | Only `codex` is accepted today. Switching GEPA, SkillOpt or ACE to `llm.cli: claude` requires shared-layer support first. |

Hermit's Claude sign-in path copies `~/.claude/.credentials.json` into the session's home when no API binding is
configured; it does not read a login stored only in macOS Keychain. The Codex-specific `login: chatgpt`, container
and plan-management settings above do not configure Claude Code. A complete Claude-plan recipe has not been
validated in this tutorial.

Continue with [Choose a method](../methods/index.md), the [OfficeQA recipe](../recipes/officeqa.md), or
[Bring your task](your-own-task.md) to use your own benchmark.
Loading
0