ruiling-smartbear · Nov 6, 2026 · Nov 6, 2026 · Nov 6, 2026 · Nov 6, 2026 · Nov 6, 2026
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -20,6 +20,9 @@ Changes are grouped by software version. Entries under Unreleased describe work
   for now, as every Docker-driving agent does.
 - A test workflow (`.github/workflows/tests.yml`): a pull request runs the test groups it touches
   (`pytest --changed`); main, a daily schedule and a manual run run the whole suite.
+- The OPRO method (Yang et al., ICLR 2024, arXiv:2309.03409): the `opro` Evolver on the `prompt-only` template, one
+  OPRO step per training step, with the authors' meta-prompt verbatim; the optimizer is the run's `llm`
+  (`docs/methods/opro.md`). OPRO on OfficeQA: `examples/officeqa/opro/`, in the GPT-5.6-Luna setting.
 
 ### Changed
 

diff --git a/README-pypi.md b/README-pypi.md
@@ -13,8 +13,8 @@ hermit --help
 ```
 
 The package holds the framework, the adapter templates and the built-in learning methods (prompt-only, GEPA,
-SkillOpt, MetaWorld Model, Harness², Continual Harness, ReHarness, Meta-Harness Adapter, ACE, AHE Adapter, RRSI Adapter
-and EvoSafeHarness). GEPA, SkillOpt, Harness² and RRSI Adapter call
+SkillOpt, MetaWorld Model, Harness², Continual Harness, ReHarness, Meta-Harness Adapter, ACE, AHE Adapter, RRSI Adapter,
+EvoSafeHarness and OPRO). GEPA, SkillOpt, Harness² and RRSI Adapter call
 their authors' packages, pinned to Git commits in the repository's `pyproject.toml` dependency groups (`gepa`,
 `skillopt`, `harness2` and `rrsi`). These groups are not installed by `pip install hermit-harness`. From a source checkout,
 use pip 25.1+ and run `python -m pip install -e . --group gepa` for GEPA, or

diff --git a/README.md b/README.md
@@ -57,6 +57,7 @@ Many harnesses are black boxes; even open-source ones have carefully designed co
 - [AHE Adapter](https://infini-ai-lab.github.io/hermit/docs/methods/ahe_adapter.html)
 - [RRSI Adapter](https://infini-ai-lab.github.io/hermit/docs/methods/rrsi_adapter.html)
 - [EvoSafeHarness](https://infini-ai-lab.github.io/hermit/docs/methods/evosafe.html)
+- [OPRO](https://infini-ai-lab.github.io/hermit/docs/methods/opro.html)
 
 Hermit provides an RL-style training loop for evolving harness adapters:
 
@@ -214,6 +215,10 @@ Train and evaluate harness adapters on benchmarks using different evolution meth
 <td><a href="examples/officeqa/rrsi_adapter/">OfficeQA</a> (whole adapters evolved round by round under regularizers against overfitting, Codex + GPT-5.6-Luna acting and learning)</td>
 </tr>
 <tr>
+<td><a href="https://infini-ai-lab.github.io/hermit/docs/methods/opro.html">OPRO</a></td>
+<td><a href="examples/officeqa/opro/">OfficeQA</a> (one instruction searched by a model that reads every earlier instruction's training score, Codex + GPT-5.6-Luna acting and optimizing)</td>
+</tr>
+<tr>
 <td><a href="https://infini-ai-lab.github.io/hermit/docs/methods/evosafe.html">EvoSafeHarness</a></td>
 <td><a href="examples/agentdyn/evosafe/">AgentDyn</a> (a defense bundle of a system prompt and tool-call hooks searched around a frozen victim, on fixed benign and injected cases)</td>
 </tr>

diff --git a/docs/faq.md b/docs/faq.md
@@ -70,7 +70,7 @@ Compatibility does not guarantee the same performance on a different model, harn
 <details class="ha-faq" id="which-harness-evolution-methods-can-i-use">
 <summary>Which Harness Evolution Methods Can I Use?</summary>
 
-Hermit includes GEPA, SkillOpt, MetaWorld Model (MWM), Harness², Continual Harness, ReHarness, Meta-Harness Adapter, ACE, and RRSI Adapter, along with Codex/Claude coding-agent learners and a prompt-only baseline. They share the Trainer interface, but have different data requirements and compatible adapter templates.
+Hermit includes GEPA, SkillOpt, MetaWorld Model (MWM), Harness², Continual Harness, ReHarness, Meta-Harness Adapter, ACE, RRSI Adapter, and OPRO, along with Codex/Claude coding-agent learners and a prompt-only baseline. They share the Trainer interface, but have different data requirements and compatible adapter templates.
 
 Use the [method selection guide](methods/index.md) to choose a method and the [recipe catalog](recipes/index.md) for runnable benchmark setups. Methods labeled **Coming Soon** in the README are planned integrations.
 

diff --git a/docs/framework/agent-layers.md b/docs/framework/agent-layers.md
@@ -202,7 +202,7 @@ accepted; a key given in two layers is refused. On the command line a layer's se
 |---|---|---|
 | OfficeQA in Codex, every method | Acting Agent with OfficeQA's task functions (`utils/task.py`) | `codex` |
 | OfficeQA in direct chat | Acting Agent running SkillOpt's chat loop | `direct`, per turn |
-| GEPA, SkillOpt, ReHarness, Continual Harness | Evolver Agent | `direct` (`ctx.agent`) |
+| GEPA, SkillOpt, ReHarness, Continual Harness, OPRO | Evolver Agent | `direct` (`ctx.agent`); OPRO in OfficeQA: `codex` through `llm: {cli: codex}` |
 | Meta-Harness Adapter, `codex` and `claude` Evolvers | Evolver Agent | `codex` or `claude` |
 | RRSI Adapter | Evolver Agent: upstream's rules and roles, the proposer's prompt | `codex` or `claude` for the proposer; `codex`, one call per upstream `generate`, for the analyst, digesters and critic |
 | CL-Bench, ServeLearnBench | Acting Agent running the benchmark's loop | `direct`, per call, through the local endpoint |

diff --git a/docs/framework/evolver.md b/docs/framework/evolver.md
@@ -134,6 +134,7 @@ For selection guidance, setup, and the complete parameter defaults, use [Choose
 | `gepa` | [GEPA](https://github.com/gepa-ai/gepa): one iteration per step. It reflects on the step's trajectories, keeps a child that does better on the same tasks, scores it on validation, and hands back the next parent from its Pareto front. | `llm`, `data.val`, `python -m pip install -e . --group gepa` |
 | `skillopt` | [SkillOpt](https://github.com/microsoft/SkillOpt): one SkillOpt step per training step. It turns the step's trajectories into a few bounded edits under its learning rate (decaying when `total_steps` is set) and keeps them only if the validation score improves; the slow update and meta skill run at each epoch's end. | `llm`, `data.val`, `python -m pip install -e . --group skillopt` |
 | `ace` | [ACE](https://github.com/ace-agent/ace): per rollout, a Reflector tags the playbook bullets the agent named and diagnoses the attempt (a wrong answer is run again with the reflection, up to `max_num_rounds`), and a Curator adds bullets; the playbook grows by deltas. See `docs/methods/ace.md`. | `llm`; `data.val` to keep the best validation playbook |
+| `opro` | [OPRO](../methods/opro.md), with `template: prompt-only@1.0`: one OPRO step per training step. The optimizer reads the instructions scored so far, ascending, and a few training problems, and writes new instructions, each scored on every training task; finalize hands back the best on training or the best validation mean of the top `top_k`. | `llm`, `data.step_size` the training split's size; `data.val` for `select: validation` |
 | `ahe_adapter` | [AHE Adapter](../methods/ahe_adapter.md), with `template: ahe@1.0`: one AHE iteration per step. It classifies each task by its k rollouts, scores the previous step's change manifest and rolls HARMFUL changes back, then runs an Agent Debugger session per task and an Evolve Agent session that edits the adapter; finalize hands back the best-ever iteration. | The Codex CLI, k rows per task sharing `task_field`, an acting agent that runs the adapter's hooks (Codex, Claude Code) |
 | `mwm` | MetaWorld Model, with `template: mwm`: one native MWM learner session per block (the step), under the MWM contract, with the task's evidence store mounted at `/evidence`, in its own network namespace. See `docs/methods/mwm.md`. | The Codex CLI, a task that keeps an MWM evidence store (`evolver_access.environment`), `task-info/mwm/`, `data.shuffle: false` |
 | `reharness` | [ReHarness](../methods/reharness.md), with `template: reharness`: one instance of the stream per step. It revises each reflection the acting agent retrieved, from that reflection's own source trajectory with the new trajectory as feedback, then writes the instance's reflection with the revised ones as demonstrations, through the run's `llm` tool with the paper's prompts; no validation data. | `llm`, `data.step_size: 1` and `data.shuffle: false`, an embedding endpoint (`evolver.params.embedding`, and `REHARNESS_EMBED_URL` for the acting side's reader; BM25 without) |

diff --git a/docs/index.md b/docs/index.md
@@ -76,6 +76,7 @@ methods/ace
 methods/ahe_adapter
 methods/rrsi_adapter
 methods/evosafe
+methods/opro
 ```
 
 ```{toctree}

diff --git a/docs/methods/index.md b/docs/methods/index.md
@@ -15,6 +15,7 @@ template–method combinations meaningful.
 | Grow an itemised playbook by incremental deltas from reflections, keeping the best validation playbook | [ACE](ace.md) | `ace@1.0` | Learner model endpoint; optional validation split |
 | Evolve an agent's harness (prompt, middleware, scripts, skills, sub-agents, memory) from k rollouts per task, with each change attributed and reversible | [AHE Adapter](ahe_adapter.md) | `ahe@1.0` | An acting agent that runs hooks (Codex, Claude Code), Codex as the learner, k rows per task |
 | Search a dual-surface defense (prompt + tool hooks) for a frozen victim against frozen attack cases | [EvoSafeHarness](evosafe.md) | `evosafe@1.0` | Hook-bridge pipeline, utility/attack judge (verdicts in `JudgeResult.extra`), optional validation split, Claude Code or Codex |
+| Search instructions with a model that reads every earlier instruction's training score | [OPRO](opro.md) | `prompt-only@1.0` | Learner model endpoint; validation split for `select: validation` |
 | Build a persistent world model from an ordered environment stream | [MWM](mwm.md) | `mwm` | Evidence store, deployment description, tools, Linux learner environment |
 | Learn a bank of reflections online, retrieved per instance of an ordered stream | [ReHarness](reharness.md) | `reharness` | The run's `llm`, an embedding endpoint (or BM25), an agent that reads the adapter through `Session.context(query, task)` |
 | Search whole adapters with a coding-agent proposer that reads every earlier candidate's files, scores and trajectories | [Meta-Harness Adapter](meta_harness_adapter.md) | `meta-harness-adapter` | A training split (the search set), a proposer CLI (Codex or Claude Code), an acting agent that delivers the adapter |
@@ -37,6 +38,7 @@ task-specific service overhead. Call counts vary with evidence, candidates, and
 | ACE | Per sample: one Reflector call, or for a wrong answer up to `max_num_rounds` Reflector calls and agent re-runs, one Curator call; validation at each epoch's end when `data.val` is set | `state/ace.json` (sample records, validation scores) and the best validation playbook | Best validation playbook, else the current one |
 | AHE Adapter | Per step: one Agent Debugger session per task (up to three attempts) and one Evolve Agent session; no candidate rollout, no validation | `state/`: stats, Debugger reports, manifests, verdicts, rollbacks, best-ever | Best-ever iteration by training pass rate, or the last |
 | EvoSafeHarness | Per trial: Designer, Criticizer and Analyzer calls, plus candidate runs on the training cases (and the parent's validation cases when a candidate qualifies); the parent reuses the step's rollouts when they match | Lessons archive and trial history | Last promoted bundle |
+| OPRO | `n_candidates` optimizer calls per step; each new instruction run on every training task; with `select: validation`, the `top_k` best run on validation at the end | `state/opro.json` (every instruction with its per-task training scores) | Best training score, or the best validation mean of the top `top_k` |
 | MWM | Maintenance and a learner session per eligible block, plus native bootstrap/retry behavior | World-model adapter plus method/session records | Current world-model adapter |
 | ReHarness | Per instance, one reflection call, one revision call per retrieved reflection (up to k) and one embedding call; no candidate rollouts or validation | Reflection bank in the adapter; source trajectories and per-instance retrieval and revision records in `state/` | Current adapter: the bank after the last instance |
 | Meta-Harness Adapter | One proposer session per round writing k candidates; each candidate is a full rollout of the search set, one step per candidate | Every candidate's files, hypothesis, base and per-task scores in `state/` | Best mean on the search set |

diff --git a/docs/methods/opro.md b/docs/methods/opro.md
@@ -0,0 +1,90 @@
+# OPRO
+
+OPRO, Optimization by PROmpting ([Yang et al., ICLR 2024](https://arxiv.org/abs/2309.03409),
+[code](https://github.com/google-deepmind/opro)), uses a model as the optimizer: it reads earlier instructions with
+their training scores and a few training problems, and writes new instructions; every instruction it writes is
+scored on the whole training set and joins the history. Choose it when the adapter is one instruction document,
+the training split is small enough to run once per candidate, and you want the search to see every score so far.
+
+## Setup
+
+```yaml
+template: prompt-only@1.0
+evolver:
+  name: opro
+  params: {n_candidates: 4, select: validation, top_k: 3}
+data:
+  step_size: 50          # the training split's size: one OPRO step per framework step
+  val: splits/val.txt    # for select: validation
+trainer: {epochs: 3}     # OPRO steps
+llm: {model: <backend>/<model>}   # the optimizer
+```
+
+The instruction is `ADAPTER.md`. The optimizer is the run's model (`llm`); on a model reachable only through a
+ChatGPT login, OfficeQA's GPT-5.6-Luna setting makes each call one Codex session (`llm: {cli: codex}`).
+
+## One framework step
+
+The Trainer's rollouts run the current instruction on every training task. Then (`run_evolution`, one `i_step`):
+
+1. at the first step the rollouts are the initial instruction's score (step −1 in the history); later they re-score
+   the current best and are recorded, not merged;
+2. `num_exemplars` training problems are drawn at random for this step, from the runs that did not fail;
+3. the meta-prompt: the best `max_num_instructions` instructions above `score_threshold`, in ascending order with
+   scores 0–100, then the problems, then OPRO's request for a better instruction between `<INS>` and `</INS>`;
+4. `n_candidates` optimizer calls on that one meta-prompt; each reply's instruction is extracted, polished and
+   compared by md5 with every instruction generated before (as upstream, not with the initial one);
+5. each new instruction passes the template check, runs on every training task (`run_agent`), and joins the history
+   with its mean score. A wrong answer scores 0; a run that fails for an infrastructure reason (a container or
+   proxy error) is run again, and an instruction whose runs fail twice is left unscored rather than scored 0;
+6. the instruction with the best training score so far becomes the adapter.
+
+```mermaid
+flowchart TD
+  R[the step's rollouts: the current instruction<br/>on every training task] --> H[history: every instruction<br/>with its training score]
+  H --> M[meta-prompt: the best old instructions, ascending,<br/>then num_exemplars training problems]
+  M --> L[n_candidates calls to the run's llm:<br/>one instruction between INS tags each]
+  L --> D{new, non-empty,<br/>passes the template check?}
+  D -- no --> S[skipped, recorded]
+  D -- yes --> A[run_agent on every training task:<br/>its mean joins the history]
+  A --> B[the best on training becomes the adapter]
+  B --> F[finalize: the best on training, or the best<br/>validation mean of the top_k on training]
+```
+
+`finalize` hands back the best on training (`select: train`, as the paper reports), or, with `select: validation`, the
+best validation mean among the `top_k` best on training. Validation tasks never reach the meta-prompt.
+`state/opro.json` holds the history, each candidate's per-task scores and one record per step; each step's
+`output/` holds the meta-prompt, the replies and `report.md`.
+
+## Parameters
+
+```{literalinclude} ../../hermit/agent/evolver/methods/opro/evolver.yaml
+:language: yaml
+```
+
+## The authors' implementation against this one
+
+The authors' repository has no build configuration, so `meta_prompt.py` ports the meta-prompt, the reply parsing and
+the de-duplication function by function (with a NOTICE), and `evolver.py` follows `run_evolution`. The meta-prompt
+is the one of the authors' GPT optimizers with the instruction at the beginning of the question and no "Q:"/"A:",
+the setting of their README's command; a test checks it against a stored output of their function.
+
+| Dimension | google-deepmind/opro | This Evolver | Kind |
+|---|---|---|---|
+| When learning happens | Each step: 8 new instructions, each scored on the whole training subset | The same; one OPRO step per framework step, the step's rollouts on every training task (the first step refuses a smaller `step_size`). After the first step, those rollouts re-score the current best instruction: one more training pass per step, recorded, not merged into its score | Framework: the Trainer rolls out before each step |
+| What the optimizer sees | Up to 20 instructions with scores, ascending; 3 random training problems with their gold answers, drawn with `np.random.seed(i_step)` from the whole training set | The same; the answer shown is the agent's answer when judged right, else the judge's feedback (OfficeQA's names the expected answer; a run with no answer shows its fail reason instead). The problems are drawn with `Random(seed × 100003 + i_step)` from the step's runs that did not fail | Task: the task has no gold field |
+| What the acting agent sees | The instruction before the question in a one-shot prompt | `ADAPTER.md` as the agent's instructions, then the task | Framework |
+| The instruction | One sentence, skipped past 500 characters; `polish_sentence` on each | One `ADAPTER.md` (OfficeQA's seed: 886 characters); `max_instruction_chars` off by default. `polish_sentence` is the authors' and runs on the whole document, so it removes Markdown's `**` bold markers | Decision: the row's starting skill |
+| Score threshold | 0.0 for text-bison scorers, 0.3 for GPT scorers | `score_threshold: 0.0` | Decision: the scorer is an agent |
+| Final selection | None in the code (it scores instructions on the eval split every 3 steps); the paper reports the best training instructions | `select: train`, or the best validation mean of the top `top_k` on training | Decision |
+| Failures | Optimizer and scorer API errors retried until they succeed; an empty instruction is scored | A failed optimizer call tried once more, then dropped (a step can make fewer than `n_candidates`); a failed run tried once more, then the instruction is left unscored (the initial instruction's failed rollouts are run again); an empty instruction or one failing the template check is skipped and not generated again; at the hard budget, the instructions scored so far are kept, and `select: validation` falls back to the best on training. A candidate's runs that were run again are only in `tools.jsonl`; the initial instruction's are in the step's record. A run without `llm` or with an unknown `select` stops in `setup`, before its first rollouts | Framework |
+| Models | GPT-3.5/GPT-4 (512 tokens) or PaLM 2 (1024) optimizer at temperature 1.0; text-bison or GPT scorer at temperature 0 | The run's `llm` as the optimizer (temperature 1.0 when the endpoint applies it; a Codex session does not); the task's acting agent as the scorer | Framework |
+| Budget | 200 steps; GSM8K's 261 training questions | Config; OfficeQA: 3 steps of 4, 50 training questions | Decision: cost |
+| Not ported | The PaLM meta-prompt, `instructions_only`, other exemplar criteria, the temperature schedule, few-shot evaluation of new instructions, the periodic eval-split scores, the GSM8K digit filter | — | Off by default or task-specific |
+
+## What it is not
+
+The scorer is an acting agent, not a one-shot model call: an instruction is scored by running the agent on every
+training task, so one OPRO step costs `n_candidates × |train|` agent sessions (and the Trainer's pass of the current
+best, `|train|` more) and a single noisy pass per instruction decides its place in the history. The exemplars' answers come from the step's rollouts and the judge's
+feedback, since the task has no gold field. Selection on validation is a reproduction decision; OPRO itself has none.