Official implementation of Scaling Self-Play with Self-Guidance.
SGS is an asymmetric self-play algorithm in which a single language model takes on three roles — Solver, Conjecturer, and Guide — with the Guide supervising the Conjecturer to produce high quality synthetic problems. Applied to DeepSeek-Prover-V2-7B and trained on ~3000 Lean4 formal math problems, SGS surpasses the pass@4 of the 671B parameter DeepSeek-Prover-V2-671B model.
All experiments are tested and can run on 8xH200 node.
-
Create the Python environment.
conda env create -f environment.yml conda activate sgs pip install -r requirements.txt pip install -e . -
Install
elan(Lean toolchain manager) if you don't already have it. See https://github.com/leanprover/elan for instructions. -
Build the Lean REPL and mathlib4.
setup.shclones the two repositories at matching Leanv4.15.0tags, runslake exe cache geton mathlib, and compiles both. Expect this to take over 10 minutes../setup.sh
This produces
./repl/and./mathlib4/at the repo root. -
Configure environment variables. Create a
.envfile at the repo root:SGS_REPL_PATH=./repl/.lake/build/bin/repl SGS_MATHLIB_PATH=./mathlib4 # Optional, for wandb logging: WANDB_API_KEY=...
The Lean verifier reads
SGS_REPL_PATHandSGS_MATHLIB_PATHat import time and will raise if either is unset. -
(Optional) Log into wandb if you want experiment tracking:
wandb login
The training data (~3000 Lean 4 formal math problems) is hosted on Hugging Face at LukeBailey181Pub/D_3k and is pulled automatically when a run starts.
Each of the experiments in the paper corresponds to one entry-point script in scripts/:
| Script | Role |
|---|---|
scripts/standard_sgs.py |
Main method — full SGS pipeline (Solver + Conjecturer + Guide) |
scripts/cispo_sgs.py |
SGS using the CISPO (grouped importance-sampled) solver objective |
scripts/cispo_solver_only.py |
CISPO baseline — solver training only, no conjecturing |
scripts/ei_solver_only.py |
Expert-Iteration baseline — solver training only |
scripts/sgs_frozen_conjecturer.py |
Ablation: freeze the Conjecturer, only train the Solver |
scripts/sgs_no_guide.py |
Ablation: run SGS without the Guide component |
scripts/sgs_no_problem_conditioning.py |
Ablation: Conjecturer generates problems without conditioning on a seed problem |
Run an experiment with:
python scripts/standard_sgs.pyEach script configures its own checkpoint_dir and wandb_tags at the bottom of the file. Checkpoints, logs, and intermediate proof datasets are written into checkpoint_dir.
48 of the 3,323 problems in D_3k (and 2 of the 100 in the small smoke-test set) had their
final := by inside a trailing -- line comment, for example:
a ≠ 0 ∧ b ≠ 0 -- boxes are non-empty := byBecause -- comments run to the end of the line, these statements had no proof body, and a
proof was only accepted if the Solver re-emitted := by on a fresh line after the comment. In
the paper's runs the Solver learned this within the first ~15 iterations, so these problems
contributed to the reported cumulative solve rate from early on. A reproduction in which the
Solver only discovers this late will lag the paper by up to ~1 percentage point until it does.
Thanks to Lorenz for reporting this.
The seed statements in data/D_3k_prover_dataset.json, data/D_3k_small_prover_dataset.json
and the Hugging Face dataset LukeBailey181Pub/D_3k have been patched to move := by onto its
own line:
a ≠ 0 ∧ b ≠ 0 -- boxes are non-empty
:= byAll 48 patched statements parse; 45 elaborate with a sorry proof under Lean 4.15 / Mathlib.
The remaining three (4429adae, cc2e08b6, d814bd4e) are malformed for unrelated reasons
(an undefined identifier or a truncated conclusion) and were unprovable before and after the
patch. They are kept so the problem count stays at 3,323. The affected ids are listed in
data/D_3k_comment_fix_ids.txt.
@article{bailey2026scaling,
title={Scaling Self-Play with Self-Guidance},
author={Bailey, Luke and Wen, Kaiyue and Dong, Kefan and Hashimoto, Tatsunori and Ma, Tengyu},
journal={arXiv preprint arXiv:2604.20209},
year={2026}
}