8000 GitHub - LukeBailey181/sgs · GitHub
[go: up one dir, main page]

Skip to content
LukeBailey181Public

About

No description, website, or topics provided.

Resources

Stars

84 stars

Watchers

1 watching

Forks

Repository files navigation

Scaling Self-Play with Self-Guidance

arXiv

Scaling Self-Play with Self-Guidance

Official implementation of Scaling Self-Play with Self-Guidance.

SGS is an asymmetric self-play algorithm in which a single language model takes on three roles — Solver, Conjecturer, and Guide — with the Guide supervising the Conjecturer to produce high quality synthetic problems. Applied to DeepSeek-Prover-V2-7B and trained on ~3000 Lean4 formal math problems, SGS surpasses the pass@4 of the 671B parameter DeepSeek-Prover-V2-671B model.

Setup

All experiments are tested and can run on 8xH200 node.

  1. Create the Python environment.

    conda env create -f environment.yml
    conda activate sgs
    pip install -r requirements.txt
    pip install -e .
  2. Install elan (Lean toolchain manager) if you don't already have it. See https://github.com/leanprover/elan for instructions.

  3. Build the Lean REPL and mathlib4. setup.sh clones the two repositories at matching Lean v4.15.0 tags, runs lake exe cache get on mathlib, and compiles both. Expect this to take over 10 minutes.

    ./setup.sh

    This produces ./repl/ and ./mathlib4/ at the repo root.

  4. Configure environment variables. Create a .env file at the repo root:

    SGS_REPL_PATH=./repl/.lake/build/bin/repl
    SGS_MATHLIB_PATH=./mathlib4
    # Optional, for wandb logging:
    WANDB_API_KEY=...

    The Lean verifier reads SGS_REPL_PATH and SGS_MATHLIB_PATH at import time and will raise if either is unset.

  5. (Optional) Log into wandb if you want experiment tracking:

    wandb login

The training data (~3000 Lean 4 formal math problems) is hosted on Hugging Face at LukeBailey181Pub/D_3k and is pulled automatically when a run starts.

Scripts

Each of the experiments in the paper corresponds to one entry-point script in scripts/:

Script Role
scripts/standard_sgs.py Main method — full SGS pipeline (Solver + Conjecturer + Guide)
scripts/cispo_sgs.py SGS using the CISPO (grouped importance-sampled) solver objective
scripts/cispo_solver_only.py CISPO baseline — solver training only, no conjecturing
scripts/ei_solver_only.py Expert-Iteration baseline — solver training only
scripts/sgs_frozen_conjecturer.py Ablation: freeze the Conjecturer, only train the Solver
scripts/sgs_no_guide.py Ablation: run SGS without the Guide component
scripts/sgs_no_problem_conditioning.py Ablation: Conjecturer generates problems without conditioning on a seed problem

Run an experiment with:

python scripts/standard_sgs.py

Each script configures its own checkpoint_dir and wandb_tags at the bottom of the file. Checkpoints, logs, and intermediate proof datasets are written into checkpoint_dir.

Dataset fix (2026-11-05): := by swallowed by a trailing comment

48 of the 3,323 problems in D_3k (and 2 of the 100 in the small smoke-test set) had their final := by inside a trailing -- line comment, for example:

  a ≠ 0 ∧ b ≠ 0       -- boxes are non-empty := by

Because -- comments run to the end of the line, these statements had no proof body, and a proof was only accepted if the Solver re-emitted := by on a fresh line after the comment. In the paper's runs the Solver learned this within the first ~15 iterations, so these problems contributed to the reported cumulative solve rate from early on. A reproduction in which the Solver only discovers this late will lag the paper by up to ~1 percentage point until it does. Thanks to Lorenz for reporting this.

The seed statements in data/D_3k_prover_dataset.json, data/D_3k_small_prover_dataset.json and the Hugging Face dataset LukeBailey181Pub/D_3k have been patched to move := by onto its own line:

  a ≠ 0 ∧ b ≠ 0       -- boxes are non-empty
  := by

All 48 patched statements parse; 45 elaborate with a sorry proof under Lean 4.15 / Mathlib. The remaining three (4429adae, cc2e08b6, d814bd4e) are malformed for unrelated reasons (an undefined identifier or a truncated conclusion) and were unprovable before and after the patch. They are kept so the problem count stays at 3,323. The affected ids are listed in data/D_3k_comment_fix_ids.txt.

Citation

@article{bailey2026scaling,
  title={Scaling Self-Play with Self-Guidance},
  author={Bailey, Luke and Wen, Kaiyue and Dong, Kefan and Hashimoto, Tatsunori and Ma, Tengyu},
  journal={arXiv preprint arXiv:2604.20209},
  year={2026}
}

About

No description, website, or topics provided.

Resources

Stars

84 stars

Watchers

1 watching

Forks

Releases

Packages

Contributors

Languages

0