Tengxiao Liu Yuqing Yang Xi Ye Danqi Chen
Blog post: https://tengxiaoliu.github.io/autoevolver/
tasks/ contains the initial setup files we provided to Claude Code for each problem. Each task directory includes:
initial.py— the seed program the agent starts fromevaluate.py— evaluatorprompt.py— task descriptionINSTRUCTION.md— instructions given to the agent
tasks/
├── ac1/ # First Autocorrelation Inequality
├── circle_packing/ # Circle Packing (n=26)
└── erdos_min_overlap/ # Erdos Minimum Overlap
results/ contains the best solutions produced by the agent for each problem.
results/
├── ac1/result.json
├── circle_packing/result.json
└── erdos/result.json
Each evaluator supports two modes:
1. Run from a program file (execute initial.py or any evolved variant):
cd tasks/circle_packing
python evaluate.py --program_path initial.py --results_dir results2. Validate a pre-computed result (verify solutions in results/):
cd tasks/circle_packing
python evaluate.py --from-result ../../results/circle_packing/result.json --results_dir resultsBoth modes produce metrics.json and correct.json in the specified --results_dir.
Our task definitions and evaluation scripts were adapted from OpenEvolve and ShinkaEvolve.
Conversation trajectories were captured and exported using DataClaw.
All experiments were conducted using Claude Code.