8000 chore: synchronize v0.15.0 publication and 0.15.1.dev0 by DaoyuanLi2816 · Pull Request #122 · DaoyuanLi2816/mini-verl · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,8 @@ jobs:
fi
test -n "$stable_tag"
git worktree add --detach "${RUNNER_TEMP}/stable-checkout" "$stable_tag"
# Release content stays pinned; navigation names the current channels.
python scripts/release_state.py --docs-overlay "${RUNNER_TEMP}/stable-checkout"
(
cd "${RUNNER_TEMP}/stable-checkout"
mkdocs build --strict --site-dir "${GITHUB_WORKSPACE}/site"
Expand Down
34 changes: 21 additions & 13 deletions PROJECT_STATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,20 +6,28 @@ current product and evidence state rather than repeating release history.

Last updated: 2026-09-10.

Canonical release state: releasing `v0.15.0`.
Canonical release state: stable `v0.15.0` (`659690d92a9ed92a01d1beec0a75873de8b860d5`), development `0.15.1.dev0`.

## Release state

- v0.15.0 is preparing the direct-config release. Implementation and installed
development-wheel PPO/GRPO rehearsals pass; formal candidate qualification
and OIDC publication are pending.
- v0.15.0 is published on PyPI and GitHub. The release uses the exact candidate
from full RTX 4080 qualification `34442828628` (attempt 1), accepted by
rehearsal `34447536443` and OIDC publication `34448233909`.
- v4 semantically accepts all seven complete pinned upstream configurations.
Five derived stress cases add two accepted and three explicit rejections.
Exact production-model execution is not claimed by static compilation.
- The installed direct rehearsal completed four actor updates in both cases,
four PPO critic updates, reward validation, exact tensor replay and handoff.
- Local full regression: 2,725 passed, 82.01% branch coverage, plus 18 pinned
upstream conformance cases and four documentation viewports.
- The installed release wheel completed four actor updates in both direct
workflows, four PPO critic updates, reward validation, exact tensor replay
and inspected handoff. Original upstream YAML bytes were retained.
- Local full regression: 2,733 passed, 28 skipped, 24 deselected and 82.21%
branch coverage. Eighteen pinned upstream conformance checks, 10 GPU tests,
15 network tests (one local upstream-v0.8 skip) and four browser viewports
also passed; the dedicated upstream bridge CI covered that local skip.
- Release asset preparation and its current-version round trip include the
direct-workflow record at `v015/direct-workflows.json` in the evidence archive.
- Public PyPI hashes, trusted-publisher attestations, a clean core installation
and all 13 GitHub Release assets were verified. The development line is now
`0.15.1.dev0`; stable documentation remains tied to `v0.15.0`.

### Prior stable release (historical)

Expand All @@ -39,16 +47,16 @@ Canonical release state: releasing `v0.15.0`.
coverage; 10 RTX 4080 GPU and 16 network tests passed. Four pinned upstream
conformance tests and four-viewport documentation browser checks passed.

- Stable release: `v0.14.0` at
- Prior stable release: `v0.14.0` at
`60afc63da1257e3e22aa6056607a63a90d4d90f1`.
- Current development line: `0.14.1.dev0`.
- Development after that release: `0.14.1.dev0`.
- Stable docs: <https://daoyuanli2816.github.io/mini-verl/>.
- Development docs: <https://daoyuanli2816.github.io/mini-verl/dev/>.
- Historical build log: [v0.1-v0.9 archive](docs/history/project-state-v0.1-v0.9.md).

## Current product boundary

Release `0.15.0` has a closed typed profile compiler: it retains v1/v2/v3 and adds
Development `0.15.1.dev0` has a closed typed profile compiler: it retains v1/v2/v3 and adds
`verl-rl-v0.9-single-gpu-v4`, direct execution of resolved RL inputs from official verl
`v0.9.0` at commit `483b8a009ba3a97563edee3a19887e4862b8094a`.
It compiles PPO/GAE into phased one-GPU actor, critic, reference and reward
Expand Down Expand Up @@ -94,8 +102,8 @@ v0.8 environment/PPO artifact bridge is migration-only.

| Evidence | Status |
| --- | --- |
| Installed PPO product workflow | Qwen3-0.6B, 8 trajectories, 4 actor/4 critic updates, 2.7266 GiB peak reserved, 15.688 s reported train duration; exact tensor replay and handoff passed |
| Installed GRPO product workflow | Qwen3-0.6B, 8 trajectories, 2 actor updates, 1.502 GiB peak reserved, 8.923 s reported train duration; exact tensor replay and handoff passed |
| Installed direct PPO workflow (v0.15.0) | Qwen3-0.6B, 8 trajectories, 4 actor/4 critic updates, 1.502 GiB peak reserved; exact tensor replay and handoff passed |
| Installed direct GRPO workflow (v0.15.0) | Qwen3-0.6B, 8 trajectories, 4 actor updates, 1.5039 GiB peak reserved; exact tensor replay and handoff passed |
| Qwen3-0.6B/1.7B developer workload | 32 prompts, 8 current-policy updates, 3.1914 GiB peak reserved on one RTX 4080; matched interruption/resume was byte-identical |
| Qwen3 sampled-k1 PG | 32 prompts, 8 updates, exact interruption/resume, 3.1914 GiB peak reserved; no quality comparison |
| SmolLM2-360M/1.7B direct GKD | 32 prompts, 8 updates, 1.4961 GiB peak reserved; exact resume, PEFT reload and materialized export passed |
Expand Down
48 changes: 24 additions & 24 deletions PYPI.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<p align="center">
<img src="https://raw.githubusercontent.com/DaoyuanLi2816/mini-verl/v0.15.0/docs/banner.svg" alt="miniVERL — lower verl experiment semantics onto one CUDA GPU" width="880">
<img src="https://raw.githubusercontent.com/DaoyuanLi2816/mini-verl/main/docs/banner.svg" alt="miniVERL — lower verl experiment semantics onto one CUDA GPU" width="880">
</p>

<div align="center">
Expand All @@ -8,15 +8,15 @@
[![Build](https://github.com/DaoyuanLi2816/mini-verl/actions/workflows/build.yml/badge.svg)](https://github.com/DaoyuanLi2816/mini-verl/actions/workflows/build.yml)
[![PyPI](https://img.shields.io/pypi/v/miniverl.svg)](https://pypi.org/project/miniverl/)
[![Python](https://img.shields.io/badge/python-3.10%20%7C%203.11%20%7C%203.12%20%7C%203.13-blue)](https://www.python.org)
[![License](https://img.shields.io/badge/license-Apache--2.0-blue)](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.15.0/LICENSE)
[![License](https://img.shields.io/badge/license-Apache--2.0-blue)](https://github.com/DaoyuanLi2816/mini-verl/blob/main/LICENSE)

</div>

<p align="center">
<a href="https://pypi.org/project/miniverl/"><strong>PyPI</strong></a> ·
<a href="https://daoyuanli2816.github.io/mini-verl/"><strong>Stable docs</strong></a> ·
<a href="https://daoyuanli2816.github.io/mini-verl/dev/">Development docs</a> ·
<a href="https://github.com/DaoyuanLi2816/mini-verl/blob/v0.15.0/README.zh-CN.md">中文</a>
<a href="https://github.com/DaoyuanLi2816/mini-verl/blob/main/README.zh-CN.md">中文</a>
</p>

**Run common verl PPO/GRPO configs directly on one NVIDIA GPU.** Hand miniVERL
Expand Down Expand Up @@ -59,13 +59,13 @@ miniverl export-verl --run runs/local-ppo --target-verl v0.9.0 --out ppo-handoff
miniverl bridge doctor ppo-handoff --json
```

The [five-minute workflow](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.15.0/docs/for-verl-users.md) explains each artifact and
The [five-minute workflow](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/for-verl-users.md) explains each artifact and
offers the matching GRPO commands. These are small length-reward exercises;
their reward is directly inspectable in `rewards.jsonl`.

`[train]` supplies the ML stack; `[cuda]` additionally supplies quantization.
Select PyTorch's CUDA build with the [PyTorch installer](https://pytorch.org/get-started/locally/).
The [single-GPU guide](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.15.0/docs/single-gpu-guide.md) covers memory planning and the
The [single-GPU guide](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/single-gpu-guide.md) covers memory planning and the
maintainer-measured RTX 4080 environment.

## What a run gives you
Expand All @@ -84,8 +84,8 @@ maintainer-measured RTX 4080 environment.
## How it works

<picture>
<source media="(max-width: 640px)" srcset="https://raw.githubusercontent.com/DaoyuanLi2816/mini-verl/v0.15.0/docs/verl-local-runtime-mobile.svg">
<img src="https://raw.githubusercontent.com/DaoyuanLi2816/mini-verl/v0.15.0/docs/verl-local-runtime.svg" alt="A resolved verl config compiles into a validated single-GPU plan; actor, critic, reference, teacher and reward roles run in phases and produce portable artifacts plus a readiness report.">
<source media="(max-width: 640px)" srcset="https://raw.githubusercontent.com/DaoyuanLi2816/mini-verl/main/docs/verl-local-runtime-mobile.svg">
<img src="https://raw.githubusercontent.com/DaoyuanLi2816/mini-verl/main/docs/verl-local-runtime.svg" alt="A resolved verl config compiles into a validated single-GPU plan; actor, critic, reference, teacher and reward roles run in phases and produce portable artifacts plus a readiness report.">
</picture>

One GPU is treated as a temporal scheduler for logical roles. RL generates
Expand All @@ -99,10 +99,10 @@ the learning signal.

| Goal | First command | Primary artifact | Next step |
| --- | --- | --- | --- |
| **Run local RL** | `miniverl run resolved-verl.yaml --dry-run` | semantic report + local plan | [Direct configs](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.15.0/docs/direct-verl-config.md) |
| **Run local OPD** | `miniverl plan --profile verl-opd-v0.8-single-gpu-v1 --config verl-opd.yaml --out plan.json` | immutable execution plan | [OPD quickstart](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.15.0/docs/opd-quickstart.md) |
| **Fit your GPU** | `miniverl plan --config verl-opd.yaml --probe` | measured placement plan | [Hardware planning](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.15.0/docs/hardware-planning.md) |
| **Hand off artifacts** | `miniverl export-verl --run runs/my-run --target-verl v0.9.0 --out scaleout` | actor, critic, Parquet + config bundle | [Compatibility contract](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.15.0/docs/compatibility.md) |
| **Run local RL** | `miniverl run resolved-verl.yaml --dry-run` | semantic report + local plan | [Direct configs](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/direct-verl-config.md) |
| **Run local OPD** | `miniverl plan --profile verl-opd-v0.8-single-gpu-v1 --config verl-opd.yaml --out plan.json` | immutable execution plan | [OPD quickstart](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/opd-quickstart.md) |
| **Fit your GPU** | `miniverl plan --config verl-opd.yaml --probe` | measured placement plan | [Hardware planning](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/hardware-planning.md) |
| **Hand off artifacts** | `miniverl export-verl --run runs/my-run --target-verl v0.9.0 --out scaleout` | actor, critic, Parquet + config bundle | [Compatibility contract](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/compatibility.md) |

Native recipes also support SFT, DPO and offline KD, plus calculator, JSON
navigation, read-only SQLite and custom tool environments.
Expand All @@ -120,7 +120,7 @@ navigation, read-only SQLite and custom tool environments.
| Direct GKD / sampled-k1 OPD | supported | pinned verl v0.8 profiles with teacher targets |
| Ray, FSDP/FSDP2, Megatron, TP/PP/DP > 1 | distributed-only | these change physical scale, not the local objective |

The [upstream compatibility corpus](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.15.0/docs/verl-compatibility-corpus.md) resolves
The [upstream compatibility corpus](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/verl-compatibility-corpus.md) resolves
real verl examples and records every field's outcome. The v4 direct compiler
adds prompt filtering, epoch scheduling and common loss reductions, separating
semantic acceptance from missing inputs and hardware capacity. v1/v2/v3 remain
Expand All @@ -132,31 +132,31 @@ The published Qwen3-0.6B/1.7B OPD workload consumed 32 distinct prompts and
completed eight current-policy updates at 3.1914 GiB peak reserved VRAM on an
RTX 4080. A matched interruption after update four reproduced byte-identical
trajectories, adapter and optimizer tensors. The SmolLM2-360M/1.7B workload
completed the same shape at 1.4961 GiB. [System records](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.15.0/docs/verl-opd-reference-workload.md)
completed the same shape at 1.4961 GiB. [System records](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/verl-opd-reference-workload.md)
contain configs, hashes and phase timings.

Rollout Runtime v2 measured `hf_cached` and managed vLLM across 24 RTX 4080
cells spanning response length, sampling mode and `n=1/4`. See the
[runtime report](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.15.0/docs/benchmarks/rollout-runtime-v2.md). Evidence for the new
[runtime report](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/benchmarks/rollout-runtime-v2.md). Evidence for the new
RL family is published through the exact-wheel release qualification rather
than presented as a task-quality comparison.

## Research record

The scientific reports preserve their original outcomes and frozen inputs:
[calculator protocol](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.15.0/docs/benchmarking.md),
[RecoveryBench](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.15.0/docs/recoverybench/recoverybench-v1.md),
[Alignment Lab](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.15.0/docs/alignment-lab/alignment-lab-v1.md), and the
[External Alignment Gate](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.15.0/docs/alignment-external/alignment-external-v1.md).
[calculator protocol](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/benchmarking.md),
[RecoveryBench](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/recoverybench/recoverybench-v1.md),
[Alignment Lab](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/alignment-lab/alignment-lab-v1.md), and the
[External Alignment Gate](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/alignment-external/alignment-external-v1.md).
They are scoped studies, separate from runtime qualification.

## Compatibility boundary

miniVERL targets one local process and one NVIDIA CUDA GPU. The compiler
distinguishes exact or conformant semantics, local physical lowering,
distributed-only settings, feasible work not yet implemented, and technically
unsupported inputs. The [compatibility policy](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.15.0/docs/compatibility.md) is the
complete matrix; [limitations](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.15.0/docs/limitations.md) collects architecture,
unsupported inputs. The [compatibility policy](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/compatibility.md) is the
complete matrix; [limitations](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/limitations.md) collects architecture,
measurement, security and generalization boundaries in one place. miniVERL is
an independent Apache-2.0 project.

Expand All @@ -169,7 +169,7 @@ python -m pip install -e ".[dev,train]"
pytest -q -m "not gpu and not network"
```

See [CONTRIBUTING.md](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.15.0/CONTRIBUTING.md), [CHANGELOG.md](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.15.0/CHANGELOG.md),
[CITATION.cff](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.15.0/CITATION.cff), the [guide for verl users](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.15.0/docs/for-verl-users.md),
[SECURITY.md](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.15.0/SECURITY.md) and the
[Apache-2.0 license](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.15.0/LICENSE).
See [CONTRIBUTING.md](https://github.com/DaoyuanLi2816/mini-verl/blob/main/CONTRIBUTING.md), [CHANGELOG.md](https://github.com/DaoyuanLi2816/mini-verl/blob/main/CHANGELOG.md),
[CITATION.cff](https://github.com/DaoyuanLi2816/mini-verl/blob/main/CITATION.cff), the [guide for verl users](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/for-verl-users.md),
[SECURITY.md](https://github.com/DaoyuanLi2816/mini-verl/blob/main/SECURITY.md) and the
[Apache-2.0 license](https://github.com/DaoyuanLi2816/mini-verl/blob/main/LICENSE).
Loading
0