8000 Advance development to 0.6.2.dev0 by DaoyuanLi2816 · Pull Request #41 · DaoyuanLi2816/mini-verl · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 10 additions & 4 deletions PROJECT_STATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ and what it printed.

Last updated: 2026-08-03.

## v0.6.1 Visual integrity and bridge correctness release candidate
## v0.6.1 Visual integrity and bridge correctness release

| item | current state |
| --- | --- |
Expand All @@ -16,10 +16,16 @@ Last updated: 2026-08-03.
| import contract | every source field is classified as exact, derived, informational only, requiring user confirmation or unsupported; unresolved data/environment, teacher, objective or schedule semantics produce `needs_user_input` plus a non-executable template |
| export status | artifact completeness, upstream parse/load, model/data smoke, reward implementation, launchability, distributed testing and algorithm parity are independent flags; the fail-closed reward scaffold remains non-launchable and uses `launch.template.sh` |
| documentation | pinned Material 9.7.7 supplies stable/dev navigation, search, dark/light modes, command-copy controls and responsive tables/images; the landing page exposes Align, Distill locally and Scale out paths |
| visual gate | Playwright checks five pages at 1440x900, 1024x768, 820x1000 and 390x844 for overflow, SVG bounds, label collisions, readable labels, tables and the responsive bridge; docs run [`30855762827`](https://github.com/DaoyuanLi2816/mini-verl/actions/runs/30855762827) is green and its 20 Linux screenshots were manually inspected |
| validation | local ruff, format, mypy, actionlint, full non-GPU/non-network, available GPU/network, strict MkDocs, browser visual, package/Twine, clean-install, extracted-sdist, bridge end-to-end, privacy, Markdown/link and generated-byte gates pass; PR-head CI [`30855762617`](https://github.com/DaoyuanLi2816/mini-verl/actions/runs/30855762617), build [`30855763132`](https://github.com/DaoyuanLi2816/mini-verl/actions/runs/30855763132) and pinned bridge [`30855762724`](https://github.com/DaoyuanLi2816/mini-verl/actions/runs/30855762724) are green |
| visual gate | Playwright checks five pages at 1440x900, 1024x768, 820x1000 and 390x844 for overflow, SVG bounds, label collisions, readable labels, tables and the responsive bridge; final PR docs run [`30856925490`](https://github.com/DaoyuanLi2816/mini-verl/actions/runs/30856925490) is green and its 20 Linux screenshots were manually inspected |
| validation | local ruff, format, mypy, actionlint, full non-GPU/non-network, available GPU/network, strict MkDocs, browser visual, package/Twine, clean-install, extracted-sdist, bridge end-to-end, privacy, Markdown/link and generated-byte gates pass; PR CI/build/bridge runs `30856925524` / `30856925481` / `30856925485` and synchronized-main CI/build/docs runs [`30857409920`](https://github.com/DaoyuanLi2816/mini-verl/actions/runs/30857409920) / [`30857409926`](https://github.com/DaoyuanLi2816/mini-verl/actions/runs/30857409926) / [`30857409918`](https://github.com/DaoyuanLi2816/mini-verl/actions/runs/30857409918) are green |
| immutable evidence | no result JSON/JSONL or frozen bridge-smoke record changed; calculator SHA-256 remains `53fc1d4d5b7adee09618d77ad62d4086ba56b78569832d6fc7c3bcd5c2695bbc` and Alignment Lab result/task SHA-256 values remain `584752dccb91654109c357b8ebb12681a12a9c1476a9ba539dd35e4d860a22ef` / `8d7fc723436d7377d196fc44046d960e3cb7f0aa81e03d49ef05b627eb84630f` |
| release state | exact `0.6.1` metadata is being finalized on focused PR [#40](https://github.com/DaoyuanLi2816/mini-verl/pull/40); merge, tag and OIDC publication remain gated on the final head being green |
| integration | focused PR [#40](https://github.com/DaoyuanLi2816/mini-verl/pull/40) was squash-merged as exact release commit `48b9e7d9231b5f6cd018f6e927f81df066258f17`; annotated tag `v0.6.1` resolves to that commit |
| release execution | tag run [`30857762954`](https://github.com/DaoyuanLi2816/mini-verl/actions/runs/30857762954) passed metadata, the full quality gate, one-time build, OIDC Trusted Publishing, public hashes/attestations, exact install and GitHub Release creation |
| published wheel | [`miniverl-0.6.1-py3-none-any.whl`](https://pypi.org/project/miniverl/0.6.1/), SHA-256 `ef9ce5378e43c0d833b782e431248a0838e3841ace76a8b3a08781dd28007918` |
| published sdist | [`miniverl-0.6.1.tar.gz`](https://pypi.org/project/miniverl/0.6.1/), SHA-256 `e5ffd7917035d1f3878b22415dd357cd47fe16b6c77ffdae260e0b85ad7e050f` |
| independent public verification | PyPI and GitHub Release expose identical file hashes; the PyPI integrity API exposes one Trusted Publisher attestation for each distribution, and a clean Windows Python 3.12 install from the public wheel reported `miniverl 0.6.1`, kept torch absent and passed core doctor |
| version transition | `v0.6.1` is immutable and public; this state-sync change identifies subsequent development as `0.6.2.dev0` |
| release state | complete; PyPI and [`miniVERL v0.6.1`](https://github.com/DaoyuanLi2816/mini-verl/releases/tag/v0.6.1) expose the verified release, and the synchronized docs workflow publishes v0.6.1 at the stable root plus current main under `/dev/` |

## v0.6.0 Verified verl Bridge release

Expand Down
46 changes: 23 additions & 23 deletions PYPI.md
10000
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<p align="center">
<img src="https://raw.githubusercontent.com/DaoyuanLi2816/mini-verl/v0.6.1/docs/banner.svg" alt="miniVERL — single-GPU LLM post-training" width="880">
<img src="https://raw.githubusercontent.com/DaoyuanLi2816/mini-verl/main/docs/banner.svg" alt="miniVERL — single-GPU LLM post-training" width="880">
</p>

<div align="center">
Expand All @@ -8,15 +8,15 @@
[![Build](https://github.com/DaoyuanLi2816/mini-verl/actions/workflows/build.yml/badge.svg)](https://github.com/DaoyuanLi2816/mini-verl/actions/workflows/build.yml)
[![PyPI](https://img.shields.io/pypi/v/miniverl.svg)](https://pypi.org/project/miniverl/)
[![Python](https://img.shields.io/badge/python-3.10%20%7C%203.11%20%7C%203.12%20%7C%203.13-blue)](https://www.python.org)
[![License](https://img.shields.io/badge/license-Apache--2.0-blue)](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.6.1/LICENSE)
[![License](https://img.shields.io/badge/license-Apache--2.0-blue)](https://github.com/DaoyuanLi2816/mini-verl/blob/main/LICENSE)

</div>

<p align="center">
<a href="https://pypi.org/project/miniverl/"><strong>PyPI</strong></a> ·
<a href="https://daoyuanli2816.github.io/mini-verl/"><strong>Stable docs</strong></a> ·
<a href="https://daoyuanli2816.github.io/mini-verl/dev/">Development docs</a> ·
<a href="https://github.com/DaoyuanLi2816/mini-verl/blob/v0.6.1/README.zh-CN.md">中文</a>
<a href="https://github.com/DaoyuanLi2816/mini-verl/blob/main/README.zh-CN.md">中文</a>
</p>

**miniVERL is a local, inspectable runtime for a documented subset of
Expand Down Expand Up @@ -44,15 +44,15 @@ optimization in about 50 seconds on the measured laptop CPU. For inspection,
schemas and reports without the ML stack, use `pip install miniverl`. For CUDA
training, install the matching CUDA-enabled PyTorch wheel first, then install
`miniverl[train,cuda]`; the extra does not select a CUDA PyTorch build. See the
[single-GPU guide](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.6.1/docs/single-gpu-guide.md).
[single-GPU guide](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/single-gpu-guide.md).

## Three paths

| Path | Start with | Concrete artifact | Next |
| --- | --- | --- | --- |
| **Align** — compare SFT, DPO, KD and OPD only when the pilot evidence supports the cost | `miniverl pilot recipes/alignment_policy_conditioned_qwen.yaml` | `alignment-card.json` | [Alignment Lab](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.6.1/docs/alignment-lab/alignment-lab-v1.md) |
| **Distill locally** — strict OPD, shared backbones and padded trajectory updates on one CUDA GPU | `miniverl train recipes/qwen_consumer_gpu_shared.yaml --dry-run` | `config.resolved.yaml` plus a revision-pinned PEFT adapter | [Bring your own GPU](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.6.1/docs/single-gpu-guide.md) |
| **Scale out** — import a documented profile, convert Parquet, export a bundle and run bridge checks | `miniverl bridge doctor scaleout-bundle` | `provenance/compatibility-report.json` | [Verified verl bridge](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.6.1/docs/verl-bridge.md) |
| **Align** — compare SFT, DPO, KD and OPD only when the pilot evidence supports the cost | `miniverl pilot recipes/alignment_policy_conditioned_qwen.yaml` | `alignment-card.json` | [Alignment Lab](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/alignment-lab/alignment-lab-v1.md) |
| **Distill locally** — strict OPD, shared backbones and padded trajectory updates on one CUDA GPU | `miniverl train recipes/qwen_consumer_gpu_shared.yaml --dry-run` | `config.resolved.yaml` plus a revision-pinned PEFT adapter | [Bring your own GPU](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/single-gpu-guide.md) |
| **Scale out** — import a documented profile, convert Parquet, export a bundle and run bridge checks | `miniverl bridge doctor scaleout-bundle` | `provenance/compatibility-report.json` | [Verified verl bridge](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/verl-bridge.md) |

The bridge import is deliberately not generic YAML conversion. If dataset or
environment, teacher identity, objective, or schedule semantics are missing,
Expand All @@ -75,12 +75,12 @@ improved it; continued SFT and both OPD variants retained measured regressions.
| standard OPD | 98.6% | 97.2% | 100.0% | 76.7 s |
| verifier-gated OPD | 97.9% | 95.8% | 46.8% | 66.0 s |

![Alignment and utility deltas from the saturated SFT checkpoint; small marks are all three seeds and large marks are means](https://raw.githubusercontent.com/DaoyuanLi2816/mini-verl/v0.6.1/docs/alignment-lab/delta-from-sft.svg)
![Alignment and utility deltas from the saturated SFT checkpoint; small marks are all three seeds and large marks are means](https://raw.githubusercontent.com/DaoyuanLi2816/mini-verl/main/docs/alignment-lab/delta-from-sft.svg)

The two sandbox safety checks tied at zero while utility still regressed.
IFEval, XSTest, HarmBench and RewardBench were **not executed**. “Preference
win rate” is a deterministic Minipolicy paired outcome, not human preference.
Read the [study, seed-level values and limitations](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.6.1/docs/alignment-lab/alignment-lab-v1.md).
Read the [study, seed-level values and limitations](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/alignment-lab/alignment-lab-v1.md).

## One measured systems result

Expand All @@ -91,13 +91,13 @@ reserved memory versus 3.035 GiB for dual model, while running 10.1% slower.
All 12 preregistered equivalence comparisons passed. These are one-workload,
one-machine measurements, not promises for other GPUs.

![Measured throughput and reserved VRAM for dual-model and shared-backbone runtime cells](https://raw.githubusercontent.com/DaoyuanLi2816/mini-verl/v0.6.1/docs/consumer-runtime-v1-pareto.svg)
![Measured throughput and reserved VRAM for dual-model and shared-backbone runtime cells](https://raw.githubusercontent.com/DaoyuanLi2816/mini-verl/main/docs/consumer-runtime-v1-pareto.svg)

[Consumer Runtime v1 methods and caveats](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.6.1/docs/consumer-runtime-v1.md)
[Consumer Runtime v1 methods and caveats](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/consumer-runtime-v1.md)

## Compatibility boundary

![Verified local runtime, portable artifact bundle and pinned upstream smoke; distributed verl execution remains untested](https://raw.githubusercontent.com/DaoyuanLi2816/mini-verl/v0.6.1/docs/verl-bridge-architecture.svg)
![Verified local runtime, portable artifact bundle and pinned upstream smoke; distributed verl execution remains untested](https://raw.githubusercontent.com/DaoyuanLi2816/mini-verl/main/docs/verl-bridge-architecture.svg)

The bridge targets official verl `v0.8.0` at commit `7aed6b23` and uses the
term **miniVERL-defined compatibility Level 3**. That means a checksummed
Expand All @@ -114,21 +114,21 @@ PPO/reward scaffold, not an executable continuation of miniVERL OPD semantics.

## Detailed studies and preserved negative evidence

- [RecoveryBench v1](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.6.1/docs/recoverybench/recoverybench-v1.md): frozen-student KD
- [RecoveryBench v1](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/recoverybench/recoverybench-v1.md): frozen-student KD
outperformed much slower fresh-state OPD on the preregistered primary view;
the verifier gate remained `insufficient_evidence`.
- [Alignment Lab v1](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.6.1/docs/alignment-lab/alignment-lab-v1.md): the starting SFT
- [Alignment Lab v1](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/alignment-lab/alignment-lab-v1.md): the starting SFT
checkpoint was at the ceiling, so no positive OPD result is claimed.
- [Calculator benchmark](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.6.1/docs/benchmarking.md): both negative controls completed
- [Calculator benchmark](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/benchmarking.md): both negative controls completed
normally and measured 0% strict success. They were not configuration
failures. Because they used the historical ambiguous protocol-v1 prompt,
their failure cannot be attributed solely to intrinsic teacher behavior.
- [Consumer Runtime v1](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.6.1/docs/consumer-runtime-v1.md): padded update batches and
- [Consumer Runtime v1](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/consumer-runtime-v1.md): padded update batches and
shared adapters preserve the measured one-update objective within declared
tolerances; rollout generation remains sequential.
- [Limitations](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.6.1/docs/limitations.md), [math](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.6.1/docs/math.md),
[reproducibility](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.6.1/docs/reproducibility.md) and
[compatibility policy](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.6.1/docs/compatibility.md).
- [Limitations](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/limitations.md), [math](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/math.md),
[reproducibility](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/reproducibility.md) and
[compatibility policy](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/compatibility.md).

New runs establish tokenizer compatibility through structural identity. The
legacy behavioral fingerprint—token IDs for one fixed probe plus metadata—is
Expand All @@ -148,7 +148,7 @@ python -m pip install -e ".[dev]"
pytest -q -m "not gpu and not network"
```

Apache-2.0 licensed. See [CONTRIBUTING.md](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.6.1/CONTRIBUTING.md) and
[SECURITY.md](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.6.1/SECURITY.md). Project records: [default GPU recipe](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.6.1/recipes/qwen_consumer_gpu_calc.yaml),
[frozen calculator JSON](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.6.1/benchmarks/results/gpu-calc-hard-equal-update-v2.json),
[changelog](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.6.1/CHANGELOG.md), [citation](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.6.1/CITATION.cff) and [license](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.6.1/LICENSE).
Apache-2.0 licensed. See [CONTRIBUTING.md](https://github.com/DaoyuanLi2816/mini-verl/blob/main/CONTRIBUTING.md) and
[SECURITY.md](https://github.com/DaoyuanLi2816/mini-verl/blob/main/SECURITY.md). Project records: [default GPU recipe](https://github.com/DaoyuanLi2816/mini-verl/blob/main/recipes/qwen_consumer_gpu_calc.yaml),
[frozen calculator JSON](https://github.com/DaoyuanLi2816/mini-verl/blob/main/benchmarks/results/gpu-calc-hard-equal-update-v2.json),
[changelog](https://github.com/DaoyuanLi2816/mini-verl/blob/main/CHANGELOG.md), [citation](https://github.com/DaoyuanLi2816/mini-verl/blob/main/CITATION.cff) and [license](https://github.com/DaoyuanLi2816/mini-verl/blob/main/LICENSE).
2 changes: 1 addition & 1 deletion docs/generated/quality.json
8000
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"schema_version": 1,
"release": "0.6.1",
"status": "validated",
"status": "released",
"measured_commit": "7fed8e79f02c0e0ba64ae6056760a75bc1b9b047",
"measured_at": "2026-08-03T14:53:12-07:00",
"cpu_non_gpu_non_network": {
Expand Down
18 changes: 14 additions & 4 deletions docs/release-checklist.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,10 +52,20 @@ after the exact release commit and its remote checks are green.

## After the tag

- [ ] Verify the annotated tag, OIDC run, PyPI hashes and attestations, a clean
public install, the GitHub Release and versioned stable documentation.
- [ ] Advance subsequent development to `0.6.2.dev0` in a separate green
state-sync pull request.
- [x] Annotated tag `v0.6.1` resolves to release commit
`48b9e7d9231b5f6cd018f6e927f81df066258f17`; OIDC publication, one
attestation per file, public hashes, a clean public-wheel install and the
GitHub Release were verified by tag run
[`30857762954`](https://github.com/DaoyuanLi2816/mini-verl/actions/runs/30857762954).
- [x] This separate state-sync change advances subsequent development to
`0.6.2.dev0`; it merges only after remote checks are green.

Published artifact identity:

- `miniverl-0.6.1-py3-none-any.whl`: SHA-256
`ef9ce5378e43c0d833b782e431248a0838e3841ace76a8b3a08781dd28007918`
- `miniverl-0.6.1.tar.gz`: SHA-256
`e5ffd7917035d1f3878b22415dd357cd47fe16b6c77ffdae260e0b85ad7e050f`

## v0.6.0 Verified verl Bridge release

Expand Down
2 changes: 1 addition & 1 deletion src/miniverl/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,6 @@

from __future__ import annotations

__version__ = "0.6.1"
__version__ = "0.6.2.dev0"

__all__ = ["__version__"]
0