8000 Advance development after v0.2.4 by DaoyuanLi2816 · Pull Request #21 · DaoyuanLi2816/mini-verl · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
8000
Diff view
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,15 @@ All notable changes to miniVERL are recorded here. The format follows

## [Unreleased]

### Fixed

- Release verification accepts an exact, tag-pinned banner expressed as
Markdown or HTML while retaining source-URL and alt-text checks.
- A browser-only PyPI challenge can be deferred after public metadata, every
pinned target, distribution hashes and attestations pass; a recovery workflow
binds already-published artifacts to their original tag run before creating a
GitHub Release.

## [0.2.4] - 2026-07-29

Correctness, adversarial-input, concurrency, packaging and privacy hardening
Expand Down
28 changes: 15 additions & 13 deletions PROJECT_STATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,23 +4,25 @@ Living build log for **miniVERL** (`mini-verl` / `miniverl` / CLI `miniverl`).
A checkbox is not evidence: every completed item names the command that was run
and what it printed.

Last updated: 2026-07-29.
Last updated: 2026-07-30.

## v0.2.4 final framework-hardening work in progress
## v0.2.4 framework-hardening release status

| item | current state |
| --- | --- |
| audited starting point | fetched local and public `main` both resolve to `ac047a6001ae0bc2935853fa4c35cedbcd9b9ee4`; working branch is `agent/v024-final-hardening` |
| release-candidate version | metadata, generated PyPI description and clean wheel installs now identify as `0.2.4`; no tag exists and public PyPI remains `0.2.3` until the PR head and merge commit pass every remote gate |
| immutable evidence | the v0.2 calculator benchmark must remain byte-identical at SHA-256 `53fc1d4d5b7adee09618d77ad62d4086ba56b78569832d6fc7c3bcd5c2695bbc`; release tags and the pinned protocol-teacher adapter are out of scope for mutation |
| strict protocol and numerics | regressions first reproduced 33 failures: Python JSON accepted non-finite/duplicate/pathological values, calculator conversion leaked `ValueError`/`OverflowError`, and prefix-only verification accepted contradictory suffixes. A shared bounded strict decoder now rejects non-finite values, duplicate keys, 128+ digit integers, depth over 32, more than 256 members and invalid surrogates; renderers use `allow_nan=False` semantics, protocol-v2 uses complete numeric/unit verification, and historical protocol/verifier-v1 remains explicit |
| lifecycle and locking | `TrainerState` enforces `ready -> running -> completed/failed/interrupted -> closed` with an atomic one-shot transition; a second call changes no bytes and concurrent threads cannot both enter. `filelock` backs stable `<output-root>/.miniverl-locks/<run-id>.lock` ownership acquired before run access/model loading and released after teardown; fail-fast, timeout, killed-owner and distinct-run multiprocessing cases pass on Windows |
| focused evidence | regressions cover strict protocol/numerics, run locking, one-shot lifecycle, exact config layers, portable reports and wheel metadata; exact local CPU/GPU/network totals and branch coverage are recorded only in [`docs/generated/quality.json`](docs/generated/quality.json), bound to implementation commit `6911acf11978cfc5f6a99375cbe7a1666fcf7a85` |
| packaging and provenance | PyPI markdown is generated with tag-stable links; the sdist carries its complete repository-test surface and self-tests outside the checkout; submitted/validated/legacy-runtime/resolved config layers have distinct bytes and digests; public reports use the canonical portable view |
| local release gate | ruff, format, mypy, actionlint, Markdown/anchor checks, generated-file checks, full CPU coverage, available RTX 4080 GPU tests, network tests, wheel/sdist build plus twine, and the complete extracted-sdist CPU suite pass; the frozen benchmark hash remains exact |
| remote PR gate | implementation head `0cbfbad2ef5ac156a521e994a7c09ee1137a2665` passed CI run [`30521741083`](https://github.com/DaoyuanLi2816/mini-verl/actions/runs/30521741083) and build run [`30521741098`](https://github.com/DaoyuanLi2816/mini-verl/actions/runs/30521741098): all ten Python/dependency/Transformers/CPU and extracted-sdist jobs are green |
| remaining gate | revalidate the checklist-only final head, merge only when green, then run the authorized tag-only public release and independent install/hash checks |
| publication authorization | the maintainer requested completion and publication of v0.2.4; tagging remains gated on the exact merge commit and all applicable checks |
| integration source | PR [#18](https://github.com/DaoyuanLi2816/mini-verl/pull/18) was squash-merged as `57dec193af88b462dcc41d82fc6fecb813e161fd`; annotated tag `v0.2.4` resolves to that exact commit |
| version transition | `v0.2.4` is immutable and public; post-release development identifies as `0.2.5.dev0` |
| immutable evidence | the v0.2 calculator benchmark remains byte-identical at SHA-256 `53fc1d4d5b7adee09618d77ad62d4086ba56b78569832d6fc7c3bcd5c2695bbc`; earlier tags and the pinned protocol-teacher adapter are unchanged |
| strict protocol and numerics | a shared bounded strict decoder rejects non-finite values, duplicate keys, 128+ digit integers, depth over 32, more than 256 members and invalid surrogates; protocol-v2 requires complete numeric/unit verification while historical protocol/verifier-v1 remains explicit |
| lifecycle and locking | `TrainerState` has atomic one-shot entry and stable terminal behavior; `filelock` protects run construction, resume, overwrite, evaluation, report and export across processes before model loading |
| packaging and provenance | tag-pinned PyPI documentation, self-testing sdist, lean wheel, exact submitted/validated/resolved configuration layers and canonical portable public artifacts passed the release gates; exact CPU/GPU/network evidence rem 10000 ains in [`docs/generated/quality.json`](docs/generated/quality.json), bound to implementation commit `6911acf11978cfc5f6a99375cbe7a1666fcf7a85` |
| release execution | tag run [`30522484949`](https://github.com/DaoyuanLi2816/mini-verl/actions/runs/30522484949) passed metadata, tests, one-time build, OIDC publication and attestation generation; it ended red only because urllib received PyPI's browser challenge before public install and GitHub Release creation |
| verified recovery | PRs [#19](https://github.com/DaoyuanLi2816/mini-verl/pull/19) and [#20](https://github.com/DaoyuanLi2816/mini-verl/pull/20) hardened recovery and HTML-banner verification. Run [`30524088015`](https://github.com/DaoyuanLi2816/mini-verl/actions/runs/30524088015) bound the original artifacts to the immutable tag, verified hashes, pinned links and Sigstore attestations, installed the exact public version, and created the GitHub Release without rebuilding or re-uploading |
| published wheel | [`miniverl-0.2.4-py3-none-any.whl`](https://pypi.org/project/miniverl/0.2.4/), SHA-256 `3f5a239bbbd2f85217cf11f691fbb63f647092f67b82da4de38bd6907c5ab0f1` |
| published sdist | [`miniverl-0.2.4.tar.gz`](https://pypi.org/project/miniverl/0.2.4/), SHA-256 `03f0e844df2c91deed5c211cdd2dd598d22f03d59d99cd8e792a58211c0b2296` |
| independent verification | GitHub Release downloads reproduce both public hashes; a no-cache Python 3.10 install from `https://pypi.org/simple` reports `miniverl 0.2.4`, passes `doctor`, and keeps torch, Transformers, PEFT and bitsandbytes absent |
| publication authorization | granted by the maintainer and completed through Trusted Publishing without a long-lived PyPI token |
| exact blocker | none |

## v0.2.3 clarity and defensive-hardening release status

Expand Down
56 changes: 28 additions & 28 deletions PYPI.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<p align="center">
<img src="https://raw.githubusercontent.com/DaoyuanLi2816/mini-verl/v0.2.4/docs/banner.svg" alt="miniVERL — on-policy distillation for tool-using agents on one GPU" width="880">
<img src="https://raw.githubusercontent.com/DaoyuanLi2816/mini-verl/main/docs/banner.svg" alt="miniVERL — on-policy distillation for tool-using agents on one GPU" width="880">
</p>

<div align="center">
Expand All @@ -15,7 +15,7 @@
<p align="center">
<a href="https://pypi.org/project/miniverl/"><strong>PyPI package</strong></a> ·
<a href="#single-gpu-quickstart">Install &amp; train</a> ·
<a href="https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.4/docs/single-gpu-guide.md">Bring your own GPU</a> ·
<a href="https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/single-gpu-guide.md">Bring your own GPU</a> ·
<a href="#measured-result-protocol-aligned-opd-matches-sft">Measured result</a>
</p>

Expand Down Expand Up @@ -57,7 +57,7 @@ validate artifacts without downloading a multi-gigabyte ML stack; use
[Run the local demo](#local-toy-demo) ·
[Train on your GPU](#single-gpu-quickstart) ·
[Inspect the measured result](#measured-result-protocol-aligned-opd-matches-sft) ·
[Read the math](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.4/docs/math.md)
[Read the math](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/math.md)

## Why miniVERL exists

Expand Down Expand Up @@ -96,7 +96,7 @@ keeps the whole lifecycle in one readable single-GPU process.
| Calculator, JSON-navigation and SQLite environments | yes, deterministic with exact verifiers |
| Exact checkpoint/resume | yes, asserted parameter-for-parameter |
| Self-contained offline HTML report with token-level divergence | yes |
| Ray, FSDP, DeepSpeed, vLLM, VLMs, cross-tokenizer, PPO/GRPO | **no** — see [limitations](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.4/docs/limitations.md) |
| Ray, FSDP, DeepSpeed, vLLM, VLMs, cross-tokenizer, PPO/GRPO | **no** — see [limitations](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/limitations.md) |

## Measured result: protocol-aligned OPD matches SFT

Expand Down Expand Up @@ -126,28 +126,28 @@ behaviour; it diagnoses the missing qualification gate in that setup.
The historical gate and benchmark reused the same 24-task v0.2 `test` set.
Candidate A was prespecified and passed first try (no fallback tuning), but the
set was not untouched. Future selection uses `eval`; reporting uses `test`.
See [limitations](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.4/docs/limitations.md).
See [limitations](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/limitations.md).

OPD only ties SFT and takes 6.1× as much continuation time here (523.8 s versus
86.4 s). The task saturates; two seeds support neither significance nor a
general OPD advantage. See the
[full result and legacy transcript diagnosis](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.4/docs/rtx4080-baselines.md).
[full result and legacy transcript diagnosis](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/rtx4080-baselines.md).

![Two-seed protocol-teacher benchmark](https://raw.githubusercontent.com/DaoyuanLi2816/mini-verl/v0.2.4/docs/gpu-calc-hard-equal-update-v2.svg)
![Two-seed protocol-teacher benchmark](https://raw.githubusercontent.com/DaoyuanLi2816/mini-verl/main/docs/gpu-calc-hard-equal-update-v2.svg)

| Artifact | Role |
| --- | --- |
| [Default recipe](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.4/recipes/qwen_consumer_gpu_calc.yaml) | protocol-qualified default |
| [Schema-v2 benchmark](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.4/benchmarks/results/gpu-calc-hard-equal-update-v2.json) | frozen five-arm result |
| [Raw-teacher recipe](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.4/recipes/qwen_consumer_gpu_calc_raw_teacher.yaml) | historical control; not default |
| [Default recipe](https://github.com/DaoyuanLi2816/mini-verl/blob/main/recipes/qwen_consumer_gpu_calc.yaml) | protocol-qualified default |
| [Schema-v2 benchmark](https://github.com/DaoyuanLi2816/mini-verl/blob/main/benchmarks/results/gpu-calc-hard-equal-update-v2.json) | frozen five-arm result |
| [Raw-teacher recipe](https://github.com/DaoyuanLi2816/mini-verl/blob/main/recipes/qwen_consumer_gpu_calc_raw_teacher.yaml) | historical control; not default |

<details>
<summary>Historical 481-second raw-teacher smoke (schema v1)</summary>

On RTX 4080, 16 updates took 481 s, peaked at **4.25/4.76 GiB
allocated/reserved**, and moved 12-task success from **0% to 100%**. Cold start
did most of it (first OPD batch: 83.3%); this proves the pipeline, not OPD over
SFT. [Trace](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.4/docs/rtx4080-baselines.md).
SFT. [Trace](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/rtx4080-baselines.md).

</details>

Expand Down Expand Up @@ -224,7 +224,7 @@ bf16, while older CUDA cards such as Titan V use fp16. RTX 3070, Titan V,
RTX 4080 and RTX 5090-class cards all enter the same code path; only the
RTX 4080 result is measured here. Exact fit is governed by VRAM, model sizes,
drivers and token budgets, not the card's marketing name. See the
[`single-GPU guide`](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.4/docs/single-gpu-guide.md) before changing the recipe.
[`single-GPU guide`](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/single-gpu-guide.md) before changing the recipe.

```bash
git clone https://github.com/DaoyuanLi2816/mini-verl.git
Expand Down Expand Up @@ -281,7 +281,7 @@ Layer boundaries are strict, and the first layer never imports torch:
6. `evaluation/`, `reporting/` — measurement.
7. `cli.py` — a thin shell that calls one library function per command.

See [`docs/design.md`](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.4/docs/design.md).
See [`docs/design.md`](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/design.md).

## Exact versus top-k + tail

Expand Down Expand Up @@ -311,7 +311,7 @@ the teacher still runs a full forward pass to produce the hidden states. Reports
therefore say `teacher_queried_position_ratio`, never "teacher compute saved".

Top-k + tail targets are not a new idea — TRL's `ServerDistillationTrainer` has
`loss_top_k` with an optional tail bucket. See [`docs/math.md`](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.4/docs/math.md).
`loss_top_k` with an optional tail bucket. See [`docs/math.md`](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/math.md).

## Tool-token masking

Expand All @@ -336,18 +336,18 @@ documented — see `tests/unit/test_token_provenance.py`.
## Benchmark results

Every number below was produced by the commands in
[`docs/benchmarking.md`](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.4/docs/benchmarking.md) on the hardware recorded in each
[`docs/benchmarking.md`](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/benchmarking.md) on the hardware recorded in each
result file. Nothing is estimated or extrapolated.

* **RTX 4080, real models** — [`docs/rtx4080-baselines.md`](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.4/docs/rtx4080-baselines.md)
* **RTX 4080, real models** — [`docs/rtx4080-baselines.md`](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/rtx4080-baselines.md)
has measured peak VRAM, decode throughput, the full-recipe run, the two-seed
schema-v2 protocol-teacher comparison, and the preserved legacy comparison.
* **CPU, toy models** — `recipes/toy_cpu.yaml` moves task success from 0.0% to
91.7% in 192 s, and `benchmarks/results/` holds the legacy equal-update parity
run.
The parity run's accuracy differences are **within noise**; it exists to show
that all seven arms run to completion under identical budgets, not to rank
them. See [`benchmarks/README.md`](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.4/benchmarks/README.md) for why the toy
them. See [`benchmarks/README.md`](https://github.com/DaoyuanLi2816/mini-verl/blob/main/benchmarks/README.md) for why the toy
backend cannot rank methods.

## Installation
Expand Down Expand Up @@ -452,11 +452,11 @@ distribution before handing it over, and asserts that the result still trains.

For a standard frozen PEFT teacher adapter, including the Qwen3 protocol-SFT
recipe, export command, compatibility checks and policy-competence gate, see
[`docs/teacher-adapters.md`](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.4/docs/teacher-adapters.md).
[`docs/teacher-adapters.md`](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/teacher-adapters.md).

## Limitations

The short version; the full list is in [`docs/limitations.md`](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.4/docs/limitations.md).
The short version; the full list is in [`docs/limitations.md`](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/limitations.md).

* Same tokenizer only. Cross-tokenizer distillation is rejected with an error.
* One trajectory per forward pass — `gradient_accumulation_steps` *is* the batch
Expand All @@ -482,8 +482,8 @@ allowlist of ones that change numerics — asserted by a test.
File-backed runs also separate exact submitted bytes, canonical validated
logic, the v0.2 resume compatibility layer, and runtime-resolved choices.

See [`docs/reproducibility.md`](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.4/docs/reproducibility.md) and the concise
[`compatibility policy`](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.4/docs/compatibility.md).
See [`docs/reproducibility.md`](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/reproducibility.md) and the concise
[`compatibility policy`](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/compatibility.md).

## Roadmap

Expand All @@ -504,7 +504,7 @@ multi-turn tool use — at cluster scale, with Ray. If you have a cluster, use i
miniVERL exists for the case where you have one personal GPU and want to read
every line of what is happening. That can be an older 12 GiB card or a current
high-end card; the repository claims measured performance only for hardware it
actually ran. See [`docs/comparisons.md`](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.4/docs/comparisons.md).
actually ran. See [`docs/comparisons.md`](https://github.com/DaoyuanLi2816/mini-verl/blob/main/docs/comparisons.md).

## Citation

Expand All @@ -518,13 +518,13 @@ actually ran. See [`docs/comparisons.md`](https://github.com/DaoyuanLi2816/mini-
}
```

See [CITATION.cff](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.4/CITATION.cff) and [CHANGELOG.md](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.4/CHANGELOG.md).
Contributions: [CONTRIBUTING.md](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.4/CONTRIBUTING.md). Security:
[SECURITY.md](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.4/SECURITY.md).
See [CITATION.cff](https://github.com/DaoyuanLi2816/mini-verl/blob/main/CITATION.cff) and [CHANGELOG.md](https://github.com/DaoyuanLi2816/mini-verl/blob/main/CHANGELOG.md).
Contributions: [CONTRIBUTING.md](https://github.com/DaoyuanLi2816/mini-verl/blob/main/CONTRIBUTING.md). Security:
[SECURITY.md](https://github.com/DaoyuanLi2816/mini-verl/blob/main/SECURITY.md).

## License

Apache-2.0. See [LICENSE](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.4/LICENSE) and
[THIRD_PARTY_NOTICES.md](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.4/THIRD_PARTY_NOTICES.md).
Apache-2.0. See [LICENSE](https://github.com/DaoyuanLi2816/mini-verl/blob/main/LICENSE) and
[THIRD_PARTY_NOTICES.md](https://github.com/DaoyuanLi2816/mini-verl/blob/main/THIRD_PARTY_NOTICES.md).

Chinese translation: [README.zh-CN.md](https://github.com/DaoyuanLi2816/mini-verl/blob/v0.2.4/README.zh-CN.md).
Chinese translation: [README.zh-CN.md](https://github.com/DaoyuanLi2816/mini-verl/blob/main/README.zh-CN.md).
2 changes: 1 addition & 1 deletion docs/generated/quality.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"schema_version": 1,
"release": "0.2.4",
"status": "validated",
"status": "released",
"measured_commit": "6911acf11978cfc5f6a99375cbe7a1666fcf7a85",
"measured_at": "2026-07-30T00:04:29-07:00",
"cpu_non_gpu_non_network": {
Expand Down
Loading
0