8000 v0.16.0: native verl Hydra input and deterministic sampling by DaoyuanLi2816 · Pull Request #123 · DaoyuanLi2816/mini-verl · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,9 @@ jobs:
miniverl doctor --json > /tmp/doctor.json
python -c "import json; d=json.load(open('/tmp/doctor.json')); assert d['verdict']['core_commands'] is True"
python -c "import miniverl, sys; assert 'torch' not in sys.modules"
python -m pip install "$(echo dist/*.whl)[hydra]"
miniverl run --example hydra-grpo --dry-run --json > /tmp/hydra-core.json
python -c "import json; d=json.load(open('/tmp/hydra-core.json')); assert d['composition_status']=='resolved' and d['semantic_status']=='accepted'"
deactivate

- name: Install the training extra into a second clean environment
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,7 @@ jobs:
python scripts/publish_verl_direct_schema.py --check
python scripts/publish_direct_examples.py --check
python scripts/publish_direct_compatibility.py --check
python scripts/publish_hydra_compatibility.py --check
python scripts/check_markdown_links.py
python scripts/check_text_integrity.py
miniverl --help
Expand Down
10 changes: 9 additions & 1 deletion .github/workflows/gpu.yml
Original file line number Diff line number Diff line change
Expand Up @@ -214,6 +214,13 @@ jobs:
--output "$full_dir/v015-direct.json" \
--work "$full_work/v015-direct" \
"${offline[@]}"
"$QUALIFICATION_PYTHON" scripts/run_v016_hydra_qualification.py \
--commit "$GITHUB_SHA" \
--wheel "$QUALIFICATION_WHEEL" \
--wheel-sha256 "$QUALIFICATION_WHEEL_SHA256" \
--output "$full_dir/v016-hydra.json" \
--work "$full_work/v016-hydra" \
"${offline[@]}"
test -z "$(git status --porcelain)"
"$QUALIFICATION_PYTHON" scripts/benchmark_rollout_runtime_v2_selected.py \
--backend hf_cached \
Expand All @@ -239,7 +246,8 @@ jobs:
--v012-rl "$full_dir/v012-rl.json" \
--v013-ppo "$full_dir/v013-ppo.json" \
--v014-product "$full_dir/v014-product.json" \
--v015-direct "$full_dir/v015-direct.json"
--v015-direct "$full_dir/v015-direct.json" \
--v016-hydra "$full_dir/v016-hydra.json"
- name: Upload full qualification records
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: ${{ success() && inputs.qualification_level == 'full' }}
Expand Down
6 changes: 4 additions & 2 deletions .github/workflows/verl-bridge.yml
Original file line number Diff line number Diff line change
Expand Up @@ -84,10 +84,12 @@ jobs:
- name: Compare GRPO, Dr.GRPO, RLOO, REINFORCE++, GAE, policy and value math
env:
MINIVERL_VERL_V09_SOURCE: _verl-v0.9.0
run: python -m pytest -q tests/conformance/test_verl_v09_rl.py -m verl_conformance
run: python -m pytest -q tests/conformance/test_verl_v09_rl.py tests/conformance/test_verl_v09_sampling.py -m verl_conformance
- name: Regenerate the real upstream compatibility corpus
run: >-
run: |
python scripts/publish_verl_compatibility_corpus.py --upstream _verl-v0.9.0 --check
python scripts/publish_verl_hydra_tree.py --upstream _verl-v0.9.0 --check
python scripts/publish_hydra_compatibility.py --check
- name: Verify packaged PPO and GRPO workflows
run: python scripts/publish_rl_product_examples.py --check
- name: Materialize and recheck the pure-OPD bundle
Expand Down
23 changes: 22 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,26 @@ All notable changes to miniVERL are recorded here. The format follows

## [Unreleased]

## [0.16.0] - 2026-09-11

### Native verl Hydra ingestion and single-GPU sampling

- Compose the pinned verl v0.9.0 tree, config name and ordered Hydra overrides
directly through `miniverl run`; retain resolved-YAML v4 and all older identities.
- Package the original 52 upstream YAML files and use Hydra 1.3.2 in a narrow
isolated adapter. Record defaults, exact overrides, tree hashes and resolved bytes.
- Add v5 actor/critic DataLoader shuffling, raw-metric group filtering,
synchronous refill and a separately bound local attempt guard. Preserve
logical batches, policy freshness and exact cycle replay through OOM downshifts.
- Distinguish unused legacy oversampling knobs from the pinned V1 refill logic;
record deterministic local tie-breaking for equal-age surplus groups.
- Add a 16-case native-composition audit, upstream ordering/filter conformance,
filtered-run inspection and provenance-preserving v5 handoff templates.
- Gate release promotion and canonical evidence archives on exact-wheel native
Hydra PPO/GRPO workflows, tensor replay, measured memory and inspected handoff.
- Update English/Chinese entry points and keep published stable docs available
while the next release is being qualified.

## [0.15.0] - 2026-09-10

### Direct verl configurations
Expand Down Expand Up @@ -1268,7 +1288,8 @@ Same-tokenizer only; one trajectory per forward pass; `swap` unavailable for
quantized models; only Qwen3 and Qwen2 architectures tested; single-seed GPU
results. The full list is in `docs/limitations.md`.

[Unreleased]: https://github.com/DaoyuanLi2816/mini-verl/compare/v0.15.0...HEAD
[Unreleased]: https://github.com/DaoyuanLi2816/mini-verl/compare/v0.16.0...HEAD
[0.16.0]: https://github.com/DaoyuanLi2816/mini-verl/compare/v0.15.0...v0.16.0
[0.15.0]: https://github.com/DaoyuanLi2816/mini-verl/compare/v0.14.0...v0.15.0
[0.14.0]: https://github.com/DaoyuanLi2816/mini-verl/compare/v0.13.0...v0.14.0
[0.13.0]: https://github.com/DaoyuanLi2816/mini-verl/compare/v0.12.0...v0.13.0
Expand Down
4 changes: 2 additions & 2 deletions CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@ cff-version: 1.2.0
title: "miniVERL: Run verl experiment semantics on one GPU"
message: "If you use miniVERL in your work, please cite it as below."
type: software
version: 0.15.0
date-released: 2026-09-10
version: 0.16.0
date-released: 2026-09-11
license: Apache-2.0
repository-code: "https://github.com/DaoyuanLi2816/mini-verl"
url: "https://github.com/DaoyuanLi2816/mini-verl"
Expand Down
17 changes: 13 additions & 4 deletions PROJECT_STATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,21 @@ Current maintainer handoff for **miniVERL** (`mini-verl` package, `miniverl`
CLI). `release-state.yaml` is the canonical version source; this page indexes
current product and evidence state rather than repeating release history.

Last updated: 2026-09-10.
Last updated: 2026-09-11.

Canonical release state: stable `v0.15.0` (`659690d92a9ed92a01d1beec0a75873de8b860d5`), development `0.15.1.dev0`.
Canonical release state: releasing `v0.16.0`.

## Release state

- The v0.16 work adds native pinned Hydra input and immutable v5 sampling
semantics. Its 16-case audit has 14 successful compositions and 12 semantic
acceptances, with explicit composition/semantic rejection cases.
- Development installed-wheel RTX 4080 rehearsal passed native PPO and GRPO,
shuffled epochs, real GRPO filtering, exact tensor replay and handoff. This
rehearsal does not replace the mandatory exact-release candidate qualification.

### Previous stable release: v0.15.0

- v0.15.0 is published on PyPI and GitHub. The release uses the exact candidate
from full RTX 4080 qualification `34442828628` (attempt 1), accepted by
rehearsal `34447536443` and OIDC publication `34448233909`.
Expand Down Expand Up @@ -56,8 +65,8 @@ Canonical release state: stable `v0.15.0` (`659690d92a9ed92a01d1beec0a75873de8b8

## Current product boundary

Development `0.15.1.dev0` has a closed typed profile compiler: it retains v1/v2/v3 and adds
`verl-rl-v0.9-single-gpu-v4`, direct execution of resolved RL inputs from official verl
Release `0.16.0` has a closed typed profile compiler: it retains v1/v2/v3/v4 and adds
`verl-rl-v0.9-single-gpu-v5`, native Hydra ingestion and deterministic sampling from verl
`v0.9.0` at commit `483b8a009ba3a97563edee3a19887e4862b8094a`.
It compiles PPO/GAE into phased one-GPU actor, critic, reference and reward
roles. The critic is independently trainable and owns its optimizer, schedule,
Expand Down
Loading
0