8000 Harden miniVERL v0.2.4 framework boundaries by DaoyuanLi2816 · Pull Request #18 · DaoyuanLi2816/mini-verl · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 25 additions & 1 deletion .github/ISSUE_TEMPLATE/benchmark_submission.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,10 @@ body:
```

`export-benchmark` strips absolute paths and keeps only the GPU model,
VRAM, OS family and library versions. Please read it before posting.
VRAM, OS family and library versions. Validate it against
`benchmarks/schema/benchmark-result.schema.json`, compute its SHA-256,
and read it before posting. Never include tokens, private paths or
proprietary data.

- type: input
id: hardware
Expand Down Expand Up @@ -52,6 +55,23 @@ body:
validations:
required: false

- type: input
id: artifact_hash
attributes:
label: Exported JSON SHA-256
description: Hash the exact JSON attached to this submission.
placeholder: "64 lowercase hexadecimal characters"
validations:
required: true

- type: textarea
id: software
attributes:
label: Software provenance
description: miniVERL, Python, torch, CUDA and driver versions from the sanitized export.
validations:
required: true

- type: checkboxes
id: honesty
attributes:
Expand All @@ -61,5 +81,9 @@ body:
required: true
- label: I ran the recipe to completion and did not select the best of several runs without saying so.
required: true
- label: I validated the JSON against the committed schema and recorded the exact artifact SHA-256.
required: true
- label: I reviewed the export and attachments for tokens, private paths and proprietary data.
required: true
- label: I am willing to open a pull request adding this file under `benchmarks/results/`.
required: false
11 changes: 6 additions & 5 deletions .github/ISSUE_TEMPLATE/bug_report.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ body:
- type: markdown 10000
attributes:
value: |
Before filing, please run `miniverl doctor --json` and paste the output
below. It rules out half of the possible causes in one step.
Before filing, run `miniverl doctor --json`. Review every attachment:
do not paste tokens, home paths, proprietary prompts or model data.

- type: textarea
id: what-happened
Expand Down Expand Up @@ -38,10 +38,11 @@ body:
- type: textarea
id: manifest
attributes:
label: Contents of `runs/<run-id>/manifest.json`
label: Sanitized config and manifest
description: >-
Optional but very useful. It contains no personal data by design -- no
username, hostname or home path.
Optional but useful: config.validated.yaml and manifest.json with local
paths, model data and identifiers replaced. Do not paste raw private run
files merely because they are JSON/YAML.
render: json
validations:
required: false
Expand Down
5 changes: 4 additions & 1 deletion .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@

<!--
Paste the actual output, not a description of it. A claim without a command is
not evidence.
not evidence. Sanitize commands and output first: do not paste tokens, home
paths, proprietary prompts or model data.
-->

```text
Expand All @@ -31,6 +32,8 @@ pytest -q -m "not gpu and not network"
`schema_version` was bumped and the reader rejects the old version.
- [ ] If this changes what a user sees, the README and the relevant `docs/` page
were updated in the same commit.
- [ ] Configs, manifests, reports and logs pasted here were reviewed for tokens,
private paths and proprietary data.

## If this touches the objective, the masks, or the cache

Expand Down
72 changes: 70 additions & 2 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,14 +26,26 @@ jobs:
python -m twine check dist/*
ls -la dist/

- name: The sdist must contain the recipes and the tests
- name: The sdist must contain every reproducibility surface
run: |
python - <<'PY'
import glob, tarfile
[path] = glob.glob("dist/*.tar.gz")
with tarfile.open(path) as archive:
names = archive.getnames()
for needed in ("recipes/toy_cpu.yaml", "tests/conftest.py", "LICENSE", "README.md"):
for needed in (
"recipes/toy_cpu.yaml",
"tests/conftest.py",
"scripts/build_pypi_readme.py",
".github/workflows/ci.yml",
"PYPI.md",
"CONTRIBUTING.md",
"CODE_OF_CONDUCT.md",
"SECURITY.md",
"PROJECT_STATE.md",
"LICENSE",
"README.md",
):
assert any(n.endswith(needed) for n in names), f"{needed} missing from the sdist"
print("sdist contents look right")
PY
Expand Down Expand Up @@ -108,6 +120,62 @@ jobs:
test -f /tmp/wheel-demo/report.html
deactivate

- name: Reproduce tests and wheel from the extracted sdist
run: |
python - <<'PY'
import glob, pathlib, tarfile
[archive_path] = glob.glob("dist/*.tar.gz")
destination = pathlib.Path("/tmp/miniverl-sdist-source")
destination.mkdir()
with tarfile.open(archive_path) as archive:
archive.extractall(destination, filter="data")
[root] = destination.iterdir()
pathlib.Path("/tmp/miniverl-sdist-root.txt").write_text(str(root))
PY
SDIST_ROOT="$(cat /tmp/miniverl-sdist-root.txt)"
python -m venv /tmp/sdist-test
. /tmp/sdist-test/bin/activate
python -m pip install --upgrade pip
python -m pip install torch --index-url https://download.pytorch.org/whl/cpu
python -m pip install "${SDIST_ROOT}[dev,train]"
cd "$SDIST_ROOT"
unset PYTHONPATH
python -c "import miniverl, pathlib; assert pathlib.Path(miniverl.__file__).is_relative_to(pathlib.Path('/tmp/sdist-test'))"
python scripts/build_pypi_readme.py --check
ruff check .
ruff format --check .
mypy src/miniverl
pytest -q -m "not gpu and not network"
python -m build --wheel --outdir /tmp/sdist-wheel
cd "$GITHUB_WORKSPACE"
deactivate

- name: The sdist-built wheel must match the repository wheel inventory
run: |
python - <<'PY'
import glob, zipfile

[repository_wheel] = glob.glob("dist/*.whl")
[sdist_wheel] = glob.glob("/tmp/sdist-wheel/*.whl")

def package_inventory(path):
with zipfile.ZipFile(path) as archive:
return {
name
for name in archive.namelist()
if name.startswith("miniverl/")
}

repository = package_inventory(repository_wheel)
from_sdist = package_inventory(sdist_wheel)
assert repository == from_sdist, (
f"repository- - from_sdist)}; "
f"sdist- - repository)}"
)
assert not any(name.startswith(("tests/", "scripts/")) for name in repository)
print(f"both wheels contain the same {len(repository)} package files")
PY

- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: dist
Expand Down
10 changes: 10 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,14 @@ jobs:
- name: mypy
run: mypy src/miniverl

- name: actionlint
if: matrix.python-version == '3.12'
run: |
curl -fsSLO https://github.com/rhysd/actionlint/releases/download/v1.7.12/actionlint_1.7.12_linux_amd64.tar.gz
echo "8aca8db96f1b94770f1b0d72b6dddcb1ebb8123cb3712530b08cc387b349a3d8 actionlint_1.7.12_linux_amd64.tar.gz" | sha256sum --check
tar -xzf actionlint_1.7.12_linux_amd64.tar.gz actionlint
./actionlint

- name: pytest (torch-free subset)
run: pytest -q -m "not torch and not gpu and not network" -ra

Expand All @@ -69,6 +77,8 @@ jobs:

- name: CLI smoke (base install)
run: |
python scripts/build_pypi_readme.py --check
python scripts/check_markdown_links.py
miniverl --help
miniverl --version
miniverl doctor --json > doctor.json
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,9 @@ jobs:
ruff check .
ruff format --check .
mypy src/miniverl
python scripts/build_pypi_readme.py --check
python scripts/check_markdown_links.py
echo "53fc1d4d5b7adee09618d77ad62d4086ba56b78569832d6fc7c3bcd5c2695bbc benchmarks/results/gpu-calc-hard-equal-update-v2.json" | sha256sum --check
pytest -q -m "not gpu and not network" \
--cov=miniverl --cov-report=term-missing --cov-fail-under=80

Expand Down
51 changes: 50 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,54 @@ All notable changes to miniVERL are recorded here. The format follows

## [Unreleased]

## [0.2.4] - 2026-07-29

Correctness, adversarial-input, concurrency, packaging and privacy hardening
release. No training objective, environment, model family or scientific result
changed.

### Added

- A bounded strict-JSON boundary for model-generated tool calls, including
duplicate-key, non-finite-number, oversized-integer, excessive-depth and
excessive-member rejection.
- An explicit one-shot trainer lifecycle and cross-platform, process-safe run
ownership acquired before mutable resume, overwrite, evaluation, report or
export work.
- Exact submitted, canonical validated and runtime-resolved configuration
provenance layers with checksums, plus a canonical portable redaction view
for shareable artifacts.
- A generated PyPI long description with immutable release links, an
extracted-sdist self-test gate and a concise public compatibility policy.

### Changed

- Protocol-v2 numeric verification consumes a complete finite answer and
validates supported unit suffixes; protocol/verifier-v1 remains explicitly
historical so existing artifacts are not reinterpreted.
- Calculator, JSON-navigation and SQLite boundaries now convert adversarial
model inputs into bounded parse, tool or verification failures instead of
leaking built-in numeric/serialization exceptions.
- Machine JSON and JSONL writers reject non-finite values before publication.
Shareable reports, summaries and benchmark exports redact paths, identities,
secrets and environment references by default.
- The source distribution now includes the repository fixtures required by its
shipped tests, while the wheel remains runtime-only.
- Exact release-quality evidence has one generated machine-readable record;
other documentation uses a stable floor or links to that record.

### Fixed

- A trainer instance can no longer train twice or admit two threads into
training, and a failed second call mutates no run artifact.
- Competing processes can no longer mutate the same run or overwrite a run
while another writer owns it; abandoned OS locks do not permanently block a
later process.
- File-backed recipes preserve their submitted UTF-8 bytes and comments instead
of labeling a normalized, path-resolved reserialization as verbatim input.
- PyPI documentation links and images no longer depend on relative repository
paths or a moving branch for stable releases.

## [0.2.3] - 2026-07-29

Clarity and defensive-hardening release.
Expand Down Expand Up @@ -317,7 +365,8 @@ Same-tokenizer only; one trajectory per forward pass; `swap` unavailable for
quantized models; only Qwen3 and Qwen2 architectures tested; single-seed GPU
results. The full list is in `docs/limitations.md`.

[Unreleased]: https://github.com/DaoyuanLi2816/mini-verl/compare/v0.2.3...HEAD
[Unreleased]: https://github.com/DaoyuanLi2816/mini-verl/compare/v0.2.4...HEAD
[0.2.4]: https://github.com/DaoyuanLi2816/mini-verl/compare/v0.2.3...v0.2.4
[0.2.3]: https://github.com/DaoyuanLi2816/mini-verl/compare/v0.2.2...v0.2.3
[0.2.2]: https://github.com/DaoyuanLi2816/mini-verl/compare/v0.2.1...v0.2.2
[0.2.1]: https://github.com/DaoyuanLi2816/mini-verl/compare/v0.2.0...v0.2.1
Expand Down
4 changes: 2 additions & 2 deletions CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ cff-version: 1.2.0
title: "miniVERL: On-policy distillation for tool-using agents on one GPU"
message: "If you use miniVERL in your work, please cite it as below."
type: software
version: 0.2.3
version: 0.2.4
date-released: 2026-07-29
license: Apache-2.0
repository-code: "https://github.com/DaoyuanLi2816/mini-verl"
Expand Down Expand Up @@ -98,4 +98,4 @@ references:
url: "https://arxiv.org/abs/2603.07079"
notes: >-
Motivates recording per-token teacher entropy. Entropy-aware divergence
mixing is a roadmap item and is not implemented in miniVERL v0.2.3.
mixing is a roadmap item and is not implemented in miniVERL v0.2.4.
15 changes: 14 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@ git clone https://github.com/DaoyuanLi2816/mini-verl
cd mini-verl

python -m venv .venv && . .venv/bin/activate # or: uv venv
pip install -e ".[dev,train]" # CPU torch is enough
pip install torch --index-url https://download.pytorch.org/whl/cpu
pip install -e ".[dev,train]" # CPU test stack
# optional, for the 4-bit path:
pip install -e ".[cuda]"

Expand All @@ -35,6 +36,8 @@ ruff check .
ruff format --check .
mypy src/miniverl
pytest -q -m "not gpu and not network"
python -m build
python -m twine check dist/*
```

If you have a CUDA device:
Expand All @@ -43,6 +46,10 @@ If you have a CUDA device:
pytest -q -m gpu
```

Start with the focused test file for the code you changed, then run the full
CPU gate above. Network and GPU tests are opt-in; state `not run` when the
required hardware or credentials are unavailable.

## What a good change looks like

* **A test that can fail.** Not `assert True`, not a test that passes because
Expand Down Expand Up @@ -103,6 +110,12 @@ library versions, and drops absolute paths and anything identifying. Read the
file before you post it. Open a pull request adding it under
`benchmarks/results/`; `benchmarks/README.md` explains the schema.

Good first contributions are bounded fixtures and compatibility checks:
documentation corrections, a deterministic adversarial case for an existing
environment, or an old-artifact reader regression. Changes to objectives,
numerical reductions, cache transactions or checkpoint semantics need a
maintainer-designed test plan first.

## Schema changes

`Trajectory`, the teacher cache and the benchmark result all carry a
Expand Down
16 changes: 16 additions & 0 deletions PROJECT_STATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,22 @@ and what it printed.

Last updated: 2026-07-29.

## v0.2.4 final framework-hardening work in progress

| item | current state |
| --- | --- |
| audited starting point | fetched local and public `main` both resolve to `ac047a6001ae0bc2935853fa4c35cedbcd9b9ee4`; working branch is `agent/v024-final-hardening` |
| release-candidate version | metadata, generated PyPI description and clean wheel installs now identify as `0.2.4`; no tag exists and public PyPI remains `0.2.3` until the PR head and merge commit pass every remote gate |
| immutable evidence | the v0.2 calculator benchmark must remain byte-identical at SHA-256 `53fc1d4d5b7adee09618d77ad62d4086ba56b78569832d6fc7c3bcd5c2695bbc`; release tags and the pinned protocol-teacher adapter are out of scope for mutation |
| strict protocol and numerics | regressions first reproduced 33 failures: Python JSON accepted non-finite/duplicate/pathological values, calculator conversion leaked `ValueError`/`OverflowError`, and prefix-only verification accepted contradictory suffixes. A shared bounded strict decoder now rejects non-finite values, duplicate keys, 128+ digit integers, depth over 32, more than 256 members and invalid surrogates; renderers use `allow_nan=False` semantics, protocol-v2 uses complete numeric/unit verification, and historical protocol/verifier-v1 remains explicit |
| lifecycle and locking | `TrainerState` enforces `ready -> running -> completed/failed/interrupted -> closed` with an atomic one-shot transition; a second call changes no bytes and concurrent threads cannot both enter. `filelock` backs stable `<output-root>/.miniverl-locks/<run-id>.lock` ownership acquired before run access/model loading and released after teardown; fail-fast, timeout, killed-owner and distinct-run multiprocessing cases pass on Windows |
| focused evidence | regressions cover strict protocol/numerics, run locking, one-shot lifecycle, exact config layers, portable reports and wheel metadata; exact local CPU/GPU/network totals and branch coverage are recorded only in [`docs/generated/quality.json`](docs/generated/quality.json), bound to implementation commit `6911acf11978cfc5f6a99375cbe7a1666fcf7a85` |
| packaging and provenance | PyPI markdown is generated with tag-stable links; the sdist carries its complete repository-test surface and self-tests outside the checkout; submitted/validated/legacy-runtime/resolved config layers have distinct bytes and digests; public reports use the canonical portable view |
| local release gate | ruff, format, mypy, actionlint, Markdown/anchor checks, generated-file checks, full CPU coverage, available RTX 4080 GPU tests, network tests, wheel/sdist build plus twine, and the complete extracted-sdist CPU suite pass; the frozen benchmark hash remains exact |
| remote PR gate | implementation head `0cbfbad2ef5ac156a521e994a7c09ee1137a2665` passed CI run [`30521741083`](https://github.com/DaoyuanLi2816/mini-verl/actions/runs/30521741083) and build run [`30521741098`](https://github.com/DaoyuanLi2816/mini-verl/actions/runs/30521741098): all ten Python/dependency/Transformers/CPU and extracted-sdist jobs are green |
| remaining gate | revalidate the checklist-only final head, merge only when green, then run the authorized tag-only public release and independent install/hash checks |
| publication authorization | the maintainer requested completion and publication of v0.2.4; tagging remains gated on the exact merge commit and all applicable checks |

## v0.2.3 clarity and defensive-hardening release status

| item | current state |
Expand Down
Loading
0