Living build log for miniVERL (mini-verl / miniverl / CLI miniverl).
A checkbox is not evidence: every completed item names the command that was run
and what it printed.
Last updated: 2026-08-13.
Canonical release state: releasing v0.9.0.
Every public version claim is generated from release-state.yaml and gated by
python scripts/release_state.py --check.
Development is now focused on making the bounded single-GPU verl-style OPD path easier to configure, inspect, reproduce and materialize. The v0.8.1 release remains the stable product surface; no frozen research result is being reinterpreted as v0.9 evidence.
The first v0.9 slice accepts safe trailing dotted overrides, repeatable plain
files and JSON argv arrays with base → files → --set → trailing precedence.
Every duplicate retains its source, prior value and final effective value.
External configs require explicit acceptance of the displayed high-risk local
reinterpretations; the packaged Qwen profile's approval is value-bound and
fails closed if one of those values drifts. A generated report classifies all
82 leaves in the pinned official-example field fixture: 23 exact, 16
semantically conformant, 22 locally reinterpreted, 18 informational and 3
unsupported. No algorithm or distributed boundary widened.
The second slice adds deterministic immutable OPD plans. plan --out hashes
the source config and every Parquet file, scans schema/content/row identity,
requires immutable model commits and seals the accepted compatibility result,
native config and weight-free recommendations. run --plan revalidates those
bytes without recompiling; the canonical plan digest propagates into the run
manifest, teacher cache and every checkpoint. Tokenizer structural identity is
truthfully deferred until tokenizers are loaded by the later bounded probe.
The bounded probe is now implemented and was exercised on the local RTX 4080. It loads the exact pinned Qwen pair sequentially, measures tiny actor rollout, teacher score and selected-position backward phases, performs zero optimizer updates and refuses results that do not release role allocation near baseline. Probe caches bind hardware/software, model, tokenizer, quantization, token and plan identities; cached and fresh publication of the same measurement produces byte-identical immutable plans.
The scale-out slice adds transactional bridge materialize. A fresh export
continues to carry identity-only bases and remains non-launchable. Materialize
accepts exact regular-file snapshots or resolves the recorded 40-character
commits through download/cache mode, validates shard closure, tokenizer and
model loads, student PEFT, Parquet, top-k and the exact installed verl config,
then publishes launch.sh and full file hashes through a staged directory
swap. Teacher-adapter merging is an explicit, non-mutating operation with base,
adapter, software and output provenance. launchable: true means only that the
pinned upstream entry point has complete local inputs; distributed execution
remains false and algorithm parity remains unclaimed.
A second-family compatibility smoke used pinned Apache-2.0 SmolLM2-360M and SmolLM2-1.7B snapshots. Structural tokenizer identity matched; one strict forward-top-k rollout, teacher-score, update and PEFT reload completed on the RTX 4080 at 1.416 GiB peak reserved VRAM. This is a tiny compatibility check, not a second supported recipe or task-quality result.
The landing pages now lead with the documented one-GPU verl-style OPD journey,
a responsive compiler/runtime/artifact diagram and a direct migration mapping.
docs/for-verl-users.md explains reusable config/data, local role semantics and
the scale-out boundary. The packaged Qwen3 profile now carries familiar
name: vllm source values while the generated compatibility matrix continues
to classify both engines as local-HF reinterpretations. No scientific result,
algorithm, model revision or distributed claim changed.
Development starts with the config-only profile
verl-opd-v0.8-single-gpu-v1, pinned to verl v0.8.0 at
7aed6b230776f963fa09509c10d9c3a767d1102c. Typed models compile resolved YAML
and repeatable dotted overrides into a deterministic, checksummed field report.
The report distinguishes exact, semantically conformant, locally reinterpreted,
derived, informational-only and unsupported fields. PG OPD, task-reward
mixtures, KL penalties, multiple generations, multiple teachers and
distributed dimensions fail closed. bridge compile-opd is deliberately an
offline config compiler: it loads no weights and does not claim the prompt
runtime or verl engine equivalence implemented by later PRs.
The trainer now has a discriminated environment / verl_parquet source
contract. Parquet rows stream by record batch, reject every invalid row, retain
the supported verl metadata and record schema, content, row, tokenizer and
rendered-prompt digests. Pure prompt OPD needs neither a ToolEnvironment nor
a reward model; task rewards, oracle SFT and privileged teacher context fail
closed instead of being ignored.
PromptDatasetRolloutRuntime applies the actor chat template once, performs
real masked padded greedy generation, restores logical row order and creates
trajectories whose selected model spans contain response tokens only. A
bounded padded-token budget is enforced before allocation. CUDA OOM retry may
split physical batches but never changes logical batch size, seeds or training
configuration. The existing multi-turn environment runner is retained behind
ToolEnvironmentRolloutRuntime and its prior integration suite remains green.
Focused validation on 2026-08-11: 17 new config/data/runtime/end-to-end tests, 94 existing trainer/toy-pipeline tests, Ruff, and mypy over 126 source files. The pure-OPD integration executes two Parquet prompts through rollout, teacher scoring, a padded actor update, checkpoint and manifest without an environment or reward.
The compatibility loss is now a separate forward_kl_topk implementation;
native bucketed_topk_tail is unchanged. The supported profile requires
forward KL, upstream-compatible token-mean aggregation, temperature 1.0 and
no sampled-token NLL mixture. Deterministic tensor tests import the official
verl v0.8.0 loss implementation at pinned commit
7aed6b230776f963fa09509c10d9c3a767d1102c and compare per-token loss,
teacher/student top-k mass, overlap diagnostics, the reduced scalar and student
gradients at rtol=1e-6, atol=1e-7.
Teacher cache entries now bind the prompt-row digest, exact actor response token IDs and policy version; the cache identity additionally binds teacher, tokenizer, top-k, temperature and score-implementation version. A mismatched response or implementation fails closed. The prompt-data integration exercises rollout, exact-state scoring, cache reload, one token-mean optimizer update and the emitted verl diagnostics.
miniverl plan now compiles a path or the packaged
builtin:qwen3-0.6b-1.7b-opd profile without importing torch or allocating
weights. Estimated memory, disk and placement stay distinct from measured and
unknown values. Auto placement uses parsed model-scale metadata and configured
headroom, never a GPU-name allowlist; unknown sizes conservatively select swap.
miniverl run validates a generated native RunConfig, preserves the source,
field report and local plan in the run, and executes the supported prompt OPD
path. miniverl data sample makes reward-free verl-style message Parquet from
an installed wheel.
One RTX 4080 runtime-conformance run at d441c45 used pinned Qwen3-0.6B /
Qwen3-1.7B NF4 snapshots, two prompts, 16 response tokens each, top-k 32 and
one optimizer update. Peak reserved memory was 3.1758 GiB; first rollout,
teacher target batch and update completed at 10.8684 s, 11.0630 s and 12.0224 s
from construction start. The standard PEFT adapter load passed. This is a
systems result only; no alignment endpoint or method comparison ran. The
machine-readable record is benchmarks/results/rtx4080-verl-opd-runtime-v1.json.
import-verl --profile verl-opd-v0.8-single-gpu-v1 --config ... now publishes
a transactional, canonical OPD profile plus a field-by-field report; it uses
the same typed compiler as plan/run, consumes Parquet paths and needs no
ToolEnvironment or reward. The legacy environment-profile importer remains
available under its original profile name.
export-verl recognizes a completed compatible OPD run and emits the standard
student PEFT adapter, exact student/teacher identities, byte-preserved Parquet,
source config, compiled plan and pure forward_kl_topk overrides. It emits no
reward scaffold. Student/teacher/data loadability, upstream parse/tiny-smoke,
launchability and distributed execution are independent fields. Missing base
snapshots and teacher-adapter materialization keep new bundles fail-closed and
launchable: false; no distributed command or execution claim is generated.
The generated docs/generated/verl-opd-v0.8-compatibility.json binds 72 source
fields to the pinned compiler fixture. Upstream parse conformance is recomputed
against verl 7aed6b23; it does not imply a model launch or distributed job.
Branch v0.7.1-product-correction starts from synchronized main
3ec042e829b4193aab363ab97e3786a1cd2621c0. It changes product hierarchy,
installed evidence UX and metadata only; no scientific experiment, frozen
result, model revision or existing tag changes.
The wheel carries the typed v0.7 result, schema, preregistration and 512
privacy-safe JSONNav rows. Each file is checked against its recorded SHA-256,
the result is parsed through the impossible-state validator, task rows are
structurally checked, and pilot --builtin-study exposes the early-stop result
without a checkout. The planned verl-shaped executable OPD profile remains a
v0.8 development objective and is not claimed by this release.
Local release-candidate validation at implementation commit 5142ed6 passes
2110 non-GPU/non-network tests with 86.19% branch coverage, 8 RTX 4080 GPU
tests, 15 network tests, Ruff over 332 files, mypy over 147 source files,
actionlint 1.7.12, strict MkDocs and 36 rendered SVG instances at four
viewports. Twenty-four screenshots are under the ignored
artifacts/docs-visual-v0.7.1/ directory and the 1440, 820 and 390 px home
views were manually inspected. Clean core and [train] wheel installs pass;
core keeps torch absent and validates the built-in evidence, while [train]
completes the real fast demo under latest resolved dependencies.
Branch v0.7.0-evidence-release starts from the exact post-PR-#52 main commit
a8272e2b5674e12107461a81d285f1a3d56588a5. PR #51 merged the public
preregistration/outcome as c50aa93b95e6fe4a6aa6251491d3c2b5a9480ebe;
PR #52 corrected the handoff boundary before merge. Every source commit author
is Daoyuan Li <94409450+DaoyuanLi2816@users.noreply.github.com>.
Current scientific state is terminal and unchanged: two declared lineages, eight candidates, 0/64 JSONNav retained utility for every candidate, no selected checkpoint, no teacher qualification, no continuation method and no final-test access. The release work is evidence projection and documentation, not a restarted experiment.
| evidence | state / SHA-256 |
|---|---|
| original fallback selection | preserved byte-for-byte; 53efeb1af196fe8a2fd3733f3f9d6a9ce101fcc76365fc45515adc47cc7d3cd3 |
| corrected fallback projection | lineage metadata only; d68ea994672c112b38149c87fed5cb069c26c1be10154187780f98151d19ed65 |
| primary / fallback selection manifests | source-run files both e1e165e3547c7784b17e93b7e665df66ea6cafa70bec093a69377bc6683bc20b; public LF projections both 2e218db65e39bc7412271e00f7043b287c402c05298bec6618d1a3c3f242a4d5; separately generated, task-identical, not independent |
| portable JSONNav selection rows | 512 rows, no prompts/responses/absolute paths; 694d68cd997bc4b2aa7dd88ebf6572616c9a140fb0df4a672c301095a4f16c7c |
| early-stop result | schema-validated; digest recorded by the final release validation |
| final test | not_accessed, zero tasks scored |
Amendment 4 is explicitly post-selection and pre-release. It changes no
quantitative value, gate, threshold, endpoint or decision. Granite Guardian is
an unqualified diagnostic only; Granite and PairRM qualification, method-level
preference evaluation and teacher qualification are not_run.
| item | state |
|---|---|
| 7.1 bundle-tree preflight | done (71863b6). Reproduced on Windows with a junction: _check_hashes walked into an outside directory and the metadata scan reported semantic_secret_key against a file the bundle did not contain. preflight_bundle_tree walks with lstat only and refuses symlinks, reparse points, non-regular files, escaping entries and trees over bounded file count / bytes / depth. Runs before any open; a refused bundle reports every check as not_inspected, never failed. 13 regressions; symlink cases skip locally for privileges and run on Linux CI |
| 7.2 privacy completeness | done (b1e5ab7). heuristic_passed_full / heuristic_failed / heuristic_incomplete / not_inspected; every gap records file and reason, bounded. --require-complete-metadata-scan fails on incomplete. 8 regressions |
| 7.6 text and CI cleanup | done (b1e5ab7). A mis-decoded multiplication sign is one character, not the three-byte CJK leaders the v0.6.3 gate looked for: U+00D7 is two UTF-8 bytes, so reading them as GBK yields a single U+8133. Two CHANGELOG.md lines survived the release that way — the State-by-Supervision diagnostic name and the two batch-4 speedup figures. Leaders extended to the Latin-1 range (the misreadings of U+00D7, U+00A7 and U+00A6), both lines repaired, 3 new gate cases. All four tensor-to-float warnings in test_chunked_equivalence.py detached. The gate then caught this very table describing the damage with the damaged characters, which is the behaviour we want |
| 7.3 sidecar v2 binding | not started |
| 7.4 source identity during conversion | not started |
| 7.5 residual O(N) metadata | not started |
| 7.7 quality provenance | not started |
version bump to 0.7.0.dev0 |
not started |
Local suite at b1e5ab7: 1853 passed, 6 skipped, 6 deselected. Ruff, format and
mypy clean.
All eleven artifacts match their v0.6.3 digests, including the calculator at
53fc1d4d5b7adee09618d77ad62d4086ba56b78569832d6fc7c3bcd5c2695bbc.
RTX 4080 16 GiB with torch 2.13+cu130; Qwen3-0.6B and Qwen3-1.7B already in the
local HF cache; google/IFEval, natolambert/xstest-v2-copy and
allenai/reward-bench reachable. No v0.7.0 tag exists, so the release number is
uncontested.
Phase A is complete and merged as 0be43b0 (PR
#49).
| item | state |
|---|---|
| endpoint registry | done (c020b14). benchmarks/external-alignment/registry.yaml pins four ungated endpoints with 40-hex revisions, licenses and redistribution decisions, every fact read from the Hub API on 2026-08-08 |
| IFEval verifier | done (c020b14). Independent implementation of all 25 instruction types; 0 of 834 instructions unscored on the real 541 rows. number_sentences and response_language are marked approximate |
| XSTest refusal classifier | done. Deterministic three-label classifier; over-refusal and appropriate-refusal reported separately, never averaged |
| record schema | done. TaskRecord refuses an evaluated row without a score and a non-evaluated row with one; generated text is never stored, only its digest |
| registry validator | done. Rejects a gated source, an unpinned revision, a judge over 3B, a gated judge, an unqualified model evaluator, or a missing category |
| pinned-source network tests | done. Six checks against the real Hub confirm 541 IFEval rows, XSTest 250/200 across 18 types, JBB 100 behaviours, RewardBench 2985 across 23 subsets |
| Granite Guardian harmful judge | done. unclear is excluded from the rate, never folded into refused; qualification floor required before the endpoint counts |
| PairRM preference judge | done. Both orderings evaluated; an order-inconsistent pair is a tie, not a coin flip, and the disagreement rate is published beside every win rate |
| deterministic generation | done. Measured on the RTX 4080: bf16 batched decoding diverges from batch size 1, float32 is byte-identical. float32 batch 8 costs 2.0 GPU h for all 8,128 final-test generations against 19.2 GPU h for bf16 at batch 1, so float32 is required and dtype is in the recorded config digest |
| CLI surface | done. alignment-suite prepare/validate/report; verified end to end against the live Hub, manifest digest e7c946b9be1bba4a8af21be342030450d5f8238dd9bc10021f88779cae3c0867, 508 generation tasks per model |
| evaluation profile | done. profile-v1.yaml: ifeval 128, xstest 252, jbb 64, jsonnav 64 generating (508, under the 512 ceiling) plus rewardbench 96 judged |
| governance doc, THIRD_PARTY_NOTICES | done |
| checkpoint and teacher gates | done. First candidate in a committed order that clears the gate, evaluated on train/eval only; a missing metric is undecidable rather than failed; no qualified teacher is a publishable outcome |
Two defects reached CI that the local machine could not show, both now covered by a local gate:
- the fake-model generation tests drove
torch.no_grad(), so they belonged in the torch-marked set rather than the torch-free subset; default_registry_pathwalked up from__file__to find a repository root, which resolves tolib/python3.12from site-packages. The registry now ships inside the wheel, and a test asserts it resolves under any install layout.
HarmBench, StrongREJECT and AdvBench are all gated; a download attempt on
2026-08-08 returned DatasetNotFoundError requesting access. Accepting dataset
terms is an authorization the maintainer gives on their own account, and a
gated source is not reproducible by a reader either. The harmful-compliance
category therefore uses JailbreakBench/JBB-Behaviors (MIT, ungated, 100
non-adaptive categorised behaviours). meta-llama/Llama-Guard-3-1B is rejected
for the same class of reason: gated: manual. The judge is
ibm-granite/granite-guardian-3.0-2b (apache-2.0, ungated, 2B).
That endpoint is not reported as HarmBench. HarmBench's official classifier is a fine-tuned 13B model and StrongREJECT's reference judge is a paid API model; both exceed the compute contract, so reusing the name would misdescribe what ran.
| phase | projected | basis |
|---|---|---|
| final-test generation | 2.0 GPU h | 0.89 s per 256-token float32 batch-8 response x 8,128 |
| judge passes | ~1.5 GPU h | Granite Guardian 2B over 64 behaviours x 16, PairRM over 96 pairs x 16 x 2 orders |
| teacher preparation and qualification | <= 8 GPU h | contract allocation, not yet spent |
| checkpoint candidates and calibration | <= 4 GPU h | contract allocation, not yet spent |
| continuation training | <= 28 GPU h | 5 methods x 3 seeds x 8 optimizer updates |
The envelope is 48 GPU hours. Nothing above has been spent except the measurement runs that produced the generation figure.
PR B merged as 08dc3c3. Contributor audit over v0.6.3..HEAD shows exactly
one author.
| item | state |
|---|---|
| endpoint overlap | done (3e689b1). RewardBench carries 404 XSTest-derived rows; excluded so the over-refusal and preference endpoints cannot move together and read as corroboration |
| training data | done (b0dbaaf). Anthropic/hh-rlhf (MIT), disjoint from every endpoint. 17,416 usable of 20,000 real rows. Over-long examples dropped, never truncated; split from a hash of the example's identity, not its row position |
| SFT candidates | done (08fe30f). One bounded run, adapters at updates 0/4/8/16, 81 s, peak reserved 4.008 GiB against the 14.5 gate |
| final-test isolation | done (08fe30f). prepare_suite(reserved_task_ids=...) withholds the frozen final-test ids from every selection suite and records the count; an exhausted pool fails closed |
| preregistration | done. PR #51 merged as c50aa93b95e6fe4a6aa6251491d3c2b5a9480ebe; the early-stop outcome is public and frozen |
| candidate evaluation | done: checkpoint selection failed. Both declared lineages completed; every candidate failed the necessary JSONNav retained-utility condition at 0/64 |
| judge qualification | not run. Granite Guardian values are unqualified diagnostics only; PairRM qualification and method preference were not run |
| teacher qualification | not run — scientifically unauthorized because no starting checkpoint was selected |
The first candidate-evaluation run scored all four candidates and none cleared
the saturation gate. That was not a finding about the candidates. The script
substituted 1 - over_refusal for retained tool utility, which is not the
preregistered endpoint: it sits near 1.0 for any policy that rarely refuses,
so it failed the [0.20, 0.90] band structurally.
Recorded as amendment 1 rather than silently fixed, because the gap was found
after eval-split values were visible. The amendment names the alignment and
utility endpoints the profile and governance document already listed; no band
was widened, no metric added or dropped to change which candidates pass. The
superseded run is preserved under
artifacts/v07-start-selection/superseded/.
Pre-amendment eval-split values, kept for the record:
| candidate | instruction following | over-refusal | appropriate refusal | proxy utility |
|---|---|---|---|---|
| update-000 | 0.411 | 0.060 | 0.174 | 0.940 |
| update-004 | 0.442 | 0.000 | 0.022 | 1.000 |
| update-008 | 0.411 | 0.000 | 0.022 | 1.000 |
| update-016 | 0.579 | 0.040 | 0.022 | 0.960 |
miniverl.alignment_external.jsonnav_utility drives RolloutRunner against
JsonNavEnvironment through an HFBackend wrapping the already-loaded frozen
PeftModel — no RunConfig reload, no proxy path anywhere. Task ids come from
the frozen selection manifest, generation is greedy and the policy version is
pinned at zero. 18 regressions plus a real-model GPU smoke: trainable
parameters 0, peak reserved 2.4 GiB.
Both were written and pushed while no candidate's JSONNav value existed. A contingency declared after a disappointing number is a rescue; the same one declared before it is a protocol.
- 2 — one fallback starting-checkpoint lineage from the public pre-v0.7
adapter
DaoyuanLi/mini-verl-qwen3-0.6b-tool-policy-sft@7b98164f, triggered only if every primary candidate fails the unchanged gate. Same data, same 0/4/8/16 order, same thresholds; only the anchor differs. Both lineages failing publishescheckpoint_selection_failed. - 3 — RewardBench agreement qualifies PairRM and nothing else; a separate
64-prompt held-out HH-RLHF set measures arm-vs-baseline preference in both
orders. XSTest 252 → 180 (10 per type) makes room under the unchanged 512
ceiling; total generation 508 → 500. The 508 profile is preserved at
profile-v1.superseded-508.yaml, sha2568dc71d10.
Neither amendment moved a band or a threshold.
Every primary candidate scores 0/64 = 0.000 JSONNav retained tool utility,
reproduced across three independent runs. That is far below the [0.20, 0.90]
band floor.
The failure is missing tool competence, not saturation: HH-RLHF is a
conversational preference corpus with no tool protocol, so the policy never
learned to emit a valid tool call — rollouts terminate at
PARSE_ERROR_LIMIT. This is precisely the case amendment 2 was written for.
HarmfulJudge.classify sliced the result of apply_chat_template as a tensor.
The real Granite Guardian tokenizer returns a BatchEncoding, so the slice
raised TypeError. All 23 judge tests passed because none of them exercised
the loaded classify path — the fixtures' fake tokenizer happened to be
sliceable. Both return shapes are now covered.
| candidate | instruction following | over-refusal | harmful compliance | retained tool utility |
|---|---|---|---|---|
| update-000 | 0.411 | 0.020 | 0.645 | 0.000 |
| update-004 | 0.442 | 0.000 | 0.774 | 0.000 |
| update-008 | 0.411 | 0.000 | 0.750 | 0.000 |
| update-016 | 0.579 | 0.020 | 0.367 | 0.000 |
All four decidable: true, passed: false, same reason: retained utility 0.000 outside [0.2, 0.9]. 40.9 GPU minutes, peak 5.246 GiB. Amendment 2's
trigger is therefore satisfied on its terms, not by judgement.
The alignment side is healthy — instruction following 0.41–0.58 in band, harmful compliance 0.37–0.77 with room to fall. Only utility is a hard zero.
Every primary-lineage rollout ended at PARSE_ERROR_LIMIT with zero valid tool
calls and exactly 128 tokens. Fallback-lineage rollouts emitted two to four
JSONNav tool calls and ended by final answer, max turns or repeated-call limit,
but still solved 0/64 for every adapter. These are distinct observed failure
behaviors, not one uniform signature.
The environment's oracle clears the identical settings 8/8, all
FINAL_ANSWER, at 64, 128 and 256 tokens per turn and at both difficulties.
The harness is sound; the zero belongs to the models. Frozen as
tests/integration/test_jsonnav_harness_validity.py (4 checks) so a later
settings change cannot quietly produce a plausible-looking zero.
The amendment 2 anchor's own provenance manifest says:
training_task.environment: tool_policy (not jsonnav)
training_task.difficulty: easy (not hard)
strict_task_success_rate: 1.0 (on tool_policy)
parse_valid_tool_call_rate: 1.0
The anchor has real tool-protocol competence — for a different environment's tool set. JSONNav has its own tools and state space. So the retained-utility endpoint measures something neither lineage ever had, and its fallback candidate 0 (the anchor itself, zero continuation updates) also scores 0/64.
The endpoint is not being changed. The preregistration named JSONNav; swapping it after seeing the result is precisely what preregistration exists to prevent. This is recorded as a study-design finding for the report.
Preregistration PR #51
merged as c50aa93b95e6fe4a6aa6251491d3c2b5a9480ebe. That commit is the public
freeze; nothing in the study may move without a further amendment.
Contributor audit over the merged range: 16 commits, sole author
Daoyuan Li <94409450+DaoyuanLi2816@users.noreply.github.com>. The two
Anthropic matches in commit bodies are the dataset repository name
Anthropic/hh-rlhf, not attribution; an attribution-shaped pattern
(co-authored-by:, generated by ai, by claude, claude code,
assisted by) returns zero hits.
GPU spent across all of v0.7 so far: 1.2 hours of the 48-hour envelope, peak reserved 5.25 GiB against the 14.5 GiB gate.
The evidence-release branch adds the schema-valid early-stop result, portable
selection rows, original/corrected fallback relation, identical-suite
disclosure, miniverl pilot --study-result, the study page and data-bound
figures. After its full green validation it will release v0.7.0, add an
evidence comment to issue #39 while keeping the issue open, then advance main
to 0.7.1.dev0 through a separate state-sync PR.
Local release gates on implementation commit 013993a8cd5002a9ed166ba1e6948305f92c1bfb
plus its quality-record-only update pass 2,098 CPU tests with 6 platform skips
and 21 deselections at 86.35% combined branch coverage. The available RTX 4080
suite passes 8 tests and the opt-in network suite passes 15. Ruff, format,
mypy, actionlint, strict MkDocs, four-viewport browser checks, package/Twine,
clean-install and extracted-sdist gates pass.
No continuation arm, no teacher qualification and no method comparison will run — none of them is scientifically authorized by the selection outcome.
Before release, correct the fallback lineage metadata with original-byte provenance, disclose that the two separately generated selection suites are task-identical, and remove every unsupported judge-qualification claim. Add an evidence comment to issue #39 and keep it open because the method-level and final-test acceptance gates remain unexecuted.
No final-test task has been scored. first_final_test_access: not_accessed.
| item | current state |
|---|---|
| starting state | origin/main and local main both at c62e14ba4229f32b3db06b293c615e94654debc6; tag v0.6.2 at bef9f0878eb3280f450aee3868b43d61f0726557; working tree clean; no open pull request; source version 0.6.3.dev0; branch v0.6.3-security-artifact-hardening cut from that commit |
| immutable baseline | all ten frozen result artifacts plus docs/generated/verl-bridge-smoke.json re-hashed at takeover and unchanged; calculator remains 53fc1d4d5b7adee09618d77ad62d4086ba56b78569832d6fc7c3bcd5c2695bbc |
| reward-code execution | reproduced: a scaffold whose top level wrote PWNED.txt created the marker during _check_reward, and the report still said side-effect-free import; scaffold intentionally not executed. Fixed by src/miniverl/bridge/reward_static.py, an AST walk that never imports; levels are not_present, syntax_valid, interface_statically_verified, trusted_dynamic_import_verified, and the last one requires --trust-and-import-reward-code |
| safetensors validation | reproduced: a header declaring a 4x4 F32 tensor with zero payload bytes returned (True, '1 tensor header(s)') while the official reader rejected it with file not fully covered. Fixed by src/miniverl/bridge/safetensors_check.py; levels are not_present, header_only, payload_structure_validated, tensor_materialization_validated, with --require-adapter-payload for strict callers |
| source/output aliasing | reproduced: import-verl --out <source> --overwrite overwrote the source on success and deleted it on rejection while publishing the report. Fixed by reject_source_output_alias, which runs before any transaction and covers exact, relative, symlink, hard-link and Windows case aliases |
| conversion semantics | extension data in miniverl_extensions, the sidecar and extra_info.miniverl is now reconciled: equal content deduplicates, different content fails closed naming row and locations but never values. Invalid rows fail the conversion unless --allow-rejected-rows is given, and a partial report carries complete_dataset_conversion: false |
| Parquet bounds | reproduced by inspection: the scan called pq.read_table().to_pylist() before applying bounds and the schema check read every row. Both now use footer metadata and iter_batches per row group; an instrumented test asserts the requested row groups are exactly [0] when the bound is two rows into an eight-group file |
| release-state drift | python scripts/release_state.py --check reported ten disagreements at takeover, including README stable v0.6.1, docs selector Stable 0.6.1 / Development 0.6.2.dev0 and quality_floor ... at v0.6.1 inside the release: 0.6.2 record. All ten are resolved and the gate runs in ci.yml and release.yml |
| canonical release state | release-state.yaml drives the package __version__, both READMEs, PYPI.md, the docs channel selector, CITATION.cff, the changelog comparison link, the release-checklist section, this file's header line and the quality record; a test drives the full release 0.6.3 → stable 0.6.3 → main 0.6.4.dev0 transition and still reports the prose sections a person must write |
| mobile visual debt | the four-figure exemption list is empty: consumer-runtime-v1-pareto, cost-quality-pareto, fresh-vs-frozen and recovery-success each ship a dedicated 390 px layout selected by a <picture> media query, and the desktop SVG bytes are unchanged at their pinned SHA-256 values |
| transactional scope | wording is narrowed to transactional publication with in-process rollback; multi-file crash atomicity across kill -9, kernel panic or power loss is explicitly not claimed, and the versioned-directory design that would provide it is named as out of scope |
| adversarial audit | tampered bundles were re-sealed with self-consistent SHA256SUMS and still rejected: hostile reward code stops at syntax_valid, a truncated adapter stops at header_only, and conflicting pinned-verl or adapter-versus-config fields fail closed. --require-verl is not a back door into execution, and a failing rollback re-raises the original cause rather than masking it |
| defect introduced and fixed in this branch | the first safetensors implementation treated a dependency gap as evidence the file was invalid. The torch-free environment installs safetensors but not numpy, and safe_open(framework="np") needs numpy; the resulting ModuleNotFoundError was caught by a generic handler and reported as the official safetensors reader rejected adapter_model.safetensors, failing the model check and nine tests against structurally sound files. This is the mirror of the defect being fixed: the old code read "only a header" as "valid", the new code read "cannot check" as "invalid". _materialize now returns materialized, unavailable or rejected; unavailable keeps payload_structure_validated with official_reader_status: dependency_missing and a passing status, because the structural pass is dependency-free. The exact condition is a regression test |
| strict adapter option | --require-adapter-payload demands tensor_materialization_validated, so an absent official reader does not satisfy it; the same file still passes without the flag |
| local gates | ruff, ruff format and mypy clean; pytest -m "not gpu and not network" passes 1,757 with 2 skipped, 6 deselected at 86.02% branch coverage on commit 85a7eb3aee46109ced405a7cbfb2106edeb82516; pytest -m gpu passes 5 on the RTX 4080; pytest -m network passes 3; mkdocs build --strict clean; the browser gate checks 28 rendered SVG instances across 1440x900, 1024x768, 820x1000 and 390x844 with zero exemptions; python -m build and twine check both pass. The two skips are symlink creation, which needs privileges on Windows; the hard-link and case-alias cases cover the same guard |
| external metadata | the GitHub repository description was changed from Auditable on-policy distillation for tool-using agents on one personal GPU. to Auditable one-GPU alignment and distillation runtime with shared-backbone training and a fail-closed verl artifact bridge., and topics alignment, llm-alignment and peft were added; no existing topic was removed |
| integration | PR #45 |
| release state | in progress on v0.6.3-security-artifact-hardening |
| item | observed state |
|---|---|
| checkout | one worktree, no submodules |
| local HEAD | 64fd6d62087185091b2f90c2a5870a3f3c83836b, identical to origin/main after git fetch --all --prune |
| remote baseline | origin/main = 64fd6d6 "Advance development to 0.6.2.dev0 (#41)"; git describe = v0.6.1-1-g64fd6d6; stable tag v0.6.1 = 48b9e7d9231b5f6cd018f6e927f81df066258f17 |
| branch | v0.6.2-bridge-visual-hardening already existed locally at 64fd6d6, unpushed; a duplicate empty local branch v0.6.2-visual-bridge-correctness also sat at 64fd6d6 |
| background GPU work | none running. Get-CimInstance Win32_Process matched no python.exe/pythonw.exe; nvidia-smi showed the RTX 4080 at 8% with only desktop/browser C+G clients and no compute process. No RecoveryBench equal-wall-time run was live, so nothing needed termination and no current artifact was at risk of being overwritten |
| GPU availability | NVIDIA GeForce RTX 4080, 16376 MiB total / 1867 MiB used, driver 596.49, CUDA 13.2 — available for the existing GPU suite |
| uncommitted work in the tree | one file, src/miniverl/bridge/config.py (+52/-2). Archived verbatim outside version control before any change |
| audit of that draft | it added a ${...} regex, _contains_interpolation, _reject_cli_interpolation and redundant non-finite string guards, but left Callable, suppress and RunLock imported-unused, added a dead tuple-returning _validate_positives, and wired none of it into the import path. Its useful ideas are carried into the shared audit module; the half-wired scaffolding was reverted rather than committed |
| run locks / temp dirs | runs/.miniverl-locks and artifacts/.miniverl-locks contain no live lock metadata, and no partial publication directory was left behind |
| immutable evidence | all ten frozen result JSON/JSONL files hash exactly to their recorded values, including calculator 53fc1d4d5b7adee09618d77ad62d4086ba56b78569832d6fc7c3bcd5c2695bbc; docs/generated/verl-bridge-smoke.json baselines at d5266adc0f3ec46cf29f96a2f4258f03d848d846e62cbbe3ccdf5e014909bda7 |
| defect | evidence on 64fd6d6 |
|---|---|
| Workstream A — unresolved interpolation reaches a runnable recipe | with every bridge choice supplied, actor_rollout_ref.model.path: ${MODEL_PATH} produced status: accepted, generated_recipe_validated: True and a recipe containing model_id: ${MODEL_PATH} at line 16. Only some numeric paths were guarded |
| Workstream B — shared, non-transactional output names | the same run wrote import-report.json, not a stem-specific name; template runs write imported.template.yaml. Two stems in one directory overwrite each other, and dataset conversion replaces the Parquet before its sidecar/report |
tests/unit/test_verl_bridge_interpolation.py, tests/unit/test_verl_bridge_outputs.py and
tests/unit/test_verl_bridge_dataset_outputs.py were added against unmodified code and reported
44 failed, 11 passed, covering mapped-field/CLI/nested interpolation, informational labelling,
stem-specific naming, collision refusal, --overwrite, fault injection and concurrency.
| workstream | outcome |
|---|---|
| A — interpolation | src/miniverl/bridge/interpolation.py holds one recursive audit over strings, lists, tuples and mappings. It runs on source fields (classified exact/derived/requires_user_confirmation fail closed, informational_only is labelled unresolved_informational_only and stays in the report), on explicit CLI choices before any path is reserved, and on the generated recipe plus its rendered bytes before publication. Detection is conservative: any ${ is a finding. Nothing is ever resolved from the environment. 1e-5 still parses; NaN/infinity stay rejected. No new runtime dependency |
| B — transactional outputs | src/miniverl/bridge/publish.py gives every invocation a stem-specific target family, a per-stem RunLock reservation, a collision check that runs before anything is modified, staging inside the destination and restore-on-failure. import-verl publishes <stem>.yaml or <stem>.template.yaml plus exactly one <stem>.import-report.json; convert-dataset publishes Parquet, sidecar and report as one family. --overwrite is required to replace; --out alone never implies it |
| C — Alignment Lab visuals | metric coverage became an accessible responsive HTML table with one column-level scope statement, replacing the SVG whose Sandbox endpoint / External safety headers collided and whose two rightmost columns repeated one value six times. The forest legend moved to its own footer band and every mean/seed value is printed in the left column instead of floating in the plot. The outcome matrix widened its method column, stacked each value above its bar and moved — not applicable to the last column. Both charts gained dedicated 390 px vertical SVGs selected by <picture> at max-width: 900px |
| D — visual gate | scripts/check_docs_visual.py now measures each figure's real rendered bounding box in the viewport under test and re-renders the SVG at exactly that width. It inspects every visible <text> node rather than only [data-role] ones, detects header-to-header and header-to-data overlap and label-to-mark occlusion, keeps the viewBox-bounds and legend/plot checks, validates responsive tables and card labels, ignores inactive <picture> branches via currentSrc, and fails below 11 px at 390 px |
| E — tokenizer levels | not_present → metadata_only → loadable_local_snapshot → structural_identity_verified. A complete snapshot is loaded with local_files_only=True and trust_remote_code=False; structural digest, vocabulary size and special tokens are compared against the manifest identity when present, and any mismatch fails closed. --require-tokenizer-load refuses metadata-only bundles |
| F — privacy scopes | portable_metadata_privacy, dataset_content_privacy and model_weight_privacy are independent. not_inspected is never rendered as passed, and the CLI says so. --scan-dataset-text adds a bounded heuristic scan that reports category/split/column/row only, caps rows and bytes, records full vs sampled, and never reads safetensors as text |
| G — claims | README, Chinese README, PYPI.md, docs/verl-bridge.md and the exported bundle README describe a verified artifact bridge at miniVERL-defined Level 3 with launchable: false, distributed execution not tested and no OPD/PPO parity, and document the new naming, overwrite, tokenizer and privacy contracts |
| gate | result |
|---|---|
| new regressions | 55 pass (test_verl_bridge_interpolation.py, test_verl_bridge_outputs.py, test_verl_bridge_dataset_outputs.py) after implementation; the same files reported 44 failures before it |
| doctor levels/privacy | 19 pass in tests/unit/test_verl_bridge_doctor_levels.py, including no-network enforcement through the deny_network fixture |
| visual gate regression | the v0.6.1 SVG is preserved verbatim at tests/fixtures/visual/legacy-metric-coverage-matrix.svg. The corrected gate rejects it at both widths with overlapping text: ['"Sandbox endpoint" x "External safety"']; all four published figures pass with zero overlap, zero occlusion and no text under the floor |
| browser gate | checked 28 rendered SVG instances across 4 viewports at 1440x900, 1024x768, 820x1000 and 390x844 over /, /alignment-lab/alignment-lab-v1/, /consumer-runtime/, /recoverybench/recoverybench-v1/ and /verl-bridge/ |
| manual inspection | the 390 px and 1440 px screenshots were read directly. Two defects the automated gate could not express were found and fixed this way: the coverage caption collapsed to one word per line in card mode, and the outcome-matrix value labels touched their own seed marks and the next column's bar |
| CPU suite | 1652 passed, 6 deselected, 85.77% branch coverage against an 80% floor |
| static gates | ruff check, ruff format --check, mypy src/miniverl, mkdocs build --strict and the Markdown/link check pass |
| GPU suite | 5 passed on the local RTX 4080 (driver 596.49, CUDA 13.2); no scientific benchmark was rerun |
| network suite | 3 passed |
| packaging | python -m build and twine check pass. miniverl-0.6.2.dev0-py3-none-any.whl SHA-256 e1cea1c2001dd96e14542d948024b920552473eef05cfbd0ba49bfbf270eabdf; miniverl-0.6.2.dev0.tar.gz SHA-256 780e96c0d11e53ac5c80dfc72ed331d2952b34ca4f30400fdbe149c3210da900 |
| clean installs | a clean core venv from the wheel reports miniverl 0.6.2.dev0 with torch absent and passes miniverl doctor; a clean [bridge] venv resolves pyarrow 25.0.0 with torch still absent |
| extracted sdist | installed with [dev,train] into a clean venv outside the checkout, resolving miniverl from that venv: 1652 passed, 6 deselected. The torch-free CI selection passes 1303 tests on the same extraction |
| immutable evidence | all ten frozen result JSON/JSONL files and docs/generated/verl-bridge-smoke.json re-hash to their recorded values after every change |
| item | state |
|---|---|
| branch | v0.6.2-bridge-visual-hardening, pushed at 1161700c19379f6c1b49fe50d4408d4d96374143 |
| pull request | draft #43 |
| CI | 30975138358 green: core no-torch py3.10/3.11/3.12/3.13, cpu ml (torch), transformers 4.51.* and 5.*, training minimum py3.10 and training latest py3.13. actionlint runs on the 3.12 core job and passed |
| build | 30975138346 green, including the extracted-sdist reproduction and the sdist-built wheel inventory match |
| docs | 30975138352 green; 20 Linux screenshots uploaded as docs-visual-screenshots |
| pinned verl bridge | 30975138353 green |
| screenshot inspection | the CI Linux screenshots were downloaded and read directly, not only trusted as green. The 390 px Alignment Lab mobile chart and metric-coverage cards render the same as the local Windows capture |
| release metadata | the version was finalized to 0.6.2, CHANGELOG.md carries a dated [0.6.2] - 2026-08-05 section with its compare link, CITATION.cff records 0.6.2 / 2026-08-05, PYPI.md is regenerated tag-pinned to blob/v0.6.2, and docs/generated/quality.json records release 0.6.2 at 1652 CPU tests, 85.77% branch coverage, 5 GPU and 3 network |
| local release build | python -m build and twine check pass at 0.6.2. These local artifacts were a sanity check only; the published distributions are built once inside the tag workflow |
| merge | PR #43 was squash-merged; the release commit is bef9f0878eb3280f450aee3868b43d61f0726557 after its message was rewritten so the sole contributor is the repository owner. Synchronized-main CI, build and docs runs 30980343307 / 30980343344 / 30980343293 are green on that exact commit |
| tag | annotated v0.6.2 resolves to bef9f0878eb3280f450aee3868b43d61f0726557 |
| release execution | tag run 30980579109 passed metadata validation, the full quality gate, a one-time build, OIDC Trusted Publishing with attestations, public PyPI verification, exact install and GitHub Release creation |
| published wheel | miniverl-0.6.2-py3-none-any.whl, SHA-256 38131c3de838b480017b2f97df3e43d53e760f272c516522a470d2812f8a3803, 330906 bytes |
| published sdist | miniverl-0.6.2.tar.gz, SHA-256 bcd30863290c9c46c1e2c27c59d8ccfd3ec7e5934243f5bc86f7424d4f05333d, 994642 bytes |
| independent public verification | the PyPI JSON API reports version 0.6.2 with a Markdown description pinned to blob/v0.6.2, and the integrity API exposes one Trusted Publisher attestation per distribution bound to DaoyuanLi2816/mini-verl, release.yml, environment pypi. The GitHub Release ships byte-identical distributions whose recomputed SHA-256 values match its SHA256SUMS and PyPI exactly. A clean Windows venv installed miniverl==0.6.2 from https://pypi.org/simple, reported miniverl 0.6.2 with torch absent, and exposes --require-tokenizer-load, --scan-dataset-text and --sentinel |
| version transition | v0.6.2 is immutable and public; this state sync identifies subsequent development as 0.6.3.dev0 |
| release state | complete |
The 11 px readability floor at 390 px is not enforced for four figures that
predate this release and have no narrow layout: consumer-runtime-v1-pareto.svg,
cost-quality-pareto.svg, fresh-vs-frozen.svg and recovery-success.svg.
The exemption is an explicit asserted set in scripts/check_docs_visual.py, is
printed on every run, and fails the gate if any other figure joins it.
Redesigning those v0.3/v0.4 figures is out of scope for this patch.
The committed docs/generated/verl-bridge-smoke.json is left byte-identical.
It was produced before tokenizer verification levels existed, and its bundle
ships only tokenizer_config.json — which the current checker correctly
classifies as metadata_only rather than the status: ok the old
presence-only check recorded.
| item | current state |
|---|---|
| scope | representation and documentation UX were rebuilt without rerunning a benchmark; import/export bridge semantics now fail closed instead of implying runnable or equivalent execution |
| Alignment Lab visuals | the four non-data-bound scatterplots are replaced by a three-seed delta forest, an outcome/cost matrix and a metric-coverage matrix; not-applicable teacher ratios remain N/A, and the zero-variance sandbox checks are not plotted as a two-dimensional safety result |
| bridge diagram | responsive desktop/mobile layouts show solid arrows only across the verified local runtime, portable bundle and pinned v0.8.0 / 7aed6b23 parse-load smoke, followed by a dashed arrow to prominent Distributed execution: NOT TESTED |
| import contract | every source field is classified as exact, derived, informational only, requiring user confirmation or unsupported; unresolved data/environment, teacher, objective or schedule semantics produce needs_user_input plus a non-executable template |
| export status | artifact completeness, upstream parse/load, model/data smoke, reward implementation, launchability, distributed testing and algorithm parity are independent flags; the fail-closed reward scaffold remains non-launchable and uses launch.template.sh |
| documentation | pinned Material 9.7.7 supplies stable/dev navigation, search, dark/light modes, command-copy controls and responsive tables/images; the landing page exposes Align, Distill locally and Scale out paths |
| visual gate | Playwright checks five pages at 1440x900, 1024x768, 820x1000 and 390x844 for overflow, SVG bounds, label collisions, readable labels, tables and the responsive bridge; final PR docs run 30856925490 is green and its 20 Linux screenshots were manually inspected |
| validation | local ruff, format, mypy, actionlint, full non-GPU/non-network, available GPU/network, strict MkDocs, browser visual, package/Twine, clean-install, extracted-sdist, bridge end-to-end, privacy, Markdown/link and generated-byte gates pass; PR CI/build/bridge runs 30856925524 / 30856925481 / 30856925485 and synchronized-main CI/build/docs runs 30857409920 / 30857409926 / 30857409918 are green |
| immutable evidence | no result JSON/JSONL or frozen bridge-smoke record changed; calculator SHA-256 remains 53fc1d4d5b7adee09618d77ad62d4086ba56b78569832d6fc7c3bcd5c2695bbc and Alignment Lab result/task SHA-256 values remain 584752dccb91654109c357b8ebb12681a12a9c1476a9ba539dd35e4d860a22ef / 8d7fc723436d7377d196fc44046d960e3cb7f0aa81e03d49ef05b627eb84630f |
| integration | focused PR #40 was squash-merged as exact release commit 48b9e7d9231b5f6cd018f6e927f81df066258f17; annotated tag v0.6.1 resolves to that commit |
| release execution | tag run 30857762954 passed metadata, the full quality gate, one-time build, OIDC Trusted Publishing, public hashes/attestations, exact install and GitHub Release creation |
| published wheel | miniverl-0.6.1-py3-none-any.whl, SHA-256 ef9ce5378e43c0d833b782e431248a0838e3841ace76a8b3a08781dd28007918 |
| published sdist | miniverl-0.6.1.tar.gz, SHA-256 e5ffd7917035d1f3878b22415dd357cd47fe16b6c77ffdae260e0b85ad7e050f |
| independent public verification | PyPI and GitHub Release expose identical file hashes; the PyPI integrity API exposes one Trusted Publisher attestation for each distribution, and a clean Windows Python 3.12 install from the public wheel reported miniverl 0.6.1, kept torch absent and passed core doctor |
| version transition | v0.6.1 is immutable and public; this state-sync change identifies subsequent development as 0.6.2.dev0 |
| release state | complete; PyPI and miniVERL v0.6.1 expose the verified release, and the synchronized docs workflow publishes v0.6.1 at the stable root plus current main under /dev/ |
| item | current state |
|---|---|
| audited upstream | official stable verl v0.8.0, source commit 7aed6b230776f963fa09509c10d9c3a767d1102c, tested with Python 3.12; the installed source reports 0.8.0.dev0 |
| compatibility contract | Levels 0-3 are explicitly miniVERL-defined; Level 2 is a fail-closed 14-field whitelist for single-gpu-online-distillation-v1; miniVERL-defined Level 3 exchanges standard HF/PEFT/safetensors/tokenizer/Parquet artifacts and a checksummed scale-out bundle |
| command surface | import-verl, convert-dataset, export-verl, bridge doctor and benchmark --export-community are implemented without importing torch in the core path |
| exact compatibility smoke | the pinned source installed successfully; OmegaConf parsed official and exported config shapes; PEFT, safetensors, train/val Parquet, reward import, privacy and all bundle hashes passed; the checksummed record is docs/generated/verl-bridge-smoke.json |
| distributed boundary | Ray, FSDP/Megatron, vLLM/SGLang and a full distributed run were not installed or launched; distributed execution remains explicitly not tested |
| community surface | four versioned recipe records bind existing measurements; the preference-teacher distillation entry is explicitly unmeasured and links an aligned-adapter scaffold that still requires a pinned preference-trained teacher; no external adoption is claimed |
| documentation | a static MkDocs site, bridge architecture diagram, launch article/demo/card copy/announcement/social preview and community submission guide are part of the release candidate |
| immutable evidence | the frozen calculator JSON remains required byte-identical at SHA-256 53fc1d4d5b7adee09618d77ad62d4086ba56b78569832d6fc7c3bcd5c2695bbc; prior tags and published adapter revisions remain immutable |
| local release gates | ruff, format, mypy, actionlint, Markdown/link and generated-artifact checks pass; the full non-GPU/non-network suite passes 1548 tests with 6 deselected at 85.53% branch coverage; the available GPU and network gates each pass 3; package build, Twine, clean Python 3.10 wheel install and exact pinned-verl smoke pass |
| integration source | Verified verl Bridge PR #36 was squash-merged as 0d43310cca47db828b10a0e12facb33e8f0fd371; synchronized main CI, build and docs runs 30794655713, 30794655722 and 30794655822 are green; the latest PR bridge smoke is 30794329109 |
| public documentation | GitHub Pages deployed successfully and returned HTTP 200 at https://daoyuanli2816.github.io/mini-verl/ with the bridge documentation visible |
| release metadata | PR #37 was squash-merged as exact commit 6cfbdbb7bbf5c6042def4cf154bfe3c3b6530eea; annotated tag v0.6.0 resolves to that commit |
| release execution | tag run 30796058250 passed metadata, the full quality gate, one-time build, OIDC Trusted Publishing with attestations, public PyPI verification, exact install and GitHub Release creation |
| published wheel | miniverl-0.6.0-py3-none-any.whl, SHA-256 e5fbb99bf410c27d22d6959f9599dfa4fbac2e940dac63f55c9676f68264abd1 |
| published sdist | miniverl-0.6.0.tar.gz, SHA-256 2e1d85556875f6d23152220897ba919d8b82bda35c5c78fd48efffa7ec22909d |
| independent public verification | PyPI and GitHub Release expose the same files and hashes; PyPI exposes one Trusted Publisher attestation bundle per distribution, bound to release.yml, environment pypi and commit 6cfbdbb7; a clean Windows Python 3.10 install from https://pypi.org/simple reported miniverl 0.6.0 and passed JSON doctor with the core path healthy and torch absent |
| version transition | v0.6.0 is immutable and public; this state-sync change identifies subsequent development as 0.6.1.dev0 |
| release state | complete; PyPI, miniVERL v0.6.0 and the public documentation site expose the verified release |
| item | current state |
|---|---|
| phase boundary | v0.4.0 was released and state-synced to 0.5.0.dev0 before final Alignment Lab execution; v0.5 is now released, and this state sync advances main to 0.6.0.dev0 before the verified verl bridge begins |
| public workflow | miniverl align resolves base → SFT checkpoint → teacher/reference → alignment → evaluation → Alignment Card; miniverl pilot exposes versioned evidence, uncertainty, cost assumptions and a bounded method recommendation |
| alignment roles | policy-conditioned self-distillation, frozen aligned-adapter teaching and shared-backbone execution are supported; continued SFT, pinned TRL DPO, offline soft distillation, standard OPD and verifier-gated OPD are explicit methods |
| policy suite | Minipolicy v1 uses deterministic sandbox tools to check authorization, confirmation, instruction hierarchy, secret exclusion, safe refusal, benign completion and safe error recovery; no real destructive action is executed |
| preregistration | public revision 1.4 digest 71307dbfe9a5bb20c686307cafce8bd254c07af8b69c1bf1c6ec0dbf53a8cde0 freezes three seeds, the shared SFT checkpoint, four continuation updates, 48 final-test tasks, the gate and the disjoint seed-1234 recovery rule |
| final artifact audit | all 18 arms completed with 48 ordered paired final-test tasks each; 864 task-level records are retained; all arms share starting checkpoint 7304922281268a687dd1c75ba918e26c64c8207b5701db78c368afd20d80ae89; strict OPD freshness, every verifier-gate decision, DPO provenance/cost and the two disjoint baseline segments pass publication checks |
| final result | the SFT checkpoint scored 100% alignment and 100% tool utility in every seed; DPO and offline soft distillation tied it; continued SFT averaged 94.4% / 88.9%, standard OPD 98.6% / 97.2%, and verifier-gated OPD 97.9% / 95.8% alignment / utility |
| preserved negative results | verifier-gated OPD seed 20260727 failed 3 safe-error-recovery tasks; continued SFT seed 20260801 failed 8; standard OPD seed 20260801 failed 2; no completed final arm was rerun, and the interrupted pre-evaluation continued-SFT construction remains preserved outside the headline result |
| State × Supervision | measured signal diagnostic 9e08129ba4cd9e460c189b94b4e421d881ba69e3938f02eac95d251f50c88788 finds 100% teacher-argmax/student-token agreement and 0.0251% fresh soft probability mass beyond argmax; it is not a separately trained hard-target result and no soft-target advantage is claimed |
| verifier-gated OPD | frozen policy-critical-span-v1 reduced mean queried positions from 100% to 46.8% and GPU time from 76.7 to 66.0 seconds, but did not improve alignment or retained utility; selected positions are not teacher-backbone FLOPs |
| pilot decision | alignment-pilot-v1 returns insufficient_evidence and the operational decision not to spend online teacher-query cost because the starting policy is already saturated and no continuation method improves it |
| frozen public artifacts | result 584752dccb91654109c357b8ebb12681a12a9c1476a9ba539dd35e4d860a22ef; task rows 8d7fc723436d7377d196fc44046d960e3cb7f0aa81e03d49ef05b627eb84630f; technical PDF adbffa967f6b9a25d2cdb0cc4464a93c13db4615a1e91499585fb199285d980b; 18 JSON plus 18 Markdown Alignment Cards |
| public teacher | the common Qwen3-0.6B SFT adapter is immutable at DaoyuanLi/mini-verl-qwen3-0.6b-tool-policy-sft@7b98164f73e493c51f2ed3fca3169fea078f47f0; starting checkpoint content digest is recorded separately above |
| demonstration assets | reviewed short CPU run completed the actual alignment path in 17.6 seconds; the committed 75-second recording plan includes pilot, stage graph, real update, typed token inspection, Alignment Card and export-ready checkpoint without hiding the toy 0% result |
| presentation | four data-bound dark SVGs passed native and 820-pixel inspection; banner and social preview passed raster inspection; the report PDF is deterministic, six pages, metadata-valid and visually inspected page by page |
| local release gates | ruff, format, mypy, actionlint, generated-artifact checks, Markdown links and package/twine checks pass; the full non-GPU/non-network suite passes 1508 tests with 6 deselected at 86.13% branch coverage; the RTX 4080 gate passes 5 and the network gate passes 3; fresh core and CPU-training wheel installs, a real no-network toy demo, and all 1508 tests from an isolated extracted sdist pass |
| immutable baseline | calculator benchmark remains byte-identical at 53fc1d4d5b7adee09618d77ad62d4086ba56b78569832d6fc7c3bcd5c2695bbc; v0.3/v0.4 tags, RecoveryBench artifacts and public adapter revisions remain unchanged |
| integration source | Alignment Lab PR #33 was squash-merged as f9dae54a203d303f6562101500343ead310d99e8; release-metadata PR #34 was squash-merged as fd755ff351fe691531ed68b4af7793c4929ed89e; annotated tag v0.5.0 resolves to the latter exact commit |
| version transition | v0.5.0 is immutable and public; this state-sync change identifies subsequent development as 0.6.0.dev0 |
| release execution | tag run 30789267409 passed metadata, all release gates, one-time build, OIDC publication with attestations, public verification, exact clean install and GitHub Release creation |
| published wheel | miniverl-0.5.0-py3-none-any.whl, SHA-256 ff60cb747e2ad1fd74575dd8920b11b48c6c16cc97743e94f9583d162d18819c |
| published sdist | miniverl-0.5.0.tar.gz, SHA-256 d3340e0526eb4b20bb9ee15960c27c740be0fdacb9a51b029627faa63ca5276d |
| independent public verification | workflow artifacts, PyPI and GitHub Release reproduce both hashes; PyPI exposes one attestation bundle for each distribution; a refreshed clean Windows Python 3.10 install from https://pypi.org/simple reported miniverl 0.5.0, passed JSON doctor, and kept torch absent |
| release state | complete; PyPI and miniVERL v0.5.0 expose the same verified wheel and sdist |
| item | current state |
|---|---|
| phase boundary | v0.3.0 was released and state-synced to 0.4.0.dev0 before this work began; v0.4 must merge, release and advance main to 0.5.0.dev0 before alignment-lab experiments begin |
| implementation | typed padded batches support variable lengths, causal mask isolation, padding-free selected-position loss, per-trajectory normalization, deterministic length bucketing, SFT/offline-KD/strict-OPD and exact/top-k-plus-tail objectives |
| local role graph | typed actor, rollout, teacher, reference, verifier, target, update, evaluation and artifact roles map existing components without importing Ray, DataProto, FSDP placement or distributed APIs |
| shared ownership | models.runtime: shared_backbone loads one quantized base with trainable student, frozen teacher and optional frozen reference adapters; optimizer visibility, failure restoration, one-base loading, reference isolation and standard PEFT student export are covered by CPU/HF tests |
| public systems adapter | DaoyuanLi/mini-verl-qwen3-0.6b-consumer-runtime-teacher@e277b92d8c1fdb76cd133f872f0ddd2c47a4ab8c is immutable and independently re-downloaded; it is a runtime benchmark artifact, not a newly qualified teacher |
| preregistration | revision 1.2 at public PR head e44584b04837a05b0dd834c7948666d843908486 retains eager attention and NF4 weights with FP32 compute after two explicitly non-headline diagnostics exposed nondeterministic SDPA and BF16 batch-shape drift |
| final measurement | all eight dual/shared × sequential/2/4/auto cells completed on one RTX 4080; trajectory and teacher-target digests match, and 12/12 declared loss/full-gradient/post-update-logit equivalence comparisons pass |
| performance result | dual batch-4 reached 3.866 trajectories/s at 3.035 GiB reserved (1.63× sequential); shared batch-4 reached 3.475 trajectories/s at 2.227 GiB (1.54× sequential and 26.6% less memory than dual batch-4, but 10.1% slower) |
| frozen artifacts | result a302da31af99f1d29f1efd4e6b3dbeb6ea4ac956bba102ca8a1bee8dff0319eb; profiler 66111cd7fc876cf1befea3297a1a51bcd99252c0bf8989c029381e1dc155a98b; SVG 98645a668a7832423d28b621262292619615917f037adf7219ff1bf071fb2fea |
| immutable baseline | calculator benchmark remains byte-identical at 53fc1d4d5b7adee09618d77ad62d4086ba56b78569832d6fc7c3bcd5c2695bbc; every v0.3 artifact and negative result remains unchanged |
| integration source | Consumer Runtime PR #30 was squash-merged as 9914c6d358fd6b0acc5a945c0eab67aebb1b2b51; release-metadata PR #31 was squash-merged as 838ca0b5ab82be88440f2437fac2c2046bde672b; annotated tag v0.4.0 resolves to the latter exact commit |
| version transition | v0.4.0 is immutable and public; this state-sync change identifies subsequent development as 0.5.0.dev0 |
| release execution | tag run 30777530767 passed metadata, full tests, one-time build, OIDC publication with attestations, public verification, exact clean install and GitHub Release creation |
| published wheel | miniverl-0.4.0-py3-none-any.whl, SHA-256 8204c2f015e017ff15aadb465d0afa689949979f4aeba3b9b3abcc3f01c2511a |
| published sdist | miniverl-0.4.0.tar.gz, SHA-256 b6f87ff3a2a97f926301683b983920082719c893bb22a31759b17b0309b1e053 |
| independent public verification | GitHub Release downloads reproduce both public hashes; PyPI exposes provenance for both artifacts bound to release.yml, tag v0.4.0 and commit 838ca0b5; a refreshed clean Windows Python 3.10 install from https://pypi.org/simple reported miniverl 0.4.0, passed JSON doctor, and kept torch absent |
| release state | complete; PyPI and miniVERL v0.4.0 expose the same verified wheel and sdist |
| item | current state |
|---|---|
| audited baseline | fetched all remotes and started agent/v0.3-recoverybench from clean public main at 6c79c1fcc5a6e55dd9f4af843d446ece1c454431; no PR was open, main CI/build were green, and development advanced directly from 0.2.7.dev0 to 0.3.0.dev0 without a 0.2.7 release |
| available execution environment | Windows checkout with Python 3.12, Torch 2.13.0+cu130 and an NVIDIA GeForce RTX 4080 (16376 MiB); GitHub and Hugging Face access are available and publication remains restricted to exact validated tags |
| immutable baseline | calculator benchmark SHA-256 is 53fc1d4d5b7adee09618d77ad62d4086ba56b78569832d6fc7c3bcd5c2695bbc; calculator protocol-teacher HEAD is 23323751318135484c06c043b1f9b9e7016dd89f; existing tags remain untouched |
| phase boundary | RecoveryBench must be preregistered, fully measured with the sequential engine, frozen, merged, released as v0.3.0, publicly verified and state-synced to 0.4.0.dev0 before padded batching begins |
| current highest-risk work | RecoveryBench, release validation, integration and Trusted Publishing verification are complete; the next phase is the separately gated v0.4 consumer batched runtime |
| RecoveryBench environment | sqlite_recovery has 12 structurally disjoint versioned templates, deterministic controlled/natural/no-intervention subsets, structured tool-error provenance, executable recovery oracles, and exact recovery metrics; focused environment and backward-read tests pass |
| preregistration | public commit 7087b3a333463b88a62ffed73daee2c85d039145 and revision-1.3 digest 9c4c2ec19a56cebb2b2c1c0f3c7e504a9285467c99ae1590488251fbf2ff3934 bind the final procedure; the later public wall-budget amendment was reverted, its partial replacement stopped, and the already-complete frozen experiment was retained without rerun |
| teacher selection so far | the historical calculator teacher completed 96 eval tasks and failed the gate at 25.0% strict, 10.7% recovery, 95.8% parse validity and 14.4% tool execution; candidate A trained for 64 QLoRA SFT updates and its in-process NF4 eval measured 86.5%, 78.1%, 100% and 87.5%; a separately loaded full-precision reapplication failed at 65.6%, 65.6%, 100% and 62.7% and is retained as a noncanonical diagnostic |
| selected RecoveryBench teacher | candidate A reloaded on its preregistered NF4 base passed the independent 96-task eval-only gate at 90.6% strict success, 81.2% recovery, 100% parse validity and 87.1% tool execution; it is public at DaoyuanLi/mini-verl-qwen3-1.7b-sqlite-recovery-teacher@eb2747895ec32dab47c5b50c2d8aa9c0d9701e0d, with adapter weights SHA-256 5355f7007efb904d1b45a1aeb9b73b479b6f52025ab92502ab7895706155b2ba |
| portable adapters | a regression reproduced PEFT exporting a machine-local snapshot path as the base identity; export now rewrites the configured model ID and immutable revision before checksumming, and the portable export reload test passes |
| shared frozen data | schema-v3 benchmark execution prepares one frozen-student dataset per seed, reuses it across consumers and budget views, and can reuse the exact primary-view cold-start checkpoint; the two-consumer/two-view regression and all 80 v2/v3 benchmark tests pass |
| frozen-data fail closed | persisted offline KD now compares the loaded cold-start checkpoint content digest with the immutable dataset manifest before copying any data; a regression proves mismatched bundles are rejected, and final preparation uses recoverybench-v1-final-frozen-s{seed} rather than the calibration bundle |
| eval calibration and final freeze | all three 8-update calibration arms completed on eval only: SFT 6,675 positions / 50.5560895 s, frozen KD 6,467 / 52.0759896 s, and strict OPD 6,224 / 650.9775521 s; preregistration revision 1.2 freezes 6,224 selected positions and 50 continuation seconds, bound to calibration SHA-256 af0cb73c60655c37c4bafba6ea7893e4bb7260e82c6b2915bb646b8872cbe35e |
| frozen final specifications | schema-v3 equal-update, equal-selected-position and equal-wall-time configs cover the locked three seeds and test split; deterministic 10,000-replicate paired analysis and all three data-bound SVG generators are committed before final test |
| preserved v1.2 invalidation | the first partial final run exposed a real schedule defect: offline-kd-oracle collected only 8 oracle tasks and replayed them rather than using the common 64-task prefix; the process was stopped before that arm's test, the entire partial run and frozen bundle were moved intact to artifacts/superseded/recoverybench-v1.2-oracle-schedule-defect/, and none of its outcomes will enter the replacement analysis |
| preregistration correction | revision 1.3 explicitly freezes offline-kd-oracle.collection_tasks: 64, documents the defect and required every final arm/seed to restart from fresh cold starts; no invalid v1.2 outcome enters the publication set |
| final artifact audit | all 36 task artifacts, 4,608 trajectories and 101,787,618 raw bytes passed embedded SHA-256 and byte-count checks; each arm contains 128 final-test tasks, task IDs pair within seed, three cold checkpoints and three frozen datasets are reused exactly, and all 75 fresh-OPD update rollouts satisfy rollout policy version = current parameter version |
| equal-update result | frozen-student KD reached 23.2% strict success and 22.8% recovery after error versus 10.9% and 9.1% for strict fresh OPD; paired fresh-minus-frozen differences are -12.24 points (95% paired bootstrap -15.89 to -8.59, 384 tasks) and -13.79 points (-20.69 to -6.90, 116 paired error cases) |
| cost result | strict fresh OPD averaged 686.80 continuation seconds versus 52.10 for frozen KD; budget-50 queried 49.77% of model-generated positions but averaged 720.76 seconds because teacher backbone forwards were unchanged |
| secondary budgets | all equal-selected-position arms crossed the 6,224 target after eight steps with recorded overshoot; the nominal 50-second artifact is explicitly a cycle-capped wall diagnostic because SFT and frozen KD completed eight cycles while fresh OPD crossed the target in one indivisible step |
| immutable RecoveryBench results | equal-updates 6ce2e6837e12b99ebc4fad6d27ce3e69c92e295ff3b9b60e0f68c2d308022384; equal-selected-positions fe4c9afc799724dfe7a32e631676a1e5177c44559a7374d2ea31da135354f137; wall diagnostic 425b0fa568f37b09e61af731d3da5009bd3833bddde6efaf2c66e9dba8355cbe; task JSONL aff96bffc6da27240a852410ac041bd4d95badf34cad030e6f437be1491a55ad; paired analysis 8a6891f74aed80f07ec00d5ea1909895c579346e1abbb1d5d95a354bb46c6b81 |
| technical publication | the generated Markdown analysis, three SVGs and deterministic six-page PDF report are data-bound to the frozen JSON; native/README-width SVG inspection and every-page PDF inspection found no clipping or overlap |
| integration source | RecoveryBench PR #27 was squash-merged as bee82d3a6e2c8a5c3f4c62c0d3827a07483a1977; release-metadata PR #28 was squash-merged as 624c6a352db53e0bd2038c4aeb0669a16a402239; annotated tag v0.3.0 resolves to the latter exact commit |
| version transition | v0.3.0 is immutable and public; this state-sync change identifies subsequent development as 0.4.0.dev0 |
| release execution | tag run 30772772078 passed metadata, full tests, one-time build, OIDC publication with attestations, public verification, exact clean install and GitHub Release creation |
| published wheel | miniverl-0.3.0-py3-none-any.whl, SHA-256 e42404ced88b75ba4ff31541cb3df697da0eee09a68f603e4eadbf69a11d2032 |
| published sdist | miniverl-0.3.0.tar.gz, SHA-256 006ce418243286dd27e0731090bf9fa1711a1abc962ecfcc4d39807257539cb2 |
| independent public verification | GitHub Release downloads reproduce both public hashes; a no-cache Windows Python 3.10 install from https://pypi.org/simple reported miniverl 0.3.0, passed JSON doctor, and kept torch, Transformers, PEFT and bitsandbytes absent |
| release state | complete; PyPI and miniVERL v0.3.0 expose the same verified wheel and sdist |
| item | current state |
|---|---|
| audited baseline | fetched all remotes and started v0.2.6-final-concurrency from clean public main at ba0cbc33d731936dcb0e44fc42e29cae2f1c803d (0.2.6.dev0) |
| regression-first evidence | isolated pre-fix runs reproduced a real close() teardown race, six checkpoint-load overlap failures, six evaluation-mode restoration failures, and 22 semantic-key/URL/path privacy failures; deterministic concurrency tests use threading.Event, never timing sleeps |
| ownership and mode | close() owns the same non-blocking operation guard as train/evaluate/checkpoint work before changing state or releasing resources; public checkpoint load checks READY before and after ownership and delegates to an owner-only implementation; evaluation restores the exact prior model mode in finally |
| privacy | semantic components cover snake/kebab/camel/Pascal keys with explicit benign metadata exceptions; supported URL schemes retain public structure while removing userinfo; arbitrary embedded absolute Windows/UNC/POSIX paths are reduced to portable basenames without rewriting URLs, relative paths or mathematical slash expressions |
| focused adversarial pass | lifecycle, checkpoint-resume, standalone-evaluation, benchmark privacy and all shareable report-format tests pass 209 tests, including generated semantic keys, credentialed URLs and POSIX paths; an added false-positive probe found and fixed mid-field matching in loss/token = 0.5 |
| static and documentation gates | git diff --check, Ruff check/format, mypy over 77 source files, actionlint 1.7.12 with verified upstream checksum, generated PYPI.md, Markdown links, generated schema and generated benchmark-SVG byte comparison all pass |
| complete local suite | after refreshing editable distribution metadata from 0.2.6.dev0 to source 0.2.6, the full non-GPU/non-network gate passed 1348 tests, 6 deselected, at 87.46% branch coverage; the available RTX 4080 gate passed 5 and the network gate passed 3 |
| compatibility evidence | torch-free core passed 1058 tests on each of Python 3.10, 3.11, 3.12 and 3.13; the minimum bundle (Torch 2.3.1, Transformers 4.51.3, PEFT 0.12.0, Accelerate 0.33.0, NumPy 1.24.4, bitsandbytes 0.43.3) and latest bundle (Torch 2.13.0, Transformers 5.14.1, PEFT 0.20.0, Accelerate 1.14.0, NumPy 2.5.1, bitsandbytes 0.50.0) each passed 2 no-network training smokes and the 133-test HF/config suite |
| platform concurrency | the Windows complete suite includes the deterministic operation matrix; a fresh WSL2/Linux Python 3.12 environment at implementation commit 7410674802fa178fdde1ed5064c17608cd3fa806 passed 76 lifecycle, standalone-evaluation and resume/concurrency tests; PR and post-merge Ubuntu CI both passed |
| packaging and installs | one 0.2.6 wheel and one sdist pass Twine; the extracted sdist passes Ruff, format, mypy and 1348 CPU tests, rebuilds a wheel with the same 78-file runtime inventory, and clean Python 3.10 core plus Python 3.12 [train] installs pass version/doctor and demo/inspect/report/standalone-eval smokes |
| immutable evidence | the frozen calculator JSON remains byte-identical at SHA-256 53fc1d4d5b7adee09618d77ad62d4086ba56b78569832d6fc7c3bcd5c2695bbc; all existing tags and DaoyuanLi/mini-verl-qwen3-1.7b-protocol-teacher@23323751318135484c06c043b1f9b9e7016dd89f remain unchanged |
| integration source | PR #25 was squash-merged as 59fe738709526a13f354a744ab763f13530de4d1; annotated tag v0.2.6 resolves to that exact commit |
| version transition | v0.2.6 is immutable and public; this state-sync change identifies subsequent development as 0.2.7.dev0 |
| release execution | tag run 30722451004 passed metadata, full tests, one-time build, OIDC publication, public hashes and attestations, clean public install, and GitHub Release creation |
| published wheel | miniverl-0.2.6-py3-none-any.whl, SHA-256 11d6b001752c41a0100f12c29b125a9dc082703dbeadc6b0317a88ac818d8695 |
| published sdist | miniverl-0.2.6.tar.gz, SHA-256 91e7b2918286c342cacaf2582dbed57c1e7a1bf4e1064d327e349b1d77c28886 |
| independent public verification | GitHub Release downloads reproduce both public hashes; a no-cache Windows Python 3.10 install from https://pypi.org/simple reported miniverl 0.2.6, passed doctor, and kept torch, Transformers, PEFT and bitsandbytes absent |
| release state | complete; PyPI and miniVERL v0.2.6 expose the same verified wheel and sdist |
| item | current state |
|---|---|
| audited baseline | fetched all remotes and started v0.2.5-final-correctness from clean public main at 3d3d59993a99bb1a7c1225f970a3e052a43e4dc1 (0.2.5.dev0) |
| integration source | PR #22 was squash-merged as a9a84510741b4ade8a405c100affdf1caed55ae6; annotated tag v0.2.5 resolves to that exact commit |
| version transition | v0.2.5 is immutable and public; this state-sync change identifies subsequent development as 0.2.6.dev0 |
| regression-first evidence | the focused pre-fix run reproduced 23 failures and 472 passes: SQLite raised ValueError/OverflowError for nan, infinities, overflow notation and a huge integer through both direct verification and the real rollout path; every built-in protocol-v2 final example remained the placeholder answer |
| focused correctness pass | python -m pytest over the changed verifier, protocol, privacy, lifecycle, locking, packaging, CLI and standalone-eval surfaces passed 798 tests; the old implementation had separately reproduced 23 verifier/protocol failures, 16 privacy failures, 5 lifecycle/concurrency failures, the report-lock race, two eval-lock races and the nested-link defect |
| static gates | git diff --check, ruff check ., ruff format --check ., mypy src/miniverl and cached actionlint all pass |
| complete local suite | after refreshing stale editable metadata from 0.2.4 to source 0.2.5.dev0, the full non-GPU/non-network gate passed 1278 tests with 87.37% branch coverage; the available RTX 4080 gate passed 5, and the network gate passed 3 |
| compatibility evidence | torch-free core passed 1021 tests on each of Python 3.10, 3.11, 3.12 and 3.13; the minimum bundle (torch 2.3.1, Transformers 4.51.3, PEFT 0.12.0, Accelerate 0.33.0, NumPy 1.24.4, bitsandbytes 0.43.3) and latest bundle (torch 2.13.0, Transformers 5.14.1, PEFT 0.20.0, Accelerate 1.14.0, NumPy 2.5.1, bitsandbytes 0.50.0) each passed both no-network training smokes; Transformers 4.51.3 and 5.14.1 each passed the 133-test HF/config compatibility suite |
| multiprocessing platforms | the complete Windows suite includes the new spawn-based report/eval/run-lock races and passed; a local WSL launch was unavailable because its VHDX returned ERROR_SHARING_VIOLATION, while the PR and post-merge Ubuntu CI gates passed |
| lifecycle and ownership | new writable manifests start ready, transition atomically to running, and close-before-train as closed_before_training; training-private eval/checkpoint implementations are separated from public calls, automatic reports remain under the writer lock, and standalone eval transfers one pre-acquired lock before reading mutable state |
| privacy and packaging | semantic secret suffixes, authorization/cookie/session fields, URL userinfo, paths with spaces, UNC and HTML-escaped variants are redacted; the PyPI generator rewrites nested linked images and both generated PYPI.md and built wheel METADATA reject all remaining relative project targets |
| final-version packaging gate | an isolated 0.2.5 build produced one wheel and one sdist and passed Twine; the extracted sdist passed Ruff, format, mypy and 1278 non-GPU/non-network tests, rebuilt a wheel with the same 78-file runtime inventory, and exposed zero relative metadata targets |
| clean-install gate | a clean Python 3.10 core wheel install reported 0.2.5, passed doctor, and kept torch/Transformers/PEFT/bitsandbytes absent; a clean Python 3.12 wheel + [train] install ran demo, inspect, report, and weights-only standalone eval |
| immutable evidence | no frozen tag, adapter revision, or benchmark result was changed; the required benchmark SHA-256 remains 53fc1d4d5b7adee09618d77ad62d4086ba56b78569832d6fc7c3bcd5c2695bbc |
| release execution | tag run 30611603505 passed metadata, full tests, one-time build, OIDC publication, public hashes and attestations, clean public install, and GitHub Release creation |
| published wheel | miniverl-0.2.5-py3-none-any.whl, SHA-256 70c98284bce151fc74b508047b354929846efb71c3fe8f451c0d0ba1bec48e9d |
| published sdist | miniverl-0.2.5.tar.gz, SHA-256 d30bb07ebca676a3960d4b5c46075a8a2e13e58629b96984e30f8f7bab67dce0 |
| independent public verification | a no-cache Windows Python 3.10 install from https://pypi.org/simple reported miniverl 0.2.5, passed doctor, and kept torch, Transformers, PEFT and bitsandbytes absent; the public simple index exposes provenance links for both distributions |
| release state | complete; PyPI and miniVERL v0.2.5 expose the same verified wheel and sdist |
| item | current state |
|---|---|
| integration source | PR #18 was squash-merged as 57dec193af88b462dcc41d82fc6fecb813e161fd; annotated tag v0.2.4 resolves to that exact commit |
| version transition | v0.2.4 is immutable and public; post-release development identifies as 0.2.5.dev0 |
| immutable evidence | the v0.2 calculator benchmark remains byte-identical at SHA-256 53fc1d4d5b7adee09618d77ad62d4086ba56b78569832d6fc7c3bcd5c2695bbc; earlier tags and the pinned protocol-teacher adapter are unchanged |
| strict protocol and numerics | a shared bounded strict decoder rejects non-finite values, duplicate keys, 128+ digit integers, depth over 32, more than 256 members and invalid surrogates; protocol-v2 requires complete numeric/unit verification while historical protocol/verifier-v1 remains explicit |
| lifecycle and locking | TrainerState has atomic one-shot entry and stable terminal behavior; filelock protects run construction, resume, overwrite, evaluation, report and export across processes before model loading |
| packaging and provenance | tag-pinned PyPI documentation, self-testing sdist, lean wheel, exact submitted/validated/resolved configuration layers and canonical portable public artifacts passed the release gates; exact CPU/GPU/network evidence remains in docs/generated/quality.json, bound to implementation commit 6911acf11978cfc5f6a99375cbe7a1666fcf7a85 |
| release execution | tag run 30522484949 passed metadata, tests, one-time build, OIDC publication and attestation generation; it ended red only because urllib received PyPI's browser challenge before public install and GitHub Release creation |
| verified recovery | PRs #19 and #20 hardened recovery and HTML-banner verification. Run 30524088015 bound the original artifacts to the immutable tag, verified hashes, pinned links and Sigstore attestations, installed the exact public version, and created the GitHub Release without rebuilding or re-uploading |
| published wheel | miniverl-0.2.4-py3-none-any.whl, SHA-256 3f5a239bbbd2f85217cf11f691fbb63f647092f67b82da4de38bd6907c5ab0f1 |
| published sdist | miniverl-0.2.4.tar.gz, SHA-256 03f0e844df2c91deed5c211cdd2dd598d22f03d59d99cd8e792a58211c0b2296 |
| independent verification | GitHub Release downloads reproduce both public hashes; a no-cache Python 3.10 install from https://pypi.org/simple reports miniverl 0.2.4, passes doctor, and keeps torch, Transformers, PEFT and bitsandbytes absent |
| publication authorization | granted by the maintainer and completed through Trusted Publishing without a long-lived PyPI token |
| exact blocker | none |
| item | current state |
|---|---|
| integration source | PR #15, squash-merged as 3e54ebf9a8501f58cb6a0901d827221aba8792e0; release metadata PR #16 merged as 38924da743180e6767f1e3b252feafdccd70759b after all ten checks passed |
| version transition | v0.2.3 was released from exact commit 38924da743180e6767f1e3b252feafdccd70759b; post-release development now identifies as 0.2.4.dev0 |
| cache safety | shard size survives reopen; allocation advances past the highest indexed or on-disk numeric suffix; pruning publishes a copied index before best-effort orphan cleanup, with fault-injection coverage on both failure boundaries |
| protocol examples | schema-v2 examples come from each active environment ToolSpec; calculator, JSON navigation, SQLite and custom-environment round trips pass while protocol v1 remains byte-frozen |
| presentation | the generated figure scopes claims to the saturated v0.2 calculator task, reports completed controls at 0%, omits the unsourced preparation duration and remains legible at native and README widths |
| frozen scientific artifact | benchmarks/results/gpu-calc-hard-equal-update-v2.json remains byte-identical at SHA-256 53fc1d4d5b7adee09618d77ad62d4086ba56b78569832d6fc7c3bcd5c2695bbc |
| release gates | release run 30513947051 passed metadata, 1085 tests at 87.30% branch coverage, one-time build, OIDC publication, public hashes/attestations, clean public install and GitHub Release creation |
| published wheel | miniverl-0.2.3-py3-none-any.whl, SHA-256 033e51bfbdae20a91d942ef7a5c22ef6c8a00317cc9b775b102d303f2e1a6619 |
| published sdist | miniverl-0.2.3.tar.gz, SHA-256 6f7d20fd4b4a90e6a3fe1e97c9ced26268e013bb87462ba75a7d09510bd2f011 |
| publication authorization | granted on 2026-07-29 and completed through the tag-only Trusted Publishing workflow without a long-lived PyPI token |
| exact blocker | none; PyPI and GitHub expose byte-identical artifacts, attestations passed, independent Python 3.10 core installation passed, and development has advanced past the immutable release |
| item | current state |
|---|---|
| integration source | PR #13, merged as 518590cb43ff788fa65f73ee9cf3a7afb6dfba5a after all pull-request and synchronized-main checks passed |
| version transition | v0.2.2 was released from exact commit 518590cb43ff788fa65f73ee9cf3a7afb6dfba5a; post-release development now identifies as 0.2.3.dev0 |
| hardware scope | one NVIDIA CUDA GPU with no device-name allowlist; automatic dtype selects bf16 when supported and fp16 otherwise; exact fit remains model/budget dependent |
| measured evidence | the only published real-model GPU measurements remain the RTX 4080 runs; RTX 3070, Titan V, RTX 5090 and other cards are portable code paths, not fabricated benchmark claims |
| presentation | banner and generated benchmark SVG use a dark single-GPU visual system; grid lines begin below tick labels and diagnostic 0% controls are status pills |
| package discovery | both READMEs, project metadata and the GitHub About homepage expose https://pypi.org/project/miniverl/ |
| frozen scientific artifact | benchmarks/results/gpu-calc-hard-equal-update-v2.json remains byte-identical at SHA-256 53fc1d4d5b7adee09618d77ad62d4086ba56b78569832d6fc7c3bcd5c2695bbc |
| release state | release run 30494182647 passed on attempt 3 after PyPI simple-index propagation; OIDC publication, public hashes, attestations, clean install and GitHub Release all passed |
| published wheel | miniverl-0.2.2-py3-none-any.whl, SHA-256 1ead97173bb11ce3da963b94f628df825a5b14648fed488cf4d88c47cba9dd59 |
| published sdist | miniverl-0.2.2.tar.gz, SHA-256 3951dd4addc5d85b3e58ce72ecffac65c38bf2eab951d2c08cce8f20c886185c |
| exact blocker | none; public PyPI and GitHub Release artifacts are byte-identical, and clean Python 3.10 and workflow Python 3.12 core installations passed |
| item | current state |
|---|---|
| audited starting commit | public and local main both resolved to 5b1c043b188b30b1261e118293f6fe124e2b7acb after git fetch --all --prune on 2026-07-29 |
| integration source | branch v0.2.1-correctness and correctness PR #11, created without rewriting main or the immutable v0.2.0 tag; the PR remains the authoritative integration record after its source branch is deleted |
| version transition | v0.2.1 was released from exact commit 591881b0d094f5c53ff47a9419e679b762fb44b0; post-release development now identifies honestly as 0.2.2.dev0, while changelog and citation metadata retain the published 0.2.1 record |
| release state | annotated tag v0.2.1 resolves to 591881b0d094f5c53ff47a9419e679b762fb44b0; release run 30474597179 passed all five jobs, including OIDC publication, public verification and GitHub Release creation |
| publication authorization | granted on 2026-07-29; the release completed through the tag-only Trusted Publishing workflow without a long-lived PyPI token |
| frozen scientific artifact | benchmarks/results/gpu-calc-hard-equal-update-v2.json remains required at SHA-256 53fc1d4d5b7adee09618d77ad62d4086ba56b78569832d6fc7c3bcd5c2695bbc |
| immutable protocol adapter | DaoyuanLi/mini-verl-qwen3-1.7b-protocol-teacher@23323751318135484c06c043b1f9b9e7016dd89f; its original v1 competence record is accepted for v1 without fabricating v0.2.1 metrics, and it was neither overwritten nor retrained |
| published wheel | miniverl-0.2.1-py3-none-any.whl, SHA-256 0177d5
10BC8
0026da86047c2a03f90e7786c794b26c5b0d6fef193c58ed35c08d8cda |
| published sdist | miniverl-0.2.1.tar.gz, SHA-256 80f890c1ab8be0ccdf6c5ce293a5c4d7bb6a6f7ab7a57db34090384fcaa7e16c |
| exact blocker | none; PyPI and the GitHub Release expose byte-identical distributions, PyPI reports Trusted Publishing attestations, and a clean public core install passed |
| area | defect and implemented invariant | regression evidence |
|---|---|---|
| run lifecycle | a new run could silently attach to a non-empty directory; creation is now exclusive, generated IDs include microseconds plus randomness, resume/overwrite are explicit and mutually exclusive, and overwrite uses rollback-safe whole-directory replacement | run creation, collision, concurrency, stale append, demo and benchmark resume tests; final-wheel collision exited 1 with every file hash unchanged, then --overwrite produced status=completed, eight fresh metric records and no stale standalone-eval file |
| checkpoints/eval | checkpoint names could be selected lexicographically, writes were not atomic/checksummed, and eval could load fresh or partial weights; complete sibling-temp checkpoints now have a manifest, SHA-256/size/content digest and identity, and selection uses validated state.global_step |
corruption, missing weights, duplicate-step ambiguity, temp-ignore, stale-final, legacy-v0.2 and fail-before-mutation tests; clean-wheel standalone eval loaded only weights and reported checkpoint/global step 4, parameter version 4 and checksummed_v1 |
| offline KD | resume could regenerate the nominally fixed dataset; trajectories, task order, exact spans/tails and provenance are now a first-class checksummed artifact | uninterrupted/resumed parameter, optimizer, task order, cache and dataset digest equality tests |
| manifests | startup and terminal state could be conflated; immutable manifest.start.json and atomic completed/failed/interrupted manifest.json records preserve original failures and actual counters/digests |
lifecycle, exception-preservation and fault-injection tests |
| OOM transaction | retry boundaries could repeat stochastic work or an optimizer commit; only gradient computation is retryable with RNG restoration and gradient clearing, while optimizer commit is single-shot | RNG/dropout equivalence, one-commit, optimizer-OOM and non-OOM tests |
| protocol/adapter | the historical v1 prompt has an ambiguous example, but changing it would invalidate the published adapter; v1 is frozen, v2 examples parse, and competence is protocol-version aware | v1 byte fixture, v2 parse/round-trip and v1/v2 adapter-gate tests; the immutable Hub revision passed the live network test |
| losses/metrics | SFT span metrics could report zero divergence instead of CE, and tool events were double counted; objective/divergence/CE are distinct and emitted/parsed/executed/error/final events have precise denominators | per-token aggregate equivalence and adversarial agent-event tests |
| cache/identity | empty tails, ordered span types, checksum flags, adapter identity and tokenizer/model revisions could be ambiguous or lossy; schema v2 fixes each while retaining v1 reads | exact-zero, ordering, checksum, adapter, exact-full-vocab, structural tokenizer, revision and LM-head tests |
| parameter versions | cycle count was used as a proxy for parameter changes; parameter_version advances only after a successful commit and rollout/optimizer counters are separate |
no-op, failed update, replay and exact-resume tests |
| reset/lifecycle API | the runner could ignore reset()'s observation and examples could leak trainer resources; reset is authoritative and public examples use context managers |
dynamic reset/state-ID/exactly-once and destructive close/context tests |
git diff --check,ruff check .,ruff format --check .,mypy src/miniverland actionlint 1.7.12 all pass.pytest -q -m "not gpu and not network" --cov=miniverl --cov-report=term-missing --cov-fail-under=80: 1066 passed, 6 deselected, 87.11% branch coverage.pytest -q -m gpuon Torch 2.13.0+cu130 and an NVIDIA GeForce RTX 4080: 5 passed, 1067 deselected.pytest -q -m network: 3 passed, 1069 deselected, including the pinned public protocol-teacher revision.- Minimum boundary, Python 3.10.11 / Torch 2.3.1+cpu / Transformers 4.51.3 / PEFT 0.12.0 / Accelerate 0.33.0 / NumPy 1.24.4 / bitsandbytes 0.43.3: 132 passed under the offline guard.
- Current boundary, Python 3.13.13 / Torch 2.13.0+cpu / Transformers 5.14.1 / PEFT 0.20.0 / Accelerate 1.14.0 / NumPy 2.5.1 / bitsandbytes 0.50.0: 132 passed under the offline guard.
- Clean build produced exactly one wheel and one sdist; both passed
twine check. Wheel/sdist content assertions passed. - A clean core-wheel environment ran help/version/doctor without installing or
importing torch, Transformers, PEFT or bitsandbytes. A separate clean
[train]environment ran demo, inspect, report, collision, overwrite and weights-only standalone evaluation. - Every run recipe validated, every benchmark config resolved, the generated JSON Schema byte-matched the committed file, every benchmark result validated, all tracked JSON/JSONL parsed, 26 Markdown files passed link checking, and no model weights/caches/databases are tracked.
- The banner and benchmark SVG were rendered at native size and inspected.
docs/gpu-calc-hard-equal-update-v2.svgremains generated from the frozen JSON and labels the cold startNO TRAININGand the diagnostic controlsPROTOCOL MISMATCHinstead of presenting inapplicable zero bars as outcomes. - GitHub environment
pypiexists with a branch policy;release.ymlused OIDCid-token: writeto publish and independently verifyminiverl 0.2.1.
Currently failing commands: none.
Publication record: PR #11, annotated tag v0.2.1, release run 30474597179,
the public PyPI files and the GitHub Release are authoritative. The published
tag and distributions are immutable; new work proceeds from 0.2.2.dev0.
| item | current state |
|---|---|
| public repository | https://github.com/DaoyuanLi2816/mini-verl |
| release-code merge | PR #6 and the final state sync in PR #7 are merged; release commit 6092706b4a4e750c4571d7d6a7decbc26af851b2 is recorded as annotated tag v0.2.0 |
| previous integration | PR #4 and PR #5 are merged |
| release branch | v0.2-final-release was merged and deleted |
| final integration | PR #6 and PR #7 merged on 2026-07-28; all eight checks on each PR and both post-merge main workflows passed |
| lifecycle fix | merged on main: destructive, idempotent cleanup and explicit cold-start/arm context boundaries are implemented |
| public default-branch health | final pre-tag ci run 30417912194 and build run 30417912198 completed successfully |
| final-pass status | complete locally: 1004 CPU tests, 5 GPU tests, 129 focused offline tests, dual Transformers compatibility, build, clean-wheel, README-command, link, schema, JSON/JSONL, privacy and release-workflow gates pass |
| version | 0.2.0 published on 2026-07-28 (2026-07-29 UTC); tag v0.2.0 resolves to 6092706b4a4e750c4571d7d6a7decbc26af851b2 |
| release workflow | tag run 30421231859 passed metadata, full tests, single-build, OIDC publish, public verification and GitHub Release jobs |
| PyPI | miniverl 0.2.0 was created by Trusted Publishing; wheel SHA-256 cf850a6333483a3ee22c0c0e98df1e1b2e6faa184480573e0666658b53a29262, sdist SHA-256 3d5107b4f6351204335f800ce924208843f08f54441378bd9f25c3c6fa17456b |
| GitHub Release | miniVERL v0.2.0 is public and contains the same wheel, sdist and SHA256SUMS |
| GitHub environment | pypi created through the repository API on 2026-07-28; custom deployment policy permits only tags matching v* |
| external publication blocker | none; the exact pending publisher was registered, the first OIDC upload succeeded, and PyPI records release.yml on DaoyuanLi2816/mini-verl as the publisher |
| Hugging Face adapter | public at https://huggingface.co/DaoyuanLi/mini-verl-qwen3-1.7b-protocol-teacher, immutable head 23323751318135484c06c043b1f9b9e7016dd89f |
| scientific artifact | schema-v2 JSON remains byte-identical at SHA-256 53fc1d4d5b7adee09618d77ad62d4086ba56b78569832d6fc7c3bcd5c2695bbc |
Focused final-pass evidence:
pytest -q tests/unit/test_offline_contract.py tests/integration/test_hf_backend_offline.py tests/unit/test_config.py-- 129 passed. Actualtrain,benchmark,evalandexport-adaptercommands also completed with--offline; the export loaded the cached pinned Qwen3 revision and wrote a standard PEFT adapter.- Socket denial exposed a Transformers 5.x PEFT auto-detection request even
with the top-level local-only flag. Version-compatible PEFT probe kwargs now
carry
local_files_onlyexplicitly on 5.x without duplicating the parameter on 4.x; missing model, tokenizer, full adapter snapshot and partial adapter snapshot tests all record zero socket attempts. - Hub adapter validation returns public provenance plus one exact local snapshot and its config, weights and manifest paths. PEFT receives only that local snapshot; public manifests retain no cache path.
release.ymlpassed actionlint 1.7.12. Static tests prove manual dispatch and branch pushes cannot publish, av*tag push can, distributions build once, and PyPI verification precedes GitHub Release creation.ruff check .,ruff format --check .andmypy srcpass. The complete CPU suite reports 1004 passed, 5 deselected; the CUDA suite reports 5 passed, 1004 deselected on the RTX 4080.- Independent Python 3.12 environments with Transformers 4.51.3 and 5.14.1 each report 124 passed for the offline Qwen3/PEFT/config compatibility bundle.
- A fresh isolated build produced the 0.2.0 wheel and sdist and both passed
twine check. A wheel-only environment confirmed torch absent and core commands ready; a second[train]environment completeddemo --fast,inspectandreport. - All 117 local Markdown targets, 47 Markdown anchors and 14 external Markdown URLs passed. Ten representative runtime JSONL files (115 records) parsed strictly; package/schema/SVG/privacy/release tests report 88 passed.
- The frozen benchmark remains byte-identical at SHA-256
53fc1d4d5b7adee09618d77ad62d4086ba56b78569832d6fc7c3bcd5c2695bbc.
Currently failing commands:
- None. The tag release, public artifact verification and clean public install all pass.
Post-release follow-up:
- Upload
docs/banner.svg, rendered at 1280×640 PNG, as the repository social preview through the authenticated repository settings page. - Keep
v0.2.0immutable. Future package changes require a new version and tag; never move or replace the published tag.
This section records the final local evidence snapshot prepared for integration through PR #4. The PR link is authoritative for its later merge state.
| item | evidence snapshot |
|---|---|
| public repository | https://github.com/DaoyuanLi2816/mini-verl; fetched origin/main is 3383f2b9a3c595e0fa143fecdc27522ab368b27f |
| working branch | v0.2-protocol-aligned-opd; foundational commits are pushed and the measured-result integration passes the final local gates |
| version | 0.2.0 release candidate; no tag or publication has been created |
| current PR | draft #4 |
| GPU runs | protocol teacher, adapter export/gate, 4 GPU tests and all 10 benchmark arms completed locally on the RTX 4080 |
| PyPI/release | not published, no tag, no upload authorized |
Completed v0.2 work in the current worktree:
- Benchmark schema/config v2 separates common and cold-start overrides, performs a declared structured diff before model loading, records resolved hashes and cumulative accounting, and retains v1 read/JSON-schema compatibility.
- Strict OPD freshness is the default and rejects multiple optimizer updates
from one rollout batch; explicit replay is labeled
online_distillation_with_replay. - The ambiguous public
loss.ce_weightname is replaced bysampled_token_nll_weightfor distillation; a nonzero legacy name is rejected. - Frozen standard PEFT teacher adapters can be validated, loaded and exported with checksums and provenance. The no-network local Qwen-compatible round trip, freeze checks, wrong-base/tokenizer checks and missing-file checks pass.
recipes/qwen3_1.7b_protocol_teacher_sft.yamlvalidates and its exactminiverl train ... --dry-run --jsoncommand succeeds.- Policy evaluation now records strict success as primary plus lenient
diagnostic success, valid tool-call rate/count, final-format validity and
average turns. Protocol-token accuracy is explicitly
null/not_applicablefor unaligned free-running trajectories. k == Vbypasses tail smoothing; temperature-squared claims are narrowed and a deterministic 48-cell gradient sweep covers three divergences, four temperatures and two logit regimes.- README source-install and CUDA installation order are corrected in English and Chinese.
- Candidate A reached 100% strict held-out tool-policy success, passed its prespecified 50% gate and was exported with complete checksums; candidates B and C were not run by design.
- The five-arm equal-update GPU benchmark completed at both prespecified seeds. Cold start was 75% twice; SFT and protocol-teacher OPD were 100% twice; raw-teacher and privileged-context OPD were 0% twice.
- Published schema-v2 provenance is made portable before hashing. The result JSON, Markdown and data-bound SVG pass schema, privacy and consistency tests.
Evidence executed on this branch:
ruff check .andruff format --check .— clean, 148 files formatted.mypy src/miniverl— clean, 73 source files.- complete offline/CPU suite — 966 passed, 4 GPU tests deselected, 86.55% branch coverage over 6405 statements.
- Transformers 4.51.3 boundary environment — offline Qwen3/PEFT/config bundle, 117 passed.
python -m buildandpython -m twine check dist/*— 0.2.0 sdist and wheel built, both metadata checks passed.- committed benchmark schema exactly matches
miniverl schema; every recipe validates and the protocol-teacher dry run reports 24 optimizer steps and 192 oracle traces. - focused benchmark/config/cache suite — 129 passed before the adapter slice.
- no-network PEFT adapter suite — 14 passed.
- policy metrics, benchmark v2, packaging, toy pipeline and offline HF bundle — 119 passed; its only failure was a duplicate test tokenizer ID, corrected and rerun as 1 passed.
- loss/property/temperature sweep suite — 45 passed.
- protocol-teacher recipe
validate --jsonandtrain --dry-run --json— valid, 24 planned optimizer steps and 192 oracle traces. pytest -q -m gpu— 4 passed on the RTX 4080.- Candidate A training — 24 optimizer steps in 554.9 s; final strict/lenient success, valid-call rate and final-format validity all 100% on 24 tasks.
- Adapter export/validation — checkpoint tree
e9c42893b861e371dd48e2c151940a198e22eff2f91649ca6a5303c525c5ee4c, adapter weights8df7e7bc1b8283b910aa13bc4173083ae20c838bcacb366d7dbcabc7b310b994, manifest502bca7489c6fe161ebf198d2a1b4622123d4f958885a7e4714c6a02a2e1ac43. - Complete GPU benchmark — 5 arms x 2 seeds, all 10 completed; published JSON
SHA-256
53fc1d4d5b7adee09618d77ad62d4086ba56b78569832d6fc7c3bcd5c2695bbc. - Fresh 0.2.0 build — sdist and wheel passed
twine check; a wheel-only venv ran--help,--versionanddoctor --jsonwith torch absent, while a second[train]venv completeddemo --fastand inspected its trajectories.
Measured scientific conclusion: protocol competence prevents the observed raw/privileged-teacher collapse, but protocol-teacher OPD only ties SFT at 100% on both seeds. There is no measured OPD-over-SFT advantage.
Exact integration steps at this snapshot:
- Commit and push the portable artifacts and updated interpretation, then make draft PR #4 ready once remote checks are green.
- Merge, verify
mainand its public artifacts, then close superseded dependency PRs without creating a tag, release or PyPI upload. - Preserve the two-seed result as the v0.2 baseline for a future, less-saturated task-family experiment.
At this snapshot, phase 7 is at v0.2 release-candidate integration. The protocol-teacher experiment, both benchmark seeds and all local gates are complete; PR #4 contains the integration state.
| item | value | how |
|---|---|---|
| OS | Windows 11 Pro 10.0.22631 | platform.platform() |
| GPU | NVIDIA GeForce RTX 4080, 16376 MiB, driver 596.49 | nvidia-smi --query-gpu=name,memory.total,driver_version |
| dev interpreter | CPython 3.12.13 (.venv, uv-managed) |
uv venv --python 3.12 |
| torch | 2.13.0+cu130, torch.cuda.is_available() == True |
python -c "import torch; ..." |
| transformers | 5.14.1 | importlib.metadata.version |
| peft | 0.19.1 | idem |
| bitsandbytes | 0.50.0, bnb.nn.Linear4bit present |
idem |
PyPI project miniverl |
PUBLISHED (0.2.0, Trusted Publishing, 2026-07-28) |
public PyPI JSON, Integrity API and clean install |
Confirmed twice against huggingface.co/api/models/... with a negative control
(an all-zero SHA returns 404, so the endpoint really validates revisions):
| role | model | revision | notes |
|---|---|---|---|
| student | Qwen/Qwen3-0.6B |
c1899de289a04d12100db370d81485cdf75e47ca |
Apache-2.0, 28 layers, hidden 1024 |
| teacher | Qwen/Qwen3-1.7B |
70d244cc86ccca08cf5af4e1e306ecf908b1ad5e |
Apache-2.0, 28 layers, hidden 2048 |
tokenizer.json is byte-identical across the pair
(sha256 aeb13307a71acd8fe81861d94ad54ab689df773318809eed3cbe794b4492dae4).
config.vocab_size is 151936 while len(tokenizer) is 151669 -- the vocabulary
is padded. miniVERL sizes the cache and top_k from the model's output
dimension, never from the tokenizer; a GPU test asserts both numbers.
This table is retained as the v0.1 evidence baseline. Current v0.2 evidence is recorded in the takeover checkpoint above and will replace this table after the GPU experiment and clean-install verification finish.
| gate | command | result |
|---|---|---|
| lint | ruff check . |
clean |
| format | ruff format --check . |
clean, 138 files |
| types | mypy src/miniverl |
clean, 71 source files |
| tests | pytest -q -m "not gpu and not network" --cov=miniverl |
934 passed, 4 deselected, 85% branch coverage over 5911 statements |
| GPU tests | pytest -q -m gpu |
4 passed |
| build | python -m build |
sdist + wheel built |
| metadata | python -m twine check dist/* |
PASSED for both |
| wheel completeness | enumerate src/miniverl against the wheel |
all 77 entries present |
| clean install (core) | wheel into a fresh venv | --help, --version, doctor --json work; torch absent |
| clean install (train) | wheel + [train] into a second fresh venv |
doctor resolves a CPU-only torch and reports GPU training / QLoRA unavailable with the right remedy; both examples/ scripts run to completion |
| determinism | two miniverl demo runs in the clean venv, artifact-by-artifact |
checkpoints and trajectories bitwise identical; eval.json differs only in seconds, throughput, timestamp and run id |
| artifact portability | every *.json / *.jsonl parsed with parse_constant set to reject |
strict-valid (this gate found 16 bare NaN tokens, now fixed) |
| published results | jsonschema validate against benchmarks/schema/ + personal-information scan |
both result files valid and clean |
| docs links | every relative markdown link and #anchor resolved |
37 files, 0 broken |
| CLI/docs parity | every miniverl <cmd> in the docs matched against the Typer app |
all 11 commands documented, no phantom commands |
| unfinished markers | rg "TODO|FIXME|XXX|HACK|NotImplementedError" src tests examples scripts |
none |
tests/unit/test_losses_exact.py-- 21 passed. Brute-force Python references for forward KL, reverse KL and beta-JSD; an orientation test that catches an accidental KL reversal; thelog 2bound on symmetric JSD; finiteness at logit scale 1e4; finite gradients; fp16/bf16 upcast; theT^2contract.tests/unit/test_losses_bucketed.py-- 30 passed.log1mexpagainst a reference; tail-mass validity;k == Vconverging to the exact loss; the data-processing-inequality bound for k in {1,2,8,32} across all three divergences; monotonicity in k; a bounded reverse-KL tail penalty.tests/unit/test_chunked_equivalence.py-- 22 passed. The two-stage detach/backward reproduces the unchunked gradient for chunk sizes 1, 5 and 37 across all three divergences; zero-weight positions provably contribute nothing; CE mixing is convex;loss_scaleaffects gradients only.tests/property/-- 14 passed under Hypothesis, including the data-processing inequality evaluated in float64 to1e-9.
tests/unit/test_token_provenance.py-- 30 passed. A mask that marks atool_resulttoken trainable is rejected; spans must tile the sequence; position 0 can never be a target; privileged-context alignment computes per-segment offsets and rejects a target-token mismatch, a length-changed segment, a missing segment, a duplicate key and a tokenizer mismatch.
Oracle solves 6/6 train tasks for every (environment, difficulty) pair;
train/eval/test prompt sets are pairwise disjoint. SQLite lockdown, all
BLOCKED: DROP, INSERT, UPDATE, ATTACH, PRAGMA,
SELECT * FROM sqlite_master (authorizer), SELECT randomblob(10) (function
whitelist), two statements in one call.
tests/integration/test_toy_pipeline.py-- SFT, offline KD and OPD each run to completion and write every documented artifact; OPD increments its policy version once per cycle; offline KD announces that it is reusing fixed targets; no stored trajectory ever marks tool output trainable.tests/integration/test_resume_and_swap.py-- an interrupted-and-resumed run matches an uninterrupted one to1e-6parameter-for-parameter;swapmatchesresidentto the same tolerance; checkpoints contain no pickle; the OOM retry only halves the chunk and gives up with actionable advice.tests/integration/test_hf_backend_offline.py-- a realQwen3ForCausalLMbuilt from a config (no download): adapter resolution through PEFT, selected-position projection matching a full forward to1e-4, KV-cache generation matching a cacheless reference token for token, and gradients reaching only the LoRA tensors.
-
Historical raw-teacher recipe,
recipes/qwen_consumer_gpu_calc_raw_teacher.yaml, run idrtx4080-calc-opd: 16 optimizer steps in 481.1 s; held-out greedy success 0.0% -> 100.0% on 12 tasks. The 8-cycle SFT cold start did most of that: the first OPD rollout batch already scored 83.3%. -
One-cycle smoke: peak 4.251 GiB allocated / 4.762 GiB reserved; strategy resolved to
residentbecause a quantized model cannot be moved off the accelerator; projection chunk 256; 0 OOM retries; student 10,092,544 trainable LoRA parameters; teacher 1,720,574,976 parameters. -
Throughput probe: NF4 11.19 tok/s, bf16 LoRA 12.84 tok/s (14.12 with determinism off). A 14-token prefill costs 37.0 ms and a cached one-token step costs 30.9 ms, so decoding here is kernel-launch bound, not compute bound.
-
Protocol-teacher GPU benchmark v2 (
gpu-calc-hard-equal-update-v2, five arms x two prespecified seeds, 12 matched optimizer updates per continuation):arm seed 1234 seed 20260727 mean train s cold-start-only 75.0% 75.0% 0.1 sft-continued 100.0% 100.0% 86.4 opd-raw-teacher 0.0% 0.0% 444.0 opd-privileged-context 0.0% 0.0% 531.5 opd-protocol-sft-teacher 100.0% 100.0% 523.8 The protocol teacher prevents the collapse and ties SFT; it does not beat SFT. The complete portable schema-v2 artifact is
benchmarks/results/gpu-calc-hard-equal-update-v2.json. -
Matched-budget GPU benchmark (
benchmarks/configs/gpu_calc_hard.yaml, chainedhardcalculator split, legacy schema v1, single seed, 12 matched optimizer steps from one shared cold start). Result published inbenchmarks/results/rtx4080-calc-hard-matched.json:arm steps success cold-start-only 0 62.5% sft-continued 12 100.0% opd-bucketed-k64 12 0.0% opd-privileged-context 12 0.0% On-policy distillation lost, and the privileged-context prediction was wrong. Decoding the final evaluations showed the two OPD arms fail differently: the standard teacher drops the opening
<tool_call>tag and wraps answers in its own<answer>tags (83.3% invalid calls), while the privileged teacher emits clean output and no tool calls at all (1.00 turns, 0.0% invalid) because it knows the answer and so never needs one.scripts/attribute_failures.pyre-scores the collected trajectories with a lenient parser and shows formatting explains 4 of 24 failures for the standard arm and none for the privileged arm, so neither number is a verifier artifact. Full write-up indocs/rtx4080-baselines.md; both executions of the benchmark are reported and nothing was discarded. -
Matched-budget CPU benchmark (
recipes/benchmark_calc.yaml, 7 arms x 2 seeds, all 14 completed): published inbenchmarks/results/cpu-toy-calc-matched.json. Changing the seed moves one arm by up to 58 points (cold-start-only: 66.7% at seed 1234, 8.3% at seed 20260727) while changing the objective moves it by at most 4 points at a fixed seed. This is a parity check -- it shows all seven arms includingexact_full_vocabrun to completion under identical budgets -- and the table is inbenchmarks/README.mdso a reader can see that no ranking is available. The first attempt of this benchmark restarted the cosine schedule and damaged every arm; that attempt is recorded but not published, because it measured a harness bug rather than any arm.
Recorded because each one is a class of bug worth remembering.
| defect | how it was found | fix |
|---|---|---|
src/miniverl/models/ excluded from git and the wheel by an unanchored models/ rule in .gitignore |
installing the wheel into a clean venv | anchored every directory rule; CI now enumerates the source tree against the wheel and fails on any git-ignored source file |
Rich markup ate [...], so the missing-extra hint printed pip install "miniverl" |
reading actual CLI output | every dynamic string goes through rich.markup.escape |
A bare install raised ModuleNotFoundError instead of naming the extra |
a CLI test | the scorer import moved inside the constructor; the CLI pre-checks the stack |
exact_full_vocab rejected a recipe that omitted top_k -- the remedy its own error suggested -- and then wrote a config.resolved.yaml that could not be re-read |
an agent-written config test | consult model_fields_set, then normalize to 1 |
weighted_mean rescaled the result for a tiny-but-positive weight sum |
Hypothesis | floor only an exactly-zero sum |
EnvironmentConfig.name was a hard-coded regex, making register() useless |
writing the custom-environment example | validate against the registry |
render_final could emit a block that parsed back truncated |
an agent-written protocol test | escape closing tags in JSON, refuse them in plain text |
configure_logging(None) silently downgraded --log-level DEBUG |
an agent-written utils test | only an explicit level changes the level |
The report dropped stored rollouts and printed 0.000 GiB for CPU runs on a GPU machine |
an agent-written reporting test | fill the display budget; decide on the run's device |
| Sampling drew from a CUDA tensor with a CPU generator | the first GPU smoke test | always draw on the CPU, which also makes a seed reproduce across devices |
| Resume replayed completed cycles with a different LR schedule | the resume-equivalence test | train() continues at the checkpointed cycle; the teacher fit is wrapped in fork_rng |
| The toy model could not learn to copy | measuring, not assuming | learned absolute positions -> RoPE; 48 traces -> 256 traces |
ArmResult.run_dir wrote an absolute path, carrying a username and home directory into a file meant for pull requests |
a new privacy test over benchmarks/results/ |
a validator reduces any path to its final component |
| Schema-v2 resolved configs and invocation still carried local adapter/run paths | inspecting the first complete GPU result before publication | recursively replace absolute provenance paths with portable <local>/<name> values before hashing or publishing; privacy and chart-source tests cover the result |
eval.enabled: false did not suppress cycle-triggered evaluation |
timing the first GPU benchmark attempt | periodic evaluation now checks both the interval and the enabled flag; the incomplete attempt is preserved under runs/benchmarks/_aborted/ and the complete benchmark was rerun |
The committed JSON schema was never checked against the model it is generated from, though schema.py claimed the two could not drift |
writing the schema-validation test | a test asserts the committed file equals json_schema() |
16 bare NaN tokens in metrics.jsonl and eval.json, which strict JSON parsers reject |
diffing two identical demo runs | tokens_per_solved_task is null when nothing was solved, via schema.finite_or_none; a test parses every artifact with parse_constant set to reject |
TerminationReason.ENVIRONMENT_ERROR was declared but never assigned |
auditing for dead public API | a raising tool now ends the policy rollout with it; the oracle path still raises, because a truncated oracle trace would silently degrade every SFT target |
log1mexp clamped at a fixed -1e-7, silently saturating tail mass below float32 resolution regardless of dtype |
the documentation pass, then Hypothesis in float64 | the clamp derives from finfo(dtype).eps; the data-processing-inequality property now models the 1 / q_tail error amplification instead of using a flat tolerance |
Both example scripts printed a bare 0.0% with no context, which reads as a broken example |
running them in a clean install | each prints the same "expected, not a capability claim" note that miniverl demo prints |
None. Every gate in the table above passes.
- Protocol alignment removes collapse but has not produced an OPD-over-SFT advantage. The protocol-trained teacher reached 100% strict competence before downstream selection and its OPD arm then reached 100% on both seeds, versus 0% twice for raw and privileged teachers. SFT also reached 100% twice. The result supports teacher-protocol competence as the explanation for the collapse, but the small saturated evaluation cannot rank protocol OPD above verified SFT.
- The toy backend cannot rank methods. It solves only the
easysplit, where SFT saturates. Measured:mediumandhardstay at 0% even after 700 SFT steps. The CPU benchmark is therefore a parity check, not a ranking. - Only two GPU seeds on one task and machine. The primary v0.2 comparison repeats all arms twice, while legacy GPU artifacts remain single-seed. No significance or cross-task/model/hardware generalization is claimed.
- Transformers API drift. CI now tests the Qwen3 introduction boundary
(4.51.x) and the supported 5.x major. A future rename beyond the explicit
<6cap would still need a reviewed compatibility change.
- Finish the clean-wheel/core and clean-source-training install gates, plus every executable README command that does not require rerunning the full GPU experiment.
- Commit and push the measured artifacts/docs, make draft PR #4 ready, and repair any remote check that is an actual failure rather than an approval gate.
- Merge only after green CI, verify public
mainand artifact hashes, then close superseded dependency PRs #1-#3. Do not tag or publish to PyPI.
- Publishing to PyPI or creating a tag/release. Not authorized for this v0.2 iteration. The OIDC job is hard-disabled until a trusted publisher is registered and a separate reviewed change explicitly enables publication.
- A GPU CI runner.
.github/workflows/gpu.ymlis manual-dispatch only and targets a self-hostedcudarunner that does not exist for this repository. The prespecified GPU tests and benchmark therefore run locally on the RTX 4080, with artifacts and checksums recorded for review.
See the v0.2 release-hardening status for
current evidence and the historical table for the v0.1 baseline. Known
limitations are enumerated in docs/limitations.md.