miniVERL v0.9 supports one pinned, fail-closed subset of verl v0.8: one actor,
one teacher, one generation per prompt, reward-free direct GKD with
forward_kl_topk, token-mean aggregation and a LoRA/QLoRA student. It is local
single-GPU execution, not Ray/FSDP or distributed verl execution.
Install a CUDA build of PyTorch that matches your machine first, then:
python -m pip install "miniverl[train,cuda,bridge]"
miniverl data sample --format verl-parquet --out data/opd-smoke.parquet
miniverl plan --profile verl-opd-v0.8-single-gpu-v1 \
--config builtin:qwen3-0.6b-1.7b-opd
miniverl run --profile verl-opd-v0.8-single-gpu-v1 \
--config builtin:qwen3-0.6b-1.7b-opdThe built-in recipe downloads pinned Qwen3-0.6B and Qwen3-1.7B snapshots when
they are not cached. Allow roughly 6 GiB of download/cache space and 0.25 GiB
for run artifacts. The plan command itself is CPU-only and weight-free; use
--offline for a zero-network compiler smoke.
The v0.9 developer workload extends this install smoke to 32 distinct consumed prompts, 64 response tokens and eight optimizer updates. See its phase-time, throughput, VRAM and resume evidence.
| GPU | student / teacher | strategy | limits / top-k | peak reserved | first update | status |
|---|---|---|---|---|---|---|
| RTX 4080 16 GiB | Qwen3-0.6B / Qwen3-1.7B, both NF4 | dual resident | 128 + 16 tokens / 32 | 3.176 GiB | 12.02 s | measured |
| 12 GiB CUDA GPU | same built-in recipe | planner-selected | same | — | — | not measured |
| 24 GiB CUDA GPU | same built-in recipe | planner-selected | same | — | — | not measured |
The measured run completed one current-policy rollout/scoring/update cycle,
exported a loadable PEFT adapter, and used one RTX 4080. It demonstrates
runtime and artifact correctness only; it did not evaluate alignment quality.
The checksummed record is
rtx4080-verl-opd-runtime-v1.json.
plan reports estimates separately from measurements. Auto placement uses
model metadata plus the configured VRAM headroom; it never branches on a GPU
product name. Unknown-size quantized roles report requires_probe because
bitsandbytes weights cannot legally swap. plan --probe performs a bounded,
cached CUDA calibration with zero optimizer updates; normal planning remains
weight-free.
Unsupported settings—including policy-gradient OPD, task rewards, reference KL, multiple teachers, multiple generations, multimodal inputs and every distributed dimension—are rejected rather than reinterpreted.
miniverl import-verl --profile verl-opd-v0.8-single-gpu-v1 \
--config verl-opd.yaml --out local-opd.yaml
miniverl run --profile verl-opd-v0.8-single-gpu-v1 \
--config local-opd.yaml --output runs
miniverl export-verl --run runs/<run-id> --target-verl v0.8.0 --out scaleout
miniverl bridge materialize scaleout --download --offline
miniverl bridge doctor scaleout --require-verlA compatible export contains the standard student PEFT adapter, tokenizer
metadata, exact student/teacher identities, original Parquet bytes, the source
config, compiled plan and pure OPD overrides. No reward scaffold is generated.
Validation data is exported only when the source declared it; an empty
data.val_files remains empty rather than duplicating training rows.
The bundle stays launchable: false until the exact student and teacher base
snapshots are materialized. A local teacher adapter adds an explicit merge
requirement because the pinned upstream profile does not consume that adapter
path directly. Materialization requires the exact installed verl pin and emits
launch.sh only after model/tokenizer/data/config checks pass. The bridge never
claims a distributed verl job ran. See the materialization contract.