Run verl-style on-policy distillation on one NVIDIA GPU, inspect every local mapping and teacher target, and carry standard artifacts into a scale-out workflow.
Install and run locally{ .md-button .md-button--primary } Start with a verl profile{ .md-button }
miniVERL compiles typed YAML and structured Parquet prompts into three local phases: actor rollout, teacher scoring and actor update. The same execution plan binds the profile version, overrides and input bytes to trajectories, checkpoints, teacher caches and the final PEFT adapter.
python -m pip install torch --index-url https://download.pytorch.org/whl/cu130
python -m pip install "miniverl[train,cuda]"
miniverl data sample --format verl-parquet --out prompts.parquet
miniverl plan --profile verl-opd-v0.8-single-gpu-v1 \
--config builtin:qwen3-0.6b-1.7b-opd \
--set 'data.train_files=["prompts.parquet"]' --out plan.json
miniverl run --profile verl-opd-v0.8-single-gpu-v1 \
--plan plan.json --dry-runPlanning is weight-free. Review plan.json, remove --dry-run on a CUDA GPU,
then inspect and export the result:
miniverl run --profile verl-opd-v0.8-single-gpu-v1 --plan plan.json \
--output runs --run-id my-opd
miniverl inspect runs/my-opd/trajectories.jsonl
miniverl export-verl --run runs/my-opd --target-verl v0.8.0 --out scaleout
miniverl bridge doctor scaleout --jsonThe [train,cuda] extra supplies the training and quantization stack. Choose
the matching CUDA PyTorch wheel separately; the
single-GPU installation guide includes flexible and
maintainer-measured setups.
Compile a pinned direct-GKD or sampled-k1 profile, execute it in local phases, and inspect strict current-policy trajectories.
miniverl plan --profile verl-opd-v0.8-single-gpu-v1 --config verl-opd.yamlArtifact: immutable plan, trajectories, teacher cache and PEFT adapter.
Next: OPD quickstart
Keep familiar field names and structured Parquet while receiving a field-by-field account of each local effect.
miniverl compat check --profile verl-opd-v0.8-single-gpu-v1 \
--config verl-opd.yamlArtifact: resolved compatibility matrix and local execution plan.
Next: For verl users
Package the local adapter, source Parquet, resolved config and provenance, then materialize exact model snapshots against the pinned upstream source.
miniverl export-verl --run runs/my-opd --target-verl v0.8.0 --out scaleoutArtifact: 483F checksummed PEFT + Parquet + config bundle with readiness states.
Next: Scale-out contract
The local scheduler keeps actor, teacher and optional reference roles explicit while choosing resident, swap or shared-backbone placement. Structured token provenance, pickle-free caches and transactional publication make a run inspectable and resumable across phase boundaries.
Two measured profiles pin official verl v0.8.0 at 7aed6b23:
| Profile | Objective | Teacher signal |
|---|---|---|
| direct GKD | forward_kl_topk |
top-k token IDs and log-probabilities |
| sampled-k1 PG | sampled k1 + vanilla policy loss |
sampled-token teacher log-probability |
Compatibility profiles describes the exact field and semantic contract. Hardware planning explains how the same device-name-agnostic CUDA path adapts to different VRAM budgets.
The RTX 4080 Qwen3 developer workload consumed 32 distinct prompts and completed eight current-policy updates at 3.1914 GiB peak reserved VRAM. A matched interruption/resume reproduced byte-identical trajectories, adapter and optimizer tensors. The companion SmolLM2 workload completed the same run shape at 1.4961 GiB peak reserved VRAM.
Qwen3 workload{ .md-button } SmolLM2 workload{ .md-button }
The studies section preserves the actual outcome of each scoped experiment. It includes a protocol-qualified teacher that tied supervised continuation, a negative RecoveryBench result, regressions from a saturated Alignment Lab starting point, and a preregistered external-alignment early stop.
miniVERL's execution scope is one local process, one NVIDIA CUDA GPU and its versioned profiles. Its scale-out path ends in a validated artifact handoff. Read compatibility for field and handoff semantics, and limitations for the consolidated architecture, measurement, security and generalization boundaries.