This experiment trains the official TinyStories 1M, 3M, and 8M architectures from random initialization.
Each model receives the same TinyStories stream, tokenizer, seed, batch size, sequence length, and step budget. Evaluation generates each prompt normally and with every token containing a or A blocked.
The NoATaskset environment uses verifiers.v1 from verifiers==0.3.0. It scores the forbidden-letter invariant, useful output length, and trigram repetition. The environment is reusable by Prime-RL.
The checkpoint names describe transformer-body scale. Results report both body and total parameter counts.
python3 -m pip install modal pytest
modal setup
python3 -m pytest -q
modal run --detach sweep.pyThe run uses three parallel Nvidia L4 containers. Each function has a 90-minute timeout, limiting scheduled GPU cost to approximately $3.60 before CPU and memory charges.
The first completed 800-step sweep cost $0.17 for the successful app and $0.18 including one failed startup attempt.
Check progress and billing:
modal app logs tinystories-no-a-sweep
modal billing report --for today --show-resourcesOutputs persist in the tinystories-no-a-sweep Modal Volume.
Rescore downloaded samples with the Prime-RL verifier:
uv run --with verifiers==0.3.0 python rescore_results.pycompression_sweep.py compares 4k, 8k, and 16k byte-level BPE tokenizers with tied embeddings and packed sequences.
The completed run chose BPE 16k by validation bits per byte. The 10,000-step model reached 0.505231 bits per byte.
The no-a constraint passed mechanically. Generated stories remained incoherent, so the experiment does not claim useful language quality.
See COMPRESSION_EXPERIMENT.md for costs, metrics, output previews, limitations, and the checkpoint incident.
Downloaded machine-readable results are in compression-results/.
whoops_neologism_train.py completed a 1,000-step fine-tune for whoops responses and invented single words.
whoops_neologism_verifier.py supplies the Prime-RL Verifiers taskset and strict output guard.
All four raw evaluation outputs were single, a-free neologisms. The optional mishap distinction remained weak.
See WHOOPS_NEOLOGISM.md for behavior, exact billing, output previews, limitations, and saved artifacts.
The rerun accepts one ordinary text prompt. It generates a short story using only whoops or invented, no-a words.
The 1,000-step run processed 32,000 synthetic examples in 99.78 seconds. Its final mean loss was 1.697892.
Raw outputs excluded a in all four evaluations. About 91% of raw words passed the strict neologism check.
The local runner repairs truncated fragments and guarantees the complete story contract.
Read INVENTED_STORY_MODEL.md for the full experiment narrative.
Download whoops-story-model.zip for standard safetensors weights, tokenizer files, and the CPU text runner.
The invented-story weights fail an English-only contract. Only 9.09% of raw evaluation words matched the English frequency lexicon.
EnglishOnlyStoryTask adds a strict Prime-RL verifier for recognized English words without the forbidden glyph or sound family.
See ENGLISH_ONLY_AUDIT.md for the contract, exact audit, known-good example, and pronunciation limitations.
english_story_train.py fine-tunes the clean BPE 16k checkpoint on 48,000 complete positive English stories.
The final decoder uses no fallback. It emits verified grammar tokens while the model selects six meaningful story slots.
All four final evaluations passed with reward 1.0. Every emitted word is English and excludes the forbidden glyph and phone family.
The brave design shapes each story around difficulty, one honest step, help, hope, joy, and reunion.
See ENGLISH_POSITIVE_STORY.md for training evidence, raw-model limitations, reasoning-token boundaries, costs, and a full example.
Download english-positive-story-model.zip for the weights and local CPU runner.
english_story_vpo_train.py applies a two-step Vector Policy Optimization update to the saved positive-story checkpoint.
The corrected L4 smoke produced nonzero advantages, gradients, and parameter changes. Its matched 24-sample comparison moved from 23 to 24 strict passes, but its expected-best score fell by 0.000250.
This validates the training path. It does not establish that VPO improves the model.
See VPO_SMOKE.md for the estimator, reward-saturation correction, controlled comparison, outputs, artifacts, runtime, and exact billing.
The approved 100-update L4 run improved held-out subject/theme coverage from 39.58% to 70.83%. Individual strict no-a story passes fell from 93.75% to 87.5%, while strict best-of-three stayed at 100%.
The run took 88.09 seconds and cost $0.02289327. Its checkpoint is stored at runs/bpe-16k-english-positive-story-vpo/model.pt in the tinystories-token-compression Modal Volume.
vpo_story_api.py serves three raw candidates from those weights at https://thoriqakbar0--tiny-vpo-stories.modal.run. The unlisted https://ta-0.com/vpo page displays the selected story and all candidate scores.
See VPO_TRAINING.md for the controlled metrics, tradeoff, cost, response contract, deployment bounds, and reproduction commands.
The widened rerun teaches 24 subject mappings instead of inserting a subject at runtime.
After 1,000 SFT updates, gated subject coverage rose from 7.64% to 96.53% across 144 held-out stories. Both language and subject best-of-three metrics reached 100%.
Another 100 VPO updates did not materially improve those results. The experiment therefore credits widened supervised data, not VPO, for the gain.
See WIDE_RETRAINING.md for the full controlled comparison, cost, raw example, reproduction commands, and artifact paths.
Download wide-vpo-story-model/ for portable safetensors weights and a prompt-to-story CPU runner.
The live /vpo playground now runs the widened SFT and VPO checkpoints with the same prompt and sampling seed. It shows their verifier-selected stories side by side.
The first widened VPO run used independent story samples and an almost-saturated single-subject reward. VPO v2 now produces three stories in one autoregressive chain and scores one subject, one item, and one place independently.
The two-update L4 smoke passed its optimization-signal gate. The subsequent 100-update run improved held-out uniform best@12 from 0.730142 to 0.740155 and reward-space diversity from 1.777728 to 1.812602.
This is a search win with a product tradeoff. Strict single-story compliance fell from 57.64% to 55.56%, strict best-of-three fell from 100% to 97.92%, and no evaluated story completed all three requested details.
See VPO_V2.md for the design, exact smoke and full-run metrics, billing, artifacts, and reproduction commands.