Video should be compiled from durable cinematic state, not hallucinated as disposable pixels.
Semantic Video Studio (SVS) is a control-first pipeline that compiles a four-plane JSON state pack into a deterministic Blender render. Every render writes a provenance manifest with SHA-256 hashes of its inputs and outputs, enabling partial regeneration, asset reuse, and a full audit trail.
- 📄 Thesis: NORTH_STAR.md
- 📑 Whitepaper: docs/whitepaper.md — full architecture, milestone trajectory, gates
- 🧪 Reliability gate: basics/ — 7 invariants the system keeps green forever (M5)
- 🚪 Import gate: tools/imported_asset_report.py — external 3D assets pass through normalize → validate → preview → register (M11)
# 1. Install host-side Python deps
pip install -r requirements.txt
# 2. (Optional) configure secrets
copy .env.example .env.local
# Edit .env.local — add OPENAI_API_KEY only if you want narration
# 3. Render the demo
python tools\build_video.py examples\alien_roverbuild_video.py does three things:
- Validate —
tools/validate_scene.pychecks all four state files against their JSON Schemas, then runs cross-file checks (everyobject_idin a timeline action exists in the scene graph, everyasset_idexists in the manifest, beats cover[0, duration]without gaps, output dir is writable). - Render — launches Blender headless on
blender/render_scene.py. The script reads the four state files, builds (or appends) the assets, bakes per-frame keyframes from the timeline, and renders a PNG sequence muxed to MP4 via ffmpeg. - Manifest — writes
outputs/manifests/<render_id>.jsonwith a sha256 of every input the render depended on and the output it produced. This is the partial-regen contract.
- Python 3.10+ (host venv)
- Blender 5.1 — install via
winget install --id BlenderFoundation.Blender(Windows) or your distro's package manager - ffmpeg — auto-discovered via
imageio_ffmpeg(installed viarequirements.txt), so no separate install is needed
Override discovery with env vars (set in .env.local):
SEMVID_BLENDER=<path-to-blender.exe>SEMVID_FFMPEG=<path-to-ffmpeg.exe>
semantic-video-studio/
README.md
NORTH_STAR.md
LICENSE # MIT
THIRD_PARTY_NOTICES.md # Vendored components and their licenses
requirements.txt
.env.example
.gitignore
docs/
whitepaper.md # Full architecture paper (M0–M11)
schemas/ # 24 JSON Schemas — every plane has one
scene_graph.schema.json
asset_manifest.schema.json
timeline.schema.json
render_plan.schema.json
production_brief.schema.json
asset_record.schema.json
asset_request.schema.json
asset_match_result.schema.json
normalized_asset.schema.json
asset_import_report.schema.json
... (and more)
examples/ # One subdir per example, each holds the 4 planes
alien_rover/
mars_greenhouse/
imported_asset_demo/ # M11: scene built around an imported asset
prompt_suite/results/ # 5 generated state packs from prompt_to_brief
blender/
render_scene.py # bpy script: state → scene → keyframes → render
normalize_imported_asset.py # M11 headless normalize step
addon.py # BlenderMCP (vendored; see THIRD_PARTY_NOTICES.md)
tools/
validate_scene.py # Dry-run gate (schemas + cross-refs + writability)
build_video.py # Orchestrator: validate → render → manifest
prompt_to_brief.py # NL prompt → ProductionBrief
brief_to_state_pack.py # ProductionBrief → 4-plane state pack
asset_match.py # Scored deterministic matcher
asset_forge.py # Procedural-proxy fallback
asset_registry.py # Registry load/find helpers
semantic_edit_to_patch.py # NL edit → typed SemanticEditPatch
validate_state_patch.py # base_hash + from_value verification
state_patch_core.py # Patch application
import_external_asset.py # M11 stage 1
normalize_imported_asset.py # M11 stage 2 (host wrapper)
validate_imported_asset.py # M11 stage 3
preview_imported_asset.py # M11 stage 4
imported_asset_report.py # M11 orchestrator
render_narration.py # M10 TTS + gating
render_captions.py # M10 caption overlay
... (~40 tools)
basics/ # M5 invariant suite
run_all.py
test_00_environment.py
test_01_state_render.py
test_02_plane_regen.py
test_03_mcp_roundtrip.py
test_04_semantic_patch.py
test_05_prompt_to_state.py
test_06_negative_fixtures.py
fixtures/ # Bad inputs for negative-fixture tests
bad/
external_assets/ # Mock imports for the M11 gate
assets/
registry/asset_registry.json # The asset backlot (the source of truth)
library/hdri/ # Polyhaven HDRIs (CC0)
imports/ # Staged external mesh assets
previews/ # Asset thumbnails
outputs/ # Mostly .gitignored — regenerable from state
manifests/ # Tracked: per-render provenance JSONs
logs/mcp_events.jsonl # Tracked: append-only event stream
basics/ # Tracked: latest gate report
imported_asset_reports/ # Tracked: per-asset import gate reports
The basics gate is the contract that every milestone must keep green:
# Fast mode: reuses prior render manifests, verifies hashes agree
python basics\run_all.py --fast
# Full mode: re-renders the alien_rover example end-to-end
python basics\run_all.pyExpected output: Result: PASS with all 7 tests green. Reports land
in outputs/basics/latest_report.{json,md}.
| File | Touching this changes... | Does NOT change... |
|---|---|---|
timeline.json |
camera motion, object motion, action ordering, timing | what objects look like |
scene_graph.json |
which assets appear, transforms, lighting | timing or motion |
asset_manifest.json |
how a proxy is generated (geometry/material) | scene composition or timing |
render_plan.json |
resolution, fps, engine, output codec | content of the scene |
If a replacement rover keeps the same asset_id, anchors, and
capabilities, the existing timeline.json and scene_graph.json work
unchanged.
- Create
examples/<name>/with the four state files. - Run
python tools/validate_scene.py examples/<name>until it returns OK. - Run
python tools/build_video.py examples/<name>. - Inspect
outputs/manifests/<render_id>.jsonand play the MP4.
- Add an asset entry in your example's
asset_manifest.jsonwith a newgeneratorslug. - Add a function with the same slug to
GENERATORSinblender/render_scene.py. It receives the asset dict and returns the active Blender object.
- Add a
$defs/action_<name>block inschemas/timeline.schema.jsonand add it to theaction.oneOflist. - Add a branch in
apply_action()inblender/render_scene.pythat handles the newtypeand writes the appropriate keyframes. - Add a cross-reference check in
tools/validate_scene.pyif the new action references an ID type that needs validating.
python tools\imported_asset_report.py `
--asset-id my_asset_v001 `
--type structure.castle_tower `
--raw-path path\to\file.glb `
--provider manual_import `
--license CC-BY-4.0 `
--promote # registers as `approved` (else `candidate`)The asset passes through five gates: import → normalize → validate → preview → register. Each stage emits a typed provenance event into
outputs/logs/mcp_events.jsonl. Only approved assets can be
auto-selected by the matcher.
These choices were forced by what actually happened, not designed up front:
- Blender 5.x dropped
BLENDER_EEVEE_NEXT— the engine is justBLENDER_EEVEEagain.render_scene.pyfalls back through a list of candidate engines so arender_plan.jsonwritten against any 4.x/5.x build still works. - Blender 5.x on Windows doesn't expose
FFMPEGat runtime inimage_settings.file_format, even though it's in the static enum. The render script detects this, renders PNG frames tooutputs/videos/_frames_<stem>/frame_####.png, and muxes via external ffmpeg. read_factory_settings()strips runtime features. Calling it (with or withoutuse_empty=True) hides the FFMPEG enum item. The script clears default objects manually instead.- The orchestrator lives outside Blender. Blender's bundled Python is
isolated and won't see your
pip installs.tools/build_video.pyruns in the host interpreter so it can useimageio_ffmpeg, pass the ffmpeg path into Blender via env, and write the manifest afterward.
MIT — see LICENSE. Third-party components are listed in THIRD_PARTY_NOTICES.md.