Repository navigation
streaming engine 8/9: streaming transactions (#473) #550
Description
Part 8 of 9 of #473. Design: docs/superpowers/specs/2026-10-11-streaming-engine-design.md (branch docs/473-streaming-engine-spec), §8.
- Streaming EDN parser yielding statements; facts materialised and validated in chunks
-
TX_CHUNK_FACTSconstant (start 4,096, set by measurement) - Transactions of at most
TX_CHUNK_FACTSstaged facts unchanged (one WAL entry at commit) - Larger transactions write chunks into working trees (copy-on-write) and commit as a checkpoint, with no WAL entry;
tx_idfixed when the transaction becomes large (CHANGELOG note) - Statement base per statement: last statement wins and
API-011across chunks (§8.3); a rejected statement restores the base - Copy-on-write delete in
btree.rs - File-backed databases only; in-memory and
BrowserDbkeep their paths (§8.5) -
WriteTransactionreads its own writes throughScanover its snapshot - Crash and fault-matrix tests: rollback and crash before the meta write leave the database unchanged
- Acceptance: a 1M-fact
transactnever holds more than one chunk of parsed facts
PRs for this part target v3 and must not use a closing keyword for #473.
Activity
docs(spec): fix two line references (#550)
Merge pull request #683 from project-minigraf/docs/550-streaming-tran…
test(parser): run the statement differential test without threads on …
Merge pull request #684 from project-minigraf/feat/550-parser
test: run native-only large-transaction unit tests off wasm32 only (#550
test: allocator tracking asserts without short-circuit branches (#550)
test: read the active generation through the handle, not the locked f…
Merge pull request #685 from project-minigraf/feat/550-large-statements
feat(db): OpenOptions::transaction_chunk_facts, default 65,536 (#550)
Merge pull request #692 from project-minigraf/feat/550-statement-bases
adityamukho commentedon Oct 23, 2026
Done on v3: PR 1 #684, PR 2 #685, PR 3 #692 (merged 0535212). #473 stays open.
ci(crash-kill): run v3's nightly rounds with test-oracle (#687) (#695)
ci(crash-kill): print the cargo features the run uses (#697)