adityamukho · Oct 6, 2026 · Oct 5, 2026 · Oct 5, 2026 · Oct 5, 2026
@@ -10,6 +10,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ### Breaking changes
 
 - **File format v8** (#287, #371). Index keys now include the encoded value and the assert/retract flag. A v7 file (written by v2.x) is re-indexed and upgraded to v8 the first time it is opened; the upgrade is one-way and v2.x cannot open a v8 file (`STG-006`). Bindings users get this upgrade silently on open.
+- **Crash-atomic checkpoints with verified pages** (v8 storage format, PR 2 of 5; #374, #388). Two alternating meta pages replace the 84-byte file header and are the only commit point: a checkpoint writes only pages the last commit does not use, syncs, then writes the other meta page and syncs. A crash or torn write at any point leaves the previous checkpoint intact, so opening never rebuilds an index, and the whole-file index checksum is gone. Every other page carries a 24-byte header (type, CRC32, its own page id, the checkpoint generation that wrote it), checked on every read: a torn, corrupted, misplaced or stale page now fails with `STG-029`, `STG-030` or `STG-031` instead of being read. If the newest meta page is damaged after its WAL was already deleted, open fails with `STG-033` rather than silently going back to the previous checkpoint. New codes `STG-029`–`STG-035`. The WAL header gains a base generation (WAL version 2); a v2.x WAL beside a migrated file is still replayed. Pages freed by a checkpoint are reused by later ones; checkpoints still rewrite all four indexes until copy-on-write insert lands (PR 4). Files written by a v3.0.0 development build before this change fail with `STG-032`.
+- **The maximum fact size for file-backed databases is 4,052 bytes** (was 4,080), so that every index key derived from a fact fits in an index page. Index pages now reject an entry that does not fit instead of writing an overlapping page.
 - **Formats v1–v6 are no longer readable.** Opening one fails with the new `STG-028`; open it once with Minigraf v2.x to upgrade it to v7 first. `STG-003` and `STG-004` are deprecated (no longer emitted, still registered, never reused). `STG-005` and `STG-006` message text changed.
 
 ### Fixed

@@ -162,18 +162,24 @@ enum Value { String(String), Integer(i64), Float(f64), Boolean(bool),
 ### File Format (v8)
 
 ```
-Page 0:  Header (84 bytes) — magic "MGRF", version, page/fact counts,
-         B+tree root pages (eavt/aevt/avet/vaet), header_checksum, index_checksum, fact_page_count
-Page 1+: Packed fact pages (postcard-encoded, ~25 facts/4KB page)
-After:   On-disk B+tree index pages (one node per 4KB page)
-Sidecar: <db>.wal — CRC32-protected WAL entries; replayed on open; deleted on checkpoint
+Page 0, 1: Meta pages A/B (alternating commits: odd generations in page 0, even in 1).
+           Magic "MGRF"/"META", generation, page_count, index roots, free-list head,
+           required_features; CRC over the whole page. The only commit point.
+Page 2+:   Any mix of fact pages (0x41, interim until covering indexes), B+tree
+           leaf/internal pages (0x61/0x62) and free-list pages (0x81). Every one has
+           a 24-byte header (type, count, CRC32, page id, generation), verified on load.
+Sidecar:   <db>.wal — v2 header records the base generation; CRC32-protected entries;
+           replayed on open; deleted on checkpoint
 ```
 
-Auto-migrates v7 → v8 on open (index rebuild). v1–v6 are rejected (STG-028).
+A checkpoint writes only pages the active meta does not reference, syncs, then writes
+the other meta page and syncs, so a crash at any point keeps the previous checkpoint.
+Auto-migrates v7 → v8 on open (spec §9, with a backup meta page). v1–v6 are rejected
+(STG-028). Design: `docs/superpowers/specs/2026-10-05-v8-storage-format-design.md`.
 
 ## Test Coverage
 
-**1225 tests passing** (1217 passing, 8 ignored; unit + integration + doc).
+**1233 tests** (1225 passing, 8 ignored; unit + integration + doc).
 See `docs/TEST_COVERAGE.md` for the full per-file breakdown.
 
 **Testing conventions** — see the Testing Conventions section below before writing any tests.

@@ -137,6 +137,13 @@ with no `CodedError` anywhere in its chain.
 | STG-026 | Database locked by another process | Storage |
 | STG-027 | Filesystem does not support file locking | Storage |
 | STG-028 | Format version no longer supported | Storage |
+| STG-029 | Page checksum mismatch | Storage |
+| STG-030 | Page id mismatch | Storage |
+| STG-031 | Page generation ahead of meta | Storage |
+| STG-032 | No valid meta page | Storage |
+| STG-033 | Meta page damaged after commit | Storage |
+| STG-034 | Unsupported file feature | Storage |
+| STG-035 | Free-list inconsistency | Storage |
 | WAL-001 | Invalid WAL magic number | WAL |
 | WAL-002 | Unsupported WAL version | WAL |
 | WAL-003 | Fact serialised size exceeds maximum | WAL |
@@ -2003,6 +2010,84 @@ See the [file format section in README](../README.md#file-format) for version hi
 
 **Scenario**: A `.graph` file created with Minigraf v1.x and never opened with v2.x.
 
+### STG-029 Page checksum mismatch
+
+**Error text**: `Page {} checksum mismatch: the page is torn or corrupted`
+
+**Cause**: A page's stored CRC32 does not match its contents. The page was partly written (a torn write) or its bytes changed on disk after it was written.
+
+**Resolution**:
+- Restore from backup. If the damaged page belongs only to an unfinished checkpoint, the file opens at the previous checkpoint and the WAL replays the rest; this error means a page reachable from the committed meta page is damaged.
+
+**Scenario**: A disk sector holding a B+tree leaf is corrupted by the storage device.
+
+### STG-030 Page id mismatch
+
+**Error text**: `Page id mismatch: requested page {} but the page records id {}`
+
+**Cause**: A page's header records a different page id from the one that was read. The storage layer wrote the page to the wrong offset (a misdirected write), or the file was edited.
+
+**Resolution**:
+- Restore from backup. Check the storage device and filesystem for faults.
+
+**Scenario**: A faulty controller writes page 812's contents at the offset of page 640.
+
+### STG-031 Page generation ahead of meta
+
+**Error text**: `Page {} has generation {}, newer than the committed generation {}`
+
+**Cause**: A page reachable from the committed meta page records a checkpoint generation newer than the meta page itself. Pages are only written by the checkpoint that commits them, so this means a stale meta page, or a page from a later checkpoint written where an older page should be.
+
+**Resolution**:
+- Restore from backup.
+
+**Scenario**: A meta page restored from an old copy of the file, next to data pages from a newer one.
+
+### STG-032 No valid meta page
+
+**Error text**: `No valid meta page: the file is not a Minigraf v7 or v8 database, or both meta pages are damaged`
+
+**Cause**: Neither meta page (page 0 or page 1) passes its checksum, page 0 is not a valid format v7 header, and no migration backup meta page can be used. Pre-release v8 files written during v3.0.0 development are also rejected this way.
+
+**Resolution**:
+- Check that the path points to a Minigraf database.
+- Restore from backup. The file is not modified by the failed open.
+
+**Scenario**: Opening a file whose first two pages were overwritten, or a file from a v3.0.0 development build.
+
+### STG-033 Meta page damaged after commit
+
+**Error text**: `Meta page damaged after commit: generation {} is valid but a later commit existed ({}). Opening would silently drop that checkpoint`
+
+**Cause**: One meta page is valid and the other is damaged, and there is evidence that the damaged one held a newer, completed checkpoint: the WAL records a newer base generation, or there is no WAL and data pages from the newer generation exist. Opening at the older generation would lose the facts of the newest checkpoint, whose WAL is already deleted.
+
+**Resolution**:
+- Restore from backup. The file is not modified by the failed open.
+
+**Scenario**: A storage fault corrupts the most recent meta page after a successful checkpoint.
+
+### STG-034 Unsupported file feature
+
+**Error text**: `Unsupported file feature bits {}: the file was written by a newer Minigraf version`
+
+**Cause**: The meta page's `required_features` bitmask has bits this version does not know. A newer Minigraf v3.x release wrote structures this version cannot read.
+
+**Resolution**:
+- Open the file with the Minigraf version that wrote it, or a newer one. The file is not modified by the failed open.
+
+**Scenario**: A file written by a later v3.x release that enabled an optional on-disk structure, opened with v3.0.0.
+
+### STG-035 Free-list inconsistency
+
+**Error text**: `Free list is inconsistent: {}`
+
+**Cause**: The chain of free-list pages loops, or holds a page id that is reserved or past the end of the file.
+
+**Resolution**:
+- Restore from backup.
+
+**Scenario**: A corrupted free-list page whose `next` pointer points back at itself.
+
 ---
 
 ## WAL — Write-Ahead Log Errors

@@ -4,9 +4,9 @@
 
 **Verified**: 2026-10-05 with `cargo test --quiet`
 
-**Result**: 1,217 passing tests and 8 ignored tests (1,225 total)
+**Result**: 1,225 passing tests and 8 ignored tests (1,233 total)
 
-This covers the unreleased file-format-v8 work (#287, #371): the index-key value-collision fix, the v7→v8 migration, and the drop of formats v1–v6. It otherwise still covers the public database API, the Datalog engine, storage and recovery paths, v2.0.0's locking and structured-error-code changes, and v2.0.1's regression test for rebuilding indexes on open after several checkpoints (#370). Merged from the v2.x line: `tests/point_query_history_test.rs` and new storage unit tests check that attribute-narrowed bound-entity lookups return exactly what a full scan returns, and that `net_asserted_facts` matches its previous implementation on randomized histories (#323). Attribute-scan tests check that the AEVT range covers exactly the queried attribute, including non-ASCII names such as `:丿` and prefix siblings such as `:ab` for `:a` (#381). Incremental B+tree rebuild tests check that checkpoints which copy untouched index leaves produce exactly the entries and file checksum a full rebuild would, including over repeated checkpoints, that single-fact checkpoints do not fragment the index, and that a damaged slot directory is rejected rather than copied (#315). B+tree cursor tests check that every scan walks the tree, not the leaf `next_leaf` chain: full scans and start positions match the input, `seek` from the current position lands where a fresh descent from the root would (randomized, plus within a leaf, to a sibling, across subtrees, past the end, into the gap after a leaf, and backwards as a no-op), a seek to the sibling leaf reads exactly one page, reads are unchanged with every `next_leaf` zeroed, a child pointer into a non-B+tree page is STG-013, and a cycle in child pointers is an error, not a hang (v8 storage format PR 1, #374). Directory-sync unit tests check that creating the `.graph` file or the WAL, and deleting the WAL, fsync the parent directory after the file operation, and that reopening an existing file does not (#389). Disjunction tests check that `or-join` returns an empty result, not INT-031, when no rows reach it: on an empty database, after a pattern that matches nothing, after a `not` that removes every row, and in a rule body (#405). A stratification unit test builds the same two-rule negative cycle 64 times and checks that the INT-054 message names the same predicate pair every time (#410). Magic-sets tests check that a recursive rule with a literal start that recurses through another rule returns the same results as without the literal (#297). API tests check that `execute()` and `WriteTransaction::execute()` reject `$slot` bind tokens in every position with API-010 (#407). The REPL `run()` test now runs in a child process with null stdin, so it no longer hangs when the test runner's stdin is an open non-TTY stream (#415). A fault-injection test crashes `save()` at each of its 47 page writes and syncs and checks that every reopen finds exactly the old or the new checkpointed facts (#374).
+This covers the unreleased file-format-v8 work (#287, #371): the index-key value-collision fix, the v7→v8 migration, and the drop of formats v1–v6. It otherwise still covers the public database API, the Datalog engine, storage and recovery paths, v2.0.0's locking and structured-error-code changes, and v2.0.1's regression test for rebuilding indexes on open after several checkpoints (#370). Merged from the v2.x line: `tests/point_query_history_test.rs` and new storage unit tests check that attribute-narrowed bound-entity lookups return exactly what a full scan returns, and that `net_asserted_facts` matches its previous implementation on randomized histories (#323). Attribute-scan tests check that the AEVT range covers exactly the queried attribute, including non-ASCII names such as `:丿` and prefix siblings such as `:ab` for `:a` (#381). Incremental B+tree rebuild tests check that checkpoints which copy untouched index leaves produce exactly the entries and file checksum a full rebuild would, including over repeated checkpoints, that single-fact checkpoints do not fragment the index, and that a damaged slot directory is rejected rather than copied (#315). B+tree cursor tests check that every scan walks the tree, not the leaf `next_leaf` chain: full scans and start positions match the input, `seek` from the current position lands where a fresh descent from the root would (randomized, plus within a leaf, to a sibling, across subtrees, past the end, into the gap after a leaf, and backwards as a no-op), a seek to the sibling leaf reads exactly one page, reads are unchanged with every `next_leaf` zeroed, a child pointer into a non-B+tree page is STG-013, and a cycle in child pointers is an error, not a hang (v8 storage format PR 1, #374). Directory-sync unit tests check that creating the `.graph` file or the WAL, and deleting the WAL, fsync the parent directory after the file operation, and that reopening an existing file does not (#389). Disjunction tests check that `or-join` returns an empty result, not INT-031, when no rows reach it: on an empty database, after a pattern that matches nothing, after a `not` that removes every row, and in a rule body (#405). A stratification unit test builds the same two-rule negative cycle 64 times and checks that the INT-054 message names the same predicate pair every time (#410). Magic-sets tests check that a recursive rule with a literal start that recurses through another rule returns the same results as without the literal (#297). API tests check that `execute()` and `WriteTransaction::execute()` reject `$slot` bind tokens in every position with API-010 (#407). The REPL `run()` test now runs in a child process with null stdin, so it no longer hangs when the test runner's stdin is an open non-TTY stream (#415). v8 page-format tests (PR 2, #374, #388) check the 24-byte page header (a flipped bit, a wrong page id and a future generation each give their own STG code, legacy page types fail before the CRC, and a failed page is never cached), meta-page encoding, the free-list chain, WAL v2's base generation, and every row of the meta-selection table: both slots valid, a torn newest slot with the WAL's base generation (opens at the older one), a WAL newer than the valid meta, pages of the next generation on the free list or past `page_count` with no WAL (all STG-033, with the file unmodified), an unknown feature bit (STG-034), and a pre-release v8 single header (STG-032). Fault-injection tests crash `save()` at every page write (cleanly and with torn pages, including the meta write) and every sync, and crash a v7 migration at every write; each reopen finds exactly the old or the new checkpoint, and every reachable page plus the free list covers the file exactly. Twenty-five consecutive saves never write a page the previous meta references. A maximum-size fact's index keys fit in an internal node, and an oversized leaf entry or separator is rejected instead of overlapping the slot directory.
 
 ### Covered Areas