adityamukho · Oct 10, 2026 · Oct 10, 2026 · Oct 10, 2026
@@ -40,6 +40,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Queries no longer lose or duplicate facts when more than 65,535 facts are waiting for a checkpoint (#445; also fixed in v2.0.3).
 - **A file shorter than its meta page count is refused at open with `STG-044`** (#497). Open never compared the meta page's `page_count` with the file's length. A truncated file (an interrupted copy, a partial download) opened, and its queries and checkpoints then failed with `INT-049` after the WAL had accepted writes. A meta with a `page_count` far past the end opened too: its next checkpoint wrote at that page id, leaving a sparse file of up to 4 EiB, and `verify()` scanned to that count and never finished. Open now refuses both, read-write and read-only, without modifying the file. It does not fall back to the other meta page, because a valid newer meta means that checkpoint happened. A checkpoint now fails with `INT-049` instead of writing a meta that counts pages the backend does not hold, and `verify()` reports a `page_count` past the end as `STG-044` and stops its page scan at the end of the file. `BrowserDb` is not checked: it keeps only live pages (#440), so its store has no length to compare. New code `STG-044`.
 - **A file shorter than one page is refused at open with `STG-045` unless it is a torn first write** (#506). Open treated any file under 4 KiB as a new database, because it holds no whole page, and a read-write open wrote a new empty database over it. A database cut below one page by an interrupted copy, or an unrelated file at the path, was silently replaced. Now such a file opens as a new database only when its bytes are zeros or a prefix of the first meta page a new database writes (what a crash during creation can leave, the rule a torn full page 0 already followed); anything else is `STG-045`, read-write and read-only, and the file is not modified. A zero-length file is still a new database. New code `STG-045`.
+- **A failed checkpoint is never retried on the same handle** (#390). A checkpoint that failed after it may have written (a failed write, a full disk, a failed `fsync`) could be retried by the next checkpoint. After a failed `fsync` the operating system may have dropped the written pages, and the checkpoint's meta page may have reached the disk although the call failed; a retry could then overwrite pages that meta references, and a crash during the retry left a file that failed to read with `STG-029`. Now every later write, checkpoint and index rebuild through that handle fails with the new `STG-046`, queries keep working, and the close-time checkpoint is skipped. Reopening the database recovers everything committed: it is all in the WAL. A `rebuild_indexes()` refused with `STG-041` writes nothing and does not count as failed. A fault-injection matrix now covers EIO, torn and short writes, a full disk and failed fsyncs at every write and sync of transacts, checkpoints, the close-time checkpoint and index rebuilds.
 - **A commit made after a torn WAL entry is no longer lost on recovery** (#513). Recovery replays the WAL up to its first partial or damaged entry, but the writer appended after the end of the file. After a crash during an append, the next session's commits went in after the partial entry, so a second crash before any checkpoint lost them, although each had been fsynced and reported as committed. A failed append (short write, full disk, I/O error) did the same to the next commit through that handle. Opening a database for writing now cuts the WAL back to the end of its last valid entry, and syncs it, before anything is appended. After a failed append, every later write through that handle fails with the new `WAL-007` until the database is reopened or a `checkpoint()` succeeds. The transaction whose append failed is not applied; after a reopen it is either there in full (its entry reached the disk) or not at all. Affected every release since v0.5.0.
 - **A v7 fact page with a damaged type byte fails the upgrade with `STG-014`** (#496). Opening a v7 file skipped, without an error, any page in its fact-page range whose type byte was not the fact-page type. The read-write open then upgraded the file to v8 and freed the old v7 pages, so that page's facts were lost for good; the read-only open returned the same answer with them missing. A v2.x writer fills that range with fact pages only, so any other type there is damage: open now fails with `STG-014`, read-write and read-only, and leaves the v7 file as it was. v7 fact pages have no checksum of their own, so damage elsewhere in a page can still decode to wrong facts or fail with another code; keep a copy of a v7 file until it has opened under v3.0.0.
 - Reopening next to a WAL whose entries were all already checkpointed no longer rewinds the transaction counter, so later transactions no longer reuse `tx_count` values and `:as-of` history stays correct (#447; also fixed in v2.0.3).

@@ -108,6 +108,7 @@ cargo run < demos/demo_negation.txt
    - `backend/file.rs`: Single `.graph` file backend (4KB pages, cross-platform)
    - `backend/memory.rs`: In-memory backend for testing
    - `backend/fault_inject.rs`: `FaultInjectingBackend` — injects I/O errors for durability tests (test builds only)
+   - `fault.rs`: thread-local fault plan (test builds only, #390) — EIO, torn/short writes, sticky ENOSPC, failed fsync with or without lost writes, at the k-th write or sync of `FileBackend`, the WAL and `sync_parent_dir`; driven by `src/fault_matrix.rs`
    - `index.rs`: pending (uncheckpointed) EAVT/AEVT keys over UUIDs and strings, `encode_value`
    - `keys.rs`: byte-comparable v8 keys (FDB integers, value tags, `tx↓`, FOREVER), DICT keys, `MAX_VALUE_BYTES`, `MAX_IDENT_BYTES`
    - `node.rs`: prefix-compressed leaf and shortest-separator internal node codecs
@@ -197,7 +198,7 @@ Auto-migrates v7 → v8 on open (spec §9, with a backup meta page). v1–v6 are
 
 ## Test Coverage
 
-**1393 tests** (1384 passing, 9 ignored; unit + integration + doc).
+**1406 tests** (1397 passing, 9 ignored; unit + integration + doc).
 See `docs/TEST_COVERAGE.md` for the full per-file breakdown.
 
 **Testing conventions** — see the Testing Conventions section below before writing any tests.

@@ -131,7 +131,7 @@ let r2 = pq.execute(&[("tx", BindValue::TxCount(2)), ("entity", BindValue::Entit
 
 ```bash
 cargo run          # interactive Datalog REPL
-cargo test         # run 1393 tests
+cargo test         # run 1406 tests
 cargo run < demos/demo_recursive.txt   # recursive rules demo
 ```
 

@@ -157,6 +157,7 @@ with no `CodedError` anywhere in its chain.
 | STG-043 | Log writer target already exists | Storage |
 | STG-044 | File shorter than its meta page count | Storage |
 | STG-045 | File shorter than one page and not a new database | Storage |
+| STG-046 | Earlier checkpoint failed on this handle | Storage |
 | WAL-001 | Invalid WAL magic number | WAL |
 | WAL-002 | Unsupported WAL version | WAL |
 | WAL-003 | Value size exceeds maximum | WAL |
@@ -2307,6 +2308,22 @@ let mut out = LogWriter::create("memory.v2.graph", OpenOptions::new())?;
 
 ---
 
+### STG-046 Earlier checkpoint failed on this handle
+
+**Error text**: `an earlier checkpoint failed on this handle and is not retried; reopen the database`
+
+**Cause**: A checkpoint (automatic, `checkpoint()`, `rebuild_indexes()`, or the one on close) failed with an I/O error after it may have written pages: a failed write, a full disk, or a failed `fsync`. After a failed `fsync` the operating system may have dropped the written pages while marking them clean, so a second `fsync` can report success for data that never reached the disk (the PostgreSQL "fsyncgate" problem). The checkpoint's meta page may also have reached the disk although the call failed; a retry would then overwrite pages that page references. So the handle never retries: every later write, checkpoint and index rebuild through it fails with STG-046.
+
+Nothing committed is lost. Every committed transaction is still in the WAL, and the `.graph` file holds either the previous checkpoint or the failed one complete. Queries through the handle keep working. The close-time checkpoint is skipped.
+
+**Resolution**:
+- Fix the cause (free disk space, check the device and the kernel log), then drop every clone of the handle and open the database again. Opening picks the newest complete checkpoint and replays the WAL.
+- Do not copy the `.graph` file without its `.wal` while the handle is in this state.
+
+**Scenario**: The disk fills during an automatic checkpoint, so the `transact` that triggered it fails with the I/O error (its transaction is already in the WAL and applied). The next `transact` fails with STG-046 until the database is reopened.
+
+---
+
 ## WAL — Write-Ahead Log Errors
 
 WAL errors relate to the sidecar `.wal` file written alongside the `.graph` file.

diff --git a/docs/TEST_COVERAGE.md b/docs/TEST_COVERAGE.md
@@ -371,6 +371,24 @@ impl Drop for Inner {
     }
 }
 
+/// Refuse a write, before it takes a `tx_count`, on a handle where an earlier
+/// write failed: a checkpoint (STG-046) or a WAL append (WAL-007). See #390
+/// and #513.
+fn refuse_if_failed(ctx: &WriteContext) -> Result<()> {
+    #[cfg(not(target_arch = "wasm32"))]
+    if let WriteContext::File { pfs, wal, .. } = ctx {
+        if pfs.failed() {
+            bail_coded!(ErrorCode::Stg046);
+        }
+        if wal.as_ref().is_some_and(WalWriter::failed) {
+            bail_coded!(ErrorCode::Wal007);
+        }
+    }
+    #[cfg(target_arch = "wasm32")]
+    let _ = ctx;
+    Ok(())
+}
+
 // ─── Fact size validation (moved to WAL serialization) ────────────────────────
 
 // ─── Minigraf ─────────────────────────────────────────────────────────────────
@@ -696,6 +714,7 @@ impl Minigraf {
             let tx_id = crate::graph::types::tx_id_now();
             crate::graph::storage::check_valid_windows(&stamped, tx_id)?;
             crate::graph::storage::check_one_window_per_triple(&stamped)?;
+            refuse_if_failed(&ctx)?;
             let tx_count = self.inner.fact_storage.allocate_tx_count();
 
             let stamped: Vec<Fact> = stamped
@@ -891,6 +910,16 @@ impl Minigraf {
     /// # Errors
     ///
     /// Returns an error if the write lock is poisoned or the checkpoint I/O fails.
+    ///
+    /// A failed checkpoint is never retried on the same handle: after a failed
+    /// `fsync` the operating system may have dropped the written pages, and a
+    /// retry could report success over them. Every later write, checkpoint
+    /// and index rebuild through the handle fails with STG-046; queries keep
+    /// working. Reopen the database: every committed transaction is still in
+    /// the WAL, and the file holds the previous checkpoint or the failed one
+    /// complete. The same applies to an automatic checkpoint, which fails the
+    /// `transact` or `commit` that triggered it after that transaction is
+    /// already durable in the WAL.
     pub fn checkpoint(&self) -> Result<(), MinigrafError> {
         self.checkpoint_inner().map_err(MinigrafError::from)
     }
@@ -1012,6 +1041,18 @@ impl Minigraf {
         self.inner.fact_storage.current_tx_count()
     }
 
+    /// End the handle as a killed process would: no close-time checkpoint and
+    /// no save on drop. Whatever reached the file stays; the lock is released.
+    #[cfg(all(test, not(target_arch = "wasm32")))]
+    pub(crate) fn kill(self) {
+        let mut ctx = self.inner.write_lock.lock().unwrap();
+        if let WriteContext::File { mut pfs, .. } =
+            std::mem::replace(&mut *ctx, WriteContext::Memory)
+        {
+            pfs.discard();
+        }
+    }
+
     /// Internal checkpoint logic (operates on an already-held write-lock guard).
     fn do_checkpoint(_fact_storage: &FactStorage, ctx: &mut WriteContext) -> Result<()> {
         match ctx {
@@ -1500,6 +1541,7 @@ impl<'a> WriteTransaction<'a> {
             // are checked again here (#436).
             let tx_id = crate::graph::types::tx_id_now();
             crate::graph::storage::check_valid_windows(&facts_to_commit, tx_id)?;
+            refuse_if_failed(&self.guard)?;
             let tx_count = self.inner.fact_storage.allocate_tx_count();
 
             // Stamp facts with tx_id and tx_count
@@ -2425,9 +2467,10 @@ mod tests {
         let db = Minigraf::open(&db_path).unwrap();
         db.execute("(transact [[:a :name \"A\"]])").unwrap();
 
-        crate::wal::tear_next_append(9);
+        crate::storage::fault::arm(crate::storage::fault::Fault::Torn(9), 0);
         db.execute("(transact [[:b :name \"B\"]])")
             .expect_err("the torn append fails");
+        crate::storage::fault::disarm();
         assert_eq!(names(&db), 1, "the failed transaction is not applied");
 
         let err = db

@@ -189,6 +189,7 @@ pub(crate) enum ErrorCode {
     Stg043,
     Stg044,
     Stg045,
+    Stg046,
     Wal001,
     Wal002,
     Wal003,
@@ -1101,6 +1102,12 @@ pub(crate) const REGISTRY: &[(ErrorCode, &str, &str, ErrorCategory)] = &[
         "File is {} bytes, shorter than one page, and is not a new Minigraf database",
         ErrorCategory::Storage,
     ),
+    (
+        ErrorCode::Stg046,
+        "STG-046",
+        "an earlier checkpoint failed on this handle and is not retried; reopen the database",
+        ErrorCategory::Storage,
+    ),
     (
         ErrorCode::Wal001,
         "WAL-001",
@@ -2022,6 +2029,7 @@ mod tests {
                 | ErrorCode::Stg043
                 | ErrorCode::Stg044
                 | ErrorCode::Stg045
+                | ErrorCode::Stg046
                 | ErrorCode::Wal001
                 | ErrorCode::Wal002
                 | ErrorCode::Wal003
@@ -2246,6 +2254,7 @@ mod tests {
             ErrorCode::Stg043,
             ErrorCode::Stg044,
             ErrorCode::Stg045,
+            ErrorCode::Stg046,
             ErrorCode::Wal001,
             ErrorCode::Wal002,
             ErrorCode::Wal003,