Repository navigation
WAL entries of more than 1M facts are never replayed: the transaction and every later one are lost #686
Description
Summary
A transact or retract (or a WriteTransaction commit) of more than 1,000,000 facts succeeds, but the WAL entry it writes is never replayed. WAL replay stops at that entry, so after a crash, kill, or close without a checkpoint, that transaction and every later transaction in the WAL are lost.
Cause
src/wal.rs: the writer (serialize_entry) writes any number of facts, but the reader treats an entry with more than MAX_FACTS_PER_ENTRY (1,000,000) facts as corrupt and stops reading:
// Sanity cap: no legitimate entry has more than 1M facts
const MAX_FACTS_PER_ENTRY: usize = 1_000_000;
if num_facts > MAX_FACTS_PER_ENTRY {
break; // treat as corrupt entry
}The cap came in with 91fd04c (2026-03-21, first released in v0.5.0). It is the same on main (src/wal.rs:363) and v3 (src/wal.rs:454).
Reproduction (main, v2.0.9)
let db = OpenOptions::new().wal_checkpoint_threshold(usize::MAX).path(&path).open()?;
let facts: String = (0..1_000_001).map(|i| format!("[:e{i} :n {i}]")).collect();
db.execute(&format!("(transact [{facts}])"))?; // Ok
db.execute("(transact [[:after :n 1]])")?; // Ok; current_tx_count() == 2
drop(db); // no checkpoint: the WAL keeps both entries
let db = Minigraf::open(&path)?;
// current_tx_count() == 0; neither transaction's facts are thereThe same happens after a crash or SIGKILL before the next checkpoint, with the default threshold. A checkpoint before the crash writes the facts to the file, so the window is between the commit and the next checkpoint.
Affected
- v2.x (
main): every release from v0.5.0 through v2.0.9. v3:Minigraf::executeof a statement overTX_CHUNK_FACTSno longer writes a WAL entry (streaming engine 8/9: streaming transactions (#473) #550 PR 2, feat: large single statements commit as a checkpoint (#550 PR 2) #685, commits as a checkpoint), but aWriteTransactioncommit still does, until streaming engine 8/9: streaming transactions (#473) #550 PR 3 gives it the large path too. The reader cap should be fixed onv3either way, since a WAL from v2.x can hold such an entry.
Fix options
- Reject the transaction before it takes a
tx_count, with a structured error, when it would write an entry the reader refuses, as WAL-003 does for oversized facts. On v2.x that is the smallest change that never reports a lost commit as success. - Or remove the count cap from the reader. It guards against a corrupt count before the CRC is checked, so it would need another bound, such as the entry's byte length against the file's remaining length.
Data-integrity bug: stays open until the fix is in a published release (CONTRIBUTING.md). Needs a v3 fix and a v2.x backport.
Activity
fix(wal): bound the entry count without new error branches (#686)
fix(wal): bound the entry count without new error branches (#686)
test(wal): entries whose facts are cut short end replay (#686)
test(wal): entries whose facts are cut short end replay (#686)
fix(wal): replay WAL entries of more than 1,000,000 facts (#686) — ba…
Merge pull request #698 from project-minigraf/fix/686-wal-entry-cap
release: v2.0.10
adityamukho commentedon Oct 23, 2026
Fixed in v2.0.10, released 2026-10-23 (crates.io, CHANGELOG). Also fixed on
v3for v3.0.0. Moved to "Fixed in v2.0.10" in #421.