Repository navigation
fix(wal): replay WAL entries of more than 1,000,000 facts (#686) — backport - #699
Merged
Merged
Conversation
…ckport Replay took an entry of more than 1,000,000 facts as damage, and the writer's tail cut at open removed it and every later entry, so a transact, retract or WriteTransaction commit of that size, and every later one, was lost after a crash or close without a checkpoint. A fact count is now damage only when the bytes after it cannot hold that many facts (each has at least its 4-byte length), checked before the count is converted to usize. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0169pNQsLcCEn48te9X9WuZW
Take the entry's offset from valid_end rather than stream_position, and fall back to no bound when the file length is unavailable (a corrupt count still ends at EOF), so the coverage gate's branch floor holds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0169pNQsLcCEn48te9X9WuZW
The byte bound now stops the truncated-entry tests before the fact loop; cover the loop's two truncation exits directly. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0169pNQsLcCEn48te9X9WuZW
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
v2.x backport of the v3 fix for #686 (data-integrity; the issue stays open until v2.0.10 is published).
On v2.x a
transact,retractorWriteTransactioncommit of more than 1,000,000 facts succeeded and wrote a WAL entry that replay took as damage. Replay stopped there, and the writer's tail cut at the next open (#513) truncated the WAL at that entry. After a crash, kill or close without a checkpoint, that transaction and every later one in the WAL were lost. Present since v0.5.0.src/wal.rsscan_entries: the 1,000,000-fact cap is replaced by a byte bound. A count is damage only when the bytes left in the file cannot hold that many facts (4-byte length prefix each). The check runs on the rawu64before theusizeconversion. No new error code and no format change; WALs written by any v2.x release replay.Tests
test_wal_entry_of_more_than_a_million_facts_replays: an entry of 1,000,001 facts and the entries after it replay, and reopening the writer keeps them.test_wal_entry_with_more_facts_than_bytes_left_ends_replay(replaces the cap test): an impossible count (left/4 + 1,u64::MAX) ends replay at that entry, and reopening the writer cuts the WAL off there.cargo clippy --all-features -- -D warningsis clean.Docs
CHANGELOG (Unreleased → Fixed),
docs/TEST_COVERAGE.md(counts and entry), CLAUDE.md test count.🤖 Generated with Claude Code
https://claude.ai/code/session_0169pNQsLcCEn48te9X9WuZW