Repository navigation
fix(storage): index pending facts past 65,535 correctly (#445) - #446
Merged
Merged
Conversation
Each pending fact was indexed with FactRef { page_id: 0, slot_index }, where
slot_index is its position in the pending list. The position was a
saturating u16, so every fact from position 65,535 on shared one slot and
entity lookups and attribute scans silently lost or duplicated facts. This
affected in-memory databases and file databases before a checkpoint.
slot_index is now a u32, and pending positions are checked rather than
saturated. postcard writes u16 and u32 as the same varint, so v7 index pages
are byte-for-byte unchanged; a test pins this.
Refs #445
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015GgLEs8YozJjYhwPEYgTUn
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
This was referenced Oct 6, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fix for #445 on v2.x. Once a database held more than 65,535 facts that had not been checkpointed, entity lookups and attribute scans silently lost or duplicated facts. Stored data was never affected.
Cause. Each pending fact is indexed with
FactRef { page_id: 0, slot_index }, whereslot_indexis the fact's position inFactStorage's pending list. The position was a saturatingu16, so every fact from position 65,535 on shared one slot and resolved to the same fact.Fix.
FactRef::slot_indexis now au32.pending_refs, which returns an error (INT-045) instead of saturating, before anything is inserted.load_factchecks for a duplicate before claiming a slot, so an error leaves no partial state.read_slottakes au32; a committed slot is still bounded by the page's record count.File format unchanged. postcard writes
u16andu32as the same varint, so v7 index pages on disk are byte-for-byte unchanged.fact_ref_encoding_matches_the_u16_layoutpins this.Tests
pending_facts_past_u16_resolve_to_themselves: 70,000 pending facts viatransact_batch, plustransact,retractandload_factpast slot 65,535. Entity, entity+attribute and attribute lookups each return exactly the fact's own record, with no duplicates.pending_refs_are_distinct_and_checked: slots are distinct across theu16boundary, and a position pastu32::MAXis an error.edge_cases_test.rs: two tests run 70,000 facts through the public API, one in memory and one file-backed (before and aftercheckpoint()). The file-backed run failed on v2.0.2 before this fix (count65,536 instead of 70,000;:e65536returned[]).cargo clippy --all-features -- -D warningsis clean.Docs
CHANGELOG (Unreleased → Fixed), TEST_COVERAGE, and the test counts in CLAUDE.md and README.
Per CONTRIBUTING.md, #445 stays open until the fix ships in a published release, so this PR uses
Refs #445.Refs #445
🤖 Generated with Claude Code
https://claude.ai/code/session_015GgLEs8YozJjYhwPEYgTUn