Repository navigation
fix!: file format v8 — keep same-transaction multi-valued facts (#371, #287) - #377
Merged
Merged
Conversation
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PuKbXn6NaHBMDVkScdBNQZ
… for v6/v7, fixtures Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PuKbXn6NaHBMDVkScdBNQZ
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PuKbXn6NaHBMDVkScdBNQZ
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PuKbXn6NaHBMDVkScdBNQZ
…287) Generated by the v7 writer at 84bf375 (v2.0.0 format). Generator source is recorded in tests/multi_value_test.rs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PuKbXn6NaHBMDVkScdBNQZ
Formats older than v7 now fail to open with STG-028, which tells the user to open the file once with v2.x. Removes the v1→v2, one-per-page and v5→v6 migrations and the legacy v5 B+tree module. STG-003/STG-004 are retired. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PuKbXn6NaHBMDVkScdBNQZ
…371, #287) EavtKey/AevtKey gain value_bytes and asserted; AvetKey/VaetKey gain asserted. Fields are appended so range-scan order is unchanged. All key construction goes through per-key constructors. A v7 file is always re-indexed from its fact pages on open and rewritten with a v8 header. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PuKbXn6NaHBMDVkScdBNQZ
… barrier (#371, #287) Round-1 review findings on c74f89d: - Empty transactions (`(transact [])`, `(retract [])`) allocate a tx_count without producing facts, so restoring the tx_counter from `max(fact.tx_count)` alone could rewind it below what the v7 file's header already recorded. Floor it at the old header's `last_checkpointed_tx_count` so migration never lets a new transaction reuse a tx_count that `:as-of N` already served facts for. - The rebuild path wrote the header before syncing the freshly built index pages. Add the same `backend.sync()` barrier `save()` uses, after the B+tree builds and before the header write, so the header (the crash-safe commit point) is never durable ahead of the data it points to. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PuKbXn6NaHBMDVkScdBNQZ
#371) selective_fact_fetch deduplicated on (entity, attribute, tx_count, asserted), which collapsed two values written in one transaction and left one half of a batched retract un-cancelled. The key now includes the encoded value. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PuKbXn6NaHBMDVkScdBNQZ
… dedup key (#371) batched_retract_of_both_values_hides_both_on_every_path passes even against the pre-fix (entity, attribute, tx_count, asserted) dedup key: retracting exactly the asserted value set is symmetric under that key, since the smallest-encoded-byte value always wins the collision on both the assert side and the retract side and correctly cancels itself, regardless of statement order or whether the retracts are batched or split across separate transactions (verified empirically, not just argued). Add two tests where the retracted value set includes a value that was never asserted (:k/A, which sorts before :k/a): under the old key this non-asserted value wins the retract-batch collision and silently drops the real retraction records for :k/a and :k/b, leaving :k/a visible. Confirmed RED against a reverted copy of executor.rs and GREEN against the fix from the prior commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PuKbXn6NaHBMDVkScdBNQZ
…#287) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BedF9ZmqVZLkvDwn91fuKy
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BedF9ZmqVZLkvDwn91fuKy
…287) `selective_fact_fetch`'s dedup key (entity, attribute, tx_count, asserted, value_bytes) had no valid-time component, so two facts differing only in their per-fact valid-time window — same tx_count (one transact call), same value bytes, same asserted flag — collapsed into a single arbitrary survivor in both the entity-bound and attribute-bound loops. A `:valid-at` query landing inside the dropped window's range returned zero rows instead of one, and `:any-valid-time` returned one stint instead of two. Add fact.valid_from/valid_to to the key in both loops and update the doc comment. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BedF9ZmqVZLkvDwn91fuKy
…287) Both targets hardcoded header version 7 with an all-zero page_count, so `Minigraf::open()` always either failed STG-007 immediately (page_count 0) or took the version<8 forced-rebuild path, which rebuilds indexes from fact pages and never reads the fuzzed page at all. - fact_page.rs: version 8, page_count 2, fact_page_count 1, eavt_root_page left at 0 so `load()` still rebuilds from fact pages — opening alone now decodes the fuzzed page as a packed fact page (postcard `Fact`). - btree_page.rs: version 8, page_count 3, eavt_root_page pointing at the fuzzed page so `load()` wires it up as the EAVT B+tree root instead of rebuilding. `open()` only wires the reader lazily and never reads the root page itself, so the target now also runs a minimal entity-bound query to force an EAVT range scan, which decodes the fuzzed page as a v8 B+tree leaf/internal node (with value bytes + asserted flag). fuzz/Cargo.lock and fuzz/target/ remain untracked/uncommitted. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BedF9ZmqVZLkvDwn91fuKy
…ure (#371, #287) The example's doc header said "do not regenerate" the v7 compat.graph fixture, but the code still wrote straight to tests/fixtures/compat.graph and renamed over it — running the example silently replaced the frozen v7 file with a v8 one, invalidating the v7->v8 migration coverage. Write to target/generate_compat_fixture/compat_generated.graph instead, and fix the doc header to describe the new behavior. Also assert in fixture_readable_by_native (cross_platform_compat_test.rs) that the embedded fixture bytes are version 7 before opening, so a future regeneration-in-place is caught by the test instead of silently changing what's being covered. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BedF9ZmqVZLkvDwn91fuKy
) unsupported_version_fails_loudly checked that opening a v99 file errors with STG-006, but not that Minigraf::open() left the file alone on that error path (spec Testing 5). Compare the file bytes before and after the rejected open. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BedF9ZmqVZLkvDwn91fuKy
#287) near_max_size_values_checkpoint_and_read_back leaves ~230 bytes of headroom below the real per-fact maximum (MAX_FACT_BYTES minus the fixed postcard overhead of the rest of the Fact struct, which isn't a public constant so tests can't compute it directly). Add a case that locates the exact edge at runtime via a binary search over transact attempts, then checkpoints/reads back two values of that exact length through EAVT, AEVT and AVET, and confirms one byte past the boundary is rejected. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BedF9ZmqVZLkvDwn91fuKy
The field doc still listed 0x00/0x01 as if they were live decode options. Only 0x02 (packed, FACT_PAGE_FORMAT_PACKED) is ever written or read; v1-v6 files (the only ones that could carry 0x00/0x01) are rejected outright by STG-028, not migrated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BedF9ZmqVZLkvDwn91fuKy
…ty (#371, #287) The 7-tuple dedup key added for the valid_from/valid_to fix trips clippy::type_complexity under the CI gate (cargo clippy --all-features -- -D warnings). Factor it into a local type alias. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BedF9ZmqVZLkvDwn91fuKy
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BedF9ZmqVZLkvDwn91fuKy
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
…g them (#287) Error codes are a stable public contract and are never removed or reused. STG-003 and STG-004 can no longer be emitted (pre-v7 headers are rejected with STG-028 before any size check), so they stay in ErrorCode, REGISTRY and ERROR_REFERENCE.md, marked deprecated in v3.0.0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BedF9ZmqVZLkvDwn91fuKy
This was referenced Sep 25, 2026
adityamukho
added a commit
that referenced
this pull request
Sep 25, 2026
Main must stay releasable as v2.x so v2.1–v2.3 (and #370's unreleased fix) can ship without a forced one-way format migration. The v8 work (#371, #287) is preserved unchanged on the long-lived `v3` branch (at b8a9762), which will merge main regularly and becomes main when v3.0.0 is cut. This reverts commit b8a9762. Claude-Session: https://claude.ai/code/session_01BedF9ZmqVZLkvDwn91fuKy Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
When one transaction wrote two values of the same attribute for one entity, reads returned only one of them, and EAVT and AEVT paths could return different ones. A batched retract could leave a retracted value visible. This PR fixes all three layers that caused it, moves the file format to v8, and drops support for formats v1–v6.
Closes #371
Closes #287
Root causes
EavtKey/AevtKeyhad no value bytes, so two values in one transaction collided and the in-memoryBTreeMapkept only the last.selective_fact_fetchdeduplicated on(entity, attribute, tx_count, asserted), which kept one of the colliding facts and dropped one half of a batched retract. The final review also found it dropped facts that differ only in valid time.sort_unstable_byordered them differently in EAVT and AEVT, so "first seen" differed between paths.Changes
value_bytes+asserted; AVET/VAET gainasserted. Fields are appended, so range-scan order is unchanged. All key construction goes through per-key constructors. Lookups usetake_whileinstead of"zzzz"/u64::MAXsentinels.tx_counter/last_checkpointed_tx_count, even when the last transactions were empty. The upgrade is one-way.selective_fact_fetchkey is now the full EAVT identity(entity, attribute, tx_count, asserted, value, valid_from, valid_to).btree.rsand the v1→v2, one-per-page and v5→v6 migrations are removed. STG-003/STG-004 are deprecated (no longer emitted; still registered and documented, never reused), and the STG-005/STG-006 message text changed.tests/fixtures/v7_multivalue.graph(written by the v2.0.0 v7 writer).compat.graphis now the frozen v7 fixture; its tests assert v7 before opening and v8 after.generate_compat_fixtureno longer overwrites it.fact_page/btree_pagetargets use v8 headers (btree_pagestill doesn't reach node decoding; see fuzz: btree_page target never reaches B+tree node decoding #375).Tests
tests/multi_value_test.rs(10 tests) covers:cargo test: 1149 passed, 8 ignored, 0 failed.cargo fmt --checkandcargo clippy --all-features -- -D warningsare clean.fuzzcratecargo checkis clean.Performance / size (subset,
--measurement-time 5,mainvs this branch)The file for 10,000 facts, checkpointed, grows from 2,641,920 to 2,965,504 bytes (+12.2%), because index keys now carry value bytes. This is a short, subset run; treat it as indicative.
Breaking changes (v3.0.0 milestone)
Follow-ups
btree_pagefuzz target never reaches B+tree node decoding (it didn't onmaineither).selective_fact_fetchdoc comment omits valid time from the dedup key.Architecture.md(key layout, format v8) will be updated after merge.page_cache_capacity, and CI doesn't build it, so the new browser v8 assertion is unverified.🤖 Generated with Claude Code
https://claude.ai/code/session_01BedF9ZmqVZLkvDwn91fuKy