Repository navigation
feat(query): queries, cursors and BrowserDb read from pinned snapshots (#543, PR 3 of 3) - #580
Merged
Merged
Conversation
#543) PR 3 of 3 for #543 (plan steps 5, 8, 9, 10). - Snapshot gains Clone (pin in an Arc), now, and the read methods (live_facts_of_scans, all_facts, facts_as_of); a snapshot's pending reads stop at its length on every scan. FactStorage's reads go through one. - DatalogExecutor reads from one snapshot (set_snapshot); Minigraf::execute and query, PreparedQuery::execute and query, and BrowserDb.query take one. "now" for valid-time is the snapshot's time. - Cursor holds its snapshot, so it pins the committed generation until it is dropped, like a fact log. - BrowserDb: free pages in force are withheld from release (no dropped bytes, no IndexedDB delete) and released by a later commit once out of force; pins::split_released decides. - Crash test: a crash at every write point of a save with a pinned generation loses no fact or page, and the reopened database reuses the quarantined ids. - Docs: CHANGELOG, DURABILITY, CLAUDE.md, TEST_COVERAGE. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HyLscCUuJ1GQpdAeX79EdR
Codex review: now was read after the pending lock was released, so an append in between was left out of a snapshot whose now was past it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HyLscCUuJ1GQpdAeX79EdR
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
# Conflicts: # CLAUDE.md # docs/TEST_COVERAGE.md
Collaborator
Author
|
#394 perf envelope (
Point queries ( |
Collaborator
Author
|
Follow-up to the envelope comparison above, re-measured on an idle machine with alternating and swapped runs:
|
7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #543 (streaming engine 1/9, #473): plan steps 5, 8, 9, 10 of
docs/superpowers/plans/2026-10-11-543-snapshots-and-quarantine.md. PR 1 was #578, PR 2 was #579.Changes
SnapshotisClone(pin in anArc), carriesnowand the read methods (live_facts_of_scans,all_facts,facts_as_of); its pending reads stop at its length on every scan path.DatalogExecutor::set_snapshot:Minigraf::execute/query,PreparedQuery::execute/queryandBrowserDb.queryread one snapshot per query. Valid-time "now" is the snapshot's time, taken together with the pending cutoff.Cursorholds its snapshot until dropped (decision 7), so checkpoints and rebuilds run but do not reuse its generation's pages.pins::split_releaseddecides; a withheld id the commit wrote again is live and dropped from both lists. This fixes the PR 2 Codex note (released = listedincluded quarantined ids).verify()is clean, space is accounted for, and later checkpoints reuse the quarantined ids.Deviations from the plan
Cursor::from_resultwithout a snapshot for BrowserDb. Instead, browser cursors pin too, so step 8's withholding is actually used and tested (otherwise it would be dead code).split_releasedtakes awrittenpredicate (the buffer's dirty set) besideslisted/withheld/in_force. An id withheld earlier can be handed out by the commit that sees it out of force, and its bytes must not be dropped.@since v3.0.0(never released), so it is edited in place rather than wrapped in@until/@since.Tests
tests/cursor_test.rs: a cursor open across writes, checkpoints and a rebuild returns its answer at open; the file grows over 4× the no-cursor growth while it is open (checked to fail with the pin removed) and stops growing after close; a prepared cursor does the same.graph::storage: a snapshot leaves out later appends on all four scan kinds.storage::pins: foursplit_releasedunit tests.browser: a#[wasm_bindgen_test]keeps pages pinned by an open cursor readable across 20 commits and releases them afterwards (checked to fail with withholding removed).wasm-pack test --headless --firefox --features browser --lib: 33 passed.cargo test: 1456 passed, 10 ignored.snapshot_atomicity_testpassed 15 runs in a row. Clippy is clean on native (--all-features), wasm32-unknown-unknown withbrowser, and wasip1.Review
Codex (
codex exec -s read-only) found one issue:nowwas read after the pending lock was released. It is fixed in 1aa2bea. A transaction stamped before a snapshot but appended after it is still left out. That state really existed, and the behaviour is the same as before.Pending
🤖 Generated with Claude Code
https://claude.ai/code/session_01HyLscCUuJ1GQpdAeX79EdR