Repository navigation
feat: raw log writer — LogWriter (#431) - #466
Merged
Merged
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015GgLEs8YozJjYhwPEYgTUn
Builds a new database from explicit FactRecords, keeping each record's tx_count, tx_id, valid-time window and assertion flag: the load step of an offline migration after fact_log() on a read-only source. Records come in transaction order with gaps allowed, so a purged transaction stays a hole and :as-of on it shows the state before. advance_tx_count keeps the source's counter after trailing purges. Checks are structural: order (new API-015), one tx_id per transaction (new API-016), one valid-time window per fact per transaction (API-011) and size (WAL-003). A rejected record changes nothing. No WAL: records are committed in batches (65,536 facts) with the normal copy-on-write checkpoint, at transaction boundaries only. The file is built at <path>.partial and renamed into place by finish(), so a crash or an abandoned build never leaves part of a log at the target; an existing target or WAL is the new STG-043. No page-range copy: v8 pages share one dictionary and are not self-contained. Bindings follow in the binding repositories. Design: docs/superpowers/specs/2026-10-07-log-writer-design.md Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015GgLEs8YozJjYhwPEYgTUn
7 tasks done
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #431 (Rust API; exposing
LogWriterandFactRecordin the bindings is #467).What
LogWriterbuilds a new database from explicitFactRecords, keeping each record'stx_count,tx_id, valid-time window and assertion flag. It is the load step afterfact_log()on a read-only source (#430, #429), so extract, transform, load is complete; a purge is "do not append".tx_id. Leaving a transaction out leaves a hole;:as-ofon it shows the state before.advance_tx_countcloses the open transaction and keeps the source's counter after trailing purges, so the next write takes the source's next number.tx_idin one transaction the new API-016, two windows for one fact in one transaction the existing API-011 (bug: a later assertion does not replace an earlier valid-time window of the same [e a v]; :as-of N should see exactly one window per triple #435), oversize values or idents WAL-003. A rejected record changes nothing and the writer stays usable. Exact repeats are written once, as in a normal write.finish. Memory is O(batch + largest transaction).<path>.partialunder the exclusive lock and renamed into place byfinish(then the directory is synced). A crash or an abandoned build (dropping the writer deletes the.partial) never leaves part of a log atpath. An existing target or<path>.walis the new STG-043, atcreateand again before the rename. A stale.partialfrom a crashed build is truncated and reused; a live one is STG-025/026.read_onlyoptions are API-014.copy_pages. v8 trees share one dictionary and copy-on-write pages, so a page range is not self-contained (same conclusion as enhancement: streaming fact-log iterator with full history (tx, asserted, valid bounds) and coarse filters #430 decision 5).Design:
docs/superpowers/specs/2026-10-07-log-writer-design.mdTests
1345 total (1336 passing, 9 ignored), up 16.
tests/log_writer_test.rs(13 tests):.partialor WAL, verifies clean, and matches the source'stx_count, fact log and queries (current,:valid-at, every:as-of, entity/attribute/value-bound lookups) read-write, read-only and after reopen (bug: after a kill during an active run the EAVT index silently loses most entities' entries while AEVT stays intact — entity-bound lookups return [] and no rebuild path exists #370). It takes new writes at the source's next number.:as-of.:as-ofthe one before; trailing holes keep the counter; an empty log gives an empty database with the counter.read_only; a stale.partialis reused; dropping leaves nothing;finishrefuses a target created mid-build and leaves it untouched;LogWriter: Send.FileBackend::truncateempties a file.Docs updated: CHANGELOG, ROADMAP, README, CLAUDE.md, TEST_COVERAGE, ERROR_REFERENCE (STG-043, API-015, API-016, plus notes on API-011 and WAL-003). The wiki is left for the v3.0.0 doc sync, as with the earlier v3 PRs.
🤖 Generated with Claude Code
https://claude.ai/code/session_015GgLEs8YozJjYhwPEYgTUn