adityamukho · Oct 7, 2026 · Oct 7, 2026 · Oct 7, 2026
@@ -21,6 +21,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ### Added
 
 - **Fact log: `Minigraf::fact_log()`** (#430). Streams every fact version in the database as `FactRecord`s (entity, attribute, value, `tx_count`, `tx_id`, valid-time bounds, and whether it is an assertion or a retraction), without Datalog. `FactFilter` selects attributes (exact or by prefix), entities and a `tx_count` range, checked on index keys so excluded records are never decoded. `FactOrder::Tx` (the default) returns records in ascending `tx_count`; format v8 has no index in transaction order, so it holds at most `FactFilter::window` records (default 262,144) and scans the selected keys once per window. `FactOrder::Storage` reads in one pass. The log reads the database as it was when it opened; while one is open, checkpoints are deferred (writes stay in the WAL, automatic checkpoints wait, and `checkpoint()` or `rebuild_indexes()` with work to do fail with the new `API-013`). Close, drop or finish the log to release it. Like `Cursor`, a `FactLog` is owned and `Send`. Rust API only for now; the bindings follow.
+- **Raw log writer: `LogWriter`** (#431). Builds a new database from explicit `FactRecord`s, the load step of an offline migration after `fact_log()` on a read-only source. Each record keeps its `tx_count`, `tx_id`, valid-time window and assertion flag. Records come in transaction order, gaps allowed: leaving a transaction out leaves a hole, and `:as-of` on it shows the state before. `advance_tx_count` keeps the source's counter after trailing purged transactions. The checks are structural only: order (new `API-015`), one `tx_id` per transaction (new `API-016`), one valid-time window per fact per transaction (`API-011`) and size (`WAL-003`); a rejected record changes nothing. There is no WAL: records are committed in batches with the normal copy-on-write checkpoint, never splitting a transaction. The file is built at `<path>.partial` and renamed into place by `finish`, so a crash or an abandoned build never leaves part of a log at `path`; an existing target or WAL is the new `STG-043`. The page-range copy the issue proposed is not provided, since v8 pages share one dictionary and are not self-contained. The bindings follow.
 - **Read-only open: `OpenOptions::read_only`** (#429). Opens a database so that nothing done through the handle writes to the `.graph` file or its WAL. The file is locked in shared mode: any number of read-only handles, in this process or others, can hold it at once, and they exclude a read-write open (and it excludes them) with the existing STG-025/STG-026. The handle shows the same data as a read-write open: a WAL left by an earlier session is applied in memory but not checkpointed or deleted, and a format v7 file is read into memory instead of migrated. A missing file is the new `STG-042` (nothing is created). `transact`, `retract`, `begin_write`, `checkpoint` and `rebuild_indexes` fail with the new `API-014`; queries, cursors, `fact_log`, `verify` and rule registration work, and dropping the handle does not checkpoint. `page_cache_size` applies as usual, so a tool can keep the whole file resident. Rust API only for now; the bindings follow.
 - **Cursor API: `Minigraf::query()` and `PreparedQuery::query()`** (#432). Both return a `Cursor` that hands a query's rows back in batches: `vars()`, `next_batch(max_rows) -> Result<Option<Batch>, _>`, `close()`, and an iterator over rows. The answer is fixed when the cursor opens, so writes that commit while it is open do not change it. A cursor borrows nothing from the database handle: it can outlive it and move to another thread. For now the answer is still computed when the cursor opens, under the same limits as `execute()`; the API is shaped so the engine can later produce rows as they are pulled without breaking callers. `query()` rejects `transact`, `retract` and `rule` with the new `API-012`, and a query with bind slots with `API-010`. `execute()` is unchanged. Rust API only for now; the bindings follow.
 - **`Minigraf::verify()` and `Minigraf::rebuild_indexes()`** (#373). Page checksums turn a torn or rotted page into an error, but they cannot see a page that is valid and says the wrong thing. `verify()` walks every committed page and returns an `IntegrityReport` with one structured error per finding: an index that holds different facts than EAVT or a different count than the meta page (new `STG-038`), keys out of order or a page reached twice (new `STG-039`), dictionary maps that disagree (new `STG-040`), ids with no dictionary entry (`STG-036`), and leaked, doubly used or wrongly free pages (`STG-035`). It keeps no per-fact state: indexes are compared by an order-independent digest. `rebuild_indexes()` rebuilds EAVT, AEVT, AVET and VAET from an intact index that agrees with another one (or the only intact one), derives the free list again, and commits like a checkpoint, including uncheckpointed writes; a crash at any point leaves the previous checkpoint. If no index can serve as the source, or the dictionary is damaged, it fails with new `STG-041` and writes nothing. Both are Rust API only for now; the bindings follow.

@@ -141,6 +141,7 @@ cargo run < demos/demo_negation.txt
 6. **`src/db.rs`** — Public API: `Minigraf::open/execute/query/fact_log/prepare/begin_write/checkpoint/save/verify/rebuild_indexes`, `WriteTransaction`, `IntegrityReport`, `OpenOptions::page_cache_size`, `OpenOptions::read_only` (#429: shared lock, nothing written, writes are `API-014`)
    - `src/cursor.rs`: `Cursor` / `Batch` — owned, `Send` cursor returned by `Minigraf::query` and `PreparedQuery::query` (#432); answer computed at open today, shaped for streaming operators
    - `src/fact_log.rs`: `FactLog` / `FactFilter` / `FactRecord` / `FactOrder` — every fact version streamed by `Minigraf::fact_log` (#430): tx order (multi-pass, bounded window) or storage order, key-level filters; an open log pins the committed generation, deferring checkpoints (`API-013`)
+   - `src/log_writer.rs`: `LogWriter` — builds a new file from `FactRecord`s with their tx and valid-time kept (#431): tx-ordered appends with holes (`API-015`/`API-016`), batched copy-on-write checkpoints at tx boundaries, no WAL, built at `<path>.partial` and renamed by `finish` (`STG-043` if the target exists)
 
 7. **`src/wal.rs`** — Fact-level sidecar WAL, CRC32-protected entries, crash recovery
 
@@ -196,7 +197,7 @@ Auto-migrates v7 → v8 on open (spec §9, with a backup meta page). v1–v6 are
 
 ## Test Coverage
 
-**1329 tests** (1320 passing, 9 ignored; unit + integration + doc).
+**1345 tests** (1336 passing, 9 ignored; unit + integration + doc).
 See `docs/TEST_COVERAGE.md` for the full per-file breakdown.
 
 **Testing conventions** — see the Testing Conventions section below before writing any tests.

@@ -66,7 +66,7 @@ v2.x gets data-integrity and security fixes for 12 months after v3.0.0 ships. Se
 ## Quick Start
 
 ```rust
-use minigraf::{FactFilter, Minigraf, OpenOptions};
+use minigraf::{FactFilter, LogWriter, Minigraf, OpenOptions};
 
 // Open or create a file-backed database
 let db = OpenOptions::new().path("myapp.graph").open()?;
@@ -100,6 +100,16 @@ while let Some(batch) = log.next_batch(1000)? {
 // a read-write handle (like `db` on myapp.graph) excludes them.
 let archive = Minigraf::open_with_options("archive.graph", OpenOptions::new().read_only(true))?;
 
+// Log writer — copy records into a new file with their transactions and valid-time
+// windows kept; leave records out to purge them
+let mut out = LogWriter::create("archive-clean.graph", OpenOptions::new())?;
+for rec in archive.fact_log(&FactFilter::new())? {
+    let rec = rec?;
+    if rec.attribute != ":person/ssn" { out.append(&rec)?; }
+}
+out.advance_tx_count(archive.current_tx_count())?;
+out.finish()?;
+
 // Explicit transaction — all-or-nothing
 let mut tx = db.begin_write()?;
 tx.execute(r#"(transact [[:alice :person/age 31]])"#)?;
@@ -121,7 +131,7 @@ let r2 = pq.execute(&[("tx", BindValue::TxCount(2)), ("entity", BindValue::Entit
 
 ```bash
 cargo run          # interactive Datalog REPL
-cargo test         # run 1329 tests
+cargo test         # run 1345 tests
 cargo run < demos/demo_recursive.txt   # recursive rules demo
 ```
 

@@ -70,6 +70,7 @@ Scope — query API:
 - Cursor API: `Minigraf::query` and `PreparedQuery::query` return an owned `Cursor` that delivers rows in batches, with the answer fixed at open (#432; Rust API done on `v3`, with the current evaluator computing the answer at open). The binding cursor objects follow. Streaming operators replace the evaluator behind the cursor in 3.x without further API changes; how operators bound their memory (spill, scratch pages in the `.graph` file, or a clear error) is still open.
 - Fact-log iterator: `Minigraf::fact_log(&FactFilter)` streams every fact version (assertions and retractions, with `tx_count`, `tx_id` and valid-time bounds) in transaction order or storage order, filtered by attribute, attribute prefix, entity and tx range, with memory bounded by a window rather than the database (#430; Rust API done on `v3`). An open log defers checkpoints (`API-013` for an explicit one). A raw log writer (#431) completes the extract-transform-load path; the binding objects follow with #462.
 - Read-only open: `OpenOptions::read_only` takes a shared lock, so several readers can hold a file at once, writes nothing to the file or its WAL (a WAL is applied in memory, a v7 file is read without migrating), and refuses writes with `API-014` (#429; Rust API done on `v3`). The bindings' `open_with_options` follows.
+- Raw log writer: `LogWriter` builds a new database from `FactRecord`s with their `tx_count`, `tx_id`, valid-time windows and assertion flags kept, gaps allowed (a purged transaction stays a hole for `:as-of`), at `<path>.partial` renamed into place by `finish` (#431; Rust API done on `v3`). Completes extract (#429, #430), transform, load. The binding objects follow.
 
 Scope — release-gate testing and documentation:
 

@@ -151,6 +151,7 @@ with no `CodedError` anywhere in its chain.
 | STG-040 | Dictionary inconsistency | Storage |
 | STG-041 | Cannot rebuild indexes | Storage |
 | STG-042 | Database file not found (read-only open) | Storage |
+| STG-043 | Log writer target already exists | Storage |
 | WAL-001 | Invalid WAL magic number | WAL |
 | WAL-002 | Unsupported WAL version | WAL |
 | WAL-003 | Value size exceeds maximum | WAL |
@@ -171,6 +172,8 @@ with no `CodedError` anywhere in its chain.
 | API-012 | Non-query command passed to query() | Database API |
 | API-013 | Checkpoint while a fact log is open | Database API |
 | API-014 | Write on a read-only handle | Database API |
+| API-015 | Log writer record out of transaction order | Database API |
+| API-016 | Log writer record with a second tx_id for one transaction | Database API |
 | INT-000 | Unclassified internal error | Internal |
 | INT-001 | WriteTransaction already in progress on this thread | Internal |
 | INT-002 | Invalid entity (API layer) | Internal |
@@ -2199,6 +2202,25 @@ A read-only handle (`OpenOptions::read_only`) in another process also refuses a
 
 **Scenario**: A migration tool is pointed at `memory.graph` from the wrong working directory.
 
+
+### STG-043 Log writer target already exists
+
+**Error text**: `{} already exists; a log writer only creates a new database`
+
+**Cause**: `LogWriter::create` was given a path where a file, or that path's WAL (`<path>.wal`), already exists; or `LogWriter::finish` found that a file appeared at the path while the build ran. A log writer never overwrites or extends a database, and a stale WAL beside the new file would be replayed onto it. The `{}` is the path that exists. At `finish`, the existing file is left untouched and the build (`<path>.partial`) is deleted.
+
+**Resolution**:
+- Write to a new path, then replace the old database yourself once the new one is checked (`verify()`, your own audit).
+- If the old file and WAL are no longer needed, delete them first.
+- Do not create the target while a build runs.
+
+```rust
+let mut out = LogWriter::create("memory.v2.graph", OpenOptions::new())?;
+// ... append, then out.finish()?; then swap files
+```
+
+**Scenario**: A migration is re-run after a successful first run, with the same output path.
+
 ---
 
 ## WAL — Write-Ahead Log Errors
@@ -2235,7 +2257,7 @@ The WAL is replayed on open and deleted on checkpoint.
 
 **Error text**: `Value of {} bytes exceeds the maximum of {} bytes. Store large payloads externally and reference them with a Value::String URL/path or Value::Ref entity ID.`
 
-**Cause**: A string value is longer than `MAX_VALUE_BYTES` (4,068 bytes, one value page), or an attribute name or keyword value is longer than 1,024 bytes. This typically means a `Value::String` contains very large content such as raw document text, a base64-encoded image, or binary data. In-memory databases have no WAL and no limit.
+**Cause**: A string value is longer than `MAX_VALUE_BYTES` (4,068 bytes, one value page), or an attribute name or keyword value is longer than 1,024 bytes. This typically means a `Value::String` contains very large content such as raw document text, a base64-encoded image, or binary data. `LogWriter::append` checks the same limits. In-memory databases have no WAL and no limit.
 
 **Resolution**:
 - Store large payloads in an external file or object store.
@@ -2456,7 +2478,7 @@ let result = pq.execute(&[
 
 **Error text**: `one transaction asserts the same value of {} with two valid-time windows; assert it once with its final window`
 
-**Cause**: One transaction asserts the same entity, attribute and value twice with different `:valid-from`/`:valid-to` bounds. At any transaction time a fact has exactly one current valid-time window, and the window of the latest transaction applies. Two windows in the same transaction leave no latest one, so the transaction is rejected and nothing is written. The `{}` is the attribute. This can come from one `(transact ...)` with per-fact bounds, or from several `execute()` calls inside one `WriteTransaction`.
+**Cause**: One transaction asserts the same entity, attribute and value twice with different `:valid-from`/`:valid-to` bounds. At any transaction time a fact has exactly one current valid-time window, and the window of the latest transaction applies. Two windows in the same transaction leave no latest one, so the transaction is rejected and nothing is written. The `{}` is the attribute. This can come from one `(transact ...)` with per-fact bounds, from several `execute()` calls inside one `WriteTransaction`, or from `LogWriter::append` records of one transaction (that record alone is rejected).
 
 **Resolution**:
 - Assert the fact once per transaction, with the window it should have.
@@ -2531,6 +2553,31 @@ let mut log = src.fact_log(&FactFilter::new())?;
 
 **Scenario**: An offline migration tool opens its source read-only and, by mistake, runs a cleanup `retract` against it instead of against the destination.
 
+
+### API-015 Log writer record out of transaction order
+
+**Error text**: `tx_count {} is out of order: the log writer is at {}, and a record must open a later transaction or join the open one`
+
+**Cause**: `LogWriter::append` received a record whose `tx_count` is 0, lower than the writer's current `tx_count`, or equal to it after that transaction was closed (by `advance_tx_count`, or by a record of a later transaction). The same code is returned by `advance_tx_count` with a value below the writer's. Records must come in non-decreasing `tx_count` order, each transaction's records together. The record is not written and the writer stays usable.
+
+**Resolution**:
+- Read the source with `fact_log()` in the default `FactOrder::Tx`, which returns records in non-decreasing `tx_count` with each transaction's records together. `FactOrder::Storage` does not, so sort its records by `tx_count` first.
+- Gaps are allowed: skip a transaction's records to purge it.
+- Call `advance_tx_count` only after the last record, or with a value above every record still to come.
+
+**Scenario**: A transform step merges two sources and appends their records one source after the other.
+
+### API-016 Log writer record with a second tx_id for one transaction
+
+**Error text**: `transaction {} has tx_id {}; a record with tx_id {} cannot join it`
+
+**Cause**: `LogWriter::append` received a record with the open transaction's `tx_count` but a different `tx_id`. A transaction has one wall-clock time, which `:as-of` with a timestamp compares against. The record is not written and the writer stays usable.
+
+**Resolution**:
+- Keep each record's `tx_id` as the source gave it. If a transform rewrites timestamps, give every record of one transaction the same new `tx_id`.
+
+**Scenario**: A transform sets `tx_id` from a per-record field instead of from the transaction.
+
 ---
 
 ## INT — Internal Errors