8000 fix: kernel file locking replaces the PID sidecar (#317, #304) by adityamukho · Pull Request #332 · project-minigraf/minigraf · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
6444c9a
docs: add kernel file locking design spec (#317, #304)
adityamukho Aug 22, 2026
023de6a
docs: add kernel file locking implementation plan
adityamukho Aug 22, 2026
b886ea4
chore: ignore .superpowers/ scratch workspace
adityamukho Aug 22, 2026
2c41f4a
docs: fix four plan defects found in the pre-flight scan
adityamukho Aug 22, 2026
75e373d
feat(storage): add lock-decision classifier and declare MSRV 1.89
adityamukho Aug 22, 2026
1fa978f
fix(storage): lock the .graph file in the kernel, not a PID sidecar
adityamukho Aug 22, 2026
be77de5
fix(storage): field drop order, ProceedUnlocked registry, message repair
adityamukho Aug 22, 2026
6b4334c
feat(db): add OpenOptions::allow_unlocked
adityamukho Aug 22, 2026
f720a3f
docs: verify clippy the way CI does, not more strictly
adityamukho Aug 22, 2026
6178faa
fix: gate FileBackend::open on tests and rewrite stale comment
adityamukho Aug 22, 2026
51878c9
chore: ignore memory.graph FTS sidecar files
adityamukho Aug 22, 2026
e102455
test: simulate crashes with real subprocesses
adityamukho Aug 22, 2026
b320f68
fix(storage): retry WouldBlock with bounded backoff before failing open
adityamukho Aug 22, 2026
19ddc94
test: restore explicit-transaction crash coverage, add cross-process …
adityamukho Aug 22, 2026
a1bfec1
test: reproduce #317 across PID namespaces
adityamukho Aug 22, 2026
8cc1419
test: fail loudly on a missing helper, kill-on-drop for container A
adityamukho Aug 22, 2026
44cb040
docs: record the kernel locking change
adityamukho Aug 22, 2026
4420bee
style: apply cargo fmt to test files
adityamukho Aug 22, 2026
330a3f6
fix(storage): make the unlocked-path registry check atomic
adityamukho Aug 22, 2026
632035f
docs: correct two factual errors found in the fix-wave re-review
adityamukho Aug 22, 2026
0d17497
docs: record kernel file locking in ROADMAP and CLAUDE.md status
adityamukho Aug 22, 2026
1814ce7
test: kill the namespaced holder by process group, not by pid
adityamukho Aug 22, 2026
e4d90b1
docs: credit @ocasazza in the CHANGELOG for #317
adityamukho Aug 22, 2026
6355daa
test: let the namespaced holder end its own life
adityamukho Aug 22, 2026
4e7712e
Merge branch 'main' into kernel-file-locking
adityamukho Aug 22, 2026
2d4dc1b
test: let the parent decide when the live holder releases
adityamukho Aug 22, 2026
6a7f926
docs: resync test counts after main was merged in
adityamukho Aug 22, 2026
b15f08d
test: pass the release marker as an argument, not an env var
adityamukho Aug 22, 2026
505f60e
fix(test): read the raw header after closing the database
adityamukho Aug 22, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,14 @@

.idea
.worktrees/
.superpowers/
.codex
.opencode/
.claude/
.mcp.json
skills/
memory.graph
memory.graph.fts.sqlite3*

# heaptrack profiling artifacts
heaptrack.*.zst
Expand Down
83 changes: 83 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,91 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## Unreleased

### Breaking-ish changes

- **Adding `OpenOptions::allow_unlocked` breaks struct-literal construction.**
`OpenOptions` has all-public fields and is not `#[non_exhaustive]`, so under
Rust's semver rules any downstream code building it as
`OpenOptions { wal_checkpoint_threshold: .., page_cache_size: .., .. }`
without `..Default::default()` no longer compiles. Callers who spread
`..Default::default()`, or who use the chainable builder methods, are
unaffected. This branch had to update two such literals in its own tree —
the `Default` impl itself and one test — and only the latter is a
"downstream literal" in the sense that matters to a consumer.

Recorded here because the release decision is deliberately deferred, and
whoever makes it needs the facts: this is the **first post-1.0 field
addition** to `OpenOptions`. The earlier `max_derived_facts` and
`max_results` fields landed in v0.19.0, before 1.0, so they set no
precedent under the stability guarantee in PHILOSOPHY.md §7. Marking
`OpenOptions` `#[non_exhaustive]` would immunise every future field
addition, but is itself a breaking change, so it is a decision to make at
the same time as the version number rather than after it.
- **Minimum supported Rust version is now 1.89** (was effectively 1.85, implied
by `edition = "2024"`). Required for `std::fs::File::try_lock`, stabilised in
1.89. Declared as `rust-version` in `Cargo.toml`, so cargo reports a clear
error rather than a confusing one.
- **On Windows, an open database can no longer be read through any other file
handle** — including another handle in the same process. `LockFileEx` locks
are mandatory rather than advisory and exclude every handle but the one
holding them, so copying or backing up an open `.graph` file now fails on
Windows where it previously succeeded, and so does opening it yourself for a
raw read while the database is live. The error is os error 33, "another
process has locked a portion of the file", even when the other process is
you. Unix locks remain advisory and are unaffected. Close the database
before reading the file directly — which also avoids a torn read.
- **`open()` now fails on filesystems that cannot lock at all** rather than
proceeding unprotected. Set `OpenOptions::allow_unlocked(true)` to accept the
risk on single-writer deployments.
- **A database genuinely locked by another process now reports its error
later than before** — about 375ms later. `FileBackend::open_with` retries a
cross-process `WouldBlock` up to 10 times with backoff doubling from 5ms and
capped at 50ms before concluding another process holds the lock. This exists
because `fork` transiently duplicates the lock-holding descriptor into a
child until `execve` closes its close-on-exec descriptors, so any process
that spawns subprocesses — the normal pattern for the Python and Node
bindings — could see a spurious `WouldBlock` from its own child. SQLite
solves the same problem with `busy_timeout`; this is the same idea, bounded
so a real conflict still fails, just not instantly. A same-process conflict
(this process already holds the file open) is unaffected: it fails
immediately, since waiting could never help.
- **A refused open on a filesystem that cannot lock may leave a 0-byte
`.graph` file behind.** The file must exist before it can be locked, so the
create-then-lock ordering now creates before it can know the lock will be
refused. Harmless — the next open sees `is_new` and initialises normally —
but worth knowing if you inspect the filesystem after a failed open.

### Bug fixes

- **A `.graph` file is no longer bricked when its holder dies in a container**
(#317): `FileLock` recorded the holder's PID in a `.graph.lock` sidecar and
refused to open when that PID equalled our own. Every container's main process
is PID 1 in its own namespace, so a container killed with SIGKILL left the
sidecar holding `1`, and the replacement container — also PID 1 — read its own
PID back and refused. The refusal was permanent; no code path could clear it,
and an operator had to delete the file by hand. The `pid == our_pid` refusal
was itself the fix for #304, so the two defects were entangled.

A PID cannot serve as a liveness token: it is not unique across PID namespaces
and is recycled within one. The sidecar is gone, and the lock now lives in the
kernel via `std::fs::File::try_lock` on the `.graph` file itself. The kernel
releases it whenever the process exits, however it exits, so a crashed holder
leaves nothing behind. Because `flock` and OFD locks attach to the open file
description rather than the process, a second open within one process is also
denied, so #304 stays fixed with no PID bookkeeping at all.

A leftover `.graph.lock` from a version before 1.3 is ignored and never
deleted — it no longer has any effect, and a still-running old process may
still depend on it. Running mixed versions against one file is not
supported; upgrade all writers together.

Reported and diagnosed by [@ocasazza](https://github.com/ocasazza) in #317,
who supplied the reproduction, identified the PID-namespace mechanism, and
correctly traced the refusal back to the #304 fix that caused it. That
analysis is what made this straightforward to verify. One of the tests from
their proposed patch is the ancestor of the regression test that now covers
this.

- **Keywords may contain `?`**: `:person/alive?` now lexes as a single keyword. Previously `?` terminated the keyword, so `[:e :alive? true]` lexed as `:alive` followed by a stray `?` symbol and was rejected as a four-element fact — reporting `Optional 4th element of a fact must be a map`, which named the value rather than the keyword. `?` is a constituent character in EDN keywords and predicate-style names (`:artist/dead?`) are idiomatic. Query variables are unaffected: a `?` not preceded by `:` still begins a symbol. `tests/grammar/grammar.pest` updated to match.

## v1.2.3 — 2026-08-10
Expand Down
4 changes: 2 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -167,14 +167,14 @@ Auto-migrates v1/v2/v3/v4/v5/v6 → v7 on open/checkpoint.

## Test Coverage

**998 tests passing** (990 passing, 8 ignored; unit + integration + doc).
**1023 tests passing** (1015 passing, 8 ignored; unit + integration + doc).
See `docs/TEST_COVERAGE.md` for the full per-file breakdown.

**Testing conventions** — see the Testing Conventions section below before writing any tests.

## Key Files for the Next Phase

Phase 8 is complete — v1.0.0 released. Wave 3 Reliability is complete. #231 Repo Split is complete — Python, Node, WASM, Java, Android, Swift, and C bindings are all in separate repos under the project-minigraf org. Wave 8 Documentation is complete (#190, #191, #192). v1.2.0 released — magic sets rewriting (#289) and per-query limits (#288). Wave 5 (Query Profiler) is next.
Phase 8 is complete — v1.0.0 released. Wave 3 Reliability is complete. #231 Repo Split is complete — Python, Node, WASM, Java, Android, Swift, and C bindings are all in separate repos under the project-minigraf org. Wave 8 Documentation is complete (#190, #191, #192). v1.2.0 released — magic sets rewriting (#289) and per-query limits (#288). Kernel file locking (#317, #304) is merged but unreleased: `std::fs::File::try_lock` on the `.graph` file replaced the `.graph.lock` PID sidecar, which bricked databases after any container restart. MSRV is now 1.89. The release version is deliberately undecided — see the CHANGELOG's breaking-changes section. Wave 5 (Query Profiler) is next.

Wave 5 relevant areas (see `ROADMAP.md` for full spec):
- `src/query/datalog/` — query executor where profiling hooks will go (#185)
Expand Down
1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ resolver = "2"
name = "minigraf"
version = "1.2.3"
edition = "2024"
rust-version = "1.89"
description = "Zero-config, single-file, embedded graph database with bi-temporal Datalog queries"
license = "MIT OR Apache-2.0"
authors = ["Aditya Mukhopadhyay"]
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ let r2 = pq.execute(&[("tx", BindValue::TxCount(2)), ("entity", BindValue::Entit

```bash
cargo run # interactive Datalog REPL
cargo test # run 998 tests
cargo test # run 1023 tests
cargo run < demos/demo_recursive.txt # recursive rules demo
```

Expand Down
3 changes: 2 additions & 1 deletion ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -1707,6 +1707,7 @@ When evaluating features, ask:
- ✅ v1.2.1 — Magic sets `fb` adornment fix: sentinel entity + 2-arg guard preserves keyword type through value-position binding (#297, #298)
- ✅ v1.2.3 — Core unchanged; cut to carry `minigraf-node`'s new `close()` (project-minigraf/minigraf-node#1), needed because v1.2.2's handle exclusion left GC'd hosts with no way to release a handle
- ✅ v1.2.2 — `FileLock::acquire` no longer treats a lock held by our own PID as stale, which had allowed two handles on one file per process with divergent page tables — the source of the intermittent `Page N out of bounds` (#304); ordering predicates compare strings lexicographically (#312)
- 🚧 Unreleased — Kernel file locking replaces the PID sidecar (#317, #304). A PID is not a liveness token: it is not unique across PID namespaces and is recycled within one, so a container killed with SIGKILL left `.graph.lock` holding `1` and the replacement container — also PID 1 — read its own PID back and refused to open, permanently. `std::fs::File::try_lock` moves liveness to the kernel, which releases on process death however it occurs; because `flock`/OFD locks attach to the open file description, a second open within one process is refused too, so v1.2.2's fix for #304 is preserved without any PID bookkeeping. The sidecar is gone. MSRV moves 1.85 → 1.89. Nightly NFS and Docker lock suites deferred to #324

**Transferred to other repos**:
- Developer tools: `minigraf-inspector` (#184), `minigraf-visualizer` (#186); #185 (query profiler) deferred to 2.0
Expand All @@ -1729,4 +1730,4 @@ See [GitHub Issues](https://github.com/project-minigraf/minigraf/issues) for spe

---

**Last Updated**: August 2026 — v1.2.3 released (core unchanged; carries the minigraf-node `close()` fix). Next up is v1.3.0 (performance + structured errors: #274, #275, #248, #250, #277)
**Last Updated**: August 2026 — v1.2.3 released (core unchanged; carries the minigraf-node `close()` fix). Kernel file locking (#317, #304) is merged but unreleased; its release version is deliberately undecided, see the CHANGELOG's breaking-changes section. Next up is v1.3.0 (performance + structured errors: #274, #275, #248, #250, #277)
47 changes: 47 additions & 0 deletions docs/ERROR_REFERENCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,9 @@ appear in runtime output today — runtime codes are tracked in
| STG-022 | Page id overflow writing fact pages | Storage |
| STG-023 | Page index exceeds u64::MAX | Storage |
| STG-024 | Pending fact count exceeds u64::MAX | Storage |
| STG-025 | Database already open in this process | Storage |
| STG-026 | Database locked by another process | Storage |
| STG-027 | Filesystem does not support file locking | Storage |
| WAL-001 | Invalid WAL magic number | WAL |
| WAL-002 | Unsupported WAL version | WAL |
| WAL-003 | Fact serialised size exceeds maximum | WAL |
Expand Down Expand Up @@ -1791,6 +1794,50 @@ See the [file format section in README](../README.md#file-format) for version hi

---

### STG-025 Database already open in this process

**Error text**: `Database is already open in this process ({path}). A second handle on one file would give each its own page table and corrupt both — reuse the existing handle instead. \`Minigraf\` is cheap to clone and all clones share the same database.`

**Cause**: This process already holds an open handle on this file. Two `FileBackend` instances on one file each cache their own `header.page_count`, allocate new pages from that count, and bounds-check reads against it, so the two page tables diverge and produce intermittent `Page N out of bounds` errors.

**Resolution**:
- Reuse the handle you already have. `Minigraf` is cheap to clone and all clones share one database.
- If you cannot find the other handle, it is usually held by a longer-lived object than you expect — a cache, a registry, or a background task.

**Scenario**: A request handler calls `Minigraf::open` per request while a connection pool already holds the same path open.

---

### STG-026 Database locked by another process

**Error text**: `Database is locked by another process ({path}). The lock is held on the file itself and is released automatically when the holding process exits, so there is no lock file to clean up.`

**Cause**: Another process holds the kernel file lock on this `.graph` file. Minigraf is single-writer: one process at a time. This is reported only after a bounded retry (up to ~375ms) rules out a transient `fork`-related false positive — see the Unreleased CHANGELOG entry on the bounded retry.

**Resolution**:
- Wait for the other process to exit; the kernel releases the lock automatically, including when the process is killed.
- There is no lock file to delete. If you find a stale `.graph.lock`, it is a leftover from a version before 1.3 and has no effect — it is not read or deleted, and should not be removed by hand in case an old process still depends on it.
- If two services genuinely need the same file, put one in front of the other. Minigraf is embedded, not a server.

**Scenario**: A rolling deployment starts the new pod before the old one has exited, and both mount the same volume.

---

### STG-027 Filesystem does not support file locking

**Error text**: `Failed to lock database at {path}: {e}. This filesystem does not support file locking (common on NFSv3 without lockd, and on some FUSE mounts). Set \`allow_unlocked\` in \`OpenOptions\` to open anyway — that accepts the risk that concurrent writers corrupt the file.`

**Cause**: The underlying filesystem rejected the lock outright. Common on NFSv3 mounts with no `lockd` running, and on some FUSE filesystems. Because the file must exist before it can be locked, this refusal can leave a 0-byte `.graph` file behind; the next open sees `is_new` and initialises normally.

**Resolution**:
- Prefer moving the database to a filesystem that supports locking — this is the safe fix.
- On NFS, ensure NFSv4, or start `lockd` for NFSv3.
- If you are certain there is exactly one writer, set `OpenOptions::allow_unlocked(true)`. This accepts the risk that concurrent writers corrupt the file; Minigraf cannot detect them without a working lock.

**Scenario**: A `.graph` file placed on an NFSv3 export whose `lockd` is not running.< 6546 /td>

---

## WAL — Write-Ahead Log Errors

WAL errors relate to the sidecar `.wal` file written alongside the `.graph` file.
Expand Down
Loading
Loading
0