8000 fix(datalog): bound parser recursion depth to prevent stack overflow by adityamukho · Pull Request #326 · project-minigraf/minigraf · GitHub
[go: up one dir, main page]

Skip to content

fix(datalog): bound parser recursion depth to prevent stack overflow - #326

Merged
adityamukho merged 2 commits into
mainfrom
worktree-fix-321-parser-recursion-depth
Aug 22, 2026
Merged

adityamukho merged 2 commits into
mainfrom
worktree-fix-321-parser-recursion-depth

Conversation

@adityamukho
Copy link
Copy Markdown
Collaborator

Summary

  • Parser::parse_value in src/query/datalog/parser.rs mutually recursed with parse_list/parse_vector/parse_map with no depth limit; deeply nested or unbalanced input exhausted the stack and aborted the process instead of returning a parse error. Found by the datalog_parser libFuzzer target.
  • Adds a depth: usize field to Parser, checked on every parse_value call, returning a graceful Err once nesting exceeds MAX_RECURSION_DEPTH = 100 — well beyond any real query (deepest nesting found anywhere in tests/demos is 6), mirroring the existing MAX_STRING_LENGTH/MAX_KEYWORD_LENGTH/MAX_SYMBOL_LENGTH limits already in this file.
  • Adds a regression unit test (test_parse_edn_deep_nesting_returns_error) and a fuzz corpus seed (fuzz/corpus/datalog_parser/seed_deep_nesting_recursion.bin) — verified to genuinely stack-overflow the pre-fix parser and to be cleanly rejected by the fix.

Fixes #321

Test plan

  • cargo test — full suite passes (668 lib tests + all integration/doc tests, 0 failures)
  • cargo clippy --lib — clean
  • Verified pre-fix build genuinely SIGABRTs (stack overflow) on the new regression test/corpus seed input, confirmed on this branch it now returns a clean Err containing "recursion depth"
  • Independent code review subagent confirmed: every recursive path in the parser routes through the guarded parse_value, depth accounting is balanced across error paths, no legitimate query in the codebase approaches the new limit, and the error propagates cleanly to Minigraf::execute with no bypass panic path

🤖 Generated with Claude Code

https://claude.ai/code/session_01YMWSutMLJJ7Y5b8hs9ZaCU

adityamukho and others added 2 commits August 22, 2026 15:59
…321)

Parser::parse_value mutually recurses through parse_list/parse_vector/
parse_map with no depth tracking, so deeply nested or unbalanced input
(e.g. thousands of unclosed parens) exhausts the stack and aborts the
process instead of returning a parse error.

The nightly datalog_parser fuzzer found this via a malformed,
multi-level-nested query string.

Add a depth counter to Parser, checked on every parse_value call, that
returns a graceful error once nesting exceeds 100 levels — well beyond
any legitimate query, per the existing MAX_STRING_LENGTH-style limits
already used in this parser.

Fixes #321

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YMWSutMLJJ7Y5b8hs9ZaCU
The reconstructed hex bytes from the issue report failed in tokenize()
on a leading control character before ever reaching the recursion-
guarded parser, so the seed never exercised the vulnerable code path
on either the pre-fix or fixed build.

Replace it with 5000 unclosed '(' characters, verified to genuinely
SIGABRT with a stack overflow against the pre-fix parser and to be
cleanly rejected with the new recursion-depth error on the fixed one.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YMWSutMLJJ7Y5b8hs9ZaCU
@codecov
codecov Bot commented Aug 22, 2026
Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@adityamukho
adityamukho merged commit 234868a into main Aug 22, 2026
17 checks passed
@adityamukho
adityamukho deleted the worktree-fix-321-parser-recursion-depth branch August 22, 2026 15:05
adityamukho added a commit that referenced this pull request Aug 22, 2026
The merge brought #325 and #326, adding two lib unit tests, so every count
this branch had just synced was stale again: 1015 passing, 8 ignored,
1023 total, with 678 in the lib binary.

Updated CLAUDE.md, README.md, and docs/TEST_COVERAGE.md, including the
itemized Test Summary — its 33 bullets sum to exactly 1023, verified
rather than eyeballed.

Co-Authored-By: Olive Casazza <olive.casazza@schrodinger.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dtz3m9Ww7SUbft1TqRSGXf
adityamukho added a commit that referenced this pull request Aug 22, 2026
* docs: add kernel file locking design spec (#317, #304)

Replaces the PID-sidecar FileLock with std::fs::File::try_lock. A PID is
not a liveness token: it is not unique across PID namespaces and is
recycled within one, so any PID-based liveness check is unsound wherever
containers run.

Supersedes PR #318, whose /proc start-time comparison cannot establish
liveness across namespaces and would convert a safe refusal into silent
cross-process corruption.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dtz3m9Ww7SUbft1TqRSGXf

* docs: add kernel file locking implementation plan

Scoped to the core fix plus the PID-namespace regression test. The nightly
NFS and Docker tiers are tracked separately in #324.

Also corrects two errors in the spec: OpenOptions exposes both public
fields and builder methods, so allow_unlocked needs both; and there are 14
crash-simulation call sites, not 23 -- the earlier figure counted comment
mentions of mem::forget alongside the actual calls.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dtz3m9Ww7SUbft1TqRSGXf

* chore: ignore .superpowers/ scratch workspace

Subagent-driven-development writes its ledger, briefs, and review packages
there. It is scratch, not source.

* docs: fix four plan defects found in the pre-flight scan

- Task 4's crash assertion required death by signal, which fails on
  Windows CI where abort() yields an exit code instead. wal_test.rs is
  gated only on not(wasm32), so it does run there.
- Task 5's Files block omitted examples/pid_ns_helper.rs, which its own
  Step 2 creates.
- Task 5's catching-power check used `git stash`, which cannot revert work
  already committed by Tasks 1-2 -- as written it would have tested the
  fixed code against itself and always passed. Uses a throwaway worktree
  at the pre-fix commit instead.
- Task 6 edited .wiki/, which does not exist in a worktree; it is a plain
  clone living only in the main checkout.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dtz3m9Ww7SUbft1TqRSGXf

* feat(storage): add lock-decision classifier and declare MSRV 1.89

Isolates the try_lock decision from the I/O so the unsupported-filesystem
branch is testable without a filesystem that cannot lock.

allow_unlocked deliberately does not apply to WouldBlock.

* fix(storage): lock the .graph file in the kernel, not a PID sidecar

A PID is not a liveness token. It is not unique across PID namespaces and
is recycled within one, so the sidecar could never distinguish a crashed
holder from a live one. A container killed with SIGKILL left the sidecar
holding pid 1, and the replacement container -- also pid 1 -- refused to
open the database forever (#317).

File::try_lock puts the decision in the kernel, which releases the lock on
process death however it occurs. Because flock and OFD locks attach to the
open file description rather than the process, this also denies a second
open within one process, covering #304 with no bookkeeping.

A leftover sidecar from v1.2.x is ignored, never deleted: a still-running
old process may depend on it.

* fix(storage): field drop order, ProceedUnlocked registry, message repair

Three fixes to the kernel-file-lock work for #317/#304:

1. `_path_guard` is now declared before `file` in `FileBackend`. Rust drops
   struct fields in declaration order, so `file` (and the kernel lock it
   holds) was previously released before the registry entry was removed,
   opening a window for another thread to acquire the lock and insert its
   own entry that this guard's drop would then incorrectly delete.

2. `LockOutcome::ProceedUnlocked` now consults `already_open_here` and
   inserts into `OPEN_PATHS` before returning, matching the `Acquired` arm.
   On an unlockable filesystem there is no kernel lock, so the registry is
   the only thing preventing a second same-process handle to the same path
   from corrupting the file with two independent page tables (#304).

3. Repaired the `ProceedUnlocked` bail! message: its line-continuation
   backslashes had been dropped, leaving literal runs of spaces at each
   line join. Restored the `\`-continuation style used by the neighbouring
   bail! calls so it reads as one clean sentence, textually identical to
   the `LockOutcome::Held` same-process message.

`test_refused_open_does_not_corrupt_path_registry` was rewritten to assert
on the second refusal's error message rather than just its Err-ness.

Not done: an in-process test for "a refused open leaves drop-removal
intact" is not attempted here. That property is only observable with an
independent process holding the kernel lock (Task 5's subprocess
machinery) — inside one process, once the holder drops, try_lock simply
succeeds and the registry is never consulted.

Also not added: a test that reaches `ProceedUnlocked`'s same-process
refusal directly through `FileBackend::open_with`. That requires
`File::try_lock()` to return a real `TryLockError::Error` from an
actually unlockable filesystem (e.g. NFSv3 without lockd) — not
reproducible with a tempdir on a normal CI runner. The `classify` unit
tests already pin that `ProceedUnlocked` is produced correctly from
that raw `Result`; the registry-consulting behavior inside the
`open_with` match arm has no unlockable-filesystem-free way to exercise.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dtz3m9Ww7SUbft1TqRSGXf

* feat(db): add OpenOptions::allow_unlocked

Fails closed by default when a filesystem cannot lock. The escape hatch is
for single-writer-by-convention deployments on mounts without working
locks; it never bypasses a lock another handle holds.

* docs: verify clippy the way CI does, not more strictly

Task 7 called for --all-targets, which lints test binaries carrying
several hundred pre-existing failures on main. CI runs --all-features.
Gating the branch on debt CI ignores would block it for unrelated
reasons.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dtz3m9Ww7SUbft1TqRSGXf

* fix: gate FileBackend::open on tests and rewrite stale comment

- Gate FileBackend::open with #[cfg(test)] since production code now calls
  open_with directly. Add comment explaining this survives for test convenience.
  Fixes clippy dead-code warning that would fail CI.
- Rewrite checkpoint comment to remove stale 'manual lock deletion' example
  from .graph.lock sidecar era. Now accurately describes legitimate use cases:
  filesystems that cannot lock at all (NFSv3 without lockd, FUSE mounts).

* chore: ignore memory.graph FTS sidecar files

.gitignore covered memory.graph but not its FTS index sidecars, so a
broad `git add` swept 72KB of local sqlite binaries into a commit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dtz3m9Ww7SUbft1TqRSGXf

* test: simulate crashes with real subprocesses

mem::forget no longer models a crash: it leaks the File, so the kernel
lock stays held for the life of the test process and no external cleanup
can release it.

A child that aborts is both the accurate model -- the kernel closes its
descriptors exactly as it would for a SIGKILLed container -- and the one
that needs no test-only hook in the shipping crate. #[cfg(test)] cannot
reach tests/, and #[doc(hidden)] pub would be callable in production.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dtz3m9Ww7SUbft1TqRSGXf

* fix(storage): retry WouldBlock with bounded backoff before failing open

Command::spawn forks the whole process before exec'ing the child; until
the child reaches execve and its close-on-exec descriptors close, it
transiently holds a duplicate of every flock its parent has open. A
concurrent try_lock() elsewhere -- another thread in this process, a
sibling process, or bindings that shell out (Python/Node commonly do)
-- can observe WouldBlock during that window even though no other
process is really contending for the file. This surfaced as flaky
"Database is locked by another process" failures on completely
unrelated tests once tests/wal_test.rs started spawning real crash
subprocesses.

SQLite solves the same class of problem with busy_timeout; this is our
version: retry try_lock() up to 10 times with backoff (5ms doubling,
capped at 50ms, ~375ms worst case) before concluding another process
genuinely holds the lock. The retry is skipped entirely when this
process already has the path open -- that conflict is unwinnable by
waiting, since we are the only thing that could release it -- so
test_second_open_in_same_process_is_refused stays instant.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dtz3m9Ww7SUbft1TqRSGXf

* test: restore explicit-transaction crash coverage, add cross-process lock test

Review round 2 fixes:

1. test_explicit_tx_all_or_nothing_commit and
   wal_recover_committed_tx_crash_before_checkpoint had been converted to
   plain db.execute() calls in the crashing child, silently dropping the
   one thing they existed to pin: that WriteTransaction::commit() itself
   writes to the WAL before a crash, and that a multi-statement commit
   recovers atomically (all facts, not a subset). Neither property was
   asserted anywhere else in the tree. Added a CrashTx::{Implicit,Explicit}
   mode to run_crashing_child/crash_child_entrypoint (env var
   MINIGRAF_CRASH_TX) so the child can wrap statements in begin_write()/
   commit() instead of auto-committing each one, and restored both sites
   to CrashTx::Explicit.

2. The retry-on-WouldBlock fix in FileBackend::open_with had no test:
   nothing on the branch spawned a second process to actually hold the
   lock while another tried to open it, so the cross-process error
   message and the retry-budget bound were both unexercised -- and an
   already_open_here that wrongly returned true would have passed the
   whole suite. Added a lock-holding child (hold_lock_entrypoint) and a
   driver test that asserts: the open fails, it fails within one second
   (not by hanging out the full retry budget plus the holder's longer
   sleep), the error is the cross-process wording and not the
   same-process one, and a subsequent open succeeds once the holder
   exits.

Also: corrected the OPEN_PATHS doc comment (it now also gates the retry
decision, not just error wording) and the LOCK_RETRY_ATTEMPTS comment
that claimed a genuinely locked database "fails fast" (it pays the full
~375ms budget). Noted CRASH_CHILD_SPAWN_LOCK is now belt-and-braces now
that the real fix lives in open_with.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dtz3m9Ww7SUbft1TqRSGXf

* test: reproduce #317 across PID namespaces

Two processes at PID 1 in separate namespaces sharing one file, via
unshare -- no container runtime needed. Verified to fail against the
pre-fix code: the killed holder's sidecar left pid 1 behind and the
replacement was refused permanently.

The second test guards the other direction: two live holders must still
be refused, which is what PR #318's start-time comparison would have
broken.

Co-Authored-By: Olive Casazza <olive.casazza@schrodinger.com>

* test: fail loudly on a missing helper, kill-on-drop for container A

Review round 1 fix for #317's regression test.

A missing helper binary used to be a silent skip. Under any filtered
`cargo test` invocation that never triggers a rebuild, that reports a
green, vacuous pass with nothing in the output to say so. Now the skip
is reserved for the case that actually warrants it -- unshare_prefix()
returning None, i.e. this environment cannot create PID namespaces at
all. If namespaces work but the helper binary is absent, that can only
mean `cargo build --examples` was not run, so the test panics with a
message naming the binary instead of asserting nothing.

Also wrap container A's Child in a kill-on-drop guard in both tests.
std::process::Child does not kill its child on Drop, and both tests
hold container A across an assert! before their explicit kill/wait; if
that assert (or the wait_with_output().expect() further down in the
second test) panics first, the panic used to unwind straight past
cleanup and orphan container A -- PID 1 in its own namespace, still
mid-sleep, still holding the flock. The guard makes that unwind-safe
without restructuring either test.

Co-Authored-By: Olive Casazza <olive.casazza@schrodinger.com>

* docs: record the kernel locking change

CHANGELOG carries the semver-relevant facts (new public option, MSRV
1.85 -> 1.89, behaviour change on unlockable filesystems, Windows
mandatory locks, bounded cross-process retry) so the release decision
can be made later.

Adds STG-025 through STG-027 with the actual error text; the lock
errors had no reference entries. Updates docs/TEST_COVERAGE.md,
CLAUDE.md, and README.md to the measured count (1021 tests: 1013
passing, 8 ignored) and reconciles the itemized breakdown -- including
the Wave 3 Conclusion section and the XTDB/Datomic compatibility
section headers, command-reference counts, and bullet lists (now one
accurate bullet per real test, matched in both directions against the
source), which had all drifted stale before this branch -- so no count
in the file contradicts another. Documents that hold_lock_entrypoint's
child process leaks a stray harness summary line into cargo test
output (easy to mistake for a second partial run of the lib unit test
binary), and gives a verified filter to exclude it when totalling.
Fixes a stale "Diagnostic only." comment on already_open_here in
src/storage/backend/file.rs -- its return value now also gates the
WouldBlock retry loop.

* style: apply cargo fmt to test files

cargo fmt --check failed on three test files touched by the kernel
file locking work (migration_matrix_test.rs, pid_namespace_test.rs,
wal_test.rs) — formatting only, no behavior change.

Task 7 verification (docs/superpowers/plans/2026-08-22-kernel-file-locking.md).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dtz3m9Ww7SUbft1TqRSGXf

* fix(storage): make the unlocked-path registry check atomic

Final-review fix wave, four items.

1. TOCTOU in the ProceedUnlocked arm. already_open_here() took OPEN_PATHS,
   read, and released it; the insert then took the lock again. Two threads
   opening the same path on a filesystem that cannot lock both observed
   "absent", both inserted, and both proceeded -- two FileBackends on one
   file with diverging page_count, which is exactly the #304 corruption.
   That arm has no kernel lock behind it, so the registry was the only
   guard, and its own comment said so while not providing it. Now one
   critical section using HashSet::insert's return value as test-and-set.
   A poisoned mutex fails closed.

2. Record the OpenOptions semver break in the CHANGELOG. Adding a public
   field to a struct that is not #[non_exhaustive] breaks struct-literal
   construction downstream. The release decision is deferred, so this
   documents the facts needed to make it: first post-1.0 field addition,
   the v0.19.0 precedent predates the guarantee, and #[non_exhaustive]
   would immunise future additions at the cost of being breaking itself.

3. Remove a flake window. The cross-process lock test held for 600ms
   against a ~375ms retry budget -- ~215ms of slack, across three CI
   platforms with 34 test binaries in flight. Now 3s, and the holder is
   killed rather than waited out, which is both faster than the original
   and tests release-on-SIGKILL instead of release-on-clean-exit.

4. Mark tiers 2 and 3 in the design spec as deferred to #324, and state
   plainly that this leaves the spec's own NFS silent-no-op risk
   unmitigated on merge.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dtz3m9Ww7SUbft1TqRSGXf

* docs: correct two factual errors found in the fix-wave re-review

- The CHANGELOG said this branch updated "three" exhaustive OpenOptions
  literals. It updated two: the Default impl and one test. Only the test
  is a "downstream literal" in the sense the semver paragraph means.
  Verified by walking every OpenOptions literal in the tree.

- Two statements in the design spec still read as though Tier 2 ships
  with this branch: the prose at the NFS discussion, and the risk-register
  row, which a skim-reader may see without ever reaching the delivery
  note. Both now say plainly that the mitigation is deferred to #324 and
  the risk is unmitigated on merge -- and the risk row now notes why
  fail-closed does not cover it: a silent no-op returns Ok(()), so there
  is no error to fail on.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dtz3m9Ww7SUbft1TqRSGXf

* docs: record kernel file locking in ROADMAP and CLAUDE.md status

Both were missed in the Task 6 doc sync. ROADMAP's release history
described v1.2.2's #304 fix with no note that this branch supersedes the
mechanism behind it, and CLAUDE.md's status line — which loads every
session — did not mention the change or the MSRV bump.

Neither states a version: the release decision is deferred, and both
point at the CHANGELOG's breaking-changes section instead.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dtz3m9Ww7SUbft1TqRSGXf

* test: kill the namespaced holder by process group, not by pid

The #317 regression test passed locally and failed on GitHub's ubuntu
runner. Both tests ran (no skip), and the two-live-holders test passed,
so namespaces work there — the difference is how container A gets killed.

Command::kill() signals only the process we spawned. When `unshare` is
that process, `--kill-child` covers the rest: it sets PR_SET_PDEATHSIG on
the process that becomes PID 1. But GitHub's runners restrict unprivileged
user namespaces, so the probe selects the `sudo` fallback, and then a
privilege wrapper sits in between. SIGKILL runs no handler, so the wrapper
forwards nothing, `unshare` and its namespaced child survive, and the lock
is still held when container B tries to open.

That failure is maximally misleading: the #317 test fails identically
whether or not the bug is present, because the lock genuinely is still
held. A regression test that cannot distinguish those two states is worse
than no test.

The holder now leads its own process group and is killed as a group, with
the same privilege escalation the prefix used — root-owned descendants
reject an unprivileged signal with EPERM. The second test uses the same
path: its kill is cleanup only and cannot fail the assertion, but an
orphan there leaves a namespaced process on the flock for 300 seconds,
which on a shared runner breaks somebody else's test.

Both asserts now print the prefix actually selected and container B's
stdout and stderr, so the next failure of this kind names its own cause
instead of requiring a bisect against CI.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dtz3m9Ww7SUbft1TqRSGXf

* docs: credit @ocasazza in the CHANGELOG for #317

Attribution existed in the commit trailers, the design spec, and the PR
body, but not in the CHANGELOG — which is the one that ships to users and
gets read at release time.

They supplied the reproduction, identified the PID-namespace mechanism,
and traced the refusal back to the #304 fix that caused it. That analysis
is why this was quick to verify rather than quick to guess at.

The co-author trailer is repeated here deliberately. This repository
squashes with squash_merge_commit_message=COMMIT_MESSAGES, so the squash
body is every commit message concatenated in order, and git only parses
trailers in the FINAL paragraph. The trailers on a1bfec1 and 8cc1419 land
mid-body and would read as prose. Carrying it on the last commit puts it
where it will actually be parsed.

Co-Authored-By: Olive Casazza <olive.casazza@schrodinger.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dtz3m9Ww7SUbft1TqRSGXf

* test: let the namespaced holder end its own life

CI named the cause once the assertions printed it:

  prefix used: ["sudo", "-n", "unshare", "--pid", "--fork", "--mount-proc",
                "--kill-child"]
  container B stdout: OPEN_ERR Database is locked by another process

GitHub's runners restrict unprivileged user namespaces, so the probe takes
the sudo fallback. Ubuntu's sudo defaults to use_pty, which runs the
command in a NEW SESSION. Signalling the process we spawned reaches only
sudo, and SIGKILL runs no handler so nothing is forwarded; killing the
process group misses the holder too, because it is no longer in that
group. The previous attempt at a group kill failed for exactly this
reason.

Killing across that privilege boundary cannot be made reliable from here,
so stop trying. The holder now aborts itself: no unwinding, no Drop, no
checkpoint, no library-side lock release -- the same state a SIGKILLed
container leaves, which is what #317 is about. Test 1 asks for a zero
hold, so A dies the moment it holds the lock; test 2 asks for a bounded
2s hold, long enough for B to be refused while A lives and short enough
that a failure cannot strand a namespaced process on the flock.

KillOnDrop goes with it. It existed to stop a panic orphaning a holder
mid-way through a 300-second sleep; holders are now self-limiting, so it
guards nothing.

Catching power re-verified after the rework, not assumed: against pre-fix
e056905 the regression test still fails with the #317 message while the
two-live-holders test still passes -- the same discrimination as before.

Co-Authored-By: Olive Casazza <olive.casazza@schrodinger.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dtz3m9Ww7SUbft1TqRSGXf

* test: let the parent decide when the live holder releases

CI moved the failure from test 1 to test 2: the #317 regression test now
passes on the runner, and the two-live-holders test failed with container
B reporting OPEN_OK where it must be refused.

Cause was the fixed 2s hold. Container B has to start through
`sudo unshare` and attempt an open, and how long that takes differs
enormously between a laptop and a loaded runner. A hold that is generous
locally expires mid-attempt on CI, B then legitimately succeeds because
nothing is holding the lock any more, and the test reports that a live
holder was not excluded — which was never what happened.

Any fixed number is a guess at someone else's machine. The holder now
waits for a release marker the test writes once B's result is recorded,
so the hold is exactly as long as it needs to be. The helper keeps a 60s
deadline purely as a backstop, so a panic in the parent cannot strand a
namespaced process on the flock.

Faster too: 0.64s against 2.16s, because nothing waits out a sleep that
was mostly slack.

Catching power re-verified against pre-fix e056905 with a freshly built
helper -- the regression test fails there with the #317 message while the
two-live-holders test passes.

Note for anyone iterating on this: `cargo build
8E23
 --tests` and
`cargo test --test <name>` do NOT rebuild `examples/`, so the helper can
go stale and silently change behaviour under you. Plain `cargo test`,
which is what CI runs, does rebuild it.

Co-Authored-By: Olive Casazza <olive.casazza@schrodinger.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dtz3m9Ww7SUbft1TqRSGXf

* docs: resync test counts after main was merged in

The merge brought #325 and #326, adding two lib unit tests, so every count
this branch had just synced was stale again: 1015 passing, 8 ignored,
1023 total, with 678 in the lib binary.

Updated CLAUDE.md, README.md, and docs/TEST_COVERAGE.md, including the
itemized Test Summary — its 33 bullets sum to exactly 1023, verified
rather than eyeballed.

Co-Authored-By: Olive Casazza <olive.casazza@schrodinger.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dtz3m9Ww7SUbft1TqRSGXf

* test: pass the release marker as an argument, not an env var

sudo defaults to env_reset: everything outside a small allowlist is
stripped before the command runs. GitHub's runners restrict unprivileged
user namespaces, so the test takes the `sudo unshare ...` fallback, and
the holder never saw MINIGRAF_NS_RELEASE_MARKER at all. It took the
"nothing to wait for" path and aborted the instant it held the lock, so
container B found the file free and opened it — and the test reported
that a live holder had not been excluded, when there had been no live
holder to exclude.

This also corrects the previous commit's diagnosis. That one blamed a
fixed 2s hold expiring on a slow runner. The hold was never the problem:
MINIGRAF_NS_HOLD_MILLIS was stripped by the same mechanism, so A aborted
immediately in that version too. The release marker was the right change
for a different reason — it removes the guess about someone else's
machine — but the failure it was meant to fix had another cause entirely.

Arguments cross the sudo boundary; environment variables do not.

Verified locally under `env -i`, which reproduces env_reset exactly: the
holder stays alive with a completely stripped environment and aborts only
once the marker appears. That is the CI condition, reproduced on a machine
with no passwordless sudo.

Catching power re-checked against pre-fix e056905: the regression test
still fails there with the #317 message, the two-live-holders test still
passes.

Co-Authored-By: Olive Casazza <olive.casazza@schrodinger.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dtz3m9Ww7SUbft1TqRSGXf

* fix(test): read the raw header after closing the database

Windows CI failed with os error 33, "another process has locked a portion
of the file", on a read whose "other process" was the test itself.

Windows file locks are mandatory, not advisory, and they exclude every
handle but the one holding them — a second handle in the same process
included. test_manual_checkpoint_deletes_wal opened the .graph file
through a fresh `File` to assert the header reflected the checkpoint,
while the database was still open and holding the lock. On Unix the lock
is advisory and that read succeeds, which is why this only ever failed on
one leg of the matrix.

Moving the read after the existing `drop(db)` fixes it and asserts the
same thing. Two other raw reads in the suite already closed their handle
first, so this was the only site.

Also corrects the CHANGELOG, which claimed only that "other processes"
cannot read an open database on Windows. That understated it: the
exclusion covers your own process's other handles, which is the case a
user is far more likely to hit — inspecting a file they themselves have
open. The wiki carried the same understatement and is corrected in its
own repository.

Co-Authored-By: Olive Casazza <olive.casazza@schrodinger.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dtz3m9Ww7SUbft1TqRSGXf

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Olive Casazza <olive.casazza@schrodinger.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Datalog parser crashes on deeply nested malformed query input

1 participant

0