8000 docs: document NFSv3 nolock as an unsupported lock deployment (#334) by adityamukho · Pull Request #335 · project-minigraf/minigraf · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
8000
Diff view
8000
18 changes: 11 additions & 7 deletions .github/workflows/nfs-lock.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,12 +40,17 @@ jobs:
sudo exportfs -ra
sudo systemctl restart nfs-kernel-server

- name: Mount the export as NFSv4 and as NFSv3 without lockd
# NFSv3 mounted with `-o nolock` is deliberately not provisioned here.
# #334 found that mode is invisible to Minigraf's lock detection: the
# Linux NFS client serves locks out of its own local, in-kernel lock
# table instead of over NLM, so `try_lock` always reports success and
# there is nothing for this suite to gate on. It is documented as an
# unsupported deployment instead — see docs/ERROR_REFERENCE.md STG-027.
- name: Mount the export as NFSv4
run: |
sudo mkdir -p /mnt/nfs-v4 /mnt/nfs-v3-nolock
sudo mkdir -p /mnt/nfs-v4
sudo mount -t nfs -o vers=4,proto=tcp 127.0.0.1:/srv/nfs/export /mnt/nfs-v4
sudo mount -t nfs -o vers=3,nolock 127.0.0.1:/srv/nfs/export /mnt/nfs-v3-nolock
sudo chown "$(id -u)":"$(id -g)" /mnt/nfs-v4 /mnt/nfs-v3-nolock
sudo chown "$(id -u)":"$(id -g)" /mnt/nfs-v4

- name: Build the PID-namespace helper binary
run: cargo build --examples
Expand All @@ -54,7 +59,6 @@ jobs:
shell: bash
env:
MINIGRAF_NFS_TEST_DIR_V4: /mnt/nfs-v4
MINIGRAF_NFS_TEST_DIR_V3_NOLOCK: /mnt/nfs-v3-nolock
RUST_BACKTRACE: 1
run: |
set -o pipefail
Expand All @@ -73,8 +77,8 @@ jobs:
echo
echo "- Run: $GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID"
echo "- Commit: $GITHUB_SHA"
echo "- Reproduce: needs a loopback NFS export mounted as vers=4 and as"
echo " vers=3,nolock; see the \"nfs\" job in .github/workflows/nfs-lock.yml"
echo "- Reproduce: needs a loopback NFS export mounted as vers=4; see"
echo " the \"nfs\" job in .github/workflows/nfs-lock.yml"
echo
echo "## Test output"
echo
Expand Down
11 changes: 11 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,17 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- **`open()` now fails on filesystems that cannot lock at all** rather than
proceeding unprotected. Set `OpenOptions::allow_unlocked(true)` to accept the
risk on single-writer deployments.
- **This detection does not cover NFSv3 exports mounted with `-o nolock`**
(#334). Linux's NFS client serves `flock`/OFD locks for a `nolock` mount out
of its own local, in-kernel lock table instead of contacting the server, so
`try_lock` reports an ordinary local success and looks identical to a mount
where locking genuinely works — `allow_unlocked` is never consulted because
the code never learns the mount can't really lock. Two separate client hosts
writing the same `nolock` export can each believe they hold the lock with no
cross-host coordination. There is no way to detect this from the lock call's
result, so `nolock` NFSv3 exports are an unsupported deployment for
multi-writer use — the same category as running mixed Minigraf versions
against one file, below. See `docs/ERROR_REFERENCE.md` STG-027.
- **A database genuinely locked by another process now reports its error
later than before** — about 375ms later. `FileBackend::open_with` retries a
cross-process `WouldBlock` up to 10 times with backoff doubling from 5ms and
Expand Down
2 changes: 2 additions & 0 deletions docs/ERROR_REFERENCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -1836,6 +1836,8 @@ See the [file format section in README](../README.md#file-format) for version hi

**Scenario**: A `.graph` file placed on an NFSv3 export whose `lockd` is not running.

**NFSv3 `nolock` is not covered by this error — it is a silent, undetectable gap (#334).** An export explicitly mounted with `-o nolock` behaves differently from plain "no `lockd`": the Linux NFS client serves `flock`/OFD locks out of its own local, in-kernel lock table instead of going over NLM to the server. `try_lock` sees an ordinary local success, `classify` takes the same branch it would on a filesystem where locking genuinely works, and this error is never raised — `allow_unlocked` is never consulted because the code never learns the mount can't really lock. Two separate client hosts writing the same `nolock` export each get a false `Ok(())` from their own kernel with zero cross-host coordination, and can silently corrupt the file. There is no way to detect this from inside `try_lock`'s result, so `nolock` NFSv3 exports are an unsupported deployment for multi-writer use, the same way running mixed Minigraf versions against one file is unsupported (see the kernel-locking CHANGELOG entry) — avoid `nolock` exports rather than relying on this check to catch them.

---

## WAL — Write-Ahead Log Errors
Expand Down
13 changes: 13 additions & 0 deletions src/storage/backend/file.rs
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,19 @@ enum LockOutcome {
Held,
/// The filesystem cannot lock, and the caller did not opt in to running
/// without one.
///
/// This branch is reachable only when the OS itself reports the lock
/// call as unsupported (e.g. some FUSE mounts, or NFSv3 with `lockd`
/// simply not running). It is NOT reachable on an NFSv3 export mounted
/// with the explicit `nolock` option: there, the Linux NFS client
/// serves `flock`/OFD locks out of its own local, in-kernel lock table
/// instead of going over NLM to the server, so `try_lock` sees an
/// ordinary local success and this crate cannot tell that mount apart
/// from one where locking genuinely works. `allow_unlocked` is never
/// consulted in that case, because `classify` never learns the mount
/// can't really coordinate locks across hosts. `nolock` NFSv3 exports
/// are an unsupported deployment for this reason — see #334 and
/// docs/ERROR_REFERENCE.md STG-027.
Unsupported(std::io::Error),
/// The filesystem cannot lock and the caller accepted the risk.
ProceedUnlocked,
Expand Down
63 changes: 47 additions & 16 deletions tests/nfs_lock_test.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,12 @@
//! is provisioned by `.github/workflows/nfs-lock.yml`, not by this file. They
//! skip cleanly when the environment variables they need are unset, so plain
//! `cargo test` never touches them.
//!
//! `MINIGRAF_NFS_TEST_DIR_V3_NOLOCK` is intentionally not set by that
//! workflow (#334): the `nolock` mount option is a documented-unsupported
//! deployment (docs/ERROR_REFERENCE.md STG-027), not a case CI gates on.
//! The three tests gated on it below are kept for local reproduction against
//! a manually mounted `-o nolock` export; nightly CI always skips them.
#![cfg(target_os = "linux")]

use std::path::{Path, PathBuf};
Expand Down Expand Up @@ -169,30 +175,20 @@ fn test_lock_is_exclusive_or_refused_over_nfsv3_without_lockd() {
assert_lock_excludes_second_holder(&mount, "nfsv3-nolock");
}

/// On a mount where locking is unsupported, the default (`allow_unlocked =
/// false`) must fail closed, and `open-unlocked` must proceed. Proves
/// `allow_unlocked` is consulted only where it should be.
/// `open-unlocked` must succeed regardless of what the mount's locking
/// support turns out to be — it is documented as accepting that risk
/// unconditionally, so this holds on the `nolock` mount too.
#[test]
fn test_allow_unlocked_gates_the_unsupported_nfsv3_mount() {
fn test_open_unlocked_mode_succeeds_on_the_unsupported_nfsv3_mount() {
let Some(mount) = nfsv3_nolock_dir() else {
eprintln!("SKIP: MINIGRAF_NFS_TEST_DIR_V3_NOLOCK not set");
return;
};
let helper = helper_binary();
require_helper_built(&helper);

let db = mount.join("allow-unlocked-gate.graph");

let default_open = run_helper(&helper, &db, "open");
let default_out = String::from_utf8_lossy(&default_open.stdout);
assert!(
!default_out.contains("OPEN_OK"),
"default open must fail closed on a mount that cannot lock.\n\
stdout: {default_out}"
);

let db2 = mount.join("allow-unlocked-gate-2.graph");
let unlocked_open = run_helper(&helper, &db2, "open-unlocked");
let db = mount.join("open-unlocked-nolock.graph");
let unlocked_open = run_helper(&helper, &db, "open-unlocked");
let unlocked_out = String::from_utf8_lossy(&unlocked_open.stdout);
assert!(
unlocked_out.contains("OPEN_OK"),
Expand All @@ -201,3 +197,38 @@ fn test_allow_unlocked_gates_the_unsupported_nfsv3_mount() {
String::from_utf8_lossy(&unlocked_open.stderr)
);
}

/// #334: `-o nolock` is a known, permanent gap in lock detection, not a bug
/// to fix here. This test documents the actual (surprising) behavior rather
/// than the behavior a naive reading of `allow_unlocked` would predict.
///
/// A `nolock` mount tells the BD0 Linux NFS client to serve `flock`/OFD locks
/// out of its own local lock table instead of going over NLM to the server.
/// `File::try_lock` sees a normal local success and returns `Ok(())` —
/// indistinguishable, from inside `classify()` in `src/storage/backend/
/// file.rs`, from a mount where locking genuinely works. So the default
/// (`allow_unlocked = false`) open does NOT fail closed here, even though
/// this mount cannot actually coordinate locks across separate client
/// hosts. `allow_unlocked` is never consulted because the code never
/// learns the mount can't really lock; see docs/ERROR_REFERENCE.md
/// STG-027 for the operator-facing writeup of why this mode is
/// unsupported rather than detected and rejected.
#[test]
fn test_default_open_is_not_detected_as_unsupported_on_the_nolock_mount() {
let Some(mount) = nfsv3_nolock_dir() else {
eprintln!("SKIP: MINIGRAF_NFS_TEST_DIR_V3_NOLOCK not set");
return;
};
let helper = helper_binary();
require_helper_built(&helper);

let db = mount.join("default-open-nolock.graph");
let default_open = run_helper(&helper, &db, "open");
let default_out = String::from_utf8_lossy(&default_open.stdout);
assert!(
default_out.contains("OPEN_OK"),
"default open is expected to succeed on a `nolock` mount — this is \
the known, undetectable gap tracked by #334, not a regression.\n\
stdout: {default_out}"
);
}
Loading
0