Repository navigation
feat(ffi): expose OpenOptions over UniFFI — the close-time checkpoint cannot be suppressed from Python #322
Description
Summary
OpenOptions is not reachable from the UniFFI surface, so an embedder cannot
opt out of the checkpoint that Drop for Inner performs on every handle close.
For a workload that opens and closes handles frequently, that turns #315's
O(graph size) checkpoint() into a mandatory per-close cost with no escape
hatch — even when the embedder is already managing compaction itself.
What is reachable today
The exported UniFFI metadata on libminigraf_ffi.so (1.2.3) is exactly four
entry points:
UNIFFI_META_MINIGRAF_FFI_CONSTRUCTOR_MINIGRAFDB_OPEN
UNIFFI_META_MINIGRAF_FFI_CONSTRUCTOR_MINIGRAFDB_OPEN_IN_MEMORY
UNIFFI_META_MINIGRAF_FFI_METHOD_MINIGRAFDB_CHECKPOINT
UNIFFI_META_MINIGRAF_FFI_METHOD_MINIGRAFDB_EXECUTE
and the Python constructor signature is MiniGrafDb.open(path: str). There is
no open_with_options, so wal_checkpoint_threshold, page_cache_size,
max_derived_facts and max_results are all Rust-only.
Why it matters
Drop for Inner (src/db.rs:196) runs a best-effort do_checkpoint on close,
and src/db.rs:203 already documents the intended escape hatch:
// Skip if wal_checkpoint_threshold is usize::MAX — that sentinel suppresses
// all checkpointing (used by benchmarks to keep WAL entries pending).
if self.options.wal_checkpoint_threshold == usize::MAX {
return;
}The sentinel exists and works. It is simply unreachable from any UniFFI
consumer, so only Rust benchmarks can use it.
Measured cost of that close, after exactly one dirty fact (so this is the
graph-size term, not the dirty-bytes term — consistent with #315):
| graph | handle drop |
|---|---|
| 100k facts | 337.5 ms |
| 1M facts | 4241.0 ms |
Downstream case: project-minigraf/temporal_reasoning runs a git-history
ingestion that takes a refcounted lease per commit and drops the handle when the
count reaches zero. Measured over 220 real commits, that is 1.02 handle drops
per commit, 19.1 s of 40.5 s of write time (47.3%), growing 3.20x within the
run — while the embedder's own checkpoint policy accounted for 4.9%. The
embedder already schedules checkpoint() on a duty-cycle budget; it just has no
way to tell minigraf to stop doing it on close.
Proposed
Expose OpenOptions over UniFFI, e.g. MiniGrafDb.open_with_options(path, options) with a record carrying the four existing fields, open() staying as
the default-options shorthand. Same shape as #180 and #181.
The minimal version, if the full record is not wanted, is a single
suppress_close_checkpoint: bool on a new constructor — but exposing the
existing struct seems cleaner than adding a one-off flag.
Notes
- This is not a duplicate of perf: checkpoint() is O(graph size) and flat in dirty bytes — it rebuilds all four covering indexes every call #315. perf: checkpoint() is O(graph size) and flat in dirty bytes — it rebuilds all four covering indexes every call #315 asks for
checkpoint()itself to
stop rebuilding all four covering indexes every call; this asks for the
ability to not call it implicitly. Fixing perf: checkpoint() is O(graph size) and flat in dirty bytes — it rebuilds all four covering indexes every call #315 shrinks the constant here but
does not give the embedder control over cadence; fixing this issue gives
control without changingcheckpoint()at all. They are independent and
either helps. - Version measured: minigraf 1.2.3.
Activity
adityamukho commentedon Oct 7, 2026
Resolved by #465 on
v3(minigraf-python#6 and the other UniFFI bindings):MiniGrafDb.open_with_options(path, OpenOptions(...))/minigraf.open(path, wal_checkpoint_threshold=...)now exposes everyOpenOptionsfield. Settingwal_checkpoint_thresholdto its maximum (2**63-1 over the FFI, which maps to theusize::MAXsentinel) suppresses the close-time checkpoint. Ships in v3.0.0, earlier than this issue's v3.1.0 milestone.adityamukho commentedon Oct 7, 2026
Reopening: my closing comment was wrong. The UniFFI shim (and Node) carry
wal_checkpoint_thresholdas a signed i64, so the largest value a binding can pass is 263-1. Core suppresses the close-time checkpoint only for exactlyusize::MAX(264-1), so the escape hatch is still unreachable from Python/Java/Android/Swift/Node. The C binding (usize JSON) can reach it. A fix follows: the bindings map i64::MAX tousize::MAX, with a test that dropping the handle leaves the WAL in place.fix: let wal_checkpoint_threshold reach the never-checkpoint sentinel (…
fix: let wal_checkpoint_threshold reach the never-checkpoint sentinel (…
fix: let wal_checkpoint_threshold reach the never-checkpoint sentinel (…
fix: let wal_checkpoint_threshold reach the never-checkpoint sentinel (…
fix: let walCheckpointThreshold reach the never-checkpoint sentinel (#6)
fix: let wal_checkpoint_threshold reach the never-checkpoint sentinel (…
Merge pull request #472 from project-minigraf/docs/spec-wal-checkpoin…
adityamukho commentedon Oct 7, 2026
Done, for real this time. On
v3,open_with_optionsexposes everyOpenOptionsfield in every binding (#465), and the never-checkpoint value now reaches core'susize::MAXsentinel:minigraf.open(path, wal_checkpoint_threshold=minigraf.WAL_CHECKPOINT_NEVER)(minigraf-python#7). Java/KotlinMinigraf.WAL_CHECKPOINT_NEVER(java#10), AndroidLong.MAX_VALUE(android#7), SwiftwalCheckpointNever(swift#7), NodeWAL_CHECKPOINT_NEVER(node#6), and the template (binding-template#2).usizeJSON, so"wal_checkpoint_threshold": 18446744073709551615already worked.Ships in v3.0.0.
docs(roadmap): move v3.0.0-milestone features into v3.0.0 scope (#733)