8000 minigraf/ROADMAP.md at main · project-minigraf/minigraf · GitHub
[go: up one dir, main page]

Skip to content

Latest commit

 

History

History
287 lines (183 loc) · 13.9 KB

ROADMAP.md

File metadata and controls

287 lines (183 loc) · 13.9 KB

Minigraf Roadmap

Planned work for Minigraf, a bi-temporal Datalog database. v2.x has known data-integrity issues (pinned list); v3.0.0 fixes them (v3.0.0 milestone).

Philosophy: Embedded graph memory for agents, mobile, and the browser — built on the SQLite approach: be boring, be reliable, be embeddable.

Completed releases and their implementation details live in the CHANGELOG. This document tracks planned, exploratory, and other future work.


v2.0.11 — Query and Write-Transaction Patch (released 2026-10-26)

File format stays v7 and the public API does not change. Details are in the CHANGELOG.

Shipped:

  • An expression's = and != equate a keyword with a ref to the entity it names, as patterns and joins do (#744)
  • A rule call with 0.0 returns -0.0 rows in every run; equal zero floats hash alike (#750)
  • checkpoint inside a WriteTransaction returns INT-001 instead of deadlocking; prepare and PreparedQuery::execute enforce the same guard (#734)

v2.0.10 — WAL and Float-Constant Data-Integrity Patch (released 2026-10-23)

File format stays v7 and the public API does not change. Details are in the CHANGELOG.

Shipped:

  • WAL entries of more than 1,000,000 facts replay, so the transaction and every later one survive a crash or close without a checkpoint (#686)
  • A float constant in a pattern matches equal floats only, not every float within f64::EPSILON (#706)
  • Prepared queries use the database's max_derived_facts and max_results (#679)
  • Kani proofs of the value encoding, backported from v3.0.0 (#627)

v2.0.9 — Rule and Parser Data-Integrity Patch (released 2026-10-21)

File format stays v7 and the public API does not change. Details are in the CHANGELOG.

Shipped:

  • A pseudo-attribute in a rule body reads the stored fact's valid time and tx metadata (#661)
  • A fact of more than four elements is rejected with PRS-108 instead of being stored truncated (#649)
  • BrowserDb rejections carry their error code (#673)
  • Every binding package is installed and used before it is published and again after; the check fixed packaging bugs in the Node, Android, Swift, C, Java and WASM 2.0.8 packages

v2.0.8 — Query Data-Integrity Patch (released 2026-10-20)

File format stays v7 and the public API does not change. Details are in the CHANGELOG.

Shipped:

  • A :db/valid-at pattern in a not-join body (#615) or a rule definition (#617) binds the query's valid time
  • An or / or-join branch whose expression reads an outer variable matches (#620)
  • An expression reading a variable only an or binds keeps its rows (#625)
  • An or / or-join branch meets a keyword value as the entity it names (#637)
  • A rule calling a rule with not, not-join, or or or-join gets all its rows (#628), and recursion through such a rule returns all its rows (#633)

v2.0.7 — Query Patch (released 2026-10-16)

File format stays v7 and the public API does not change. Details are in the CHANGELOG.

Shipped:

  • A prepared query keeps a literal :as-of, :valid-at or :any-valid-time (#603)
  • Rule queries with a literal argument return every row (#608)
  • An or that shares no variable with the earlier clauses returns its cross product (#610)

v2.0.6 — Concurrent-Query Patch (released 2026-10-14)

File format stays v7 and the public API does not change. Details are in the CHANGELOG.

Shipped:

  • A query on another thread sees each committed transaction whole (#554)
  • All patterns of one query read the same state of the database (#555)
  • A query during a checkpoint no longer misses committed facts or sees part of a transaction (#558)

v2.0.5 — Data-Integrity Patch (released 2026-10-11)

File format stays v7. Two new error codes (STG-045, WAL-007); no other API changes. Details are in the CHANGELOG.

Shipped:

  • A file shorter than one page is refused with STG-045 unless it is a torn first write (#506)
  • A commit made after a torn WAL entry is no longer lost on recovery; a failed append makes later writes WAL-007 until reopen or checkpoint (#513)
  • Backported from v3.0.0: rule queries count only derived facts against max_results (#528), checkpoint() removes an all-checkpointed WAL (#457), usize::MAX checkpoint threshold skips the close-time save (#476)
  • Weekly 10M-fact soak test (#392) and per-file coverage gates that actually gate (#396)

v2.0.4 — Browser Durability Patch (released 2026-10-08)

File format stays v7 and the public API does not change. Details are in the CHANGELOG.

Shipped:

  • BrowserDb returns an error instead of leaving an unreadable dirty page out of the IndexedDB flush (#470)
  • Golden-file compatibility corpus for format v7 (#391) and a SIGKILL crash test that checks the committed data (#384)

v2.0.3 — Data-Integrity Patch (released 2026-10-06)

First v2.x patch under the support policy. File format stays v7 and the public API does not change. Details are in the CHANGELOG.

Shipped:

  • Indexed queries return correct results with more than 65,535 uncheckpointed facts (#445)
  • The transaction counter no longer rewinds when reopening next to an already-checkpointed WAL (#447)

Further v2.x releases are data-integrity and security patches only.


v2.0.2 — Final Planned v2.x Release (released 2026-09-27)

The v2.x line (main) ends with a patch. File format stays v7 and the public API does not change. All new feature work has moved to v3.x. Work list: v3.0.0 milestone. Details are in the CHANGELOG.

Shipped:

  • Bound-entity point queries narrowed to the queried attribute (#323 mitigation), exact attribute-scan bounds (#381), and checkpoints that copy untouched index leaves (#315 mitigation)
  • fsync the parent directory after creating the .graph/WAL file and after deleting the WAL (#389)
  • or-join returns an empty result, not INT-031, when the clauses before it match no rows (#405)
  • A clear user error for $slot queries run through execute(), not internal INT-025 (#407)
  • Recursive rules with a literal start that recurse through another rule no longer fail under magic sets (#297)
  • The INT-054 negative-cycle error names its predicates in a stable order (#410)
  • Benchmark regression alerting that is not silently muted (#393)
  • CI: MSRV build, public-API semver check, dependency policy (#395)
  • Stability and support policy, including how long v2.x gets data-integrity fixes after v3.0.0 (#397)
  • Known-issues process: data-integrity issues stay open until the fix is in a published release (#399)
  • Support tiers for language bindings and platforms (#400)

Known issue on all v2.x releases: same-transaction multi-valued facts read back as one value (#371, #287). The fix needs format v8 and ships in v3.0.0. Workaround: write or retract each value of a multi-valued attribute in its own transact/retract call. All v2.x known issues are listed in the pinned issue #421.

After v3.0.0 ships, v2.x gets data-integrity and security fixes for 12 months (support policy).


v3.0.0 — File Format v8 and Data Integrity

v3.0.0 supports file format v8 only. Fixing #287/#371 (EavtKey/AevtKey carry no value bytes, so same-transaction multi-values collapse) bumps the format from v7 to v8. Support for v1–v6 is dropped at the same time. The v1–v6 → v7 auto-migration code in persistent_facts.rs can be removed when cutting this release; a v7→v8 migration replaces it. Any database opened at least once under a v1.x release will already be on v7; there are no known users on older formats. Every other layout change this release needs goes into v8 too, so there is no separate v9.

This was the GitHub milestone named “2.0” before v2.0.0 used that version number for the kernel-locking and structured-error-code breaking changes; it was consequently renumbered to v3.0.0. It is developed on the long-lived v3 branch and merges into main at the cut.

Scope — format and storage integrity:

  • v8 index keys with value bytes and v7→v8 migration (#371, #287; done on v3)
  • Per-page CRC32 checksums on fact pages and B+tree pages (#388)
  • Crash-atomic save(), with checkpoints proportional to the change (#374)
  • Index verify and public rebuild_indexes() (#373)
  • Net-assert on v8 keys before resolving facts, for O(live) point queries (#379)
  • btree_page fuzz target that reaches node decoding (#375)

Scope — query and bindings:

  • Native :limit N / :offset for plain :find queries (#306, #310)
  • Query profiler (#185)
  • Set membership predicates (#316)
  • PreparedQuery over UniFFI (#181)
  • UDF registration over UniFFI (#180)
  • OpenOptions exposed over UniFFI, so embedders can suppress the close-time checkpoint (#322; done on v3)

Scope — release-gate testing and documentation:

  • Golden-file compatibility corpus for every format version, frozen before v8 work goes further (#391)
  • SIGKILL crash test that checks committed data through EAVT, AEVT and full scan (#384; also on v2.x, without the #371/#435 cases)
  • Model-based test of transact/retract/checkpoint/reopen (#385)
  • Wider property tests: file-backed, batched, retracts, temporal, joins (#386)
  • Persistent fuzz corpus and an operation-sequence target (#387)
  • Fault injection across save/WAL/recovery (#390)
  • Long-running soak test at scale (#392; tests/soak_test.rs, 10M facts and a 5-hour churn weekly, alternating main and v3)
  • Benchmarks at 100K–10M facts and a published performance envelope (#394)
  • Higher storage/WAL coverage gates and branch-coverage gates (#396; line and branch floors per file in coverage-thresholds.toml, checked on every PR)
  • Durability, recovery and operations guide (#398)

v3.next — Features

Additive features, kept out of v3.0.0 so the #371 fix is not held up by them:

  • Rule persistence — design first (#241)
  • lag/lead window functions (#182)
  • Sliding row frames — :rows N preceding (#183; builds on #182)
  • Temporal graph-traversal research (#273; may result in documentation or a blog post rather than code)

Planned Work: Ecosystem & Tooling

Goal: Improve developer experience and grow the ecosystem without expanding the core unnecessarily.

Developer Tools

Integration Examples

Tracked in: minigraf-examples.

  • 🎯 GraphRAG example that combines Minigraf with a vector store
  • 🎯 LangChain / LangChain.js integration example
  • 🎯 LlamaIndex integration example
  • 🎯 Annotated end-to-end scenarios for agent memory, offline-first mobile, and audit logs

Ecosystem Libraries

  • 🎯 Graph algorithms as a separate crate
  • 🎯 Optional schema validation
  • 🎯 Import/export tools
  • 🎯 Backup utilities

Database Branching / Forking (Exploratory)

Allow a Minigraf database to be forked into an independent .graph file pre-populated with facts from the parent at a given transaction count.

Potential uses include speculative writes, snapshot distribution, test isolation, and agent sandboxing. The design must preserve the single-file, zero-configuration model; a fork should behave as an independent file copy with a fully flushed WAL.

Status: Exploratory. Pursue only with demonstrated user demand.

Magic Sets with Stratified Negation (Exploratory)

Magic-sets rewriting is not applied to mixed rules containing not/not-join; those use full semi-naive evaluation. Extending it to negation is well studied but requires care to avoid unsound propagation across stratification boundaries.

Pursue only if profiling demonstrates a real bottleneck in negation-heavy recursive workloads.


Performance Direction

Future performance work will be driven by production profiles and reproducible benchmarks. The benchmark documentation records measurements; completed performance work is recorded in the CHANGELOG.


Decision Framework

When evaluating features, ask:

  1. Does it align with the philosophy? (embedded, reliable, simple, bi-temporal)
  2. Is it needed for target use cases? (audit, event sourcing, knowledge graphs)
  3. Does it compromise reliability? (stability over features)
  4. Can it be a separate crate? (keep the core small)

Say no to:

  • Distributed consensus
  • Multi-datacenter replication
  • Built-in ML/AI
  • Features useful only at massive scale
  • Complex configuration
  • Breaking the single-file philosophy

Say yes to:

  • Crash safety
  • Data integrity
  • Temporal queries
  • Query performance
  • Developer experience
  • Cross-platform support

Current Focus

  • v2.x maintenance on main: data-integrity and security fixes only
  • v3.0.0 (format v8 and data integrity) on the v3 branch, then v3.next features
  • Production readiness for high-stakes deployments, tracked in the v3.0.0 milestone
  • Ecosystem work tracked in minigraf-examples
  • Developer tools tracked in minigraf-inspector and minigraf-visualizer

See GitHub Issues for specific tasks, and the CHANGELOG for completed releases.


Last updated: October 2026 — v2.0.11, a query and write-transaction patch on v2.x, is the current release; v3.0.0 work, including the file-format v8 fix for #287/#371, is developed on the long-lived v3 branch and merges into main at the v3.0.0 cut.

0