8000 minigraf/CLAUDE.md at main · project-minigraf/minigraf · GitHub
[go: up one dir, main page]
Skip to content

CLAUDE.md

Latest commit

 

History

History
241 lines (177 loc) · 16.5 KB

CLAUDE.md

File metadata and controls

241 lines (177 loc) · 16.5 KB
Raw

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

Minigraf is a tiny, portable bi-temporal graph database with Datalog queries written in Rust. Designed as the embedded graph memory layer for AI agents, mobile apps, and the browser — built on the SQLite philosophy: embedded, single-file, reliable, with time travel.

See ROADMAP.md for planned work and current direction. See CHANGELOG.md for completed-release history.

Core Philosophy - CRITICAL

Before implementing ANY feature or change, you MUST assess it against the design philosophy in PHILOSOPHY.md.

Minigraf follows the "SQLite for bi-temporal graph databases" philosophy:

  • Zero-configuration - No setup, no config files, just works
  • Embedded-first - Library not server, in-process execution
  • Single-file database - One portable .graph file
  • Self-contained - Minimal dependencies, small binary (~1.2MB budget as of #277; see PHILOSOPHY.md §4)
  • Cross-platform - Native, WASM, mobile, embedded
  • Reliability over features - Do less, do it perfectly
  • Bi-temporal first-class - Time travel is a core feature, not addon
  • Datalog queries - Simpler, more powerful for graphs than SQL/GQL
  • Stability - Backwards compatibility, stable file format
  • Long-term support - Decades-long commitment

Philosophy Compliance Check

CRITICAL INSTRUCTION: When the user requests a feature or change, you MUST:

  1. First, assess whether it aligns with the philosophy in PHILOSOPHY.md
  2. If it violates the philosophy, WARN the user BEFORE implementing:
    • Explain which principle(s) it violates
    • Explain why it's problematic
    • Suggest alternatives that align with the philosophy
    • Ask for explicit confirmation to proceed despite the violation
  3. If it aligns, proceed with implementation

Examples of philosophy violations to warn about:

  • Adding client-server architecture (violates embedded-first)
  • Requiring external services or complex setup (violates zero-configuration)
  • Large dependencies that bloat binary size (violates self-contained)
  • Features only useful for distributed systems (violates target use cases)
  • Breaking changes to API or file format (violates stability)
  • Complex features before basics are reliable (violates reliability-first)
  • Multi-file storage (violates single-file philosophy)

Your response format when detecting a violation:

⚠️ PHILOSOPHY WARNING ⚠️

The requested feature/change may violate Minigraf's core philosophy:

**Violated Principle**: [principle name from PHILOSOPHY.md]

**Why this is problematic**: [explanation]

**Philosophy-aligned alternatives**:
- [alternative 1]
- [alternative 2]

Do you want to proceed anyway? This would be a deviation from the "SQLite for graph databases" philosophy.

See PHILOSOPHY.md for complete design principles and decision framework.

Build and Run Commands

# Build
cargo build
cargo build --release

# Run the REPL
cargo run

# Run tests
cargo test

# Run specific test suite
cargo test --test bitemporal_test -- --nocapture
cargo test --test window_functions_test -- --nocapture
cargo test --test complex_queries_test -- --nocapture
cargo test --test recursive_rules_test -- --nocapture
cargo test --test concurrency_test

# Kani proofs of the value encoding (#627); ~1 GB. Never add decoder/string harnesses
# without a bench-machine run first: they went past 27 GB without finishing
cargo kani --lib -j --output-format terse -Z unstable-options --harness-timeout 10m

# Run examples
cargo run --example embedded
cargo run --example file_storage

# Run demo scripts
cargo run < demos/demo_commands.txt
cargo run < demos/demo_recursive.txt
cargo run < demos/demo_bitemporal.txt
cargo run < demos/demo_negation.txt

Architecture

Module Structure

  1. src/graph/ — EAV fact store with bi-temporal support

    • types.rs: Fact, Value, EntityId, TxId, VALID_TIME_FOREVER
    • storage.rs: FactStorage — in-memory store, transact_batch, retract, get_facts_as_of, get_facts_valid_at, net_asserted_facts, get_facts_by_entity_attribute_indexed

  2. src/storage/ — Persistence layer

    • mod.rs: StorageBackend trait, FileHeader v7 (84 bytes), CommittedFactReader / CommittedIndexReader traits
    • backend/file.rs: Single .graph file backend (4KB pages, cross-platform)
    • backend/memory.rs: In-memory backend for testing
    • backend/fault_inject.rs: FaultInjectingBackend — injects I/O errors for durability tests (test builds only)
    • index.rs: EAVT / AEVT / AVET / VAET key types, FactRef, encode_value; index/verification.rs: Kani proofs of encode_value (#[cfg(kani)], #627; kani.yml runs them on encoder PRs and nightly)
    • btree_v6.rs: On-disk B+tree (build_btree, rebuild_btree_incremental, OnDiskIndexReader, MutexStorageBackend)
    • btree.rs: Legacy v5 B+tree (migration only)
    • cache.rs: LRU page cache (PageCache, default 256 pages)
    • dir_sync.rs: sync_parent_dir — fsyncs the parent directory after creating the .graph/WAL or deleting the WAL (no-op off Unix)
    • packed_pages.rs: Packed fact pages (~25 facts/4KB page), MAX_FACT_BYTES
    • persistent_facts.rs: PersistentFactStorage — v7 save/load, auto-migration v1–v6→v7

  3. src/query/datalog/ — Datalog engine

    • parser.rs: EDN/Datalog parser — transact, retract, query, rule, :as-of, :valid-at, not, not-join
    • parser/verifier.rs: test-only (not wasm32) bounded-exhaustive check of the parser against tests/grammar/grammar.pest (#643): every 1–3 character string (parse_edn) and every balanced token filler up to MINIGRAF_GRAMMAR_BOUND (default 3) in 15 command templates (parse_datalog_command); pest rejects ⇒ parser rejects, pest accepts ⇒ parser accepts or raises a SEMANTIC code (INT-019 counts as semantic on v2.x); today's drift is KNOWN_DRIFT, one entry per cause, removed by its fix (#651; #650 fixed); grammar-verifier.yml runs bound 4 nightly
    • executor.rs: Query executor — temporal filter (tx-time → net-assert → valid-time), not/not-join post-filter
    • matcher.rs: Pattern matching with variable unification; edn_to_value, edn_to_entity_id
    • evaluator.rs: RecursiveEvaluator (semi-naive), StratifiedEvaluator (repeats a stratum whose rules recurse through a not/or rule until it derives nothing new, #633), evaluate_not_join
    • stratification.rs: DependencyGraph, stratify() — negative edges + cycle detection; a positive edge to a rule with not/not-join/or/or-join also raises the caller, off a cycle (#628)
    • rules.rs: RuleRegistry — thread-safe rule management
    • types.rs: EdnValue, Pattern, DatalogQuery, AsOf, ValidAt, WhereClause (incl. Not, NotJoin); PseudoAttr enum, AttributeSpec wrapper
    • optimizer.rs: Selectivity-based join reordering; disabled under wasm feature
    • prepared.rs: BindValue, PreparedQuery — parse-once/execute-many with named $slot bind slots; prepare_query, substitute
    • magic_sets.rs: Magic-sets rewriting for demand-driven recursive rule evaluation (not applied to rules with not/not-join)

  4. src/temporal.rs — UTC-only timestamp parsing (avoids chrono CVE GHSA-wcg3-cvx6-7396)

  5. src/repl.rs — Interactive REPL; TTY-aware (suppresses prompts/banner for piped input)

  6. src/db.rs — Public API: Minigraf::open/execute/prepare/begin_write/checkpoint/save, WriteTransaction, OpenOptions::page_cache_size

  7. src/wal.rs — Fact-level sidecar WAL, CRC32-protected entries, crash recovery

  8. src/error.rs — Structured error codes: MinigrafError, ErrorCategory, ErrorCode registry (PRS/QRY/STG/WAL/API/INT codes); must match docs/ERROR_REFERENCE.md

  9. src/browser/ — Browser WASM backend (browser feature): buffer.rs (BrowserBufferBackend, in-memory pages with dirty tracking), indexeddb.rs (IndexedDB persistence)

Data Model

struct Fact {
    entity: EntityId,  // Uuid
    attribute: String, // e.g. ":person/name"
    value: Value,
    tx_id: TxId,       // Unix ms timestamp
    tx_count: u64,     // Monotonic counter — used by :as-of N
    valid_from: i64,   // Unix ms; defaults to tx_id
    valid_to: i64,     // Unix ms; i64::MAX = forever (VALID_TIME_FOREVER)
    asserted: bool,
}

enum Value { String(String), Integer(i64), Float(f64), Boolean(bool),
             Ref(Uuid), Keyword(String), Null }

Important: tx_count (sequential 1, 2, 3…) is what :as-of N compares against. The REPL displays tx_id (Unix ms). A single (transact [...]) command increments tx_count once regardless of how many facts it contains (transact_batch).

File Format (v7)

Page 0:  Header (84 bytes) — magic "MGRF", version, page/fact counts,
         B+tree root pages (eavt/aevt/avet/vaet), header_checksum, index_checksum, fact_page_count
Page 1+: Packed fact pages (postcard-encoded, ~25 facts/4KB page)
After:   On-disk B+tree index pages (one node per 4KB page)
Sidecar: <db>.wal — CRC32-protected WAL entries; replayed on open; deleted on checkpoint

Auto-migrates v1/v2/v3/v4/v5/v6 → v7 on open/checkpoint.

Test Coverage

1335 tests (1326 passing, 9 ignored; unit + integration + doc). See docs/TEST_COVERAGE.md for the full per-file breakdown.

Testing conventions — see the Testing Conventions section below before writing any tests.

Ordinary aggregates on v2 are count, count-distinct, sum, sum-distinct, min and max; avg requires :over (#777).

Current Maintainer Context

v2.0.11 released (2026-10-26) — expression = / != equates a keyword with a ref to its entity (#744), float-literal rule calls return -0.0 rows in every run (#750), and checkpoint inside a WriteTransaction returns INT-001 instead of deadlocking (#734). v2.0.10 released (2026-10-23) — WAL entries of more than 1,000,000 facts replay (#686), float constants match equal floats only (#706), prepared queries use the database's rule limits (#679); Kani proofs of encode_value backported (#627). v2.0.9 (2026-10-21) — a pseudo-attribute in 5A1D a rule body reads the stored fact (#661), a fact of more than four elements is rejected with PRS-108 (#649), BrowserDb rejections carry their error code (#673); every binding package is consumer-tested before and after publishing. v2.0.8 (2026-10-20) — query data-integrity patch: :db/valid-at in not-join bodies and rule definitions (#615, #617), or branches with outer-variable expressions, or-bound expression variables and keyword/entity joins (#620, #625, #637), rules calling or recursing through a not/or rule (#628, #633). v2.0.7 (2026-10-16) — prepared queries keep a literal :as-of/:valid-at (#603), rule calls with a literal argument return every row (#608), an or sharing no variable returns its cross product (#610). v2.0.6 (2026-10-14) — concurrent-query data-integrity patch: queries see whole transactions (#554), one state per query (#555), no half-written checkpoint reads (#558). v2.0.5 (2026-10-11) — short-file open refused with STG-045 (#506), commit after a torn WAL entry no longer lost, WAL-007 (#513); backported rule-query max_results (#528), all-checkpointed WAL removal (#457) and usize::MAX close-time save (#476); weekly soak test (#392), real coverage gates (#396). v2.0.4 (2026-10-08) — BrowserDb IndexedDB flush returns an error for an unreadable dirty page (#470); golden-file v7 corpus (#391) and SIGKILL committed-data crash test (#384). v2.0.3 (2026-10-06) — data-integrity patch under the support policy: indexed queries with >65,535 uncheckpointed facts (#445), tx counter rewound on reopen next to an already-checkpointed WAL (#447). v2.0.2 (2026-09-27) — last planned v2.x feature/fix release: query fixes (#297, #405, #407, #410), directory fsync (#389), point-query/attribute-scan/checkpoint perf mitigations (#323, #381, #315), CI policy checks (#395), support policy and tiers (#397, #399, #400). v2.0.1 (2026-09-25) fixed index rebuild on open (#370). v2.0.0 introduced kernel file locking (#317, #304), structured runtime error codes (#277), OpenOptions #[non_exhaustive], and an MSRV of Rust 1.89. See CHANGELOG.md for the full rationale and release history.

Current plan (see ROADMAP.md and the milestones):

  • v2.x (main, format v7) now gets only data-integrity and security fixes as patch releases (v2.0.3 is the first); v2.0.2 was the last planned v2.x release.
  • v3.0.0 (format v8 and data integrity, plus :limit #306, query profiler #185, set membership predicates #316 and UniFFI PreparedQuery/UDF registration #181/#180; v3.0.0 milestone) is developed on the long-lived v3 branch. Do not merge main into v3 routinely: main only gets backports of fixes already on v3. Shared CI and workflow changes land on both branches as separate PRs. Just before the v3.0.0 cut, merge main into v3 once (merge commit; source conflicts resolved to v3, main's v2.0.x CHANGELOG sections brought over) so the cut is a fast-forward.
  • v3.next (the release after v3.0.0) holds new features (lag/lead #182, sliding row frames #183, rule persistence #241).
  • Known v2.x data issue: same-transaction multi-valued facts (#371). Fix ships in v3.0.0. All v2.x known issues are listed in pinned issue #421 (known-issue label); data-integrity/corruption/durability issues stay open until the fix is in a published release (CONTRIBUTING.md).
  • Support policy and tiers: PHILOSOPHY.md §7 and §10. v2.x gets data-integrity and security fixes for 12 months after v3.0.0. Tier 1 = Rust + Python; other bindings are Tier 2.

Testing Conventions

Never use {:?} debug format of Result, Fact, Value, EdnValue, or any type that may transitively contain Uuid in assert!/assert_eq! message strings.

CodeQL flags this as rust/cleartext-logging. It is a false positive in tests, but it pollutes the security scan and blocks CI.

// BAD — triggers CodeQL:
assert!(result.is_ok(), "parse failed: {:?}", result);

// GOOD — plain string message:
assert!(result.is_ok(), "parse failed");

// GOOD — use unwrap/expect instead:
result.unwrap();
result.expect("parse failed");

// GOOD — assert on count/bool only:
assert_eq!(results.len(), 3, "expected 3 results");

Applies to all #[cfg(test)] modules and all tests/*.rs files.

Important Reminders

  1. Always use git worktrees for new features/bugfixes — Never make changes directly on main. Create an isolated worktree (in .worktrees/ directory) using the using-git-worktrees skill before implementing any feature or fixing any issue.
  2. Datalog is the query language — no other query language
  3. Bi-temporal is first-class — not an afterthought
  4. Single file is sacred — never break this
  5. Simplicity over features — do less, do it perfectly
  6. Test everything — no untested code
  7. Think SQLite — would SQLite do this?
  8. Long-term vision — building for decades
  9. Keep documentation synchronized — when a release or planned-work item changes, update and cross-check ALL of: CLAUDE.md (status line, test count), ROADMAP.md, README.md, docs/TEST_COVERAGE.md, CHANGELOG.md. No doc should contradict another. Also update the docs site, project-minigraf/minigraf-docs (clone at ../minigraf-docs): content/architecture.md (module/format/model changes), content/datalog-reference.md (new syntax), content/comparison.md (feature matrix), content/use-cases.md (deployment targets). It documents every release at once: wrap text that applies only from or until a version in <!-- @since vX.Y.Z --> / <!-- @until vX.Y.Z --> … <!-- @end --> rather than rewriting shared text, and add each release to its site.toml (CHANGELOG.md and docs/ERROR_REFERENCE.md are read from this repo per tag, never copied). Open a PR there; its CI checks every link in every version and deploys main. The GitHub wiki is retired: its pages only link to the site.
  10. Tag every version bump — after the final doc-sync commit: git tag -a v<x.y.z> -m "<release> — <summary>" then git push origin v<x.y.z>.

When in doubt, refer to PHILOSOPHY.md and ROADMAP.md. The goal is not to be the most feature-complete graph database. The goal is to be the one that's always there when you need it, works reliably, and never gets in your way.

Be boring. Be reliable. Be Minigraf.
0