Python binding for Minigraf — zero-config, single-file, embedded bi-temporal graph database with Datalog queries.
pip install minigrafSupports Python 3.9+ on Linux (x86_64, aarch64), macOS (universal2), and Windows (x86_64).
import json
from minigraf import MiniGrafDb
# In-memory database
db = MiniGrafDb.open_in_memory()
db.execute('(transact [[:alice :name "Alice"] [:alice :age 30]])')
result = json.loads(db.execute("(query [:find ?n ?a :where [?e :name ?n] [?e :age ?a]])"))
print(result["results"]) # [["Alice", 30]]
# File-backed (persisted to disk)
db = MiniGrafDb.open("path/to/mydb.graph")
db.execute('(transact [[:bob :name "Bob"]])')
db.checkpoint()minigraf.open() takes the Rust OpenOptions as keyword arguments. An option left out
keeps its default.
import minigraf
# Read-only: shared lock, nothing written. Any number of read-only handles can share
# the file; writes raise MiniGrafError with API-014, a missing file STG-042.
src = minigraf.open("app.graph", read_only=True, page_cache_size=4096)
# Every option: read_only, page_cache_size, allow_unlocked, wal_checkpoint_threshold,
# max_derived_facts, max_results, synchronous (minigraf.SyncMode.FULL / NORMAL),
# query_memory_limit.
db = minigraf.MiniGrafDb.open_with_options("app.graph", minigraf.OpenOptions(max_derived_facts=10_000))query_memory_limit is the most bytes one query may hold, counted per entry by the
operators that keep state (join build sides, negation and or tables, distinct sets,
aggregate groups, window rows, rule-derived facts). A query that would hold more raises
MiniGrafError with QRY-012 and frees what it held. The default, and 2**63 - 1, is no
limit. Rows a cursor returns are not counted, so read a large answer with db.query().
wal_checkpoint_threshold=minigraf.WAL_CHECKPOINT_NEVER turns off automatic checkpoints
and the checkpoint when the handle closes, for applications that call checkpoint() on
their own schedule.
db.query() returns a cursor whose answer is fixed when it opens. Iterating gives
rows, decoded as in execute(); next_batch(n) gives a JSON string of up to n rows,
or None at the end.
with db.query("(query [:find ?n :where [?e :name ?n]])") as cursor:
print(cursor.vars()) # ["?n"]
for row in cursor:
print(row)db.prepare() parses a query once; execute() and query() run it with its $slots
bound, from keyword arguments or a dict (names without the $). Each run reads the
database as it is then. A prepared query keeps its database open until both are freed.
pq = db.prepare("(query [:find ?n :where [?e :age $age] [?e :name ?n]])")
print(json.loads(pq.execute(age=30))["results"])
print(json.loads(pq.execute({"age": 40}))["results"])
with pq.query(age=30) as cursor:
for row in cursor:
print(row)bool, int, float, str and None bind as values, and a uuid.UUID as an entity
(it also fits a value slot as a ref). For the other kinds use minigraf.Value.KEYWORD(":k"),
minigraf.Value.REF(uuid_str), minigraf.Entity(uuid), minigraf.TxCount(n) or
minigraf.Timestamp(ms) for :as-of, and minigraf.ANY_VALID_TIME for :valid-at. Any
other Python type raises TypeError. A missing slot raises MiniGrafError.Api with
[API-022], a slot in an attribute position [API-023], and a value of the wrong kind
for its slot [API-024].
A set or frozenset binds a set value, for in? and the other set predicates or a
set-valued attribute: pq.execute(ids={1, 2, 3}). Its elements are bool, int, float,
str, None or uuid.UUID (a ref). minigraf.Value is not hashable, so a set with keyword
elements is written minigraf.Value.SET([minigraf.Value.KEYWORD(":role/admin")]). A set
inside a set raises MiniGrafError.Api with [API-021]; a list is not a set. Set values
come back as JSON arrays from execute() and as Value.SET([...]) in fact records.
A predicate subclasses minigraf.Predicate and is called in Datalog as [(name ?v)].
An aggregate subclasses minigraf.Aggregate; its accumulator() returns a new
minigraf.Accumulator (step(value), finish(count)) for each group or window
partition, and it is used as (name ?v) in :find or with :over. Values arrive as
minigraf.Value (Value.INT64, Value.TEXT, …, with the value in .value), and
finish returns one.
from minigraf import Accumulator, Aggregate, Predicate, UdfError, Value
class Adult(Predicate):
def test(self, value):
if not isinstance(value, Value.INT64):
raise UdfError.Failed("age is not an integer")
return value.value >= 18
class Product(Aggregate):
def accumulator(self):
return ProductAcc()
class ProductAcc(Accumulator):
def __init__(self):
self.product = 1
def step(self, value):
self.product *= value.value
def finish(self, count): # count: values stepped
return Value.INT64(self.product)
db.register_predicate("adult?", Adult())
db.register_aggregate("product", Product())
db.execute("(query [:find ?n :where [?e :age ?a] [(adult? ?a)] [?e :name ?n]])")
db.execute("(query [:find ?g (product ?a) :where [?e :group ?g] [?e :age ?a]])")
db.execute("(query [:find ?a (product ?a :over (:order-by ?a)) :where [?e :age ?a]])")- Failures. A callback that raises stops the query with
QRY-014, raised asMiniGrafError.Query; it is never read asFalseor a missing value. RaiseUdfError.Failed(msg)to choose the message. Any other exception also fails the call, with itsrepr(such asValueError('bad')) as the message. AValue.REFfromfinishwhose value is not a UUID is a failing call too. - Lifecycle. The database keeps a strong reference to each registered object until
the database and every cursor and prepared query opened from it are freed. There is
no unregister. Predicates and aggregates have separate names, and registering a name
twice in either is an error (
INT-040). A query sees the functions registered when it opened. - Threading. A callback runs on the thread that calls
execute(of the database or a prepared query) or a cursor'snext_batch, including a prepared query's cursor. The binding releases the GIL while the core runs and the callback re-acquires it, so queries on different threads may call the same object at once; it must be thread-safe. One accumulator is used by one query on one thread at a time. - Cost. Each call crosses the FFI once: one Python call per value.
- Re-entry. A callback may query the same database, open a cursor or register another
function. Inside a write transaction, the calls the core refuses there still return
INT-001.
db.fact_log(filter) streams every fact version (assertions and retractions, with
tx_count, tx_id and valid-time bounds) as FactRecords. LogWriter builds a new
file from records, keeping those bounds: the load step of an offline migration.
from minigraf import FactFilter, LogWriter, OpenOptions
src = minigraf.open("old.graph", read_only=True)
with src.fact_log(FactFilter()) as log, LogWriter.create("new.graph", OpenOptions()) as out:
batch = []
for rec in log:
if not rec.attribute.startswith(":secret/"):
batch.append(rec)
if len(batch) == 10_000:
out.append_batch(batch)
batch = []
out.append_batch(batch)
out.advance_tx_count(src.current_tx_count())
# Leaving the `with` block finishes the file; an exception abandons it and leaves no file.FactFilter takes attributes, attribute_prefixes, entities (UUID strings),
tx_from/tx_to (inclusive), order (FactOrder.TX or FactOrder.STORAGE) and
window. A record's value is a minigraf.Value (TEXT, INT64, FLOAT64, BOOL,
REF, KEYWORD, NULL), so a ref and a string stay different. A valid_to of
minigraf.VALID_TIME_FOREVER means forever.
Every error is a MiniGrafError whose msg starts with its code, such as
[API-015]. Its class follows the code's category: MiniGrafError.Parse, .Query,
.Storage, .Api (a bad argument, a closed object, a bind mistake) or .Other. See the error reference.
append_batch stops at the first rejected record and ends the message with
(batch index N); the records before it stay appended.
bench/envelope.py measures the FFI round trip for minigraf's performance envelope
(minigraf#394): it runs the core harness's query shapes through db.query(...) and
next_batch(1000) and prints wall-time p50/p99 and peak RSS next to the Rust numbers.
Results are published in minigraf's docs/BENCHMARKS.md.
# In minigraf: build the fixtures and the report.
cargo run --profile envelope --example perf_envelope -- --scales 1m --json envelope.json
# Here, with the binding installed (maturin develop --release):
python bench/envelope.py ../minigraf/target/perf-envelope/fixture-v1-1m.graph \
--report ../minigraf/envelope.json --scale 1mRequires Rust stable toolchain and maturin.
pip install maturin
maturin developThis repo receives a core-release repository_dispatch from the minigraf monorepo
cascade whenever a new version of the minigraf core crate is published. The release
workflow pins the new version, commits, tags, builds wheels for all platforms, and
publishes to PyPI.
MIT OR Apache-2.0