FFFF docs(bench): reconcile #836 exclusions and row-frame costs by adityamukho · Pull Request #846 · project-minigraf/minigraf · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Documentation

- **Benchmark exclusions audit** (#836): [pinned enumeration and per-shape dispositions](docs/benchmarks/836-exclusions/README.md) reconcile 156 unselected native IDs, retain 110 manual row-frame cases, and distinguish linear full-prefix UDF accumulation from bounded replay. No new latency measurements or runtime changes.

- Retire tracker #383 in favor of the v3.0.0 milestone and its release gates. Ordinary/window value-handling consistency is completed by #789.

### Performance
Expand Down
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co

Minigraf is a tiny, portable **bi-temporal graph database with Datalog queries** written in Rust. Designed as the embedded graph memory layer for AI agents, mobile apps, and the browser — built on the SQLite philosophy: embedded, single-file, reliable, with time travel.

See `ROADMAP.md` for planned work and current direction. See `CHANGELOG.md` for completed-release history.
See `ROADMAP.md` for planned work and current direction. See `CHANGELOG.md` for completed-release history. The [#836 benchmark audit](docs/benchmarks/836-exclusions/README.md) records the 156 unselected native IDs and per-shape manual dispositions; it changes no runtime behavior or test counts.

## Core Philosophy - CRITICAL

Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ For example, `(lag ?salary :over (:partition-by ?dept :order-by ?hire-date))` re

The unreleased v3.0.0 branch supports ordinary `(avg ?score)` (#777). It skips nil and nonnumeric values, counts only numbers, and returns a Float average or nil for an existing group without numbers. No matching rows produces no result row. It uses the same exact-total-then-divide arithmetic as window `avg`; a finite total can overflow before division. V2 requires `:over` for `avg`.

The unreleased v3.0.0 branch adds bounded sliding windows such as `(avg ?salary :over (:partition-by ?dept :order-by ?hire-date :rows 2 preceding))` (#183).
The unreleased v3.0.0 branch adds bounded sliding windows such as `(avg ?salary :over (:partition-by ?dept :order-by ?hire-date :rows 2 preceding))` (#183). See the [benchmark coverage and manual-case dispositions](docs/benchmarks/836-exclusions/README.md) for the limits of nightly coverage (#836).

On v3.0.0, an ascending value-only query such as `[:find ?v :where [?e :score ?v] :order-by [?v] :limit 10]` can read directly in index order and stop early when all committed `:score` history has one value kind (integer, float, or Boolean) and its source range has no pending writes. A query whose full order cannot be proved uses the existing sort (#641). Built-in `count`, `count-distinct`, `sum`, and `avg` can release completed groups when the pinned source proves their grouping order (#781). `min`/`max` also qualify when their arguments are proved Integer/Float/Boolean/nil; `sum-distinct` requires Integer/nil. A single Integer/Boolean/Float key (including mixed numeric groups) is supported. A longer effective tuple, including hidden `:with` keys, must reduce to one varying Integer/Boolean slot plus repeated occurrences and proved Integer/Boolean/nil constants. Any requested final Sort is retained for `:with`. Small grouped point queries pay proof startup cost; see the [measurements](docs/benchmarks/781-ordered-aggregate.md). Memory follows the active group and batches; distinct members within one group still grow. A limit can stop after enough emitted groups, but dropped min/max groups do not count and a retained Sort drains the input. Callbacks, `collect-set`, aggregate+window queries and unproved arguments keep full-input evaluation. When every built-in window uses the same proved nonconstant partition key, the engine can also finish and release one partition at a time. Its memory follows the largest partition plus bounded batches; one growing partition still grows. Unpartitioned windows, incompatible keys, aggregate queries, callbacks and `collect-set` keep full-input evaluation. A surrounding sort or distinct operation can still require more memory.

Expand Down
2 changes: 1 addition & 1 deletion ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ Scope — query API:

- Optional `:distinct` result tuples, preserving bag output by default (#493; done on `v3`)
- `lag` and `lead` window functions for the previous/next row within an ordered partition (#182; v3.0.0)
- Bounded sliding row frames, `:rows N preceding`, for window aggregates (#183; v3.0.0, implemented on `v3`; builds on #182)
- Bounded sliding row frames, `:rows N preceding`, for window aggregates (#183; v3.0.0, implemented on `v3`; builds on #182). [#836 audit evidence](docs/benchmarks/836-exclusions/README.md) retains the 110 wider cases as manual/deferred; it does not expand nightly coverage.
- Cursor API: `Minigraf::query` and `PreparedQuery::query` return an owned `Cursor` that delivers rows in batches, with the answer fixed at open (#432; Rust API done on `v3`). The binding cursor objects follow. Behind the cursor, the streaming engine (#473) runs every query and produces rows as they are pulled: patterns, expressions and pseudo-attributes (#544), joins and `:db/valid-at` (#545), `not` and `or` (#546), aggregates and windows (#547), rules (#548), and transaction reads (#550); the executor's query path is gone (#551; done on `v3`). Operators do not spill: memory follows operator state, with an opt-in per-query limit (`QRY-012`, #549).
- Projected UUID lookup batches walk dense DICT ranges (#530; implemented on `v3`), keeping temporary memory bounded to a batch and query-cache charges in projection order.
- Native `:limit N`, `:order-by` and `:offset`, with a top-k `Sort` whose memory follows `N` (#306; done on `v3`, with the 10M-fact acceptance tests moved from #473; spec `docs/superpowers/specs/2026-10-24-limit-order-by-design.md`). #641 targets v3.0.0 in three slices: snapshot-bound order proofs and Sort omission for committed scalar ranges, then one-group count/count-distinct and partition-at-a-time windows. All three slices are implemented in the dependent #641 PRs for `v3`. #781 extends ordered grouping to sum/avg, fixed-scalar min/max, Integer/nil sum-distinct, and effective tuples reducible to one ordered Integer/Boolean slot plus scalar constants/repetitions, including hidden `:with` keys. Every `:with` Sort is retained. Unproved keys/arguments, callbacks, `collect-set`, rules and aggregate+window queries retain full-input evaluation. Compatible built-in windows with one proved nonconstant partition key hold one partition and bounded batches; unpartitioned/incompatible windows, aggregate queries, callbacks and `collect-set` retain full-input evaluation. See `docs/superpowers/specs/2026-10-28-641-input-order-design.md` and `docs/superpowers/specs/2026-10-29-781-ordered-aggregate-design.md`.
Expand Down
6 changes: 6 additions & 0 deletions docs/BENCHMARKS.md
Original file line number Diff line number Diff line change
Expand Up @@ -732,6 +732,12 @@ UDF case remains excluded for quadratic callback cost. An untimed local check
of the shortlist uses `cargo bench --bench minigraf_bench -- --test '<filter>'`
with the exact filter above.

The [#836 disposition report](benchmarks/836-exclusions/README.md) retains the
110 manual cases and reconciles all 156 unselected registrations at a fresh pin.
It records per-shape costs, not new timings. In particular, full-prefix UDF
frames use a running accumulator; only bounded replay has the quadratic
half-frame cost. The one-partition prefix remains manual.

## Performance Envelope (v3.0.0, #394)

**Version**: `v3` branch (format v8), unreleased · **Date**: 2026-10-11 · **Command**: `cargo run --profile envelope --example perf_envelope -- --scales 100k,1m,10m` · Intel Core i7-1065G7 (4 cores / 8 threads, 16 GB RAM), Rust 1.94.0, Linux 6.12.108, on AC power and otherwise idle; the `.graph` file on an SSD with the OS page cache warm.
Expand Down
2 changes: 1 addition & 1 deletion docs/TEST_COVERAGE.md
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,7 @@ execution was not repeated; its 2,066-test inventory was verified.

- **Partition windows** (#641, third slice; `window_order_tests.rs`, `order/tests.rs`, `order_tests.rs`, `engine/tests.rs`): compatible multiple windows with independent orders, peers, frames and neighbors; scalar/mixed-number, entity, Ref and Keyword adjacency with comparator-specific fallbacks; boundaries on and across batch edges; empty/selective ranges; callback/CollectSet/aggregate-query exclusions; bounded early-limit reads and output coalescing; pending/staged/working-reader/prepared/cursor pinning; fixed versus growing partitions; separately tracked Window/IdSpace peaks; internal input and reference-result cache eviction, including a failing retention mutation; lookahead ownership, low budgets, allocator refusal, close/drop/rewind and defensive `INT-032` before and after returned batches. The committed `MemoryBackend` fixture runs on native, WASI and browser targets.

- **Bounded row frames** (`tests/window_functions_test.rs`, `src/query/datalog/aggregate/tests.rs`, #183): `:rows N preceding` parsing and errors, physical rows with tied order keys, partition reset, descending order, width zero and maximum, all five built-in aggregates against a replay oracle with numeric edge cases, UDFs through prepared queries and cursors, grouping with `:with`, distinct output, memory-limit refusal for long min/max strings, and final `:order-by` matching. A `test-hooks` case checks answer bags with clause reordering on and off.
- **Bounded row frames** (`tests/window_functions_test.rs`, `src/query/datalog/aggregate/tests.rs`, #183): `:rows N preceding` parsing and errors, physical rows with tied order keys, partition reset, descending order, width zero and maximum, all five built-in aggregates against a replay oracle with numeric edge cases, UDFs through prepared queries and cursors, grouping with `:with`, distinct output, memory-limit refusal for long min/max strings, and final `:order-by` matching. A `test-hooks` case checks answer bags with clause reordering on and off. The [#836 benchmark audit](benchmarks/836-exclusions/README.md) reconciles registration/selection and documents deferred shapes; enumeration is not a query-correctness run and adds no Rust tests or timing claims.

- **Optional distinct output** (`tests/distinct_test.rs`, #493): bag defaults, whole tuples, section syntax, cross-batch and committed/pending values, chunked write transactions, prepared execute/cursors, joins/rules/disjunction, aggregates and hidden `:with` groups, window peers and partitioned row numbers, temporal filters, typed equality and canonical float/set output, memory limits (including retained set payloads) and cursor cleanup. A meter unit test verifies each cell/key/value estimate includes set elements and their strings. With `test-hooks`, checks both reorder settings and hashed/correlated subplans. An engine unit test checks counters and state release.

Expand Down
5 changes: 4 additions & 1 deletion docs/bench-coverage-v3.md
Original file line number Diff line number Diff line change
Expand Up @@ -362,7 +362,7 @@ shortlist. No manual timing or release sign-off is implied by this close-out.

| ID | Disposition | Remaining scope and reason |
| --- | --- | --- |
| X01 | MANUAL / EXCLUDED registered cases | Of 583 registered native IDs, 156 are unselected: 110 `window_rows/` cases are MANUAL wider width/function/partition sweeps; four history depth-500 IDs and six retract 1k/100k/1m IDs are EXCLUDED redundant intermediate/scaling points; six pending `btree_lookup/` IDs are EXCLUDED duplicates of C01; 30 synthetic SIMD IDs are EXCLUDED experimental non-engine kernels. Exact expansions are in Appendix C. Full-prefix one-partition UDF replay is expensive and stays manual; the unregistered 100k UDF half-frame is also excluded for quadratic callback cost |
| X01 | MANUAL / EXCLUDED registered cases | Of 583 registered native IDs, 156 are unselected: 110 `window_rows/` cases are MANUAL wider width/function/partition sweeps; four history depth-500 IDs and six retract 1k/100k/1m IDs are EXCLUDED redundant intermediate/scaling points; six pending `btree_lookup/` IDs are EXCLUDED duplicates of C01; 30 synthetic SIMD IDs are EXCLUDED experimental non-engine kernels. Exact expansions are in Appendix C. The [#836 pinned reconciliation and per-shape dispositions](benchmarks/836-exclusions/README.md) retain all exclusions. Full-prefix one-partition UDF execution stays manual but uses a linear running accumulator, not quadratic replay; the unregistered 100k UDF half-frames remain excluded for replay cost |
| X02 | ENVELOPE / acceptance, execution unverified | Retained-state/RSS/scaling bounds, nonlinear/multistratum recursion and derived-fact exhaustion, million-fact transactions and custom chunk bounds are not Bencher memory metrics. Observe the revised v3 smoke schedule or dispatch it and inspect exact SHA/test names/bounds before claiming these enforced in nightly CI. Accounting itself has no disabled-engine control; unlimited only disables the finite cap |
| X03 | MANUAL migration | Native migration uses one small frozen v7 fixture, not a large dictionary/scale sweep. Browser migration and migration with pinned-reader churn have no V timing; IndexedDB open/write coverage is not migration coverage |
| X04 | MANUAL binding checks | JVM/JNI, Android ART/ABI/device storage, Swift ARC/Apple storage and WASI need distinct runtime evidence/release sign-off. Options/error paths, foreign FactLog/LogWriter conversion, foreign preparation/registration latency and browser pinned-reader churn remain unmeasured. Python shared UniFFI coverage does not measure other host runtimes; no binding RSS bound is asserted |
Expand Down Expand Up @@ -1008,6 +1008,9 @@ binding/python/aggregate/1k

The enumeration/selected-set difference is exactly **156** IDs. These have no
V analysis or upload. Counts below are not omitted-feature counts.
The [#836 follow-up](benchmarks/836-exclusions/README.md) re-enumerates a later
pinned revision and records an explicit cost/disposition for every manual shape;
it adds no latency measurements and leaves this historical validation unchanged.

| Prefix / expansion | Count | Disposition and reason |
| --- | ---: | --- |
Expand Down
Loading
Loading
0