Repository navigation
perf(query): extend ordered Aggregate beyond one-key count groups #781
Description
Context
#641 delivered three v3 optimizations in core PRs #778, #779 and #780: proved Sort omission, one-group count/count-distinct, and partition-at-a-time built-in windows. Its ordered Aggregate path currently requires exactly one proved scalar grouping key, no :with or windows, and only built-in count/count-distinct. Other grouped aggregates and grouping shapes retain the full-input strategy.
Goal
Extend ordered-group execution to additional ordinary aggregates and grouping shapes only where the pinned input proves complete Value::Ord group order and the existing answers, aggregate data errors, callback behavior, and query-memory accounting can be preserved. This is a performance optimization for top-level plain queries; all current queries must keep working through the existing fallback.
Dependency on #777
Blocked by #777 for completion. #777 must add and specify ordinary avg before this issue can decide or test whether it can use the ordered-group path. Work on existing aggregates can proceed independently, but final acceptance here should cover every ordinary built-in available in v3 at that point. If #777 leaves v3.0.0, remove this dependency and drop avg from the acceptance criteria. This issue does not implement ordinary avg or alter v2 behavior.
Acceptance criteria
- Write a Claude-reviewed spec and plan that classify
sum,sum-distinct,min,max, and (after avg: correct v2 aggregate documentation and add ordinary aggregate support in v3 #777)avgby whether completed groups can be released before end of input. Include empty/nil/nonnumeric values, mixed numeric types, large sums, NaNs and signed zero,:limit/:offset, and deterministic later-error precedence. A normally fallible aggregate may stream only when a pinned-source proof rules out every data-error path for that query; otherwise it retains the full drain, with no early row or Limit stop. On every newly eligible path, upstream I/O,QRY-012andINT-032errors must keep the same precedence relative to any remaining aggregate errors (such assum-distinctallocation failure) as on full drain. - Extend snapshot-bound proofs only for safe grouping keys. Composite keys and
:with(which extends the grouping tuple) qualify only if the complete effective tuple is ascending underValue::Ord, not merely contiguous. Initially reject float components in composite keys, long strings, sets, DICT-ordered entity/keyword keys and unproved joins unless a specific sound semantic-order proof is added. - For newly eligible cases, hold the current group and bounded batches rather than all completed groups. Measure Aggregate-holder and per-query
IdSpacepeaks separately; preserveQRY-012accounting, allocation-failure behavior, cursor pinning, close/rewind, and linear single-growing-group controls forcount-distinctandsum-distinct. - Compare answers and errors with forced full-input evaluation across prepared queries, pending/staged facts, mixed key kinds and every newly eligible aggregate/
:withshape. User callbacks (including registered aggregates),collect-setanywhere in the pipeline or nested bodies, aggregate+window queries and rule-query pipelines retain the existing materialized strategy. Confirm their answers, errors and callback counts remain unchanged. - Run 100K–10M fixed-group, selective-startup and growing-group measurements plus existing perf: compute net-assert on v8 index keys before resolving facts (O(live) point queries) #379/ordinary-query guards on the bench machine, including holder and
IdSpacepeaks against the baseline-plus-two-batches allowance. Keep the release-size cap and portable target checks; add no dependency, configuration, syntax, or file/WAL format change. Synchronize repository and version-scoped docs with the implemented scope.
Related: #383 production-readiness tracker; #641 delivered subset; #777 ordinary avg prerequisite.
Activity
Merge pull request #793 from project-minigraf/perf/781-ordered-aggregate