Repository navigation
Release proved window partitions incrementally (#641 slice 3) - #780
Merged
Merged
Conversation
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
This was referenced Oct 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Compatible built-in windows can now finish and release one partition at a time. A pinned committed source must prove one nonconstant partition key shared by every window, with no pending/staged records in its range. The operator retains one partition plus charged lookahead and bounded output batches, and reuses the existing evaluator for independent window orders, peers, frames and neighbors. Unpartitioned/incompatible windows, aggregate queries, callbacks and
collect-setkeep full-input evaluation. Dictionary entity/ident order proves adjacency only; it cannot omit a semantic Sort or certify Aggregate order.Depends on #778 and #779; this is slice 3 of #641. #641 remains open. #473 was already closed on 2026-10-24, before this work, and was left unchanged. Companion docs: minigraf-docs #82, built on docs PRs #80 and #81.
Transient DICT lookups are evicted at safe internal input and reference-result conversion boundaries; provisional ids survive under the pin. Completed small partitions share a bounded output batch. One growing partition remains linear, and a surrounding Sort/Distinct can still dominate memory. Query row order across physical plans requires final
:order-by. Coalescing can encounter a read/allocation error while gathering a batch, even after completing some of its partitions. Retained lookahead and copied kept-cell entries are charged while both exist; shared payload estimates follow the existing conservative per-entry accounting. Passing output batches follow existing uncharged accounting.The initial release build exceeded the unchanged 1,259,520-byte cap. Two approved equivalent refactors share stable sorting code: legacy transaction ordering, and ascending/descending clause-cost ordering. Both preserve equal-key input order. The final native binary is 1,258,040 bytes. No flags, dependencies, public API, syntax, file/WAL formats or limits change; #777 remains separate.
On the fixed committed-group fixture (10M facts, 16 rows per value partition), the six-process median full value window takes 33.54 → 25.96 s and entity window 33.46 → 27.38 s; limit-10 value takes 28386.3 → 2.212 ms, entity 28347.6 → 2.235 ms. The fixed value/entity window peaks are exactly flat at 52,464/49,464 bytes through 100K, 1M and 10M; IdSpace peaks are flat too. Both fixed limit 10M/100K time ratios and RSS growth satisfy the approved gates. One growing partition stays linear: Window peak = 200*N+32 bytes. At 100K, the growing-value full drain is 247.919 → 266.611 ms (+7.54%) and the entity full drain is 247.649 → 255.748 ms (+3.27%); both are faster at 1M and 10M. The single-partition and unpartitioned controls are retained in BENCHMARKS.
The selective one-row value-window query adds 98.1–112.3 µs through eight optional proof seeks: about 5.1–5.7× the parent (23.4–24.1 → 121–136 µs). Ordinary points range -2.31% to +0.61%, and the worst exact #379 guard is +0.49% (both under +5%). All 252 window, 360 envelope/control, 144 history, 18 meter, and 72 startup records are committed; all 16 Criterion IDs actually execute under the existing nightly
^query/route. The matched libraries, source fingerprints, compiler and fixture/probe hashes are in the build manifest; the benchmark chapter records the build flags.Validation on the bench: 1,972 default, 2,005 test-hooks, 1,182 WASI, and 61 Firefox tests pass; strict native/browser/WASI lint, WASI release, unchanged per-module coverage floors (95.44% lines, 86.50% branches), golden/fault/migration tests and frozen answers pass. The cache-retention mutation makes its targeted test fail. Native release size is 1,258,040 bytes against the unchanged 1,259,520-byte cap. Claude reviewed the approved spec/plan, window code, coalescing/cache changes and both size refactors; Claude marked the final combined code, docs and benchmark evidence READY FOR PR.