Repository navigation
docs(551): measurement, acceptance and docs for the executor removal (#551 PR 3) - #727
Merged
Merged
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0169pNQsLcCEn48te9X9WuZW
as_of_ts and as_of_ts_scan query :as-of the time of transaction tx_half. A scale <scale>-irregular measures a LogWriter copy of that fixture whose neighbouring transactions swap tx_ids, so times fall and rise and :as-of a time cannot become a counter bound. cargo test runs the example's test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0169pNQsLcCEn48te9X9WuZW
count_holds_no_rows counts 1M facts under a 4 KiB query_memory_limit, so the meter bounds what an ungrouped count holds at any size. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0169pNQsLcCEn48te9X9WuZW
One Added entry for the streaming engine replaces the per-PR Internal entries, with three Breaking changes for the answers that changed. docs/BENCHMARKS.md records the #551 A/B against 1096e6a, the :as-of time shapes on a rising and an irregular fixture, and the Python round trip; every shape within 5% of the baseline except the point lookups, which #721 slowed and #726 restored to within 3%. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0169pNQsLcCEn48te9X9WuZW
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This was referenced Oct 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part 3 of 3 for #551 (measurement, acceptance and docs). This PR closes nothing on its own: #551 and #473's checklist are closed and ticked by hand after the merge.
Changes
examples/perf_envelope.rs):as_of_tsis theas_ofpoint query with:as-of "<time>".as_of_ts_scanis an:agescan at that time, and its rows are checked.<scale>-irregularfixture: aLogWritercopy whosetx_ids fall and rise. It has a unit test, whichcargo testruns through[[example]] test = true.count_holds_no_rows(ignored; run nightly by smoke, which already runs the file):countover 1M facts finishes under a 4 KiBquery_memory_limit.notbody now fails withQRY-007at open).:limit(No working result-limiting/pagination mechanism for plain (non-recursive) :find queries #306) is the one enhancement: streaming, pull-based operator pipeline with bounded memory behind the cursor API (follow-up to #432) #473 item left, in v3.next.A/B (base
1096e6a2with this PR's envelope shapes, against this branch)Both sides ran in parallel on two sockets of an idle server, swapping sockets for the second round, at 100k, 1m, 1m-irregular and 10m.
:as-ofwith a time is what streaming engine 9/9: remove the old executor, measure, docs (#473) #551 changes: at 10M, 17.2 s and 3.6 GiB → 0.9 ms and 5 MiB; the scan 17.8 → 1.40 s.recursive_ancestors(35.7 s → 1.9 ms at 10M) andjoin_3_selectivegot faster through Recursive rule query 50–10,000× slower since #700: delta run hash-joins a whole attribute (v3) #721/fix(engine): probe joins on a bound value through AVET (#721) #722, which merged after the baseline.Streamsize, fixed by perf(engine): keep every query's stream in the allocator's thread cache (#721 follow-up) #726 down to about 3%. Per-query loop: 14.8–15.1 µs before Recursive rule query 50–10,000× slower since #700: delta run hash-joins a whole attribute (v3) #721, 15.3–15.8 µs after perf(engine): keep every query's stream in the allocator's thread cache (#721 follow-up) #726.not_probeat 100K (+4%; within 1% at 1M and 10M).Python (minigraf-python
v3, built against this branch and against the base):attribute_scanat 1M is 355 → 358 ms at 19 MiB, andcount_groupedis 64.9 → 65.2 ms at 18 MiB.#473 acceptance mapping (plan decision 22)
docs/superpowers/specs/2026-10-11-streaming-engine-design.mdcountin constant memory:count_holds_no_rows. The 10M RSS is the envelope'scountrow (7 MiB).countwith 1M groups, and the memory cap:grouped_count_1m_groupsmax_derived_facts:recursive_acceptance_test::reach_5mcursor::tests::first_batch_then_closeandtests/cursor_test.rs::close_after_first_batchlarge_transaction_acceptance_test::one_million_fact_transact_holds_one_chunkdocs/BENCHMARKS.mdtests/cursor_test.rsis unchanged apart from two earlier changes: since enhancement (v3.0.0): rework the query and transaction engine as a streaming, pull-based operator pipeline with bounded memory #432 it gained tests (streaming engine 1/9: snapshots and page quarantine (#473) #543), and one test moved frommax_resultstomax_derived_facts(max_results limits derived facts, not results: merge into max_derived_facts (v3.0.0) #553), which the alias still accepts.:limitstays open for No working result-limiting/pagination mechanism for plain (non-recursive) :find queries #306.Testing
On the bench machine: the full suite passes, 1,703 passing / 13 ignored plain and 1,719 / 14 with
--features test-hooks. A Codex review of the diff found no issues.Docs site: project-minigraf/minigraf-docs PR to follow, to merge after this one.
🤖 Generated with Claude Code
https://claude.ai/code/session_0169pNQsLcCEn48te9X9WuZW