Repository navigation
Bench: qualify the #841 shape-sweep nightly candidates #863
Description
Follow-up of #841. The manual shape sweeps (report, merged in #860) proposed six small representatives. Each guards one behavior the sweep found, at a cost that looks compatible with a nightly route. None was added to nightly CI by #841, and none is qualified yet.
Candidates
| Candidate | Behavior it guards | Bench-host warmed median |
|---|---|---|
| 100k facts, 90%-skewed running sum, committed | Partition-at-a-time window retention follows the largest partition | 718.680 ms |
10k entities, K=256 skewed set membership, committed |
Set membership over large value-page sets | 1,124.625 ms |
| 100k facts, descending half offset + 1,000, committed | Sort heap of offset + limit rows, paired with ascending input-order Sort omission |
62.810 ms |
1,110-node tree ancestors, k=1, pending |
Small in-memory ancestor alternative (5.2× slower without it) | 0.205 ms |
1,110-node tree ancestors, k=16, committed |
Bound-value AVET probes (16× slower with attribute scans) | 54.969 ms |
| 10k facts, one held cursor across the churn schedule | Quarantine of freed pages, release and reuse | 0.86–0.94 s of checkpoints per run |
Work
For each candidate:
- Qualify it on CI hardware. Measure fixture construction, warm-up, process start-up and run-to-run variance on the nightly runner. The figures above come from an idle bench host and don't transfer.
- Pin more than time. Assert the answer (row count and checksum) and the relevant path counters (e.g.
bound_value_avet_range,ancestor_alternative_selected,entity_probe_cache_hit), plus charged query memory where that is the behavior guarded (window retention,Sortheap). A path regression must fail distinctly from a slower runner. - Choose the route. Decide between Criterion and the envelope profile (
profile.envelope, opt 3; the bench profile is opt-level z). Follow the alternating main/v3 schedule and per-suite alerting. - Reject candidates that fail. Record why for any candidate that doesn't qualify (cost, variance, or a fixture too large for the runner).
Keep conclusions distinct from #841: a nightly route detects regressions in one representative, and it doesn't re-establish the sweep's envelope.
Acceptance criteria
- Each candidate is qualified with a recorded cost and variance on the nightly runner, or rejected with a reason.
- Qualified candidates run nightly, asserting answers, path counters and (where relevant) charged memory as well as time.
-
docs/bench-coverage-v3.mdlists the new routes and marks the Bench: large-scale shape sweeps and recursive optimization isolation #841 proposals as resolved.
Activity