Repository navigation
Bench: wider row-frame sweeps and registered-case exclusions #836
Description
Follow-up of #776: X01 in the remaining exclusions and follow-ups table, finalized in #834.
Disposition: MANUAL / EXCLUDED registered cases.
Milestone: v3.next — These are deliberate exclusions and wider future sweeps beyond the release's representative nightly coverage.
Remaining scope recorded by the audit:
Of 583 registered native IDs, 156 are unselected: 110 window_rows/ cases are MANUAL wider width/function/partition sweeps; four history depth-500 IDs and six retract 1k/100k/1m IDs are EXCLUDED redundant intermediate/scaling points; six pending btree_lookup/ IDs are EXCLUDED duplicates of C01; 30 synthetic SIMD IDs are EXCLUDED experimental non-engine kernels. Exact expansions are in Appendix C. Full-prefix one-partition UDF replay is expensive and stays manual; the unregistered 100k UDF half-frame is also excluded for quadratic callback cost.
C09/C10/C28/C38/C41 and Appendix C account for every unselected registration. The 583 native registrations split into 427 selected and 156 unselected; none of the 156 has an analysis/upload in the validation run.
The exact remaining sets are:
- 110 manual
window_rows/cases. All 128 registered cases expand aswindow_rows/{10000,100000}/{one,hundred}/{sum,count,min,max,avg}/{default,zero,two,sqrt,half,prefix}(120), pluswindow_rows/10000/{one,hundred}/frame_bench_sum/{zero,two,half}(six) andwindow_rows/100000/{one,hundred}/frame_bench_sum/prefix(two). Subtract these 18 nightly cases to get the exact 110:
window_rows/10000/one/sum/zero
window_rows/10000/one/sum/two
window_rows/10000/one/sum/prefix
window_rows/10000/hundred/sum/zero
window_rows/10000/hundred/sum/two
window_rows/10000/hundred/sum/prefix
window_rows/10000/hundred/frame_bench_sum/two
window_rows/100000/one/sum/zero
window_rows/100000/one/sum/two
window_rows/100000/one/sum/prefix
window_rows/100000/hundred/sum/zero
window_rows/100000/hundred/sum/two
window_rows/100000/hundred/sum/prefix
window_rows/100000/hundred/min/two
window_rows/100000/hundred/min/prefix
window_rows/100000/hundred/max/two
window_rows/100000/hundred/max/prefix
window_rows/100000/hundred/frame_bench_sum/prefix
- Four redundant intermediate-depth cases:
point_query_chain_depth/{churned_attr,sibling_attr,attr_scan,churned_attr_as_of}/500; depth 1/2000 endpoints already run. - Six extra retract scales:
retract/{single_fact,batch_100}/{1k,100k,1m}; 10k and chunk-boundary retractions already run. - Six duplicate pending-memory cases:
btree_lookup/{entity_point,attribute_scan}/{1k,10k,100k}. Despite their names these do not measure committed B+tree reads; C01 already covers pending scalar lookup. - Thirty experimental vector-slice kernels:
simd_{temporal,as_of,aggregate}/{scalar,simd}/{100,1k,10k,100k,1m}; they are not shipped engine paths.
Row-frame fixtures have 2n facts and one/100 partitions. Widths are zero, two, square root, half or full prefix of partition size; default uses the peer frame. Full-prefix one-partition registered-aggregate replay stays manual because replay is expensive. The unregistered 100k UDF half-frame is separately excluded for quadratic callback work.
Why outside the nightly shortlist: 18 row-frame representatives bound routine runtime; redundant scales and duplicate pending lookups add little distinct coverage, while synthetic SIMD kernels are not database feature evidence.
Acceptance criteria:
- Re-enumerate a pinned revision and reconcile all 156 unselected IDs against the expansions above, separately identifying the unregistered 100k UDF half-frame.
- Publish a reproducible manual sweep report for the 110 row-frame cases, or an explicit per-shape deferred/excluded disposition with cost and rationale; keep expensive full-prefix one-partition replay manual.
- Record fixture shape, frame/function/partition, build/core revision, hardware conditions, elapsed time and correctness checks for every measured case.
- Confirm the depth/retract/lookup/SIMD exclusions still hold; propose any newly useful representative separately and retain an explicit disposition for every excluded family.
Run heavy builds, enumeration, measurements and comparisons on idle hardware; retain reproducible evidence and clean temporary scratch afterward. This is benchmark/verification work aligned with PHILOSOPHY.md's reliability, portability and simplicity principles; any production change needs its own review.
Evidence: docs/bench-coverage-v3.md; validation run 41642356471. The validation run establishes only its recorded representative latency coverage; it does not complete this follow-up.
Activity
Merge pull request #846 from project-minigraf/bench/836-exclusions