8000 Add reproducible benchmarks and documentation branding by rafatahmed · Pull Request #35 · rafatahmed/Flahax · GitHub
[go: up one dir, main page]

Skip to content

Add reproducible benchmarks and documentation branding - #35

Merged
rafatahmed merged 4 commits into
mainfrom
codex/docs-branding
Oct 20, 2026
Merged

rafatahmed merged 4 commits into
mainfrom
codex/docs-branding

Conversation

@rafatahmed
@rafatahmed rafatahmed commented Oct 20, 2026 •
Copy link
Copy Markdown
Owner

FlahaX now has a versioned, reproducible benchmark protocol that separates formulation fit, stored-reference chemistry agreement, and performance measurements. Reports preserve provenance, raw samples, and failures; a manual CI workflow collects evidence without imposing speed thresholds or publishing results automatically.

The branch also integrates the approved FlahaX banners and transparent color/black/white logos into the README and Read the Docs theme, with asset-generation provenance.

Validation

  • 20 focused measurement, documentation, and release-hygiene tests passed.
  • Strict Sphinx documentation build passed.
  • Clean wheel and source-distribution installation checks passed.
  • Local development baseline: 21/21 stored chemistry comparisons pass; 563/636 explicit nutrient targets and 29/52 whole recipes meet the specified target-fit tolerance.
  • Fifteen parameterized timing measurements completed with ten samples each. Before/after input manifests match.

Known limitations requiring review

  • The near-dependent NNLS case fails its predefined objective-gap tolerance (approximately 1.50e-12 against 1e-12). The scientific report exits nonzero and that workload supplies no successful timing. Solver code and tolerances were not changed to obtain a pass.
  • The acid-target timing exceeds the 900-second ASV process budget, including repeated setup calculations. Its missing timing is retained; this does not establish that one solve takes 900 seconds or fails to converge.
  • These measurements came from a dirty development checkout on Windows/Python 3.14, not a clean release or the hosted Python 3.13 runner. Generated local measurement files are not committed as release evidence.
  • No physical validation, new live PHREEQC replay, or new objective-aligned competitor comparison was performed.

This PR remains a draft so the retained numerical limitation and measurement budget can be reviewed before merge. Unrelated publication edits and unused branding alternatives are excluded.

Follow-up tracking

Benchmark limitations and subsequent validation work are tracked in #42, with individual issues #36, #37, #38, #39, #40, and #41. This PR does not close those follow-ups.

Historical branch naming clarification

The original source branch codex/docs-branding remains recorded here to preserve the audit history. Under the repository's current naming convention, the appropriate descriptive name is docs/branding. This clarification does not rename the historical PR source branch or change any commits, hashes, merge records, reviews, or checks. Branch names are not authorship attribution. Future branches use purpose-based prefixes, without codex/, and future commits must not add Codex co-author attribution.

Separate formulation, stored chemistry reference checks, and informational performance measurements. Record provenance, raw samples, and failed cases; add a manual CI workflow and reproduction protocol.

Validation: 20 focused tests, strict Sphinx build, and wheel/sdist installation checks passed. Local baseline retains a near-dependent NNLS tolerance failure and an acid timing budget timeout; these are not hidden or treated as successful measurements.
@read-the-docs-community
read-the-docs-community Bot commented Oct 20, 2026 •
Copy link
Copy Markdown

Documentation build overview

📚 Flahax | 🛠️ Build #35265528 | 📁 Comparing 6373d23 against latest (4c0ea9c)

  🔍 Preview build  

1 file changed
+ evidence/benchmark-v1.html

@rafatahmed
rafatahmed marked this pull request as ready for review October 20, 2026 09:45
@rafatahmed
rafatahmed merged commit 3b6d304 into main Oct 20, 2026
6 checks passed
@rafatahmed
rafatahmed deleted the codex/docs-branding branch October 20, 2026 09:46
@rafatahmed rafatahmed self-assigned this Oct 20, 2026
rafatahmed added a commit that referenced this pull request Nov 7, 2026
Add reproducible benchmarks and documentation branding
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

0