Repository navigation
Competitor comparison: FlahaX vs Horticalc, PHREEQC, pyEQL and an Irriblend-method baseline (#40) - #63
Merged
Conversation
… baseline Development-only harness under benchmarks/competitors (never imported by flahax). Every engine gets the same 30-product catalogue, 26 formulas and two waters; each recipe is re-scored by the Benchmark v1 evaluator. - Horticalc pinned at v0.6.7 (db4e68c), run as shipped and aligned (separate N forms, S objective). The catalogue translation inverts Horticalc's own conversion rules (max error 3.6e-16). - Irriblend-DSW has no licence, so the baseline is an independent LP after the published method (exact NO3/NH4/P/K/micros, Ca/Mg/S at or above target, minimum grams), labelled as not the authors' code. - A second scoreboard counts recipes that meet every target AND stay within agronomic limits: incidental Cl and Na <= 3 mmol/L (De Kreij et al. 2003), NH4-N <= min(10% of N, 1.5 mmol/L), micronutrients <= 2x. - Timings: best of three, engines interleaved per case.
Runs each fixture's own PHREEQC input (the _target input for acid-target captures) through PhreeqPython's embedded IPhreeqc on the pinned, merged minteq.v4 database, replays the live rows against the stored PHREEQC 3.8.6 rows, then scores FlahaX with Benchmark v1's evaluate_chemistry at the live prescribed pH. The database is reloaded for every fixture: IPhreeqc keeps SOLUTION_SPECIES and PHASES from earlier inputs, and mixed_product_phases redefines ammonium species, which had corrupted later fixtures (struvite). Replay compares log quantities absolutely and molalities relatively above 1e-20; charge and pct_err are convergence residuals and are not compared.
…KCl standards Accuracy is scored against the IUPAC molality-based primary standards (Pratt, Koch, Wu and Berezansky 2001, Pure Appl. Chem. 73, 1783, Table 1, KCl 0.01/0.1/1.0 mol/kg at 25 C), transcribed into a fixture with its provenance. Catalogue recipes are compared for agreement only (no measured EC): every tool gets the same macronutrient totals, phosphate as H2PO4- for the tools that do not speciate. PHREEQC uses PhreeqPython's bundled phreeqc.dat, since minteq.v4.dat has no diffusion coefficients.
README gives the tool pins, licences, track protocols and run commands. requirements.txt pins the direct development tools and constraints.txt the full environment of the run. Tests cover the IUPAC transcription (checked against the paper's own Table 2 polynomial), replay scoring, acid-target input selection, EC input mapping and the incidental-ion limits, without the competitor packages installed.
Snapshot 2026-10-21-competitors-ed568c5: formulation, chemistry and EC track outputs from one run at ed568c5 (after #62), with the environment record and a checksummed evidence.zip. tools/publish_competitor_snapshot.py refuses a commit whose src/ or benchmarks/ differ from HEAD. The new evidence page reports every engine, including results unfavourable to FlahaX; the Benchmark v1 page links to it.
This was referenced Oct 20, 2026
rafatahmed
added a commit
that referenced
this pull request
Nov 7, 2026
Competitor comparison: FlahaX vs Horticalc, PHREEQC, pyEQL and an Irriblend-method baseline (#40)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #40.
A pinned, reproducible comparison of FlahaX with other open tools on shared inputs, published as a versioned evidence snapshot on Read the Docs (
docs/site/evidence/competitor-comparison.md). Results unfavourable to FlahaX are kept.Tools
db4e68cpyfarm-nutrients is excluded, since it calibrates EC heuristically and does not formulate. All of these tools are development-only (
benchmarks/competitors/requirements.txtplusconstraints.txt). FlahaX runtime dependencies stay empty.Results (snapshot
2026-10-21-competitors-ed568c5)Formulation, 52 recipes re-scored by the Benchmark v1 evaluator. "Acceptable" adds incidental limits: Cl and Na ≤ 3 mmol/L, NH4-N ≤ min(10 % N, 1.5 mmol/L), micronutrients ≤ 2× target.
Chemistry.
EC against the IUPAC molality-based KCl primary standards (Pratt et al. 2001):
Changes from the approved design
phreeqc.dat, becauseminteq.v4.dathas no diffusion coefficients.Harness bugs found and fixed while building it
SOLUTION_SPECIES/PHASESbetween inputs, somixed_product_phasescorrupted later fixtures (struvite). The database is now reloaded per fixture.charge,pct_err). Log quantities now compare absolutely, molalities relatively above 1e-20, and the residuals are not compared._targetinput, as the golden test does.The comparison also found the solver slowdown fixed in #61 / #62. These results were measured after #62.
Files
benchmarks/competitors/: the three tracks, Horticalc adapter, README/protocol, pins.tests/fixtures/condu 8000 ctivity/iupac2001_kcl_table1.json, with provenance.tests/test_competitor_benchmark.py: offline tests, no competitor packages needed.tools/publish_competitor_snapshot.py: copies the outputs byte for byte, adds the environment record and a checksummed zip, and refuses a commit whosesrc/orbenchmarks/differ from HEAD.docs/site/evidence/competitor-comparison.mdand its snapshot; linked from the evidence toctree and from Benchmark v1.Verification