8000 Fix NNLS early stopping; add chloride salts and incidental-ion ceilings by rafatahmed · Pull Request #51 · rafatahmed/Flahax · GitHub
[go: up one dir, main page]

Skip to content

Fix NNLS early stopping; add chloride salts and incidental-ion ceilings - #51

Merged
rafatahmed merged 2 commits into
mainfrom
feat/catalogue-chloride-sources
Oct 20, 2026
Merged

rafatahmed merged 2 commits into
mainfrom
feat/catalogue-chloride-sources

Conversation

@rafatahmed
Copy link
Copy Markdown
Owner

Two commits from the root-cause audit of the open benchmark issues (tracker #42).

1. NNLS early stopping (#36)

Root cause: every threshold in nnls/_lstsq was a fixed 1e-10, so behaviour depended on problem scale. Near-dependent columns stopped early (Benchmark v1 near_dependent: objective 1.5e-12 instead of ~0), and a problem scaled by 1e-8 returned all zeros.

Fix:

  • The optimality (dual) test is now at round-off level, scaled by |A| and |b|.
  • Rank checks are relative to the largest column norm.
  • A column rejected as dependent is skipped until the solution changes, as in Lawson-Hanson.

Evidence:

  • All 1,266 NNLS problems that recommend() solves for the 52 benchmark recipes match SciPy before and after the change.
  • No recipe dose changes.
  • Two tests that fail before the fix cover the near-dependent and scale cases.

2. Chloride salts, data-sheet sources and incidental-ion ceilings (#39)

Root cause of 10 recipe misses: Calcium Nitrate was the only calcium product, so Ca and NO3 came in a fixed ratio. A linear-programming check showed 16 of the 23 failing recipes have no solution within 1% from the old catalogue.

Catalogue (28 → 30):

  • Potassium Chloride (Sigma-Aldrich P3911) and Calcium Chloride (Dihydrate) (Sigma-Aldrich 223506), with identity-theoretical assays.
  • Ammonium Sulfate and Urea now cite ChemicalBook data sheets (SHA-256 recorded); their assays are unchanged.

Engine: recommend(..., maximum_concentrations={"Cl": ...}), also passed through by the workflow.

  • An ion with a ceiling is admitted and is not penalised up to the ceiling.
  • When a fit exceeds the ceiling, the ion is pulled back to it; any remaining excess is a reported limit violation, never a silent one.
  • Chloride needs consent: recovery never admits it, so all 52 default recipes are identical to main.
Cl ceiling Targets Whole recipes
none (default) 563/636 29/52
100 mg/L 579/636 31/52
150 mg/L 580/636 31/52

Fixes:

  • Chloride assays were dropped in the catalogue chemistry conversion.
  • A calcium salt was assumed to carry nitrate.
  • Chlorine now uses 35.45 g/mol everywhere. The manual's Shomar et al. station 104 example now closes chloride by -9.53% instead of -9.52%; the 0.4.0 release-evidence record keeps -9.52%.

Not covered

  • The live PHREEQC replay has no fixtures for the two chloride salts; they are covered by conservation and charge tests.
  • Silicon (Rice) remains unsupported, by decision.

Verification

  • 325 tests pass (1 skipped).
  • sphinx -W builds cleanly.

Closes #36. Partly addresses #39: the remaining misses are coupled Mg/S targets and Pepper's multi-target conflict.

Add Potassium Chloride and Calcium Chloride (Dihydrate) with
identity-theoretical assays sourced to Sigma-Aldrich P3911 and 223506.
Calcium chloride is the first calcium source without nitrate (#39).
Chloride needs consent: recovery never admits it, so no default recipe
changes.

Record the ChemicalBook data sheets as sources for Ammonium Sulfate and
Urea; their assays are unchanged.

recommend() and plan_fertilizer_workflow() now use
maximum_concentrations in the fit: an ion with a ceiling is admitted,
is not penalised up to it, and is pulled back to it when exceeded. Any
remaining excess is reported as a limit violation.

Fix chloride in catalogue chemistry conversion (assay dropped, and
calcium assumed to carry nitrate) and use 35.45 g/mol for chlorine
everywhere.
The optimality test and the rank checks used a fixed 1e-10 threshold,
so behaviour depended on problem scale. Near-dependent columns stopped
before the optimum (Benchmark v1 near_dependent: objective 1.5e-12
instead of ~0), and a problem scaled by 1e-8 returned all zeros.

The optimality test is now at round-off level, scaled by |A| and |b|,
rank checks are relative to the largest column norm, and a column
rejected as dependent is skipped until the solution changes, as in
Lawson-Hanson. All 1,266 NNLS problems solved for the 52 benchmark
recipes match SciPy as before, and no recipe dose changes.
@read-the-docs-community
Copy link
Copy Markdown

Documentation build overview

📚 Flahax | 🛠️ Build #35274498 | 📁 Comparing 28ee9e8 against latest (a628be7)

  🔍 Preview build  

6 files changed · ± 6 modified

± Modified

@rafatahmed
rafatahmed merged commit a1e9659 into main Oct 20, 2026
6 checks passed
@rafatahmed
rafatahmed deleted the feat/catalogue-chloride-sources branch October 20, 2026 16:31
rafatahmed added a commit that referenced this pull request Nov 7, 2026
Fix NNLS early stopping; add chloride salts and incidental-ion ceilings
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Investigate NNLS objective discrepancy for near-dependent columns

1 participant

0