8000 Irrigation EC M2b: phosphate coefficients fitted to Pethybridge et al. 2006 by rafatahmed · Pull Request #30 · rafatahmed/Flahax · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

- Add experimental irrigation EC25 from speciation (McCleskey 2012 Eq. 8) with explicit partial results, envelope checks and provenance; no accuracy claim.
- Irrigation EC (M2a): species without a McCleskey coefficient now give a bounded `estimated` result (`bounded`, `ec25_range_ms_cm`, midpoint `ec25_ms_cm`, `uncertainty_ms_cm`, audit `uncovered_bound_ms_cm`; upper limit from 80 S cm2/mol per charge) instead of `partial`; `MODEL_VERSION` is now m2a and `compare_measured_ec` reports `ec25_range_ms_cm` and `within_range`.
- Irrigation EC (M2b): H2PO4- and HPO4-2 now contribute through 25 C coefficients fitted to Pethybridge, Talbot and House (2006) Table III (`tools/fit_phosphate_conductivity.py`, `supplementary` block of the coefficient file, data to I = 0.014); the audit records `supplementaryCoefficients` and `phosphate_fit_max_ionic_strength`, and `MODEL_VERSION` is now m2b. Minor phosphate pairs and PO4-3 stay under the bound.
- Add the material traceability API (`docs/materials.md`): shipped identity records (`load_identities`, `identity_for_product`), user grades and lots (`make_grade`, `make_lot`), `resolve_material` (per-nutrient basis, band ends, flags, provenance) and `nutrient_band`; `catalogue_dose_totals` and `calculate_irrigation_ec` take `materials=`, undisclosed blends raise or report `composition_undisclosed` in chemistry, and EC and equilibrium-delivery audits carry material provenance.
- Link every catalogue product in `library.json` to its chemical identity (`identityId`) and record its `analysisBasis` (`declared_catalogue`, `identity_theoretical` or `grade_typical`) and `source` document reference. Product `id` and `name` values are unchanged.
- Correct Mg Nitrate to Mg(NO3)2·6H2O theory (owner decision D1): formula `Mg(NO3)2` → `Mg(NO3)2.6H2O`; N_NO3 11 → 10.92564 %, Mg 9.65 → 9.47931 %; basis `identity_theoretical`. Its declared-form charge residual at 1 g/kg water goes from +0.5532 % to ~0 %, so it no longer trips the EC O5 gate on its own.
Expand Down
36 changes: 22 additions & 14 deletions docs/conductivity.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,10 +125,10 @@ still be reported when EC evidence is missing; that does not approve the recipe
or authorize EC-driven correction. Verify EC after preparation/dilution using
meters rated for the relevant stock and irrigation ranges.

## Calculated irrigation EC (experimental, M2a)
## Calculated irrigation EC (experimental, M2b)

Purpose: estimate the EC of an irrigation solution before mixing, and compare
it with a meter reading after mixing. This is **experimental (M2a)** and Python
it with a meter reading after mixing. This is **experimental (M2b)** and Python
only: there is no CLI flag and no grower-facing number. It is separate from the
calibration-based planner above.

Expand All @@ -141,7 +141,7 @@ applied.

| Status | Meaning |
|---|---|
| `estimated` | Every charged species above 1e-12 mol/kgw has a McCleskey coefficient, or the species without one are covered by the bound below. `ec25_ms_cm` is the estimate. |
| `estimated` | Every charged species above 1e-12 mol/kgw has a McCleskey coefficient, or the species without one are covered by the bound below, or a supplementary 25 C coefficient (H2PO4-, HPO4-2). `ec25_ms_cm` is the estimate. |
| `partial` | With a `supplied` pH, there are unrecorded counterions (`unrecorded_counterions`, rule O5) reported in eq/kgw. Only a known-species subtotal is reported, with no total. |
| `not_supported` | Outside the envelope (temperature, ionic strength, non-closed carbon) or the assay charge residual is too large. `reason` says which; no EC is reported. |

Expand All @@ -154,7 +154,7 @@ it is over the speciated absolute equivalents. Above it, a `supplied` pH gives
`not_supported` (`assay_charge_residual_too_large`).

Bounded estimates (decision O2): charged species with no McCleskey coefficient
(H2PO4-, MgOH+, CaOH+, charged chelates) no longer make the result `partial`.
(KHPO4-, CaH2PO4+, MgOH+, CaOH+, charged chelates) no longer make the result `partial`.
Their contribution is non-negative, so the known subtotal is a lower limit. The
upper limit adds sum(|z| x 80 x m) over those species, because no ion's limiting
molar conductivity exceeds 80 S cm2/mol per unit charge at infinite dilution
Expand All @@ -164,18 +164,26 @@ molar conductivity exceeds 80 S cm2/mol per unit charge at infinite dilution
- `ec25_range_ms_cm`: `(known subtotal, known subtotal + bound)`; the range extends only upward from the known subtotal, because missing contributions cannot be negative.
- `ec25_ms_cm`: the midpoint; `uncertainty_ms_cm`: half the bound.
- `audit["uncovered_bound_ms_cm"]`: the bound; `missing_species` still lists every uncovered species.
- `transport_numbers` is `{}`, and `MODEL_VERSION` is `irrigation-ec-mccleskey2012-m2a-experimental`.
- `transport_numbers` is `{}` for bounded results, and `MODEL_VERSION` is `irrigation-ec-mccleskey2012-m2b-experimental`.

The midpoint and `uncertainty_ms_cm` describe only the uncovered-species interval; they are not an accuracy statement and exclude model error. Because 80 S·cm²/mol per charge is conservative, the true value tends to lie in the lower half of the range.

`partial` arises only from unrecorded counterions; temperature, carbon,
ionic-strength domain and undisclosed blends give `not_supported`. All 26 shipped formulas return `estimated`, with ranges up to 14.4 % of
the midpoint (Generic Wet Season). Phosphate coefficients are not yet
measured, so phosphate stays under the bound until M2b. `compare_measured_ec`
ionic-strength domain and undisclosed blends give `not_supported`. All 26 shipped formulas return `estimated`, with ranges up to 5.4 % of
the midpoint (Generic Wet Season, mostly citrate pairs; 14.4 % before the
phosphate coefficients).

**Phosphate (M2b).** H2PO4- and HPO4-2 are covered at 25 C by coefficients
fitted to the measured KH2PO4 and K2HPO4 conductivities of Pethybridge,
Talbot and House (2006), Table III, through FlahaX speciation (lambda0 and A
fitted, B = 1.5; RMS 0.13 % over 22 points). The data reach I = 0.014 mol/kgw;
above that the fit is extrapolated, and the audit records
`phosphate_fit_max_ionic_strength` and `supplementaryCoefficients`. PO4-3 and
minor phosphate pairs such as KHPO4- stay under the bound. KHPO4- is not covered; the fitted HPO4-2 coefficients absorb part of its current, so the upper end of a mixture's range may count it twice (small and conservative). `compare_measured_ec`
also returns `ec25_range_ms_cm` and `within_range` (True, False, or None when the
prediction is not bounded); it is a fact, not a pass/fail verdict.
prediction is not bounded); it is a fact, not a pass/fail verdict. `within_range` does not include model or meter error, so when the range is narrow a False value says nothing about agreement.

Example (KH2PO4 at 1 g/kg water, calculated pH):
Example (KH2PO4 at 1 g/kg water, calculated pH). With the phosphate coefficients the remaining bound is below 0.001 mS/cm, so a 0.80 reading lies outside the range:

```python
from flahax import calculate_ec_from_totals, compare_measured_ec
Expand All @@ -197,10 +205,10 @@ Output:

```text
estimated True
[0.524, 1.113]
0.818 0.295
['H2PO4-', 'HPO4-2', 'KHPO4-', 'PO4-3']
True
[0.738, 0.738]
0.738 0.0
['KHPO4-', 'PO4-3']
False
```

```python
Expand Down
2 changes: 1 addition & 1 deletion docs/development-status.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ The matrix below describes the 0.3.1 capabilities and their boundaries.
| Industrial acid selection and product mass/volume | Standard workflow HNO3/H3PO4, `acid_selection.py`, four preserved lower-level PHREEQC acid fixtures | Supplied assay/density/provenance, nutrient and volume limits; no automatic dosing approval |
| Source-specific CLI | `planning_cli.py`, `__main__.py`, `test_site_planning.py` | Explicit JSON/interactive inputs, no default water or equipment model |
| No-new-measurements EC screening | `calculated_ec.py`, exact SQM anchors with source hash, regression tests | Pre-acid dilute approximation at 25 C, restricted loading, unknown accuracy; incomplete products yield no total |
| Calculated irrigation EC (McCleskey 2012 on speciation) | `irrigation_ec.py`, transcription and rule tests | Experimental M2a; phosphate and chelate species give a bounded range; no measured accuracy |
| Calculated irrigation EC (McCleskey 2012 on speciation) | `irrigation_ec.py`, transcription and rule tests | Experimental M2b; H2PO4-/HPO4-2 fitted to Pethybridge 2006 Table III; minor pairs and chelate species give a bounded range; no measured accuracy |
| Optional measured-profile EC method | `conductivity.py`, synthetic regression profiles | Requires matching measured evidence when used; example profiles are fictional |
| A/B and acid report clarity | `pepper_stock_review.py`, HTML/Markdown/JSON examples and tests | Candidate grouping only; trace locations unresolved; no approved full pepper StockPlan |
| Stock planning API | Existing sourced compatibility/solubility/capacity checks | Caller must supply complete applicable records; absence from a pairwise table is not evidence of compatibility |
Expand Down
37 changes: 21 additions & 16 deletions docs/superpowers/plans/2026-10-19-irrigation-ec-m2.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,25 +70,30 @@
- `test_band_ends_carry_own_bounds` (Review Focus 4): a graded KH₂PO₄-like case where the ends differ.
- [ ] **Implement, run, full suite, then commit** with the message "Report range membership for bounded EC estimates".

### Task 3 (M2b): Phosphate coefficients from measured data. BLOCKED until the owner supplies data.
### Task 3 (M2b): Phosphate coefficients from Pethybridge et al. 2006 (amended O1)

**Precondition (controller checks; if missing, stop and report BLOCKED):**
- `tests/fixtures/conductivity/crc_phosphate_lambda0.json`: owner-checked CRC λ° for H2PO4-, HPO4-2 and PO4-3, with the edition and page.
- `tests/fixtures/conductivity/muccitelli1994_25c.json` and an independently typed copy `..._independent.json`. They hold every 25 °C KH₂PO₄ and K₂HPO₄ point (concentration in mol/L, κ in µS/cm or as printed, with units) from Muccitelli & DiAngelo (1994), plus the paper's DOI and the PDF SHA-256.
**Inputs:**
- `tests/fixtures/conductivity/pethybridge2006_table3.json` is the owner's first transcription. The controller writes it from the owner's message, with the citation, doi, units and the ½K₂HPO₄ heading convention.
- `..._independent.json` is typed by a separate agent from the PDF in `docs/sources/` (step 3b). **Merge gate:** both copies must agree.

**Files:**
- Create: `tools/fit_phosphate_conductivity.py` (reads the fixtures, converts mol/L to mol/kgw with a stated density of water at 25 °C, speciates with FlahaX, and fits A and B per phosphate ion by least squares on κ with λ° fixed).
- Modify: `src/flahax/data/conductivity_mccleskey2012.json`. Add a top-level `"supplementary"` section with `{ion: {lambda0_25c, A_25c, B, source, fit_rms_percent, points}}`. Do not add these ions to the McCleskey `ions` table, because they are not in the paper.
- Modify: `irrigation_ec.ionic_molal_conductivity` / `covered_species`, so supplementary ions are covered at 25 °C only.
- Test: `tests/test_irrigation_ec.py` (class `PhosphateCoefficientTests`).

- [ ] **Failing tests:**
- `test_fixture_transcriptions_agree`;
- `test_fit_reproduces_measurements` (each point within the published RMS, which is ≤ 2 %);
- `test_phreeqc_cross_check` (CRC λ° within 15 % of the `phreeqc.dat`-derived 31.8 / 103.6 / 206.8; report, don't hide);
- `test_kh2po4_now_covered`: KH₂PO₄ totals give an unbounded `estimated` result, or a much smaller bound if minor pairs remain.
- [ ] **Run the tool**, commit its output, then run the full suite. Recount the 26 formulas and record the narrowed ranges in the design §13.
- [ ] **Commit** with the message "Add phosphate conductivity coefficients fitted to measured data".
- Create `tools/fit_phosphate_conductivity.py`.
- **Inputs:** the fixture. Concentrations in mmol/dm³ convert to mol/kgw using a water density of 0.997047 kg/dm³ at 25 °C, a recorded constant.
- **Conductivity:** κ (mS/cm) = Λ × c(mmol/dm³) / 1000. The K₂HPO₄ rows use ½-formula units, so salt molarity = c/2.
- **Speciation:** each point uses `calculate_equilibrium_ph({'K+': n·m, 'PO4-3': m}, complete_analysis=True, carbon_boundary='closed', phases=[])`.
- **Fit:** weighted linear least squares (weight 1/κ²) for λ° and A of H2PO4- and HPO4-2, with B = 1.5 fixed.
- **Output:** the supplementary block, JSON on stdout, written with `--write`.
- Modify `src/flahax/data/conductivity_mccleskey2012.json`. Add a top-level `"supplementary"` block: `{"H2PO4-": {"lambda0_25c", "A_25c", "B": 1.5}, "HPO4-2": {...}, "source": {citation, doi, table: "III"}, "fit": {rms_percent, points, max_ionic_strength, density_kg_dm3}, "cross_checks": {...}}`. The McCleskey `ions` table stays untouched. Update `COEFFICIENTS_SHA256` / the literal pin test.
- Modify `irrigation_ec.py`. `ionic_molal_conductivity` and `covered_species` include the supplementary ions (25 °C only; any other temperature raises). The audit gains `phosphate_fit_max_ionic_strength` and `supplementaryCoefficients: true` whenever a supplementary ion contributes. Bump `MODEL_VERSION` to `irrigation-ec-mccleskey2012-m2b-experimental`.

- [ ] **3a. Failing tests** (class `PhosphateCoefficientTests`):
- `test_fit_reproduces_table3`: running the tool on the fixture reproduces the shipped supplementary block to 1e-9, and every point is within 0.5 %.
- `test_cross_checks_reported`: fitted λ° vs CRC (36, 114), Pethybridge (33.33, 115.14) and phreeqc.dat (31.8, 103.6). Report the percent gaps and assert ≤ 15 %, or document a larger gap in the block.
- `test_kh2po4_now_mostly_covered`: KH₂PO₄ totals (1/136.09 mol/kgw) are bounded only by minor pairs, and the range width is below 5 % of the midpoint (it was ±36 % under M2a).
- `test_supplementary_not_in_mcCleskey_table`.
- `test_all_shipped_formulas_estimated`: tighten the width ceiling to the new observed maximum plus margin, and record the numbers.
- [ ] **3a. Implement, run the tool, commit its output, run the full suite** (in the background, waiting for it). Recount the 26 formulas into design §13. Commit with the message "Add phosphate conductivity coefficients fitted to Pethybridge et al. 2006".
- [ ] **3b (needs the PDF):** a separate agent types `pethybridge2006_table3_independent.json` from the PDF. Add `test_table3_transcriptions_agree`. If any value disagrees, stop and report to the owner. Commit with the message "Check Pethybridge Table III against an independent transcription".

### Task 4: Docs, exports, verification

Expand Down
Loading
Loading
0