8000 Close catalogue follow-ups: formulation provenance, counterion sodium, band-end chemistry by rafatahmed · Pull Request #27 · rafatahmed/Flahax · GitHub
[go: up one dir, main page]

Skip to content
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,9 @@
- Calcium Nitrate (ag grade) keeps N_NO3 14.4 / N_NH4 1.1 / Ca 19 % and now cites the Haifa Cal NG technical data sheet (October 2018) with basis `grade_typical`. Metadata: Urea formula `Urea` → `CO(NH2)2` (the formula check now reads urea's amide N as N_UREA, so Urea stays eligible); display names "Iron(II) Sulfate (Heptahydrate)" and "Potassium Citrate (anhydrous)".
- Recommended doses (g/L, `recommend` with all eligible products and the formula's water) change for 20 of the 26 shipped formulas: Chilli, DEFAULT, Generic Bloom, Generic Dry Season, Generic Grow, Generic Wet Season, Generic for Berries, Hoagland, Lettuce 2, Lettuce General, Melons, Pepper (Howard Resh), Rice, Strawberry Drip Irrigation, Strawberry Fruiting, Tomato (Howard Resh), Tomatoes Fourth/Third Cluster and to termination, Tropical Lettuce. Pepper regression recipe (Test01, Ca 20 ppm water): Mg Nitrate 0.414486 → 0.421953, Phosphoric Acid (75%) 0.117392 → 0.151738; pepper water workflow: Mg Nitrate 0.414483 → 0.421949, Phosphoric Acid (75%) 0.098687 → 0.126809, incidental N_NH4 4.845114 → 4.845193 ppm.
- Chelates now carry their sodium counterions in chemistry totals (D2); recipes with chelates show incidental Na in equilibrium and EC results. Per formula unit: Fe-EDTA 1, Mn/Zn/Cu-EDTA 2, Fe-DTPA 2, Fe-o,o-EDDHA 1 Na, also added to CH - micro's borax and molybdate sodium; product `elements` are unchanged, so formulation is unaffected. Na+ at 1 g/kg water: Iron EDTA 0 → 53.52 mg, Zn EDTA 0 → 98.46 mg, Mn EDTA 0 → 108.80 mg, Copper EDTA 0 → 101.30 mg, Iron DTPA 0 → 90.57 mg, Iron EDDHA 0 → 24.70 mg, CH - micro 14.06 → 63.23 mg. Declared-form charge residuals fall to ~0 % (Iron EDTA −14.29 %, Zn/Mn/Cu EDTA −33.33 %, Iron DTPA −25.00 %, Iron EDDHA −14.29 %, CH - micro −12.76 %), so chelate recipes no longer trip the EC O5 gate. Borax declares its ion forms (`Na+`, `B(OH)4-`, `B(OH)3`; −0.5 charge per B in the residual): residual +100 % → +0.006 %; its equilibrium totals are unchanged (boron stays the H3BO3 basis).
- `recommend` results (and the workflow recommendation) now carry per-salt `provenance` (recordId, identityId, analysisBasis, documents, flags; `unattributed` when the record has none) and `identitiesSha256`; `solve_weights` and `forward` are unchanged.
- Formulation rows now report formula counterion sodium not already assayed as elemental Na (chelate sodium, D2, and CH - micro's borax and molybdate sodium), added to the `Na` row and incidental review after the fit, with an `Na` target's `deltaPct` recomputed (pepper workflow incidental Na 0.048 -> 1.722 ppm; Tomatoes U of Florida clusters 2.1923 -> 2.7548, Lettuce General 2.2258 -> 2.5774, Generic Wet Season 2.5403 -> 2.7161 mg/L with CH - micro's borax/molybdate share); a tight Na maximum can make such recipes require review or be infeasible.
- Add `calculate_irrigation_ec_band`: irrigation EC25 chemistry (pH, precipitation, EC) at nominal and nutrient band minimum and maximum ends; an end with an open band returns `not_supported` / `band_end_open`.

## 0.4.0 — 2026-10-17

Expand Down
74 changes: 70 additions & 4 deletions docs/materials.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,8 @@ re-run, and chemistry (pH, precipitation, EC) is reported at the nominal assay
only. A lot with stated uncertainty gives its own band; otherwise the grade's
minimum and maximum apply. An open bound (a grade with a minimum only) gives
`None` for that end. FlahaX shows the band; it does not pick a "safe" end.
For pH, precipitation and EC at the band ends, use `calculate_irrigation_ec_band`
(below).

## Undisclosed blends

Expand Down Expand Up @@ -187,8 +189,72 @@ Band applies to nutrient amounts only; chemistry is reported at the nominal assa
The lot supplies N-NO3 (`lot_measured`); K comes from the grade typical. The
lot gave no uncertainty, so the band is the grade's minimum to maximum.

## Follow-ups (not in this release)
## Provenance, counterion sodium and band-end chemistry

**Provenance on formulation results.** `recommend` results (and so the
workflow recommendation) attach `provenance` to every salt entry: `recordId`,
`identityId`, `analysisBasis`, `documents` and `flags`, taken from the record
that produced the salt. A record without provenance reports
`analysisBasis: "unattributed"`; a basis is never guessed. `analysisBasis` is a
string for shipped records and a per-nutrient dict for resolved materials. The
result also carries `identitiesSha256`, which pins the shipped catalogue.
`solve_weights` and `forward` are unchanged.

**Counterion sodium in formulation rows.** Formula counterion sodium not
already assayed as elemental Na (chelate sodium, D2, and CH - micro's borax and
molybdate sodium) is added to the `Na` row and to the incidental-ion review
after the fit, with a warning naming the amount; an `Na` target's `deltaPct` is
recomputed. It never enters `elements`, eligibility, the fit or the
grams. Na stays a restricted ion, so a tight Na maximum can make a recipe with
chelates require review or be infeasible.

**Chemistry at the band ends.** `calculate_irrigation_ec_band` takes the
arguments of `calculate_irrigation_ec` and returns `nominal`, `minimum` and
`maximum` results (each with its own pH, warnings and EC) plus a `note`. The
minimum end sets every dosed product to its band minimum and the maximum end
to its band maximum.

- The ends are corner cases (every product at its minimum, or every product at
its maximum), not statistical bounds.
- A non-proportional grade (nutrients that do not move together) can leave an
end charge-unbalanced, so that end returns `assay_charge_residual_too_large`.
- Each end's `audit['materials']` shows the nominal provenance; `audit['band_end']`
tells which end it is.
- An end where any dosed product has an open band end (a shipped product has
none) returns `not_supported` / `band_end_open`. Nothing is substituted.

- Provenance on `recommend` and forward formulation results.
- Reporting chelate sodium in formulation rows.
- Chemistry (pH, precipitation, EC) at the band ends.
```python
from flahax import make_grade, resolve_material, calculate_irrigation_ec_band
from flahax.irrigation_ec import DeclaredPureWater

doc = {"supplier": "Example Fertilisers Ltd", "title": "Potassium nitrate TDS rev 3",
"issued": "2026-03-01", "entered": "2026-10-19"}
grade = make_grade({
"grade_id": "example-kno3-tds", "identity_id": "idn-potassium-nitrate",
"supplier": "Example Fertilisers Ltd", "product": "KNO3 crystal", "document": doc,
"analysis": {"N_NO3": {"minimum": 13.8, "typical": 13.85, "maximum": 13.9},
"K2O": {"minimum": 46.4, "typical": 46.6, "maximum": 46.8}},
})
material = resolve_material("Potassium Nitrate", grade=grade)
band = calculate_irrigation_ec_band(
{"Potassium Nitrate": 0.5}, materials={"Potassium Nitrate": material},
recipe_id="band-example", water=DeclaredPureWater("ro-1"),
ph_mode="calculated", phase_policy="dissolved")
for end in ("minimum", "nominal", "maximum"):
r = band[end]
print(end, r.status, r.ec25_ms_cm, r.ph, r.audit.get("band_end"))
print(band["note"])
```

Output:

```text
minimum estimated 0.6892812265428483 6.515030771493912 minimum
nominal estimated 0.6924419215785783 8.454879373311996 None
maximum estimated 0.6957751961226507 8.77560493350029 maximum
Band ends are corner cases (every product at its minimum, or every product at its maximum), not statistical bounds.
```

In unbuffered pure water the pH follows the assay charge residual, so the small
residual differences between the ends move pH strongly (6.52, 8.45 and 8.78
here). This is not a buffered-solution prediction.
110 changes: 110 additions & 0 deletions docs/superpowers/plans/2026-10-19-catalogue-followups.md
8BD0
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
# Catalogue Follow-ups (F1–F3) Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Close the three documented follow-ups of the traceable catalogue: provenance on formulation results, chelate sodium in formulation rows, and chemistry at the nutrient-band ends.

**Architecture:** Small changes to existing modules. `resolve_material` attaches `record['provenance']`. `engine.recommend` reports per-salt provenance and chelate sodium after the fit. `irrigation_ec` gains `calculate_irrigation_ec_band`, which builds the minimum and maximum corner records from `ResolvedMaterial` band ends and reuses `calculate_irrigation_ec`.

**Tech Stack:** Python 3.13, standard library, `unittest`.

**Spec:** [docs/superpowers/specs/2026-10-18-traceable-catalogue-design.md](../specs/2026-10-18-traceable-catalogue-design.md), §12.

## Global Constraints

- No new runtime dependency. Existing signatures stay unchanged; every addition is optional or additive.
- `forward()` is unchanged.
- Chelate sodium (D2) never enters `elements`, eligibility or the fit. Na stays in `RESTRICTED_IONS`.
- Records lacking provenance report `analysisBasis: "unattributed"`, never a guessed basis.
- An open band end returns `not_supported` / `band_end_open`; no substitution.
- Work on branch `feat/catalogue-followups`, cut from `feat/traceable-catalogue`. Stage only named files. Never use `git stash`. Never stage `README.md`, `docs/publication/*`, `docs/landscape-comparison-swot.md` or `outputs/`.

## Review Focus

1. **`recommend` with a mix of shipped and resolved records.** Each salt entry carries its own provenance; no entry borrows another product's. Pinned in Task 1.
2. **A recipe with no chelates.** The `Na` row and `incidentalContributions` are byte-identical to before. Pinned in Task 2.
3. **Band ends with only shipped products.** Both ends are `not_supported` / `band_end_open`, and the nominal equals `calculate_irrigation_ec`. Pinned in Task 3.
4. **A band end that pushes ionic strength over 0.1.** That end alone is `not_supported` / `ionic_strength_outside_domain`. Pinned in Task 3.

---

### Task 0: Branch

- [ ] `git switch -c feat/catalogue-followups` from `feat/traceable-catalogue`. Commit the spec addendum and this plan: `git add docs/superpowers/specs/2026-10-18-traceable-catalogue-design.md docs/superpowers/plans/2026-10-19-catalogue-followups.md && git commit -m "Plan catalogue follow-ups F1-F3"`.

### Task 1: Provenance on formulation results (F1)

**Files:** Modify `src/flahax/materials.py` (`resolve_material` sets `record['provenance']`) and `src/flahax/engine.py` (`recommend` output). Test: `tests/test_materials.py` (class `FormulationProvenanceTests`).

**Interfaces:**
- Each `recommend(...)['salts'][i]` gains a `provenance` dict with these keys:
- `recordId`
- `identityId`
- `analysisBasis`: a string, or a per-nutrient dict for resolved materials
- `documents`: a list of document-reference dicts
- `flags`: a list
- `recommend(...)` gains `identitiesSha256`.

**Rules:**
- **Source of provenance:** use the record's own `provenance` key when present; else its `identityId`, `analysisBasis` and `source` keys (shipped `library.json`); else `analysisBasis: "unattributed"`, `identityId: None`.
- **Isolation:** provenance must be built from the record that produced that salt entry, matched by `id`.

- [ ] **Step 1: Failing tests.**
- `test_shipped_recommend_has_catalogue_provenance`: every salt entry has an `identityId` starting `idn-`. Calcium Nitrate has `analysisBasis` `grade_typical` and a Haifa document.
- `test_resolved_record_provenance_flows` (Review Focus 1): mix the shipped library with one resolved KNO3 grade record. The KNO3 entry carries the grade's per-nutrient basis and `recordId` `<cuid>+<grade>`; the other entries keep catalogue provenance.
- `test_unattributed_record`: a hand-built `{id, name, formula, elements}` gives `analysisBasis` `unattributed`.
- `test_identities_hash_on_result`.
- [ ] **Step 2: Run** `PYTHONPATH="src;." python -m unittest tests.test_materials -v`. Expect FAIL.
- [ ] **Step 3: Implement.** Then regenerate the API reference if docstrings changed.
- [ ] **Step 4: Run** `tests.test_materials` and the full suite. Expect OK.
- [ ] **Step 5: Commit** with the message "Report material provenance on formulation results".

### Task 2: Chelate sodium in formulation rows (F2)

**Files:** Modify `src/flahax/engine.py` (post-fit reporting only), reusing `product_conversion`'s counterion rule through a small helper `chelate_sodium_mg_per_l(salts_with_doses) -> float`, defined in `product_conversion.py`. Test: `tests/test_materials.py` (class `ChelateSodiumReportTests`).

**Rules:**
- Sodium (mg/L) = Σ over chosen chelate and CH - micro products of the D2 counterion moles at dose g/L, treated as g/kg water, × 22.98977 × 1000.
- Add it to the `Na` row's `final`; create the row with `target: None` if it is absent.
- Add it to the `incidentalContributions` `Na` entry. Re-run the existing incidental assessment so `requires_review` and limit checks see it.
- Append the warning `"Incidental Na includes chelate sodium counterions (D2): {x:.4f} mg/L."` only when x > 0.
- No change to eligibility, `_fit` or grams.

- [ ] **Step 1: Failing tests.**
- `test_chelate_sodium_reported_not_fitted`: the pepper/Chilli formula with Mn EDTA chosen. Assert the `Na` row increases by exactly the helper value, grams are identical to a run with the reporting disabled (patch the helper to return 0), and chelates remain chosen.
- `test_no_chelate_recipe_unchanged` (Review Focus 2): a formula with no chelates. `rows` and `incidentalContributions` equal the pre-change output, captured as a literal.
- `test_helper_matches_chemistry`: the helper value equals Na+ from `catalogue_dose_totals` × 22.98977 × 1000 for the same chelate doses.
- [ ] **Steps 2–4:** run, implement, run, then the full suite.
- [ ] **Step 5: Commit** with the message "Report chelate sodium counterions in formulation rows".

### Task 3: Chemistry at the band ends (F3)

**Files:** Modify `src/flahax/irrigation_ec.py`. Test: `tests/test_irrigation_ec.py` (class `BandChemistryTests`).

**Interfaces:** `calculate_irrigation_ec_band(doses_g_per_kg_water, *, materials=None, **same keyword args as calculate_irrigation_ec) -> dict` with keys `nominal`, `minimum` and `maximum`, each an `IrrigationECResult`, plus `note`: "Band ends are corner cases (every product at its minimum, or every product at its maximum), not statistical bounds."

**Rules:**
- **Nominal:** `calculate_irrigation_ec(...)` with the given materials.
- **Building an end:** for each dosed product, take its `ResolvedMaterial` from `materials`, or `resolve_material(name)` for shipped products. Build a copy of its record whose `elements` are the band end for every nutrient.
- **Open end:** if any band end needed is `None`, that end is an `IrrigationECResult` with status `not_supported`, reason `band_end_open`, all numbers `None`, and audit `band_end`.
- **Computing an end:** otherwise call `calculate_irrigation_ec` with the end materials and add audit `band_end`: `minimum` or `maximum`.
- **Error codes** from an end's call are mapped exactly as in `calculate_irrigation_ec`.

- [ ] **Step 1: Failing tests.**
- `test_shipped_only_band_ends_open` (Review Focus 3).
- `test_band_ends_bracket_ec`: a KNO3 grade with min/max plus a K2SO4 grade with min/max, in declared pure water, calculated mode. Expect min EC < nominal EC < max EC, and each audit's `band_end` is correct.
- `test_band_end_outside_domain` (Review Focus 4): doses near I = 0.1 so only the maximum end exceeds it.
- `test_note_text`.
- [ ] **Steps 2–4:** run, implement, run, then the full suite.
- [ ] **Step 5: Commit** with the message "Evaluate irrigation chemistry at nutrient band ends".

### Task 4: Exports, docs, verification

**Files:**
- `src/flahax/__init__.py`: export `calculate_irrigation_ec_band`.
- Regenerate `API_REFERENCE.md`.
- `docs/materials.md`: move the three follow-ups from "follow-up" to documented behaviour, with one runnable band-chemistry example and its real output.
- `CHANGELOG.md`: one `## Unreleased` line per follow-up.

- [ ] **Run the full checks:** API `--check`, `tests.test_release_hygiene`, full suite, `compileall`, `git diff --check`. All must pass.
- [ ] **Commit** with the message "Document and export catalogue follow-ups".
36 changes: 35 additions & 1 deletion docs/superpowers/specs/2026-10-18-traceable-catalogue-design.md
Original file line number Diff line number Diff line change
Expand Up @@ -196,7 +196,41 @@ File layout and JSON schema of identity, grade and lot records; the
validation error codes; the alias mechanism; how the band is attached to
existing result types.

## 12. Follow-ups (addendum, 2026-10-19)

The three follow-ups listed below are now in scope. These are designer's
calls, made for owner review.

**F1. Provenance on formulation results.** Each entry in `recommend()`'s
`salts` list gains `provenance`: `recordId`, `identityId`, `analysisBasis`
(the per-nutrient `basis` for resolved materials), `source` or the
document references, and `flags`. Shipped records take these from
`library.json`. Resolved materials carry them in `record['provenance']`,
which `resolve_material` now sets. A record with neither is recorded as
`analysisBasis: "unattributed"`, never guessed. The `recommend` result also
gains `identitiesSha256`. `forward()` stays a pure mapping of achieved
nutrients, because its inputs are the caller's own records. Documented, not
changed.

**F2. Chelate sodium in formulation rows.** After the fit, `recommend` adds
the D2 chelate sodium for the chosen doses to the `Na` row and to
`incidentalContributions`. It is computed with the same counterion rule as
the chemistry, treating dose g/L as g/kg water for this report only. It
never enters eligibility or the fit, so Na stays a restricted ion and
chelates stay eligible. A warning line names the sodium source.

**F3. Chemistry at the band ends.** `calculate_irrigation_ec_band(...)` has
the same inputs as `calculate_irrigation_ec`. It returns `nominal`,
`minimum` and `maximum` results, each a full `IrrigationECResult` with pH,
supersaturation warnings and EC.
- The minimum end sets **every** product to its band minimum, and the
maximum end sets every product to its band maximum. These are corner
cases, not statistical bounds, and the docs say so.
- If any dosed product has an open band end, that end returns
`not_supported` / `band_end_open`. It is never computed from a
substituted value.
- Each end's audit records which end it is.

## Related, out of scope

- Chemistry evaluated at band ends (pH, precipitation, EC).
- A methods-article section on traceable materials, after this ships.
6 changes: 4 additions & 2 deletions src/flahax/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -149,9 +149,11 @@
from .measured_water import BalancedWaterAnalysis, balance_water_analysis, carbonate_alkalinity
__all__ += ["BalancedWaterAnalysis", "balance_water_analysis", "carbonate_alkalinity"]
from .irrigation_ec import (IrrigationECResult, DeclaredPureWater, calculate_irrigation_ec,
calculate_irrigation_ec_from_plan, calculate_ec_from_totals, compare_measured_ec, assay_charge_residual)
calculate_irrigation_ec_from_plan, calculate_ec_from_totals, compare_measured_ec, assay_charge_residual,
calculate_irrigation_ec_band)
__all__ += ["IrrigationECResult", "DeclaredPureWater", "calculate_irrigation_ec",
"calculate_irrigation_ec_from_plan", "calculate_ec_from_totals", "compare_measured_ec", "assay_charge_residual"]
"calculate_irrigation_ec_from_plan", "calculate_ec_from_totals", "compare_measured_ec", "assay_charge_residual",
"calculate_irrigation_ec_band"]
from .materials import (DocumentReference, Grade, Lot, ResolvedMaterial, make_grade, make_lot,
resolve_material, nutrient_band, load_identities, identity_for_product)
__all__ += ["DocumentReference", "Grade", "Lot", "ResolvedMaterial", "make_grade", "make_lot",
Expand Down
Loading
Loading
0