8000 feat(ai): evidence-bound observations (partial #92) by shroominic · Pull Request #70 · shroominic/OpenGlucose · GitHub
[go: up one dir, main page]

Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations< 8000 /strong>
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 87 additions & 0 deletions docs/architecture/adr/0004-ai-evidence-safety-contract.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
# ADR 0004: evidence-bound AI observation contract

- Status: Proposed, pending independent R2 review
- Date: 2026-08-24
- Accountable privacy/safety owner: `@shroominic`
- Related: issue #39

## Context

AI output can be plausible but incorrect, unsafe, or based on health data that
the user did not expect to share. The previous foundation stored arbitrary
provider prose. It also described an Anthropic-compatible provider while always
serializing an OpenAI-style chat-completions request.

## Decision

OpenGlucose uses a fail-closed, versioned observation boundary:

1. Deterministic local analytics produce a MetabolicContextSnapshot of
aggregate-only EvidenceRef values. It contains no raw readings, journal
note text, identifiers, or API keys.
2. A generated observation is valid only when it is structured JSON matching
ObservationDraft version 2. The only accepted statement template cites one
known evidence value and one matching numeric claim. OpenGlucose renders
the visible number and unit locally; the provider cannot supply prose.
3. Unsupported fields, unsupported templates, inconsistent values or units,
and malformed output are rejected before persistence or display. This is a
declarative allowlist, not a keyword deny-list for medical or prompt-
injection language.
4. The shipped remote serializer is named OpenAI-compatible. Native Anthropic
Messages is not claimed or accepted until it has a separate serializer and
contract tests.
5. Provider metadata declares execution location, structured-output support,
availability reason, lo 8000 cale, model/version, runtime version, and resource
limits. The host must not represent unavailable on-device inference as
available.
6. A connection test uses a fixed synthetic request and never reads health
data or persists an insight. Real remote generation first shows the exact
endpoint and aggregate data categories, then needs user consent.
The confirmation creates a one-use receipt bound to the endpoint, model,
prepared aggregate snapshot, and categories before a provider call.
7. A valid insight persists statement-level evidence/numeric mappings plus
prompt, provider, model, and runtime provenance. It keeps the wellness
disclaimer. Public provider failures redact raw response content, and the
transport bounds response bytes before parsing.

## Threat model and controls

| Threat | Control in this decision | Residual risk |
| --- | --- | --- |
| Remote provider receives more data than expected | Aggregate-only snapshot, recipient/category disclosure, no redirects, explicit real-generation consent | Provider retention and legal basis need owner review |
| Prompt injection from journal content | The AI snapshot excludes free-text notes; the response grammar has no provider prose field | New context sources need the same review |
| Hallucinated statistics or citations | Typed evidence IDs, local canonical numeric/unit persistence, and local rendering | A selected aggregate can still be a weak observation |
| Medical, dosing, treatment, or emergency guidance | Declarative evidence-only statement allowlist; adversarial free-prose fixtures fail closed | This is not a substitute for human R2 safety review |
| Unreviewed or replayed outbound request | One-use receipt bound to endpoint, model, exact snapshot, and categories | UI/device confirmation semantics need independent review |
| Oversized or reflected provider error | Bounded response bytes and redacted public exceptions | Provider availability and service terms remain external risks |
| API-key disclosure | Platform secure storage, no preference serialization, no logging in the contract | Platform/device compromise is outside this contract |
| Misleading provider claims | Explicit OpenAI-compatible name and capability metadata | Native provider adapters require independent tests |

## Rollback

Set AI disabled, remove the stored API key, and stop calling the provider. New
insights are local records and can be deleted through the repository contract.
The complete product delete/export UI, retention policy, and device-level
verification remain separate work. No remote automatic fallback exists.

## R2 local evidence

The checked-in redacted consent-dialog screenshot is generated by the widget
test fixture at `openhealth/test/ai/ai_settings_test.dart`. It uses synthetic
readings and the test key `sk-preview`; the visible dialog contains no glucose
values, user identifiers, or secret. The same test checks the dialog's
semantics label contains the recipient endpoint, the disclosed aggregate
categories, and the raw-reading exclusion.

![Redacted remote-generation consent dialog](ai-remote-generation-disclosure-redacted.png)

## Consequences and follow-up

This decision intentionally adds validation friction and can reject a provider
response that was otherwise readable. That is preferred to persisting an
unsupported health claim.

This partial implementation does not close issue #39. Remaining acceptance
work includes an approved data receipt, verified full export/delete behavior,
exact OS-runtime capture in the host, expanded adversarial evaluation, and
independent R2 review.
1 change: 1 addition & 0 deletions docs/architecture/adr/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ enforcement detail is already implemented.
| [0001](0001-package-boundaries.md) | Accepted | Separate domain, BLE transport, vendor protocol, platform adapter, and app composition |
| [0002](0002-local-first-health-data.md) | Accepted | Keep health data local by default and make movement explicit |
| [0003](0003-platform-release-model.md) | Accepted | Build source-bound mobile artifacts and release only through fail-closed platform lanes |
| [0004](0004-ai-evidence-safety-contract.md) | Proposed | Require typed evidence-bound AI observations and explicit remote-data consent |

## Adding an ADR

Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions docs/engineering/standards.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ unrecorded failed-check bypasses are not waivers. Expired waivers block release.
| Dependency/secret scanning | Supply chain | Enforced in repository; external execution unverified | `.github/workflows/security.yml` and `.github/dependabot.yml` | `@shroominic` | — |
| Private vulnerability intake | Security reports | Enabled and verified | GitHub repository API reported the setting enabled 2026-08-14 | `@shroominic` | — |
| Accessibility verification | Mobile UI | Planned | No established semantics/device lane | `@shroominic` | Before production-readiness claim |
| AI remote-generation boundary | Restricted health data/AI | Contract and local consent tests enforced; independent R2 review pending | `docs/architecture/adr/0004-ai-evidence-safety-contract.md`, redacted disclosure evidence | `@shroominic` | Before public distribution |
| Protected branch/required checks | Integration | Unverified external | Requires repository-admin evidence | `@shroominic` | Before merging the feature train |

Update this register as controls become enforced, verified, waived, or no longer
Expand Down
41 changes: 31 additions & 10 deletions docs/privacy/data-handling.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,9 @@
OpenGlucose is local-first, not data-free. This document separates required
policy from the checked-in implementation status and outstanding verification.
Platform store disclosures and applicable legal obligations require an assigned
privacy owner before public distribution.
privacy owner before public distribution. The accountable privacy/safety owner
for the AI remote-generation boundary is `@shroominic`; independent R2 review
is still required before public distribution.

## Classification and flows

Expand Down Expand Up @@ -128,12 +130,31 @@ review. Follow

## AI

AI is off by default and requires a user-supplied key. The current Generate
action sends a 24-hour aggregate to the HTTPS provider URL shown in settings:
reading count, average, range, standard deviation, time in/below/above range,
estimated A1c, meal/exercise/note counts, and total logged carbohydrates. It
does not send raw readings or note text. The UI discloses those categories and
that provider retention terms apply. Redirects, credentials in URLs, query
parameters, and non-HTTPS endpoints are rejected. Before public distribution,
the prompt/model/policy still requires representative medical/dosing
adversarial evaluation and an accountable privacy/safety owner.
AI is off by default and needs a user-supplied key held in platform secure
storage. The shipped remote path is OpenAI-compatible chat-completions only;
it does not claim native Anthropic Messages support.

The connection test sends only a fixed synthetic request. It does not read
health data and does not persist an insight. A real generation first shows the
exact recipient endpoint and these exact possible categories: time window, aggregate
glucose statistics, journal event counts, and total logged carbohydrates when
present. It does not send raw readings, identifiers, or journal note text. The
confirmation creates a one-use receipt that is bound to the endpoint, model,
exact prepared aggregate snapshot, and categories; a changed or reused receipt
cannot start a provider call. The remote provider's retention terms apply after
the user confirms.

The generated output must use a typed, evidence-only JSON declaration.
OpenGlucose renders visible values and units from the local structured claim;
the remote provider cannot supply free-form display text. Unknown citations,
unsupported fields/templates, and inconsistent numeric values or units are
rejected before persistence. A stored validated insight records statement-level
evidence/numeric mappings and prompt/provider/model/runtime provenance. Public
provider errors do not include raw provider output, and response bytes are
bounded before parsing. Redirects, credentials in URLs, query parameters,
fragments, and non-HTTPS endpoints are rejected.

This is not a production-safety claim. Full delete/export verification, exact
host OS-version capture, approved data receipts, expanded adversarial
evaluation, and independent R2 review are still required before public
distribution.
Loading
Loading
0