8000 feat(cbio): stream GS1 history and live readings into the app by shroominic · Pull Request #142 · shroominic/OpenGlucose · GitHub
[go: up one dir, main page]

Skip to content

feat(cbio): stream GS1 history and live readings into the app - #142

Closed
shroominic wants to merge 11 commits into
feature/cbio-gs1-glucose-livefrom
feature/cbio-gs1-app-live
Closed

shroominic wants to merge 11 commits into
feature/cbio-gs1-glucose-livefrom
feature/cbio-gs1-app-live

Conversation

@shroominic
Copy link
Copy Markdown
Owner

Stacks on feature/cbio-gs1-glucose-live. Refs #135, #137.

What this proves on hardware

The installed debug app on the Pixel 6 reaches the live sensor screen and shows
both halves of the goal: a live value and fetched history.

Surface Rendered
Hero 5.3, no unit suffix, stage pill Live
Hero subtitle Sensor raw value · index 10067
Stored history 10067 readings stored · sensor minutes 1–10067
Provisional marker Provisional reading. Sensor raw / 10, scale unverified.
History card 10067 readings, same marker, Sensor history complete. 1 records.

Screenshots: /tmp/og-cbio-live-raw-800.png (current) and
/tmp/og-cbio-live-800.png (earlier capture that still claimed mg/dL).
adb logcat shows [FBP] onCharacteristicChanged: chr: ff31 roughly every
500 ms while that screen is up. Reproduction is in
docs/testing/cbio-gs1-app-live.md.

Unit and time provenance

  • The 0x08 record's raw counter is shown divided by ten (raw 56 → 5.3).
    The capture harness read raw 47..53 from the same field at index
    9940..9992, i.e. 4.7..5.3; app and harness agree on field and scale.
  • No mg/dL or mmol/L claim is made anywhere the value is rendered: the model
    keeps isUnitVerified = false and the marker travels with every number.
  • The sensor's second counter has no epoch, so readings no longer carry an
    invented wall-clock timestamp; positions are the protocol's index counter.

Verification

  • packages/cgm_cbio: flutter analyze clean, flutter test 75 passed.
  • openhealth: flutter analyze clean, flutter test 480 passed.
  • Device run above, on the installed com.openglucose.app.debug build.

Depends on

The nearby-sensor scan in this repo hangs forever because the
FlutterBluePlus transport never closes its scan stream; the focused fix is in
its own PR. This branch assumes discovery completes.

The GS1 driver now owns one authenticated session: connect, subscribe on
FF31, authenticate, ingest the pushed 08 history, then poll the live edge
and publish every reading as an explicitly provisional raw record.

App surface changes:
- the nearby-sensor list can finally complete, so a real sensor is
  selectable from the dashboard flow,
- the hero value renders the sensor's raw field / 10 with no glucose unit,
- the hero reports how many records were stored and their sensor-minute
  range, so the fetch is verifiable without opening a chart,
- the provisional marker is on every rendered number and reads "sensor raw
  / 10, scale unverified".

Reading positions come from the protocol's own index counter; the sensor's
second counter has no epoch, so no wall-clock timestamp is invented from it.

Evidence, screenshots, and the reproduction steps are in
docs/testing/cbio-gs1-app-live.md.
@shroominic
Copy link
Copy Markdown
Owner Author

Correctness items raised against this work are now tracked separately, and this PR carries their fix:

Screenshots for the current screen: /tmp/og-cbio-live-raw-800.png (raw/10, no unit, 10067 readings stored) and /tmp/og-cbio-live-800.png (the earlier capture that still claimed mg/dL).

@shroominic
Copy link
Copy Markdown
Owner Author

Cross-lane gate note from the release assembly (aggregate #163 @ 212237c).

Merging this lane into the aggregate together with the open hardening PRs leaves the tree non-compiling, and the break is on this lane. Two independent causes, both mechanical:

  1. fix(cbio): require injected vendor material instead of compiled constants #166 (injected vendor material, tip 7fa477c) removes cbioVendorAuthMaterial and makes the buildMasked* / unmaskCbioFrame key and material arguments required. packages/cgm_cbio/lib/src/cbio_glucose_session.dart (added by this lane) still reads the removed constant and calls those builders without a key: 10 analyzer errors, including lib/src/cbio_glucose_session.dart:82, :341, :368, :437, :461, :525, :603, :671, plus lib/src/cbio_driver.dart:72. test/cbio_driver_test.dart:53,54 and test/cbio_glucose_session_test.dart:161,178,306,347,351,375,397,423,637,660,705,783,787 follow.

  2. fix(cbio): drop the unverified mg/dL claim from the raw record #168 (drop the unverified mg/dL claim) removes the derivedMillimolesPerLitre / derivedMilligramsPerDecilitre getters from CbioRawGlucoseRecord. This lane still consumes them: lib/src/cbio_glucose_session.dart:755 feeds CgmReading.valueMgdl from the derived getter, and test/cbio_glucose_session_test.dart:838,841,842,845,846 assert on them.

I am not editing this lane's files. The aggregate currently carries an explicit revert of both merges so the assembled candidate compiles and CI can run; the revert is recorded, not a silent drop. The aggregate cannot absorb #166/#168 until this lane adopts the injected CbioCredentialSource and decides how the unverified value reaches the app surface. Please land that adoption (a commit on this lane or a new focused PR) and I will re-merge and re-run the full gate set.

Also: this lane's openhealth/test/cbio_app_surface_test.dart (a53b00f) was committed unformatted; format-check stops before lint there. The aggregate carries the formatter output in a separate commit; the lane still needs it.

@shroominic
Copy link
Copy Markdown
Owner Author

Cross-lane note: the app-live branch no longer compiles against the redacted vendor API, so #192 carries the adoption (fix/cbio-credential-source-adoption, based on this branch's head 17db5b4). It contains #166 verbatim plus one adoption commit; a rebase of this lane onto a post-#166 feature/cbio-gs1-glucose-live will see byte-identical content, not a conflict.

One reviewable decision is flagged in #192: the platform registry now registers the GS1 driver only when the build supplied the vendor material, so registeredDriverIds is {aidex} by default instead of {aidex, cbio}.

…ants

The vendor GS1 stream key, link credential, and authentication prompt were
plaintext constants in packages/cgm_cbio/lib/src/cbio_crypto.dart. The
repository is public, the file is on the 0.4.0 release aggregate, and the
Secret Scan job is red on the same lane for a fixture derived from it.

Remove the constants and make the material an explicit input:

* cbio_credentials.dart adds CbioCredentials (validated, immutable, redacting
  toString), CbioCredentialSource, CbioMapCredentialSource,
  CbioDefineCredentialSource, and the fail-closed CbioCredentialUnavailable.
* cbioRc4Keystream, maskCbioFrame, unmaskCbioFrame, and every builder in
  cbio_vendor_frames.dart now take the key and material as required arguments,
  so a caller that has not resolved material cannot compile a call.
* the device harness reads the material from the process environment or
  --dart-define and aborts with vendor-material-missing before touching radio.
* tests use clearly synthetic fixtures; the real-material checks skip unless
  the values are injected.
* .gitleaks.toml adds a rule for this exact class of literal and a narrow
  allowlist for synthetic fixtures only.

Refs #164

(cherry picked from commit 7fa477c)
…ource

The live session still called the pre-redaction crypto API: `maskCbioFrame`,
`unmaskCbioFrame`, and every masked builder defaulted to compiled constants,
and the session read its link credential from `CbioCompiledAuthMaterialProvider`.
Those defaults are gone, so the lane has to resolve the material explicitly.

- `CbioGlucoseSession` takes a `CbioCredentialSource` and resolves it once per
  session, before the transport is touched. An unconfigured build fails closed
  with `cbio.auth.material` and a configuration status text instead of opening
  a link it could never authenticate or unmask.
- Every mask, unmask, and masked builder receives the resolved key explicitly,
  and the authentication frame receives the resolved material.
- `CbioSensorDriver` exposes its source and a `canAuthenticate` predicate.
- The app's platform registry registers the GS1 driver only when the build
  supplied the material, so a plain build does not offer a sensor it cannot
  read. The capture harness reads the material from the environment or
  `--dart-define` and reports `vendor-material-missing` when it is absent.

Tests: cgm_cbio 81 passed (2 skipped), openhealth 480 passed; `flutter analyze`
clean in both, including `openhealth/integration_test`.
@shroominic
Copy link
Copy Markdown
Owner Author

Lane head moved: #192 landed as a rebase merge, so feature/cbio-gs1-app-live is now 05785b0 (a7c21bc + 05785b0) on top of 17db5b4. The lane is on the injected-credential path.

17db5b4 is still an ancestor, so the five open lane PRs (#175, #177, #182, #186, #188) keep their merge-base; they will need a rebase before landing, and each touches cbio_glucose_session.dart where #192 touched it, so expect conflicts in the credential-resolution hunk and resolve them by keeping the resolved _resolved/_streamKey form.

Verified on the new head in a clean worktree: cgm_cbio +81 (2 skipped), openhealth +480, flutter analyze clean in both including integration_test. CI reds are unchanged from #142: Quality (format-check), Unit Tests (private-bench), Integration Tests (host/device roots) - all pre-existing.

shroominic and others added 8 commits September 18, 2026 16:24
…ishes

#146 recorded a +7h28m skew on screen: the hero read `Latest reading at 09:04`
and the 12 h axis ended at `09:01` while the phone clock read `02:34`. The
counter advances 60 s per stored record, so 448 stored records rendered as
7 h 28 m of apparent elapsed time.

The rendering half of that fix lives in the app-live branch: the session
publishes `recordedAt: null` for these records, the hero reports the protocol
position (`Sensor raw value · index 10067`), and the chart's null path renders
`m<minute>`. Nothing on the lane pins that, so this adds the guard:

- the removed counter-to-clock formula is asserted to be the on-screen skew
  (448 x 60 s == 7 h 28 m), so the arithmetic cannot be re-derived as harmless;
- a GS1 reading is asserted to carry no `recordedAt`, and the hero's only time
  formatter is asserted to refuse to invent one;
- the real dashboard is rendered with those readings and asserted to show the
  protocol position with no `HH:mm` anywhere in the hero, and to hand the chart
  only timestamp-free readings.

Mutation check: reinstating
`recordedAt: DateTime.fromMillisecondsSinceEpoch(record.rawTime * 1000, isUtc: true)`
in `CbioGlucoseSession._historyReadings` fails
`cbio_glucose_session_test.dart` with `Expected: null / Actual: DateTime:<...>`.

Tests: openhealth 483 passed; `flutter analyze` clean; `dart format` clean on
the added file.
The GS1 surface published no time at all: the `08` counter is epoch-less, so
#146/`#193` forbid rendering a wall clock from it. That left the hero with
`Sensor raw value · index 10070` and nothing else, and any surface that did ask
for a time showed a placeholder.

The one reference the app establishes itself is the sensor clock it writes
once per session. `CbioIndexTimeAnchor` turns that into absolute time honestly:

- an anchor exists only when the session wrote the clock and the sensor's own
  newest record stamp agrees with the app's clock within
  `cbioAnchorTolerance` (3 min);
- the anchored position carries its own stamp, and every older position steps
  60 s per index, as the vendor layout reports the counter (`base + 60 * i`);
- the range the anchor speaks for ends at the first hole or counter jump, so
  positions the archive cannot support keep no timestamp;
- the anchor travels in the snapshot metadata (`anchorIndex`,
  `anchorEpochSeconds`, `anchorCoveredFrom`, `referenceEpochSeconds`) and the
  diagnostics name it, so provenance is readable without the radio.

The counter is never the source: a sensor that did not take the written clock
(the 7 h 28 m skew of #146) publishes no anchor, and the hero says
`Sensor clock unsynced · ordered by sensor index, not by clock` instead of a
placeholder or an invented minute. With an anchor the hero reads
`Sensor clock set by this app · latest 10:05 (±1 min)`, localised by the device.

Refs #146, #193.

Tests: cgm_cbio 91 passed (2 skipped), openhealth 485 passed; `flutter analyze`
clean in both, including `openhealth/integration_test`.
Adopt the archive fix from `fix/cbio-archive-unit-claim` (#168, e81e335) into
the app-live lane. The record used to expose the raw `08` field three ways:
`rawCurrent`, a derived mmol/L accessor (`rawCurrent / 10`) and a derived mg/dL
accessor (`rawCurrent / 10 * 18.0182`). Nothing in this repository establishes
that either unit belongs to the field, so the record now exposes `rawCurrent`
and `rawCurrentScaled` and no unit-bearing accessor at all.

The six `cgm_cbio` analyzer errors that blocked the adoption were the session
and its tests still reading the removed accessors:

- `_historyReadingsFor` published the mg/dL derivation into
  `CgmReading.valueMgdl`; it now publishes `rawCurrentScaled`, the same
  unit-free `/10` scale the hero already renders.
- the session tests asserted the removed accessors; they now pin the scale
  itself and the number the session publishes (`raw 97 -> 9.7`).

This changes the number a cbio record carries, so it changes what the dashboard
chart plots (`raw 53`: `95` before, `5.3` now). The hero is unchanged - it
already rendered `rawValue / 10` with no unit - and provisional readings stay
excluded from Apple Health, so no surface claims the number as mg/dL.
Documented in `docs/testing/cbio-gs1-app-live.md`.

Refs #168, #145
@shroominic
Copy link
Copy Markdown
Owner Author

Carried by the GS1 consolidation in PR #209 (branch feature/cbio-gs1, head 3eca4f4).

This PR is contained in the consolidated line; the merge that carries it is 128d952.

Closing as subsumed by the consolidation - not as won't fix. The per-defect evidence stays in this thread, and the carrying commit above is where the change now lives.

@shroominic shroominic closed this Sep 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
0