8000 Add CBIO GS1 sensor integration by shroominic · Pull Request #209 · shroominic/OpenGlucose · GitHub
[go: up one dir, main page]

Skip to content

Add CBIO GS1 sensor integration - #209

Open
shroominic wants to merge 178 commits into
mainfrom
feature/cbio-gs1
Open

shroominic wants to merge 178 commits into
mainfrom
feature/cbio-gs1

Conversation

@shroominic
@shroominic shroominic commented Sep 18, 2026 •
Copy link
Copy Markdown
Owner

Current status — 2026-09-22

Published source head: 7bc307c750fb5756a3cc9bbabeca15f8e6c65a2b on the existing
feature/cbio-gs1 branch. This section supersedes current-head, verification,
and readiness wording below; the historical body remains preserved for
traceability. Source publication to this PR branch does not publish a mobile or
release artifact.

The reviewed aggregate at commit
4560dc1a832629226649b7d31f87024ba0ca149d records 369 CBIO tests passed with
2 skipped, 188 N5P tests passed, 765 app host tests passed, and one bounded
host-only integration test passed: 1,323 passed and 2 skipped in total. Its
three analyzers are clean, and its normal and gated-N5P Android debug builds
pass. This is aggregate verification evidence, not a standalone GS1 app-suite,
physical-device, production-artifact, or release proof. The debug APKs are
compile evidence only.

GS1 remains not production-ready. Current identity and firmware, verified
normalized glucose decoding, complete history, lifecycle/reconnect/restart and
screen-off behavior, normal-app physical/UI and handoff evidence, physical
acceptance, and a source-bound production artifact with signing, distribution,
and release publication approval all remain open.

Status: work in progress, not release-ready

Single GS1 integration carrier for issue #208 on feature/cbio-gs1, targeting main. Current reviewed source: 8733e96fbcbe521a8b2bee47b8504aee621b354c. No main merge, release or binary publication is requested by this update. Verified GS1 glucose decoding is still missing.

The Libre2/Anytime multi-sensor base 821f9a34bfc8af12e6a31438b88b6ed75ca370ee remains an ancestor. This does not imply inclusion of all later work on other branches.

Current implementation boundary

  • GS1 uses the unchanged shared normalized AiDEX/Libre2 hero, chart, Patterns, settings, lifecycle and archive presentation. GS1-only raw-value, support-reference and recovery UI has been removed. Existing Libre-specific safeguards remain.
  • Raw acquisition, clock/checkpoint state and resume-witness proof are driver-private. All public reading fields, including rawHistory, are normalized-only. Until a verified decoder exists, actual GS1 latest is null, history/rawHistory are empty, normalized counters are zero/unknown and decoded/raw history capabilities are false. No unverified advertisement glucose fallback is admitted.
  • Driver-owned raw-v1 envelopes keep the existing sensor binding and format. A restricted adapter durably preserves original legacy blobs/indexes and archive descriptors in a private migration manifest before changing public routes. Shared normalized history uses a separate namespace. Malformed or colliding state fails closed without deleting original bytes.
  • Generic target preparation and durable old-state flushing protect handoff, retry and supersession. Raw ownership is not in shared controller consumers; normalized GS1 fixtures follow the same analytics/lifecycle policy as other sensors. Actual GS1 start/elapsed are null, so the shared UI remains honestly unknown.
  • Authenticated BLE acquisition retains paced single-flight polling, bounded individual operations, exact checkpoint-witness admission and terminal-state protections. No reset, activation, calibration, threshold, key-registration or firmware command is enabled. No witness-equality guard was relaxed.

The raw integer and its historical /10 representation are not verified mg/dL or mmol/L. Empty normalized output is a safety boundary, not a completed glucose integration.

Fresh local verification

Combined code at 816e8a6 plus independently reviewed factual docs at 8733e96:

Gate Evidence
Full app tests 694 passed
CBIO package tests 207 passed, 2 existing real-material skips
Analysis / formatting App and package analyzers clean; 132 app and 34 package Dart files format-clean
Independent source review Private storage/driver boundary, controller cleanup and renderer accepted; factual compatibility/storage/changelog docs independently reviewed
Visual contract evidence 144 Playwright matrix screenshots plus 12 screenshots from six current-details/archive navigation flows; zero page errors
Security Updated 23-commit range secret scan clean; vendor-material tree/canary guard passed

Visual source e46ccd9 is integrated as 61392b1; final controller cleanup adds neutral lifecycle policy and private metadata projection without changing renderer files. Screenshots cover AiDEX/Libre2/GS1 synthetic inputs, EN/ZH, mg/dL/mmol/L, normal/empty/error/stale/warmup and details/archive. They do not establish actual GS1 decoding, phone operation or release readiness. Inherited narrow hero overflow and Chinese timeframe-segment clipping are documented shared-layout limitations, not claimed fixed.

The first full pre-restoration run exposed a Yuwell test-fixture scheduling issue: a const empty-stream cancellation future crossed fake_async zones. A test-only real-zone drain fixed it with original timeout/security assertions retained; no production radio workaround was added.

Native and physical evidence remain separate

Native builds and hosted checks for this new head are pending. Earlier green CI at ac016e3 and older heads is historical evidence only, not validation of 8733e96. No new local native build is claimed for this source update.

The phone still has prior private normal-release build 34 and its observed GS1-H03B checkpoint-time-witness mismatch. That observation does not prove a physical reset. Earlier build 32 acquired raw history but did not demonstrate reliable completed live/history operation. Shared UI restoration and private-state containment do not fix or explain the physical witness mismatch. This source has not been installed or demonstrated on the phone.

A private bounded clock-witness probe is separately planned/reviewed; it is not a production feature or authorization to issue new radio commands. A separate synthetic ABI experiment failed closed without positive ABI evidence; it contributes no glucose, calibration, model or licensing claim and no vendor implementation is added here.

Release gates still open

  1. Verified glucose decoder, per-sensor sensitivity/calibration requirements and independent target/reference evidence.
  2. Exact model/software admission and safe unsupported-revision handling; FF30 alone cannot distinguish GS1 from GS3.
  3. Actual sensor lifecycle evidence; shared unknown presentation is honest but not sensor-derived activation/warmup/expiry verification.
  4. Vendor-material/algorithm distribution clearance, configured artifact/signing provenance and artifact review. Build-time configuration is not secure runtime provisioning or distribution permission.
  5. Current-source native builds and real-phone proof: sustained live freshness, complete history, restart, reconnect and screen-off operation, with preserved user data and no destructive sensor commands. The observed witness-time mismatch remains unresolved.

One existing open PR is retained. No new GS1 carrier PR, main merge, force push or release is part of this update.

Snapshot the local libre-protocol-capture worktree so contest cloud agents
can branch from a shared remote: cgm_driver_registry, cgm_libre2,
cgm_libre2_glucose, cgm_yuwell_anytime, debug protocol-capture harness,
and Anytime 5P passive-capture runbook. No raw captures or health data.
Do not await the in-flight connect before the promotion gate releases, and
type ControlledSession.disconnectError as Exception? for only_throw_errors.
Add a live Cbio/SIBIONICS GS1 session on top of the existing offline frame
contract: discover the FF30 service, connect, enable FF31 notifications, and
decode every received frame with parseCbioPlaintextFrame.

The session stays fail-closed. The only write it can emit is the bounded
`06 08 01 00 00 00 F1` read the protocol evidence supports, and only after a
passive window with no notification. Activation (0x07), clock (0x03), reset,
threshold, and calibration writes are never issued; calibration submission
reports unsupported instead of attempting a write.

Register the driver in the platform registry behind its own pure
advertisement classifier so an FF30 candidate routes to the FF30 driver, and
declare the package dependency plus the device-test dev dependencies the
capture harness needs.

Verified: packages/cgm_cbio `dart test` 24/24, `dart analyze` clean,
openhealth driver registry/connection-flow tests green.
Add an integration_test entry point that drives the production
FlutterBluePlusTransport through RecordingBleTransport and the unmodified
CbioSensorDriver against the live sensor, without going through the Nearby
sensors UI. Every phase is bounded, so a silent radio cannot hang the harness,
and all evidence lands in the app-private trace sink that already backs the
protocol capture lane.

Discovery runs three ways: the driver's own FF30-filtered scan, an unfiltered
transport scan, then a bounded raw-plugin acquisition loop that waits for an
intermittent advertisement. The raw path also primes the platform's address
type for the peripheral. An address recovered from advertisement evidence can
be supplied at run time via --dart-define=CBIO_TARGET_DEVICE_ID so a capture is
possible while the Dart scan surfacing defect is open.

The harness asserts the safety contract from the trace itself: at most one
write, and that write must be the allowed 0x08 read. It prints only redacted
identifiers and never prints payload bytes.

Pin the debug-only androidx.test espresso artifacts that the integration_test
package's pinned 3.2.0 release makes ambiguous for AGP 8 namespace validation.
The platform registry now registers the Cbio driver alongside AiDEX, so the
OG_DEMO selection test's expected driver set has to include it.

Verified: openhealth `flutter test --exclude-tags integration` 473/473.
Android hands the app short-form UUIDs (FF30/FF31/FF32) while the driver
declared the full 128-bit base form, so _locateCharacteristics never
matched the FF30 service on a live link and every real session failed
closed with missingNotifyCharacteristic.

Canonicalise both sides onto the Bluetooth base UUID before comparing.
Observed on the GS1 sensor: connect ok, service FF30 with FF31 (notify)
and FF32 (write), and the previous code produced notify.enabled=[].
The first live FF31 packet is five bytes whose leading byte is the size a
full frame would need, so disconnecting on the first packet could drop
every later fragment. Linger for six seconds after the first packet.
FlutterBluePlusTransport.scan's stream can stay open when flutter_blue_plus stops the physical scan without emitting isScanning == false, so the discovery loop never returned and the Connect-a-sensor panel kept scanning forever.

The registry now owns the scan deadline (_activeScanTimeout + _completeScanAtDeadline) so an overrun scan still ends for every consumer, bounds transport cancellation so a wedged cancel cannot hold the lifecycle open, and keeps an assert-gated, identity-free scan diagnostic for debug builds.
The single bounded 08 read on the live SIBIONICS/CBio GS1 sensor returns one
five-byte FF31 notification, byte-identical across every recorded session.
Add a read-only structural inspection that reports which plaintext hypotheses
those bytes survive, with tests for the exact contract, orientation, constant
mask, and truncation searches.

The captured payload is not a complete plaintext frame: byte zero declares 36
bytes rather than 5, the additive sum is 0x56 rather than 0, byte one is not a
mapped opcode, and no rotation, reversal, or constant mask exposes a valid
frame. No decode is produced and none is claimed.
A connected sensor whose frames never decode parked setup on "Syncing
sensor history" forever: the Cbio session enables FF31 notifications, one
five-byte reply fails the offline parser with reason `length`, and the
passive timer has already been cancelled, so no further state arrives.

Sessions now arm a 45-second bounded wait for the first readable result
while they sit in the syncing stage and have no decodable data. When the
deadline elapses the host publishes a terminal error snapshot carrying
`cgm.session.syncStalled`, blocks automatic reconnect, and the connect
screen shows a next action instead of a spinner. Drivers that own their
terminal outcome (Libre 2 phases) or an in-progress history sync are left
untouched, and the Cbio driver's fail-closed write discipline is
unchanged.
Add the vendor V120 read builders and a strict 0A glucose decoder, plus a
transport-free sync session that polls the newest index and pages stored
history under explicit budgets. Every failure path returns a status instead of
a measurement, records stay raw with isUnitVerified false, and the 08
raw-data layout is refused.

The device-backed harness sends only the documented read queries
(03 F0 04 09, 03 F0 03 0A, 06 0A <index> 00 00 C). On the live sensor all
four reads return the same five-byte frame 23 F7 6F D9 F4, which fails the
plaintext contract, so the capability currently terminates in decodeFailed.
No activation, clock, reset, threshold, key, or authentication write is sent.
The vendor storage, time, and glucose read queries all return the same five
bytes, so the reply is not a per-command answer and the in-flight fragment
reading is refuted for this path.
…tory

The live sensor answers the vendor's own read queries once the link is
authenticated with the vendor's per-frame stream mask, and the answer is its
stored record stream: 9981 raw records, one per minute, contiguous from
2026-09-10T19:57:00Z to 2026-09-17T18:17:00Z on the last run.

- cbio_crypto.dart carries the vendor stream mask and the frame transforms.
  The five-byte prompt the sensor sends unmasks to the valid control frame
  `04 00 00 00 FC`, which is what identified the encoding.
- cbio_vendor_frames.dart builds the masked link frames the sensor accepts:
  device information, authentication, clock, and the two data reads. No
  activation, reset, threshold, key-registration, or firmware frame.
- cbio_history_archive.dart assembles the raw `08` stream into an ordered
  archive with one record per index, plus explicit gap, 
8000
overlap, and
  contiguity reporting.
- The authenticated harness sends the vendor's masked sequence, records every
  write byte-for-byte in masked and plaintext form, and resumes the raw read
  at a given index so a bounded window can reach the newest record.

Proven on the sensor: authentication succeeds with the 2A25 address octets as
read (`result=1 status=0`); the `0A` path holds 1782 records that are all zero
and stop at 2026-09-12T01:38:00Z; the `08` path streams to the present with
raw `current` 39..97 and `temp` 274..460. Derived values stay marked
`isUnitVerified == false` because no reference measurement confirms the scale.
Additive-only conflict resolution in packages/cgm_cbio: the barrel file,
CHANGELOG, and README keep both lanes' content (union). No lane content was
dropped or rewritten.
Lane #136 (7f5888d) committed integration_test/cbio_glucose_query_test.dart
without running the repository formatter, so `make format-check` failed on the
aggregate. This commit is formatter output only: no token, literal, or control
flow was changed.
The GS1 driver now owns one authenticated session: connect, subscribe on
FF31, authenticate, ingest the pushed 08 history, then poll the live edge
and publish every reading as an explicitly provisional raw record.

App surface changes:
- the nearby-sensor list can finally complete, so a real sensor is
  selectable from the dashboard flow,
- the hero value renders the sensor's raw field / 10 with no glucose unit,
- the hero reports how many records were stored and their sensor-minute
  range, so the fetch is verifiable without opening a chart,
- the provisional marker is on every rendered number and reads "sensor raw
  / 10, scale unverified".

Reading positions come from the protocol's own index counter; the sensor's
second counter has no epoch, so no wall-clock timestamp is invented from it.

Evidence, screenshots, and the reproduction steps are in
docs/testing/cbio-gs1-app-live.md.
FlutterBluePlusTransport.scan returned a stream that outlived the scan.
flutter_blue_plus stops the radio at its own timeout, but the transport's
cleanup awaited cancelResults, cancelScanning, awaitPendingStart, and
stopScan before closeController. On a stack that leaves any of those
pending, controller.close() never ran, so callers waited forever and
cancelling the subscription hung for the same reason.

- close the caller's stream first in closeFlutterBluePlusScanResources,
- bound each cleanup step so a pending plugin future cannot hold a caller,
- arm a scan deadline so a scan still closes when the plugin stops the
  radio without publishing isScanning = false,
- cover it with a regression test that drives cleanup with futures that
  never complete.

Measured before the fix on a Pixel 6: a 6 s window left isScanningNow
false at ~7 s while the transport stream was still open at 12 s, and a
25 s timeout on the stream fired, while the plugin's raw result stream
delivered 429 advertisements in the same period.

Fixes #143.
The Secret Scan gate fails on this lane with one gitleaks `generic-api-key`
finding in `packages/cgm_cbio/test/cbio_crypto_test.dart`. The value is a
published synthetic masking vector, but it is a 50-character high-entropy
string assigned to a name containing "auth", which is exactly the shape the
detector is built to catch.

The five masking vectors are now byte lists. The vectors and the assertions are
unchanged; `dart test` still passes 60 tests, and the same package tree now
reports no leaks to gitleaks 8.24.3.
`make format-check` fails on this lane in the Quality job: the new
`openhealth/integration_test/cbio_glucose_query_test.dart` was committed
unformatted, so the CI gate exits 2 before lint runs.

This is the formatter's own output, nothing else: `dart format` reports one
changed file and `make format-check` then exits 0 across every project.
`make test-integration` handed `test/integration/*` and `integration_test/*`
to one `flutter test` call, which Flutter rejects:

  Integration tests and unit tests cannot be run in a single invocation.

The two roots now run as separate lanes. The host lane keeps its behaviour
(`test/integration/*` plus tag-marked files, and `integration_test/*` for plain
Dart packages). The device lane runs `integration_test/*` in its own
invocation and only against hardware named by `FLUTTER_TEST_DEVICE_ID`;
otherwise it reports the deferred files and the knob that enables them, so a
host without a device stays green instead of failing closed on a lane it
cannot run.

Adds scripts/test-workspace-integration-runner.rb, which drives the real
script against a fixture repository with stubbed Flutter tooling and pins the
lane split, the deferral notice, the device-id plumbing, and the existing
nothing-found diagnostic. The check runs from `make tooling-check`.

(cherry picked from commit d8c9acb)
@shroominic
Copy link
Copy Markdown
Owner Author

Source-progress update: private full-input preservation

Reviewed source: 6fc01ae6c28e345862f91425e84f56193ef3f584 (five commits after
8733e96). This remains the existing GS1 issue/PR; it is not release-ready.
Risk: R2, restricted device/health data and restart correctness.

  • Added optional opaque CbioFullRecordStore; the app uses its existing
    restricted atomic history-blob store and existing SHA256 dependency.
  • Preserve all seven observed raw08 fields privately in one pending/observing
    envelope with its current checkpoint. Original raw-v1 bytes/checkpoint stay
    frozen; no invented missing fields, dual writes, or legacy fallback.
  • One acquisition lineage, strict binding/witness/full-word equality and
    first-observed reindex provenance, read-only preparation, old-owner durable
    drain, and pending adoption committed before BLE. Existing in-memory target
    selection and later verified durable selection policy are unchanged.
  • Fixed row/byte bounds, no truncation/rotation/backfill, fail-closed storage
    errors and retryable dirty candidates. Native recovery preserves a complete
    old or new envelope, never mixed rows/checkpoint.
  • No shared UI, controller, core, probe, normalized glucose, or protocol-command
    expansion. Actual GS1 glucose/history remain unavailable in the shared UI.

Input preservation is not complete algorithm replay or calibrated glucose.
Legacy missing fields cannot be recovered by this change; exact model/software,
sensor initialization/sensitivity, usable decoder provenance and sequential
same-index reference conformance remain unresolved. New-source physical sensor
validation and platform/build checks remain separate gates.

Evidence boundary

The private phone was updated without launch to normal, non-debuggable
0.4.0+35, built from 8733e96, not this full-input source. Same signer,
package identity, app-data ownership/inodes and stable nonempty restricted-state
bytes were verified before/after. The app remained stopped; there was no launch,
new radio operation or UI demonstration. This does not validate full-input
storage on the physical sensor.

Independent source review and canonical post-integration checks at 6fc01ae:
708 app tests and 299 CBIO package tests pass
(two existing material-dependent skips); both analyzers and formatting clean.
Real controller/driver/adapter ordering, frozen legacy bytes, private namespace,
native atomic rollback/restart and backup exclusion are covered by synthetic
tests. Canonical formatting checked 133 app and 38 package Dart files with no
changes. The five-commit secret and vendor-material scans found no leaks; the
vendor-material canary guard passed. Reviewed and canonical source trees match,
and the existing Libre2/multi-sensor base remains an ancestor. No new-source
build, hosted CI, physical launch or full-input sensor run is claimed.

@shroominic
Copy link
Copy Markdown
Owner Author

Test-only follow-up: throwing BLE cleanup

Reviewed source: 70b86b869c56a30ebc7258cf472910bc44b34629.

Five regression cases exercise notification-cancellation and disconnect errors,
preserving the original terminal error, settling pending callers, preventing
polling restart, and retaining/releasing private-state ownership only after a
successful durable drain. Delayed and failed drains are covered.

Only the existing session test file changed; production code is unchanged.
Two temporary mutations demonstrated the tests detect skipped link cleanup and
unreleased ownership; both mutations were restored before commit.

Fresh canonical evidence: five focused tests and 304 package tests pass, with
two existing material-dependent skips. Analyzer, formatting and the one-commit
secret/vendor scans are clean. Independent specification and quality review
accepted the change. No hardware, glucose-decoding or release-readiness claim
is added. The PR remains open/WIP, and the phone remains on build35 from
8733e96, installed without launch.

@shroominic
Copy link
Copy Markdown
Owner Author

Source-bound update for 7e81a73a965d7f2df65def232522c02958cbba28, superseding the older source/test snapshot above: reviewed changes preserve private acquisition state, bound recovery to one independent successor, and add private acquisition diagnostics with tracing off by default. The shared normal UI is unchanged.

At this commit, portable verification passed: tooling contracts, formatting, analysis, vendor-material guard, the six-commit secret scan, 1,640 unit/widget tests, one host integration test and JVM checks. Two credential-dependent tests were skipped; four physical harnesses were explicitly deferred. Fresh hosted native checks must be assessed on this head; this is not a full local make check claim.

Verified normalized glucose decoding, exact model/lifecycle support and physical live/history/reconnect support remain unverified. Empty normalized output is containment, not completed sensor support. This update is for source review and CI, not merge or release approval.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

0