8000 The camera preview: thicket-camera-preview and -android (docs/17 step 2) by rleibman · Pull Request #149 · rleibman/thicket · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -322,6 +322,7 @@ modules/
pickers/ photo and document pickers, save dialog; pickers-gtk/, pickers-android/
camera/ taking a photo with the camera app; camera-android/, camera-apple/ (GTK, macOS: Camera.unavailable)
share/ the share sheet; share-android/, share-apple/ (GTK: Share.unavailable)
camera-preview/ a live camera preview in the app's layout; camera-preview-android/ (GTK: CameraPreview.unavailable)
renderer-gtk/ GTK4 (Scala Native)
renderer-android/ android.view (Scala on ART)
renderer-apple/ AppKit + UIKit (Scala Native + Swift shim)
Expand Down Expand Up @@ -362,9 +363,9 @@ SELFTEST=1 examples/todo-android/build.sh
xvfb; the Apple self-tests run on a self-hosted Mac (`apple.yml`). Counts as of 2026-11-02:
signals 30, core 172, effect-zio 25, storage 14, sqlite 2 and secrets 2 per backend,
storage-gtk 4, storage-apple 4, secrets-gtk 2, http-gtk 5, http-android 5, device 8,
location 7, motion 5, clipboard 1, haptics 2, pickers 4, camera 2 and share 5 per backend, shim-gen 15,
sbt-thicket 9; GTK 144 + 69 self-test checks, Android 95 + 84 (2026-11-02, the gallery's with
the HTTP checks, every device section's and the Platform section's),
location 7, motion 5, clipboard 1, haptics 2, pickers 4, camera 2, share 5 and camera-preview 2 per backend, shim-gen 15,
sbt-thicket 9; GTK 148 + 69 self-test checks, Android 102 + 84 (2026-11-03, the gallery's with
the HTTP checks, every device section's, the Platform section's and the Camera preview's),
the HTTP checks on Linux 12.

Coverage over the JVM-tested modules:
Expand Down
27 changes: 26 additions & 1 deletion build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -832,6 +832,27 @@ lazy val camera = crossProject(JVMPlatform, NativePlatform)
.jvmSettings(libraryDependencies ++= zioTestJvm, scalacOptions ++= Seq("-release", "17"))
.nativeSettings(libraryDependencies ++= zioTestNative)

/** A live camera preview in the app's layout, with a still capture from it (`docs/17`): the handle, its widget through
* `core`'s embedding point, and a fake. Only an app that shows a preview links a platform camera stack.
*/
lazy val cameraPreview = crossProject(JVMPlatform, NativePlatform)
.crossType(CrossType.Full)
.in(file("modules/camera-preview"))
.dependsOn(core, device)
.enablePlugins(AutomateHeaderPlugin, com.github.sbt.git.GitVersioning)
.settings(commonSettings, zioTestFramework, name := "thicket-camera-preview")
.jvmSettings(libraryDependencies ++= zioTestJvm, scalacOptions ++= Seq("-release", "17"))
.nativeSettings(libraryDependencies ++= zioTestNative)

/** The camera preview on Android: Camera2 into a `TextureView` (not CameraX, which is AndroidX), and a JPEG
* `ImageReader` on the same session for the capture.
*/
lazy val cameraPreviewAndroid = project
.in(file("modules/camera-preview-android"))
.enablePlugins(AutomateHeaderPlugin, com.github.sbt.git.GitVersioning)
.dependsOn(cameraPreview.jvm, rendererAndroid)
.settings(commonSettings, name := "thicket-camera-preview-android", scalacOptions ++= Seq("-release", "17"))

/** The camera on Android: the camera app's `ACTION_IMAGE_CAPTURE`, writing to a MediaStore entry. */
lazy val cameraAndroid = project
.in(file("modules/camera-android"))
Expand Down Expand Up @@ -1027,7 +1048,7 @@ lazy val galleryShared = crossProject(JVMPlatform, NativePlatform)
.in(file("examples/gallery/shared"))
// effect-zio too: the gallery runs one real effect, so every host proves it installed a UI-thread poster.
// storage and sqlite: the device services' section; each host supplies its platform's implementation.
.dependsOn(core, effectZio, storage, sqlite, secrets, device, location, motion, clipboard, haptics, pickers, camera, share)
.dependsOn(core, effectZio, storage, sqlite, secrets, device, location, motion, clipboard, haptics, pickers, camera, share, cameraPreview)
.enablePlugins(AutomateHeaderPlugin, com.github.sbt.git.GitVersioning, BuildInfoPlugin)
.settings(thicketBuildInfo("example.gallery.buildinfo", buildTime = true))
.settings(commonSettings, name := "thicket-gallery-shared", publish / skip := true)
Expand Down Expand Up @@ -1069,6 +1090,7 @@ lazy val galleryAndroid = project
hapticsAndroid,
pickersAndroid,
cameraAndroid,
cameraPreviewAndroid,
shareAndroid,
httpChecks.jvm,
httpAndroid
Expand Down Expand Up @@ -1295,6 +1317,7 @@ lazy val platforms = project
pickersApple,
cameraAndroid,
cameraApple,
cameraPreviewAndroid,
shareAndroid,
clipboardApple,
hapticsApple,
Expand Down Expand Up @@ -1358,6 +1381,8 @@ lazy val root = project
pickers.native,
camera.jvm,
camera.native,
cameraPreview.jvm,
cameraPreview.native,
share.jvm,
share.native,
httpGtk,
Expand Down
9 changes: 6 additions & 3 deletions docs/12-component-status.md
Original file line number Diff line number Diff line change
Expand Up @@ -251,6 +251,7 @@ every remaining entry before it is built, not after.
| Haptics | `modules/haptics`, `haptics-android`, `haptics-apple` | done on Android (2026-10-30) — `performHapticFeedback`, which `build.sh` finds in the system's vibrator history; GTK uses `Haptics.none`. Apple (#123, 2026-10-30): iOS's feedback generators and macOS's `NSHapticFeedbackManager` (a Force Touch trackpad); the self-test can only show the calls return — the simulator plays nothing. `modules/haptics/README.md` |
| Pickers | `modules/pickers`, `pickers-gtk`, `pickers-android`, `pickers-apple` | done on GTK (`GtkFileDialog`) and Android (the photo picker, the Storage Access Framework) (2026-10-30) — `Photos.pick(limit)`, `Documents.open(types)` and `save(name, bytes)`; every picked file is copied into the app's cache and handed back as a `Path`, by callback. The gallery's self-tests answer the real pickers: `build.sh` taps Android's, the GTK test answers the in-process chooser. Apple (2026-11-02, #125): `thicket-pickers-apple`, its own Swift library — `PHPickerViewController` and `UIDocumentPickerViewController` on iOS, tapped in the iOS gallery by the XCUITest tapper; `NSOpenPanel` and `NSSavePanel` sheets on macOS, whose OK cannot be pressed from the app on macOS 27 (the panels are out of process), so the self-test answers an accept through the panel's own completion with a known file and cancels with the panel's own `cancel(_:)`. `modules/pickers/README.md` |
| Camera capture | `modules/camera`, `camera-android` | done on Android 10+ (2026-10-30) — `Camera.capturePhoto`: the camera app's screen, the photo copied into the app's cache as a JPEG, the MediaStore entry it was written to deleted; GTK uses `Camera.unavailable`. The gallery's self-test presses the emulator camera's shutter and Done, and backs out. iOS (2026-11-02, #129): `thicket-camera-apple` over `UIImagePickerController` (its own Swift library), the photo written as a JPEG and copied into the cache; the app's `Info.plist` must declare `NSCameraUsageDescription` (else `IllegalStateException`). The iOS 27 simulator has a camera screen whose shutter takes nothing: the gallery opens it, checks one request at a time, and closes it with its X; the capture itself is unmeasured (no device). macOS uses `Camera.unavailable`. `modules/camera/README.md` |
| Camera preview | `modules/camera-preview`, `camera-preview-android` | done on Android 8+ (2026-11-03, `docs/17` step 2) — `CameraPreview`: Camera2 into a `TextureView` through the embedding point (`dsl.Platform`), `capture` a JPEG from the same session, `facing`, `running`; runs only while mounted and in front. The gallery's self-test runs the emulator's back camera: frames, a capture of the session's size, Home closes it, front reopens it, no front camera says so. GTK uses `CameraPreview.unavailable` (§17.8.2: later). Apple is #148. `modules/camera-preview/README.md` |
| Share | `modules/share`, `share-android`, `share-apple` | done on Android (2026-11-01) — `Share.text` and `Share.file` through the chooser; a file is copied and served by Thicket's own `ThicketFileProvider`, read-only, with a grant for the chosen app; the app declares the provider in its manifest. GTK uses `Share.unavailable`. The gallery's self-test shares to its own share target, tapped in the real chooser. Done on macOS and iOS (#134) — `NSSharingServicePicker` and `UIActivityViewController`, a C helper, no Swift; the subject through a `UIActivityItemSource` made at run time; on iOS an XCUITest taps "Copy" in the sheet and `simctl pbpaste` reads the text; no macOS service is driven. `modules/share/README.md` |
| UI-thread seam | `UiThread`, `Host.install` | done — every host installs it since 2026-10-20; before that none did, so a ZIO effect's result was written from a worker thread. The default now throws |
| Apple ABI description + consistency check | `tools/shim-gen` | done — every `sui_*` function (110 externs in `Shim.scala`) and all 26 kind codes described; `thicket_apple.h` and `Shim.scala` are generated from it (adopted 2026-09-29, #3) and checked byte-for-byte; the two Swift shims' signatures and per-platform widget choice are checked against it, on any machine. Since #90 also `thicket-http-apple`'s 12 `thk_http_*` functions (`HttpAbi`): `thicket_http.h` and `HttpShim.scala` generated, `HttpShim.swift` checked |
Expand Down Expand Up @@ -363,6 +364,7 @@ someone who did not write the framework, running on all three targets.
| `locationJVM` / `locationNative` | 7 — updates follow the lifecycle (stop in the background, restart in front with the last known position), stop with their owner, wait for availability; `current` once; `Location.unavailable` (2026-10-30) |
| `clipboardJVM` / `clipboardNative` | 1 — `MemoryClipboard` (2026-10-30); the real clipboards are checked in the galleries |
| `cameraJVM` / `cameraNative` | 2 — `FakeCamera`, `Camera.unavailable` (2026-10-30) |
| `cameraPreviewJVM` / `cameraPreviewNative` | 2 — `CameraPreview.unavailable`, `FakeCameraPreview` (2026-11-03) |
| `shareJVM` / `shareNative` | 5 — `SharedFiles` serves a copy under its name and type, never outside its directory or the type file, prunes old copies; names kept to one segment; the fakes (2026-11-01) |
| `pickersJVM` / `pickersNative` | 4 — `PickedFiles` copies byte for byte into a directory of its own, closes the stream on failure, keeps names inside it; the fakes (2026-10-30) |
| `hapticsJVM` / `hapticsNative` | 2 — `RecordingHaptics`, `Haptics.none` (2026-10-30) |
Expand All @@ -373,14 +375,15 @@ someone who did not write the framework, running on all three targets.

Measured 2026-10-20 with `sbt --error testFull`, which `sbt checkLinux` and CI run; the
storage and sqlite rows 2026-10-21, the secrets rows and `storageGtk` 2026-10-26. On top of
these, the self-tests drive the real toolkits: GTK gallery **144** and todo **69**
(`bin/gtk-selftests.sh`, which also fails on any `Gtk-CRITICAL`), Android gallery **95** and
these, the self-tests drive the real toolkits: GTK gallery **148** and todo **69**
(`bin/gtk-selftests.sh`, which also fails on any `Gtk-CRITICAL`), Android gallery **102** and
todo **84** on the `s2test` emulator (2026-10-30, the storage section's checks each included,
secrets among them, the 14 HTTP checks, the Device section's 5 on GTK and 9 on Android, and
the Location section's 1 on GTK and 7 on Android, the Motion section's 1 and 8, and the
Clipboard and haptics section's 3 and 5, the Pickers and camera section's 6 and 10, the
Share section's 1 and 4, and, 2026-11-02, the Platform section's 6 on GTK, its heading among
them, and 5 on Android).
them, and 5 on Android, and, 2026-11-03, the Camera preview section's 4 on GTK, its heading
among them, and 7 on Android).
The Android gallery's `build.sh` also checks that Back runs the terminate hook. HTTP on Linux: `examples/http/selftest.sh linux`,
**12**. Apple gallery (2026-10-22, #98): macOS **174**, iOS simulator **168**,
each with six storage checks; 2026-10-27 (#108): macOS **185**, iOS simulator **179**, the two
Expand Down
4 changes: 3 additions & 1 deletion docs/14-releasing.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ The artefacts, with explicit suffixes, because sbt 2 has no `%%%`:
| `net.leibman:thicket-camera_3` / `_native0.5_3`, `thicket-camera-android_3` | taking a photo with the camera app; on GTK, `Camera.unavailable` from `thicket-camera` |
| `net.leibman:thicket-camera-apple_native0.5_3` | taking a photo on iOS with `UIImagePickerController`, with its own Swift library inside the jar (`ios-sim/libthicketcamera.a`); the app's `Info.plist` needs `NSCameraUsageDescription`; on macOS, `Camera.unavailable` from `thicket-camera` |
| `net.leibman:thicket-share_3` / `_native0.5_3`, `thicket-share-android_3` | the share sheet; on Android the app declares `ThicketFileProvider`; on GTK, `Share.unavailable` |
| `net.leibman:thicket-camera-preview_3` / `_native0.5_3`, `thicket-camera-preview-android_3` | a live camera preview in the app's layout (`docs/17`): Camera2 on Android, the app declaring `CAMERA`; elsewhere, `CameraPreview.unavailable` from `thicket-camera-preview` |
| `net.leibman:thicket-renderer-android_3` | the Android renderer (plain JVM bytecode, `-release 17`; R8 dexes it) |
| `net.leibman:thicket-renderer-apple_native0.5_3` | the AppKit and UIKit renderers, with the Swift shim inside the jar |
| `net.leibman:thicket-storage-apple_native0.5_3` | app directories and preferences on macOS and iOS (`thicket-storage` comes with it); no Swift |
Expand Down Expand Up @@ -181,4 +182,5 @@ renderers and the platform modules that are not in `root`: `thicket-http-apple`,
`thicket-location-android`, `thicket-location-apple`, `thicket-motion-android`, `thicket-motion-ios`, `thicket-clipboard-gtk`,
`thicket-clipboard-android`, `thicket-clipboard-apple`, `thicket-haptics-android`,
`thicket-haptics-apple`, `thicket-pickers-gtk`, `thicket-pickers-android`, `thicket-pickers-apple`,
`thicket-camera-android`, `thicket-camera-apple`, `thicket-share-android` and `thicket-share-apple`.
`thicket-camera-android`, `thicket-camera-apple`, `thicket-share-android`, `thicket-share-apple` and
`thicket-camera-preview-android`.
3 changes: 2 additions & 1 deletion docs/16-device-services.md
Original file line number Diff line number Diff line change
Expand Up @@ -486,4 +486,5 @@ Every service gets a gallery section and self-test checks on every host where it
closed, but no service can be driven (its list is drawn out of process).*
6. **Tier 3** as separate designs, live camera preview first if wanted. *The owner wants tier 3
(2026-11-01); the camera preview's design is `docs/17-camera-preview.md`, approved with every
recommendation (2026-11-01); the embedding point is next.*
recommendation (2026-11-01). The embedding point is built (2026-11-02, #146), and the preview
on Android (2026-11-03, `thicket-camera-preview`, `-android`); Apple is the Mac's, GTK later.*
5 changes: 3 additions & 2 deletions docs/17-camera-preview.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@

Status: **approved, 2026-11-01; step 1 of §17.7, the embedding point, built on GTK and
Android 2026-11-02** (`WidgetKind.Platform`, `Prop.View`, `GtkPlatformView`,
`AndroidPlatformView`; the Apple half is #145). **The preview itself is not built yet.** `docs/16` §16.3 puts the live camera
`AndroidPlatformView`; the Apple half is #145). Step 2, **the preview on Android, built 2026-11-03** (`thicket-camera-preview`,
`-android`; `modules/camera-preview/README.md`). Apple (step 3) is the Mac's; GTK (step 4) later. `docs/16` §16.3 puts the live camera
preview first in tier 3 ("each its own design"), and the owner said yes to tier 3 on
2026-11-01. This is that design. The owner answered §17.8's questions the same day, taking
every recommendation (decision log, 2026-11-01).
Expand Down Expand Up @@ -100,7 +101,7 @@ The largest piece in the plan (`docs/16` §16.3 says so). Proposed order, each i

1. The embedding point (§17.3) on GTK and Android, with the gallery's trivial platform view;
Apple as a Mac issue. *Done on GTK and Android, 2026-11-02 (decision log).*
2. `thicket-camera-preview` and `-android`, measured on the emulator.
2. `thicket-camera-preview` and `-android`, measured on the emulator. *Done, 2026-11-03.*
3. `-apple`, the Mac's (iOS and macOS).
4. `-gtk`, after Android and Apple (§17.8.2).

Expand Down
26 changes: 26 additions & 0 deletions docs/decisions.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,32 @@ Go decision (see `08` §8.1) is made after S1, S2, S3 have reports.

## Decision log

- 2026-11-03 — **The camera preview: `thicket-camera-preview` and `-android`** (`docs/17` step 2;
Apple is #148, after #145; GTK later, §17.8.2).
- **API:** `CameraPreview` with `availability`, `facing: Var[Facing]`, `running: Signal[Boolean]`,
`view(fit)` (through `dsl.Platform`; `Cover` by default) and `capture` (a JPEG in the cache,
`None` when not running or busy). `CameraPreview.unavailable(reason)` for GTK; a fake.
- **Android:** Camera2 into a `TextureView`, the camera's callbacks on the UI thread, a JPEG
`ImageReader` on the same session (the size both offer, at most 1920×1080; 1280×720 on
`s2test`). Orientation by hand: the view's transform turns the frame back by the display's
rotation and scales it to the fit. `createCaptureSession(List, …)`, deprecated, since
`SessionConfiguration` needs API 28 and the framework's minimum is 26. The widget takes the
frame's aspect when the layout leaves its height open.
- **Runs only while mounted, its surface ready, in front and permitted;** one preview at a
time. Each close records why (`testing.stopReasons`): Android takes the camera from a
background app by itself (`error 3`, then `disconnected`), so "closed in the background"
alone could not tell whether the preview's own lifecycle did it; the check asks for the
lifecycle's stop first.
- **The gallery declares `CAMERA`** (granted by `build.sh`'s `install -g`). The Device
section's "an undeclared permission throws" check moved to the microphone.
- **Measured.** `CameraPreviewSpec` 2/2, JVM and Native. GTK gallery **148** (144 + 4), todo
69. Android gallery **102** (95 + 7) on `s2test`: the back camera runs, frames arrive and
the picture is not blank, a capture is a 1280×720 JPEG in the cache, Home closes the camera,
front reopens it, the missing front camera says so, flipping back reopens the back one.
- **Falsified.** Without the lifecycle's background stop, the background check fails (stops
`facing…, error 3, disconnected`); with `capture` stubbed, the JPEG check fails.
- **Found on the way:** #147, `galleryAndroid` depended on `pickersApple` and `cameraApple`.

- 2026-11-02 — **The embedding point: `WidgetKind.Platform`, `Prop.View`, built on GTK and
Android** (`docs/17` §17.3, step 1 of §17.7; the Apple half is #145).
- **Shape:** `trait PlatformView { def description: String }` in `renderer-api`, and
Expand Down
8 changes: 8 additions & 0 deletions examples/gallery/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -275,6 +275,14 @@ reads the services it was offered and closes it; on iOS it finds the `UIActivity
asks its item for the text and subject, and has an XCUITest tap "Copy" in it (`DRIVE share-copy`),
after which `simctl pbpaste` must read the shared text (#134).

The Camera preview section (`docs/17`) shows a live preview with "Take from the preview" and
"Flip the camera". On Android it is Camera2 on the emulator's back camera (the gallery declares
`CAMERA`, which `install -g` grants): the self-test brings it on screen, waits for frames, checks
the picture is not blank, takes a JPEG of the session's size, sends the app Home and back
(`DRIVE home`, `DRIVE front`) to see the camera closed by the preview's own lifecycle and opened
again, then flips to the front camera `s2test` does not have, which must say so. On GTK the
section shows `CameraPreview.unavailable`; on Apple it says there is none yet (#148).

When the desktop is locked, GNOME suspends every window and the Device section's foreground
check fails. A private headless shell avoids that:
`dbus-run-session -- gnome-shell --headless --wayland --no-x11 --virtual-monitor 1024x768 --wayland-display thicket-headless &`,
Expand Down
2 changes: 2 additions & 0 deletions examples/gallery/android/app/src/main/AndroidManifest.xml
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@
<!-- The Location section: fine for the GPS provider, which the self-test moves with `adb emu geo fix`. -->
<uses-permission android:name="android.permission.ACCESS_FINE_LOCATION" />
<uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION" />
<!-- The Camera preview section: Camera2 needs it (docs/17). build.sh installs with it granted. -->
<uses-permission android:name="android.permission.CAMERA" />
<application android:label="Thicket gallery" android:allowBackup="false"
android:networkSecurityConfig="@xml/network_security_config"
android:enableOnBackInvokedCallback="true">
Expand Down
Loading
Loading
0