8000 Apple camera capture: thicket-camera-apple over UIImagePickerController (iOS); macOS Camera.unavailable (#129) by rleibman · Pull Request #143 · rleibman/thicket · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .scalafmt.conf
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ project.excludeFilters = [
"modules/device-apple/src/main/scala/thicket/device/apple/DeviceShim.scala"
"modules/motion-ios/src/main/scala/thicket/motion/ios/MotionShim.scala"
"modules/pickers-apple/src/main/scala/thicket/pickers/apple/PickersShim.scala"
"modules/camera-apple/src/main/scala/thicket/camera/apple/CameraShim.scala"
]

maxColumn = 120
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -320,7 +320,7 @@ modules/
clipboard/ text on the system clipboard; clipboard-gtk/, clipboard-android/, clipboard-apple/
haptics/ haptic taps; haptics-android/, haptics-apple/ (GTK: Haptics.none)
pickers/ photo and document pickers, save dialog; pickers-gtk/, pickers-android/
camera/ taking a photo with the camera app; camera-android/ (GTK: Camera.unavailable)
camera/ taking a photo with the camera app; camera-android/, camera-apple/ (GTK, macOS: Camera.unavailable)
share/ the share sheet; share-android/, share-apple/ (GTK: Share.unavailable)
renderer-gtk/ GTK4 (Scala Native)
renderer-android/ android.view (Scala on ART)
Expand Down
2 changes: 2 additions & 0 deletions bin/verify-apple.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
# shim-device compile thicket-device-apple's Swift library, macOS and simulator (#114)
# shim-motion compile thicket-motion-ios's Swift library, simulator only (#120)
# shim-pickers compile thicket-pickers-apple's Swift library, macOS and simulator (#125)
# shim-camera compile thicket-camera-apple's Swift library, simulator only (#129)
# abi shim-gen's consistency checks: ABI, C header, Scala externs, both shims
# secrets thicket-secrets-apple's tests against the real Keychain (#108): conformance, isolation
# link sbt nativeLink for the six Apple apps, after deleting their old outputs
Expand Down Expand Up @@ -188,6 +189,7 @@ run shim-ios modules/renderer-apple/shim/build-shim.sh ios-sim
run shim-device sh -c 'modules/device-apple/shim/build-shim.sh macos && modules/device-apple/shim/build-shim.sh ios-sim'
run shim-motion modules/motion-ios/shim/build-shim.sh ios-sim
run shim-pickers sh -c 'modules/pickers-apple/shim/build-shim.sh macos && modules/pickers-apple/shim/build-shim.sh ios-sim'
run shim-camera modules/camera-apple/shim/build-shim.sh ios-sim
run abi sbt --error shimGen/testFull
run secrets sbt --error secretsApple/testFull
run link link_all
Expand Down
33 changes: 32 additions & 1 deletion 8 10BC8 build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -839,6 +839,35 @@ lazy val cameraAndroid = project
.dependsOn(camera.jvm, rendererAndroid)
.settings(commonSettings, name := "thicket-camera-android", scalacOptions ++= Seq("-release", "17"))

/** The camera on iOS (#129): `UIImagePickerController`, through a Swift library of its own (`shim/`,
* `libthicketcamera.a`, iOS only) carried in the jar like `motionIos`'s, so an app without it links none of it. The
* permission is `deviceApple`'s. macOS has no camera screen and uses `Camera.unavailable` (decision 7). In
* `platforms`; building the Swift needs Xcode.
*/
lazy val cameraApple = project
.in(file("modules/camera-apple"))
.enablePlugins(AutomateHeaderPlugin, com.github.sbt.git.GitVersioning, ScalaNativePlugin)
.dependsOn(camera.native, deviceApple, core.native)
.settings(commonSettings, name := "thicket-camera-apple")
// Generated by shimGen and compared byte for byte, like the renderer's `Shim.scala`.
.settings(headerSources / excludeFilter := HiddenFileFilter || "CameraShim.scala")
.settings(
publish / skip := !isMac,
Compile / packageBin / mappings ++= {
val shim = (ThisBuild / baseDirectory).value / "modules" / "camera-apple" / "shim"
val conv = fileConverter.value
if !isMac then Seq.empty
else {
val code = scala.sys.process.Process(Seq("bash", (shim / "build-shim.sh").getAbsolutePath, "ios-sim"), shim).!
if code != 0 then sys.error(s"camera-apple build-shim.sh ios-sim failed with exit $code")
Seq(
shim / "build-ios" / "libthicketcamera.a" -> "thicket-camera-apple/ios-sim/libthicketcamera.a",
shim / "include" / "thicket_camera.h" -> "thicket-camera-apple/include/thicket_camera.h"
).map((f, path) => conv.toVirtualFile(f.toPath) -> path)
}
}
)

/** Handing text or a file to another app through the platform's share sheet (`docs/16` §16.3, §16.10 item 5), and the
* shared copies' store. GTK uses `Share.unavailable`.
*/
Expand Down Expand Up @@ -1041,6 +1070,7 @@ lazy val galleryAndroid = project
pickersAndroid,
pickersApple,
cameraAndroid,
cameraApple,
shareAndroid,
httpChecks.jvm,
httpAndroid
Expand Down Expand Up @@ -1085,7 +1115,7 @@ lazy val galleryIos = project
.in(file("examples/gallery/ios"))
.enablePlugins(AutomateHeaderPlugin, com.github.sbt.git.GitVersioning, BuildInfoPlugin, ScalaNativePlugin)
.settings(thicketBuildInfo("example.gallery.ios.buildinfo"))
.dependsOn(rendererApple, galleryShared.native, storageApple, secretsApple, deviceApple, locationApple, motionIos, clipboardApple, hapticsApple, shareApple, pickersApple)
.dependsOn(rendererApple, galleryShared.native, storageApple, secretsApple, deviceApple, locationApple, motionIos, cameraApple, clipboardApple, hapticsApple, shareApple, pickersApple)
.settings(commonSettings)
.settings(iosNativeSettings)
.settings(galleryAppleSharedSources)
Expand Down Expand Up @@ -1266,6 +1296,7 @@ lazy val platforms = project
pickersAndroid,
pickersApple,
cameraAndroid,
cameraApple,
shareAndroid,
clipboardApple,
hapticsApple,
Expand Down
9 changes: 7 additions & 2 deletions docs/12-component-status.md
Original file line number Diff line number Diff line change
Expand Up @@ -249,7 +249,7 @@ every remaining entry before it is built, not after.
| Clipboard | `modules/clipboard`, `clipboard-gtk`, `clipboard-android`, `clipboard-apple` | done on GTK (`GdkClipboard`, read asynchronously) and Android (`ClipboardManager`) (2026-10-30), and on macOS and iOS (`NSPasteboard`, `UIPasteboard`, #123, 2026-10-30) — text only. The gallery copies and pastes twice, each paste showing that round's copy; on Apple `bin/verify-apple.sh` reads the last copy from outside the app. On iOS, reading text this process did not write shows "Allow Paste". `modules/clipboard/README.md` |
| Haptics | `modules/haptics`, `haptics-android`, `haptics-apple` | done on Android (2026-10-30) — `performHapticFeedback`, which `build.sh` finds in the system's vibrator history; GTK uses `Haptics.none`. Apple (#123, 2026-10-30): iOS's feedback generators and macOS's `NSHapticFeedbackManager` (a Force Touch trackpad); the self-test can only show the calls return — the simulator plays nothing. `modules/haptics/README.md` |
| Pickers | `modules/pickers`, `pickers-gtk`, `pickers-android`, `pickers-apple` | done on GTK (`GtkFileDialog`) and Android (the photo picker, the Storage Access Framework) (2026-10-30) — `Photos.pick(limit)`, `Documents.open(types)` and `save(name, bytes)`; every picked file is copied into the app's cache and handed back as a `Path`, by callback. The gallery's self-tests answer the real pickers: `build.sh` taps Android's, the GTK test answers the in-process chooser. Apple (2026-11-02, #125): `thicket-pickers-apple`, its own Swift library — `PHPickerViewController` and `UIDocumentPickerViewController` on iOS, tapped in the iOS gallery by the XCUITest tapper; `NSOpenPanel` and `NSSavePanel` sheets on macOS, whose OK cannot be pressed from the app on macOS 27 (the panels are out of process), so the self-test answers an accept through the panel's own completion with a known file and cancels with the panel's own `cancel(_:)`. `modules/pickers/README.md` |
| Camera capture | `modules/camera`, `camera-android` | done on Android 10+ (2026-10-30) — `Camera.capturePhoto`: the camera app's screen, the photo copied into the app's cache as a JPEG, the MediaStore entry it was written to deleted; GTK uses `Camera.unavailable`. The gallery's self-test presses the emulator camera's shutter and Done, and backs out. Apple is a Mac issue. `modules/camera/README.md` |
| Camera capture | `modules/camera`, `camera-android` | done on Android 10+ (2026-10-30) — `Camera.capturePhoto`: the camera app's screen, the photo copied into the app's cache as a JPEG, the MediaStore entry it was written to deleted; GTK uses `Camera.unavailable`. The gallery's self-test presses the emulator camera's shutter and Done, and backs out. iOS (2026-11-02, #129): `thicket-camera-apple` over `UIImagePickerController` (its own Swift library), the photo written as a JPEG and copied into the cache; the app's `Info.plist` must declare `NSCameraUsageDescription` (else `IllegalStateException`). The iOS 27 simulator has a camera screen whose shutter takes nothing: the gallery opens it, checks one request at a time, and closes it with its X; the capture itself is unmeasured (no device). macOS uses `Camera.unavailable`. `modules/camera/README.md` |
| Share | `modules/share`, `share-android`, `share-apple` | done on Android (2026-11-01) — `Share.text` and `Share.file` through the chooser; a file is copied and served by Thicket's own `ThicketFileProvider`, read-only, with a grant for the chosen app; the app declares the provider in its manifest. GTK uses `Share.unavailable`. The gallery's self-test shares to its own share target, tapped in the real chooser. Done on macOS and iOS (#134) — `NSSharingServicePicker` and `UIActivityViewController`, a C helper, no Swift; the subject through a `UIActivityItemSource` made at run time; on iOS an XCUITest taps "Copy" in the sheet and `simctl pbpaste` reads the text; no macOS service is driven. `modules/share/README.md` |
| UI-thread seam | `UiThread`, `Host.install` | done — every host installs it since 2026-10-20; before that none did, so a ZIO effect's result was written from a worker thread. The default now throws |
| Apple ABI description + consistency check | `tools/shim-gen` | done — every `sui_*` function (110 externs in `Shim.scala`) and all 26 kind codes described; `thicket_apple.h` and `Shim.scala` are generated from it (adopted 2026-09-29, #3) and checked byte-for-byte; the two Swift shims' signatures and per-platform widget choice are checked against it, on any machine. Since #90 also `thicket-http-apple`'s 12 `thk_http_*` functions (`HttpAbi`): `thicket_http.h` and `HttpShim.scala` generated, `HttpShim.swift` checked |
Expand Down Expand Up @@ -367,7 +367,7 @@ someone who did not write the framework, running on all three targets.
| `hapticsJVM` / `hapticsNative` | 2 — `RecordingHaptics`, `Haptics.none` (2026-10-30) |
| `motionJVM` / `motionNative` | 5 — each reading follows its sensor, attitude as roll/pitch/yaw; stop in the background and with the owner; the rate asked for; a missing sensor; `Motion.unavailable` (2026-10-30) |
| `secretsApple` | 4 (2026-10-27, a Mac only: `bin/verify-apple.sh`'s `secrets` step) — an unsigned binary gets the login keychain; the 11 checks; services are separate; a status in Security's words |
| `shimGen` | 44 (the `thk_dev` ABI's 7 included, 2026-10-30; the `thk_mot` ABI's 7, 2026-10-31; the `thk_pick` ABI's 7, 2026-11-02) |
| `shimGen` | 51 (the `thk_dev` ABI's 7 included, 2026-10-30; the `thk_mot` ABI's 7, 2026-10-31; the `thk_pick` ABI's 7 and the `thk_cam` ABI's 7, 2026-11-02) |
| `sbtThicket` | 9 |

Measured 2026-10-20 with `sbt --error testFull`, which `sbt checkLinux` and CI run; the
Expand All @@ -392,6 +392,11 @@ through a wrapping provider and in `libthicketdevice.a`), macOS **192** with the
(191 + 1, location unavailable).
2026-11-02 (#125), on this branch's base: iOS simulator ("iPhone 17e") **218** (207 + 11, the
Pickers section, driven through the real pickers), macOS **209** (196 + 13), screen unlocked.
2026-11-02 (#129, `bin/verify-apple.sh "iPhone Air"`, 19/19 steps): iOS simulator **213** (main's
checks + 6, the camera: ready, the screen up, one at a time, the X twice, a shutter that takes
nothing; the Device section's camera check flipped to `Granted`), macOS **197** with the screen
locked (+ 1: the desktop has no camera screen).
The pickers and the camera were each measured on their own branch; the gallery with both has not been run yet.
2026-11-02 (#134): macOS **208**, the Share section's 8 included (**204** with the screen locked,
in `bin/verify-apple.sh`); iOS simulator ("iPhone 18 Pro") **216**, its 9 included.
Each measured on its own branch; the gallery with both sections has not been run yet.
Expand Down
3 changes: 2 additions & 1 deletion docs/14-releasing.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ The artefacts, with explicit suffixes, because sbt 2 has no `%%%`:
| `net.leibman:thicket-pickers-gtk_native0.5_3` / `thicket-pickers-android_3` | the photo picker, the document picker and save dialog (`thicket-pickers` comes with them) |
| `net.leibman:thicket-pickers-apple_native0.5_3` | the same on macOS and iOS: `PHPickerViewController` and `UIDocumentPickerViewController`, `NSOpenPanel` and `NSSavePanel` sheets, with its own Swift library inside the jar (`macos/` and `ios-sim/libthicketpickers.a`); the Apple renderer comes with it |
| `net.leibman:thicket-camera_3` / `_native0.5_3`, `thicket-camera-android_3` | taking a photo with the camera app; on GTK, `Camera.unavailable` from `thicket-camera` |
| `net.leibman:thicket-camera-apple_native0.5_3` | taking a photo on iOS with `UIImagePickerController`, with its own Swift library inside the jar (`ios-sim/libthicketcamera.a`); the app's `Info.plist` needs `NSCameraUsageDescription`; on macOS, `Camera.unavailable` from `thicket-camera` |
| `net.leibman:thicket-share_3` / `_native0.5_3`, `thicket-share-android_3` | the share sheet; on Android the app declares `ThicketFileProvider`; on GTK, `Share.unavailable` |
| `net.leibman:thicket-renderer-android_3` | the Android renderer (plain JVM bytecode, `-release 17`; R8 dexes it) |
| `net.leibman:thicket-renderer-apple_native0.5_3` | the AppKit and UIKit renderers, with the Swift shim inside the jar |
Expand Down Expand Up @@ -180,4 +181,4 @@ renderers and the platform modules that are not in `root`: `thicket-http-apple`,
`thicket-location-android`, `thicket-location-apple`, `thicket-motion-android`, `thicket-motion-ios`, `thicket-clipboard-gtk`,
`thicket-clipboard-android`, `thicket-clipboard-apple`, `thicket-haptics-android`,
`thicket-haptics-apple`, `thicket-pickers-gtk`, `thicket-pickers-android`, `thicket-pickers-apple`,
`thicket-camera-android`, `thicket-share-android` and `thicket-share-apple`.
`thicket-camera-android`, `thicket-camera-apple`, `thicket-share-android` and `thicket-share-apple`.
7 changes: 5 additions & 2 deletions docs/16-device-services.md
Original file line number Diff line number Diff line change
Expand Up @@ -323,7 +323,7 @@ this means many modules. Names follow the published ones:
| clipboard | `thicket-clipboard` (built) | `-gtk`, `-android`, `-apple` (built) |
| haptics | `thicket-haptics` (built) | `-android`, `-apple` (built); GTK uses `Haptics.none` |
| pickers: photos, documents | `thicket-pickers` (built; `Documents`, not `Files`) | `-gtk`, `-android` (built); `-apple` (built, #125) |
| camera capture | `thicket-camera` (built) | `-android` (built), `-apple` (Mac issue); GTK and macOS use `Camera.unavailable` |
| camera capture | `thicket-camera` (built) | `-android` (built), `-apple` (built, iOS; #129); GTK and macOS use `Camera.unavailable` |
| share | `thicket-share` (built) | `-android` (built; Thicket's own `ThicketFileProvider`, declared in the app's manifest), `-apple` (Mac issue); GTK uses `Share.unavailable` |

The implementations need the host (`ThicketActivity`, `GtkApp`, a Swift library), so each
Expand Down Expand Up @@ -471,7 +471,10 @@ Every service gets a gallery section and self-test checks on every host where it
- *Camera capture done on Android, `Unavailable` on GTK, 2026-10-30
(`modules/camera/README.md`):* the camera app writes to a MediaStore entry Thicket makes,
and the photo is copied into the cache and the entry deleted. Android 10+. `build.sh`
presses the shutter and Done in the emulator's camera app.
presses the shutter and Done in the emulator's camera app. *iOS done 2026-11-02 (#129):*
`thicket-camera-apple` over `UIImagePickerController`; macOS uses `Camera.unavailable`. The
iOS 27 simulator has a camera screen, but its shutter takes nothing, so the copy into the
cache is unmeasured on Apple.
- *Share done on Android, `Unavailable` on GTK, 2026-11-01 (`modules/share/README.md`):*
text and files through the chooser. Files are served by Thicket's own `ContentProvider`
(owner's decision 2026-11-01: not AndroidX's `FileProvider`), which the app declares in its
Expand Down
Loading
0