8000 Apple camera preview: thicket-camera-preview-apple over AVCaptureSession (#148) by rleibman · Pull Request #158 · rleibman/thicket · GitHub
[go: up one dir, main page]

Skip to content

Apple camera preview: thicket-camera-preview-apple over AVCaptureSession (#148) - #158

Open
rleibman wants to merge 12 commits into
mainfrom
issue/148-apple-camera-preview
Open

rleibman wants to merge 12 commits into
mainfrom
issue/148-apple-camera-preview

Conversation

@rleibman
@rleibman rleibman commented Nov 9, 2026
Copy link
Copy Markdown
Owner

Closes #148: docs/17 step 3, the live camera preview on iOS and macOS. Built on #145's ApplePlatformView (merged in #152).

What it adds

  • thicket-camera-preview-apple: AppleCameraPreview(device, cache) over AVCaptureSession.
    • The preview layer is in a UIView on iOS and an NSView on macOS. Capture uses AVCapturePhotoOutput and writes a JPEG to <cache>/thicket-preview/.
    • It has its own Swift library, libthicketpreview.a, with the thk_prev_* ABI: PreviewAbi, 12 functions, checked by PreviewConsistencySpec.
    • The session runs only while the preview is mounted, the app is in front and the permission is granted, the same rules as on Android. stopReasons and the 10 s capture timeout work as Android's do.
    • Fit: Cover, Contain and Fill map to .resizeAspectFill, .resizeAspect and .resize. The view gives itself an aspect-ratio intrinsic size, as Apple embedding point: host a platform view (WidgetKind.Platform / Prop.View) in the AppKit and UIKit shims #145 requires.
  • verify-apple gains a shim-preview step. sbt-thicket gains thicketCameraPreviewApple.
  • The macOS gallery declares NSCameraUsageDescription in an Info.plist embedded in the executable (-sectcreate). Because of that, the Device section's "missing usage string throws" check moves to the microphone.
  • The galleries pass cameraPreview = Some(AppleCameraPreview(...)). The preview stays unmounted until its own checks run, and a "Hide or show the preview" button checks unmounting.

The full account is in the decision-log entry dated 2026-11-03.

Measured

  • iOS 27 simulator: AVFoundation lists 0 capture devices. The simulated camera Apple camera capture: thicket-camera-apple over UIImagePickerController (iOS); macOS Camera.unavailable #129 found belongs to UIImagePickerController only. So both facings report Unavailable, and the check confirms it.
  • macOS: the back camera is Unavailable("no back camera"). The front camera (FaceTime HD) reports NeedsPermission(Camera), because camera permission is still undecided.
    • One run asked through the system's own dialog, and nobody answered it in 120 s.
    • Access was not granted any other way.
    • No frame or photo was taken from the Mac's camera.
  • On f07d3a3, before merging main: bin/verify-apple.sh "iPhone 18 Pro" passed 21/21 steps. macOS gallery 230 ok on each of two runs, iOS gallery 245 ok.
  • On the merge with main (e1ba31a): 19 of 21 steps passed. macOS gallery 251 ok and iOS 256 ok, with 2 failures on each, both Stepper checks that fail on main itself:
    • "Stepper: … showing its value to its step's precision": the label shows 1.5, not 1.50.
    • "Stepper: − moves it back a step".
    • Their cause is in main, and the fix is in a separate PR, linked below. Every camera-preview check passed.
  • Controls, on macOS (228 ok, 2 failed, exactly the two expected checks):
    • Contain mapped to .resizeAspectFill;
    • the aspect constraint left inactive.

Not measured

  • The live preview on either platform: frames, aspect, the JPEG, stopping in the background, restarting, and stopping on unmount. The checks are written but have not run.
  • What it takes to run them: grant camera access to the terminal app that runs the galleries (System Settings → Privacy & Security → Camera), then re-run the macOS gallery, or use a physical iPhone.
  • The two controls from the issue (no background stop, a stubbed capture) need a running session, so they have not run either.

🤖 Generated with Claude Code

Roberto Leibman and others added 9 commits November 3, 2026 19:09
…t_view, ApplePlatformView, gallery checks

macOS gallery 228 ok, 0 FAIL (the 6 Platform checks included); shimGen/testFull 51/51.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… freed

Falsified on macOS, each 223 ok / 1 failed on the matching check: release skipped,
release after the shim dropped the view, empty 'not available' label, shim keeping the +1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…12 and the gallery README

macOS gallery 228 ok / 0 failed (224 with the screen locked); iOS simulator (iPhone 17e) 239 / 0
on the warm simulator. verify-apple.sh 19/20: ios-gallery failed only on the fresh boot's
notifications 'not yet asked' check (known trap), Platform checks passing; alone it passed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hk_prev ABI

Both Swift targets build (libthicketpreview.a: macOS 121480 bytes, ios-sim 113120);
shimGen/testFull 58 passed, 0 failed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…reviewChecks

galleryMacos/compile and galleryIos/compile succeed; nothing run yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… unpacks thicket-camera-preview-apple

macOS gallery: 229 ok, 0 failed (the preview's back camera Unavailable, front NeedsPermission).
First iOS run: the camera screen's DRIVE photo failed with the preview's session open underneath.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…r, formatted

shimGen/testFull 58 passed; shimGen/run --check current. iOS 27 simulator: AVFoundation lists
0 capture devices (a simctl spawn probe), so both facings are Unavailable there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…4, 16, 17 and the READMEs

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Conflicts kept both sides: the decision log (both entries), the gallery
self-test chain, docs counts. main's AppleRenderer had lost #152's mapping in
#151's merge (Platform back to a Label, Prop.View back to "not available"),
and its generated thicket_apple.h was stale; both restored here, as in
fix/apple-after-151.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI left a comment
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Copilot was unable to complete this review at the selected tier due to model policy or plan restrictions.

🟡 Changes recommended

The change still contains a critical active issue where previewHeader redeclares sui_handle in Emit.scala:538 despite it already being defined in thicket_apple_types.h, which would cause a C header redeclaration error when both headers are included together.

1 open issue
What changed in this PR

This pull request adds an Apple (iOS + macOS) implementation of thicket-camera-preview backed by AVFoundation via a dedicated Swift static library/shim, and wires it into build tooling, galleries, self-tests, and documentation.

Changes:

  • Introduces thicket-camera-preview-apple with a new thk_prev_* ABI (generated/validated via shim-gen + consistency spec) and a Swift AVCaptureSession-based implementation.
  • Integrates the preview into Apple renderer platform-view hosting, sbt-thicket, and verify-apple, and links the shim where needed (including macOS usage-string embedding).
  • Updates galleries/self-tests and docs to exercise preview mount/unmount, fit mapping, and device/permission behavior (with live-session measurement noted as pending).
File Review
tools/​shim-gen/​src/​test/​scala/​thicket/​tools/​shim/​PreviewConsistencySpec.scala Reviewed
tools/​shim-gen/​src/​main/​scala/​thicket/​tools/​shim/​PreviewAbi.scala Reviewed
tools/​shim-gen/​src/​main/​scala/​thicket/​tools/​shim/​Parse.scala Reviewed
tools/​shim-gen/​src/​main/​scala/​thicket/​tools/​shim/​Main.scala Reviewed
tools/​shim-gen/​src/​main/​scala/​thicket/​tools/​shim/​Emit.scala Reviewed
tools/​sbt-thicket/​src/​main/​scala/​thicket/​sbt/​ThicketApplePlugins.scala Reviewed
tools/​sbt-thicket/​shared/​src/​main/​scala/​thicket/​sbt/​ThicketNativeFlags.scala Reviewed
tools/​sbt-thicket/​README.md Reviewed
README.md Reviewed
modules/​renderer-apple/​src/​main/​scala/​thicket/​renderer/​apple/​AppleRenderer.scala Reviewed
modules/​renderer-apple/​shim/​include/​thicket_apple.h Reviewed
modules/​renderer-apple/​README.md Reviewed
modules/​camera-preview/​README.md Reviewed
modules/​camera-preview-apple/​src/​main/​scala/​thicket/​camera/​preview/​apple/​PreviewShim.scala Reviewed
modules/​camera-preview-apple/​src/​main/​scala/​thicket/​camera/​preview/​apple/​AppleCameraPreview.scala Reviewed
modules/​camera-preview-apple/​shim/​Sources/​PreviewShim.swift Reviewed
modules/​camera-preview-apple/​shim/​include/​thicket_preview.h Reviewed
modules/​camera-preview-apple/​shim/​build-shim.sh Reviewed
modules/​camera-preview-apple/​.gitignore Reviewed
examples/​gallery/​shared/​shared/​src/​main/​scala/​example/​gallery/​Gallery.scala Reviewed
examples/​gallery/​shared/​shared/​src/​main/​scala/​example/​gallery/​CameraPreviewDemo.scala Reviewed
examples/​gallery/​README.md Reviewed
examples/​gallery/​macos/​src/​main/​scala/​example/​gallery/​GalleryMac.scala Reviewed
examples/​gallery/​macos/​Info.plist Reviewed
examples/​gallery/​ios/​src/​main/​scala/​example/​gallery/​GalleryIos.scala Reviewed
examples/​gallery/​ios/​ios-app/​build-app.sh Reviewed
examples/​gallery/​apple-shared/​src/​main/​scala/​example/​gallery/​GallerySelfTest.scala Reviewed
examples/​gallery/​apple-shared/​src/​main/​scala/​example/​gallery/​GalleryPreviewChecks.scala Reviewed
examples/​gallery/​apple-shared/​src/​main/​scala/​example/​gallery/​GalleryDeviceChecks.scala Reviewed
docs/​decisions.md Reviewed
docs/​17-camera-preview.md Reviewed
docs/​16-device-services.md Reviewed
docs/​14-releasing.md Reviewed
docs/​12-component-status.md Reviewed
build.sbt Reviewed
bin/​verify-apple.sh Reviewed
.scalafmt.conf Reviewed

Review effort: Balanced

Clarify comment regarding NSView and UIView, and add include guard for thicket_apple_types.h.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
rleibman added a commit that referenced this pull request Nov 13, 2026
… step (#159)

Two faults left on main after #151's merge and #156.

Prop.View: #151's merge brought back the placeholder that set "<description>:
not available" text, so a Platform host never got its view. Control on the
macOS gallery without this line: 4 Platform checks fail (the NSTextField is
not in the window, the "not available" label, the replacement, the release).

Stepper: StepperView.show() printed NSDecimalNumber's stringValue, which drops
trailing zeros, so a step of 0.25 showed "1.5", not "1.50", on AppKit and
UIKit. It now rounds with NSDecimalNumber and prints exactly `digits`
decimals, as Steps.format does. Before: both galleries failed "showing its
value to its step's precision" and "- moves it back a step" (main's own CI on
9f2a3bd, and #158's run on its merge with main).

Measured: bin/verify-apple.sh "iPhone 17e" on this tree, 20/20 steps, at a
load average of 80-180 (Apple CI ran beside it). macOS gallery 237 ok per
run, iOS 252 ok.

Co-authored-by: Roberto Leibman <rleibman@magrathea.local>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
rleibman and others added 2 commits November 12, 2026 16:42
One conflict: docs/decisions.md, where main's two 2026-11-12 entries and this
branch's 2026-11-03 camera-preview entry were both added at the top. All kept,
newest first.

Brought in line with rules main gained since this branch:
- thicket-camera-preview-apple sets `description` (#157: every published module
  does; the POM's description is what Central's search shows).
- The Swift-carrying Apple modules are eight with this one, not seven:
  docs/14 §14.5 and bin/release.sh's header. docs/14's 2026-11-09 rehearsal
  record keeps "seven", which was true then.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
f3173b1 (a Copilot autofix) put an include guard around `sui_handle` in
Emit.scala's preview header but did not regenerate the header, so shimGen's
"the checked-in generated files are exactly what Abi generates" failed:
stale modules/camera-preview-apple/shim/include/thicket_preview.h. This is
`sbt shimGen/run`'s output, and the only file it changes.

The guard is kept: thicket_apple_types.h defines THICKET_APPLE_TYPES_H and
typedefs sui_handle too, and the iOS bridging headers include it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

0