Repository navigation
bench: build and validate Swift/iOS Device Farm harness - #16
Merged
Merged
Conversation
5 tasks done
adityamukho
marked this pull request as ready for review
November 5, 2026 02:59
7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the Swift/iOS harness for project-minigraf/minigraf#839: six common binding round trips plus eight preparation, registration, options, typed-log and storage cases. Every invocation validates actual results; storage cases reopen and compare contents. The benchmark rejects simulator execution.
GitHub build/correctness run 41851223675 passed 44 Rust tests and both XCTest correctness tests, including all fourteen smoke cases with zero benchmark samples. Release IPA/XCTest checksums, embedded build identity, test binaries and the harness hash were verified. Regular binding/package/consumer CI also passes.
Physical iPhone 15 / iOS 18.5 smoke and oracle tests passed. Three complete physical timing series each passed fourteen cases and returned 140 validated samples (420 total), using identical packages from d89603d. All recorded thermal snapshots were nominal; no continuous stability, idle, power or fixed-physical-unit claim is made.
The first smoke job used an ineffective parameters.filter and ran oracle, smoke, then benchmark in fresh app processes; its complete benchmark is retained as series 1. Later runs used test.filter and ran only testBenchmark. Customer JSONL file attachment failed; complete records were recovered from one automation log per series and checked for exact build identity, case/sample completeness and timing arithmetic. The core evidence PR #852 retains all samples and documents the exact timing boundaries, run IDs, provenance and collection limitations.
Existing nightly benchmarks remain unchanged. These are source-build measurements from Device Farm, with no GitHub/macOS timing evidence or release acceptance implied. Both maintainer sign-offs and the proposed remaining dispositions await the requesting maintainer's completed-evidence review.