Flank is a massively parallel Android and iOS test runner for Firebase Test Lab.
- Use JetBrains Toolbox to install
IntelliJ IDEA Community - Clone the repo
git clone https://github.com/TestArmada/flank.git - Open
test_runner/build.gradle.ktswithIntelliJ IDEA Community
Run test_runner/flank.ios.yml with flank to verify iOS execution is working.
cd test_runner/./gradlew clean assemble fatJarjava -jar ./build/libs/flank-*.jar firebase test ios run
# gcloud args match the official gcloud cli
# https://cloud.google.com/sdk/gcloud/reference/alpha/firebase/test/ios/run
gcloud:
# results-bucket: earlgrey-swift
record-video: true
timeout: 30m
async: false
# project: delta-essence-114723
# test and xctestrun-file are the only required args
test: ./src/test/kotlin/ftl/fixtures/tmp/EarlGreyExample.zip
xctestrun-file: ./src/test/kotlin/ftl/fixtures/tmp/EarlGreyExampleSwiftTests_iphoneos11.2-arm64.xctestrun
flank:
# test shards - the amount of groups to split the test suite into
# set to -1 to use one shard per test.
testShards: 1
# test runs - the amount of times to run the tests.
# 1 runs the tests once. 10 runs all the tests 10x
testRuns: 1
# always run - these tests are inserted at the beginning of every shard
# useful if you need to grant permissions or login before other tests run
test-targets-always-run:
- a/testGrantPermissions
# test targets - a list of tests to run. omit to run all tests.
test-targets:
- b/testBasicSelectionRun test_runner/flank.yml with flank to verify Android execution is working.
cd test_runner/./gradlew clean assemble fatJarjava -jar ./build/libs/flank-*.jar firebase test android run
# gcloud args match the official gcloud cli
# https://cloud.google.com/sdk/gcloud/reference/firebase/test/android/run
gcloud:
results-bucket: tmp_bucket_2
record-video: true
timeout: 60m
async: false
project: delta-essence-114723
# test and app are the only required args
app: ../../test_app/apks/app-debug.apk
test: ../../test_app/apks/app-debug-androidTest.apk
auto-google-login: true
use-orchestrator: true
environment-variables:
clearPackageData: true
directories-to-pull:
- /sdcard/screenshots
performance-metrics: true
test-targets:
- class com.example.app.ExampleUiTest#testPasses
device:
- model: NexusLowRes
version: 28
flank:
# test shards - the amount of groups to split the test suite into
# set to -1 to use one shard per test.
testShards: 1
# test runs - the amount of times to run the tests.
# 1 runs the tests once. 10 runs all the tests 10x
testRuns: 1
# always run - these tests are inserted at the beginning of every shard
# useful if you need to grant permissions or login before other tests run
test-targets-always-run:
- class com.example.app.ExampleUiTest#testPassesDownload Flank from GitHub releases. The snapshot jar tracks master and is always up to date. Alternatively, rehost the jar to avoid automatically updating to new versions of Flank.
wget --quiet https://github.com/TestArmada/flank/releases/download/flank_snapshot/flank.jar -O ./flank.jar
In CI, it may be useful to generate the file via a shell script:
cat << 'EOF' > ./flank.yml
gcloud:
app: ../../test_app/apks/app-debug.apk
test: ../../test_app/apks/app-debug-androidTest.apk
EOF
Follow the test lab docs to create a service account.
- Save the credential to
$HOME/.config/gcloud/application_default_credentials.jsonor setGOOGLE_APPLICATION_CREDENTIALSwhen using a custom path. - Set the project id in flank.yml or set the
GOOGLE_CLOUD_PROJECTenvironment variable.
For continuous integration, base64 encode the credential as GCLOUD_KEY. Then write the file using a shell script. Note that gcloud CLI does not need to be installed. Flank works without any dependency on gcloud CLI.
GCLOUD_DIR="$HOME/.config/gcloud/"
mkdir -p "$GCLOUD_DIR"
echo "$GCLOUD_KEY" | base64 --decode > "$GCLOUD_DIR/application_default_credentials.json"flank.yml is compatible with the gcloud CLI.
gcloud firebase test android run flank.yml:gcloudgcloud alpha firebase test ios run flank.ios.yml:gcloud
Use the doctor command to check for errors in the YAML.
flank firebase test android doctorflank firebase test ios doctor
Flank supports printing the current version.
$ flank -v
v3.0-SNAPSHOTAccess Not Configured. Cloud Tool Results API has not been used in project 764086051850 before or it is disabled.
This error means authentication hasn't been setup properly. See Authenticate with a service account in this readme.
How do I use Flank without typing long commands?
Place the flank bash helper on the path to use flank instead of java -jar flank.jar. Examples:
flank android runflank ios run