Prerequisites
Before you set it up, make sure you have already:- Created mobile project (can be without uploaded app)
- Created at least one mobile test
Required environment variables
| Environment variable | Description |
|---|---|
TESTERARMY_API_KEY | API key for TesterArmy |
TESTERARMY_PROJECT_ID | Project ID for project you want to run tests on |
TESTERARMY_GROUP_ID | Group ID for group you want to run tests on |
TESTERARMY_DYNAMIC_AGENT_ENABLED | Optional. Defaults to true; set to false to skip the exploration agent on pull requests |
- API key - Settings → API Keys
- Project ID - Project Settings
- Group ID - Test Tab, then click on three dots on the group you want to run tests on and copy the ID
Set
TESTERARMY_API_KEY as an EAS secret or sensitive environment variable. Do not commit it, and
do not expose it through an EXPO_PUBLIC_* environment variable.Instructions for AI agents
Below you will find instructions for AI agent to set up EAS integration with TesterArmy for you.Setup Expo EAS integration with TesterArmy
Configure mobile build profiles
TesterArmy requires an iOS Simulator build profile and an Android APK build profile in EAS. Add the following snippet to youreas.json file:
eas.json
{
"build": {
"testerarmy-ios-simulator": {
"ios": {
"simulator": true
}
},
"testerarmy-android-apk": {
"android": {
"buildType": "apk"
}
}
}
}
For Android builds, make sure your Expo config has an Android package name, for example
android.package in app.json.Add the EAS workflow
- Basic
- Fingerprint + Repack
- With exploration agent
Here is an example workflow file that:
- Builds the iOS Simulator app and Android APK
- Uploads each app to TesterArmy
- Runs the saved test group once for iOS and once for Android
- Deletes each uploaded app from TesterArmy after the tests complete
.eas/workflows/testerarmy-mobile-tests.yml
name: TesterArmy Mobile Tests
on:
push:
branches:
- main
pull_request:
branches:
- main
workflow_dispatch: {}
jobs:
build_ios:
name: Build iOS Simulator app
type: build
environment: preview
params:
platform: ios
profile: testerarmy-ios-simulator
build_android:
name: Build Android app
type: build
environment: preview
params:
platform: android
profile: testerarmy-android-apk
upload_ios_app:
name: Upload iOS app to TesterArmy
needs: [build_ios]
environment: preview
outputs:
app_id: ${{ fromJSON(steps.upload_app.outputs.upload_result).uploadedAppId }}
steps:
- uses: eas/checkout
- uses: eas/download_build
id: download_build
with:
build_id: ${{ needs.build_ios.outputs.build_id }}
extensions:
- app
- name: Upload app
id: upload_app
run: |
set -euo pipefail
APP_PATH="${{ steps.download_build.outputs.artifact_path }}"
mkdir -p .testerarmy
npx --yes testerarmy@latest upload-app \
--app-path "$APP_PATH" \
--project "$TESTERARMY_PROJECT_ID" \
--output .testerarmy/upload.json
set-output upload_result "$(tr -d '\n' < .testerarmy/upload.json)"
upload_android_app:
name: Upload Android app to TesterArmy
needs: [build_android]
environment: preview
outputs:
app_id: ${{ fromJSON(steps.upload_app.outputs.upload_result).uploadedAppId }}
steps:
- uses: eas/checkout
- uses: eas/download_build
id: download_build
with:
build_id: ${{ needs.build_android.outputs.build_id }}
extensions:
- apk
- name: Upload app
id: upload_app
run: |
set -euo pipefail
APP_PATH="${{ steps.download_build.outputs.artifact_path }}"
mkdir -p .testerarmy
npx --yes testerarmy@latest upload-app \
--app-path "$APP_PATH" \
--project "$TESTERARMY_PROJECT_ID" \
--output .testerarmy/upload.json
set-output upload_result "$(tr -d '\n' < .testerarmy/upload.json)"
run_ios_tests:
name: Run iOS TesterArmy tests
needs: [upload_ios_app]
environment: preview
steps:
- uses: eas/checkout
- name: Run tests
run: |
set -euo pipefail
APP_ID="${{ needs.upload_ios_app.outputs.app_id }}"
COMMIT_SHA="${{ github.sha }}"
EVENT_NAME="${{ github.event_name }}"
PR_NUMBER="${{ github.event.pull_request.number || '' }}"
TIMEOUT_MS="1800000"
POLL_INTERVAL_SECONDS="10"
mkdir -p .testerarmy
args=(
ci
--group "$TESTERARMY_GROUP_ID"
--project "$TESTERARMY_PROJECT_ID"
--platform ios
--app-id "$APP_ID"
--commit-sha "$COMMIT_SHA"
--timeout "$TIMEOUT_MS"
--poll-interval-seconds "$POLL_INTERVAL_SECONDS"
--output .testerarmy/ci-result.json
--delete-app-after-run
)
if [ "$EVENT_NAME" = "pull_request" ] && [ -n "$PR_NUMBER" ]; then
args+=(--pr-number "$PR_NUMBER")
fi
npx --yes testerarmy@latest "${args[@]}"
run_android_tests:
name: Run Android TesterArmy tests
needs: [upload_android_app]
environment: preview
steps:
- uses: eas/checkout
- name: Run tests
run: |
set -euo pipefail
APP_ID="${{ needs.upload_android_app.outputs.app_id }}"
COMMIT_SHA="${{ github.sha }}"
EVENT_NAME="${{ github.event_name }}"
PR_NUMBER="${{ github.event.pull_request.number || '' }}"
TIMEOUT_MS="1800000"
POLL_INTERVAL_SECONDS="10"
mkdir -p .testerarmy
args=(
ci
--group "$TESTERARMY_GROUP_ID"
--project "$TESTERARMY_PROJECT_ID"
--platform android
--app-id "$APP_ID"
--commit-sha "$COMMIT_SHA"
--timeout "$TIMEOUT_MS"
--poll-interval-seconds "$POLL_INTERVAL_SECONDS"
--output .testerarmy/ci-result.json
--delete-app-after-run
)
if [ "$EVENT_NAME" = "pull_request" ] && [ -n "$PR_NUMBER" ]; then
args+=(--pr-number "$PR_NUMBER")
fi
npx --yes testerarmy@latest "${args[@]}"
This version uses Expo’s fingerprint,
get-build, and repack jobs to avoid full native rebuilds when the current native runtime matches an existing iOS Simulator build or Android APK.The workflow:- Calculates the iOS and Android fingerprints
- Looks for compatible iOS Simulator and Android APK builds with matching fingerprints
- Repacks a matching build with the current JavaScript bundle when a match exists
- Falls back to full platform builds when no compatible build exists
- Uploads each resulting app to TesterArmy
- Runs the saved test group once for iOS and once for Android
Keep the fingerprint job’s
environment aligned with the build profile environment. Expo’s
fingerprint job is intended for CNG/managed projects; if your repository commits native ios or
android directories, use the basic workflow or verify your Expo fingerprint setup first..eas/workflows/testerarmy-mobile-tests.yml
name: TesterArmy Mobile Tests
on:
push:
branches:
- main
pull_request:
branches:
- main
workflow_dispatch: {}
jobs:
fingerprint:
name: Calculate app fingerprints
type: fingerprint
environment: preview
get_ios_build:
name: Find matching iOS Simulator build
needs: [fingerprint]
type: get-build
params:
platform: ios
profile: testerarmy-ios-simulator
simulator: true
fingerprint_hash: ${{ needs.fingerprint.outputs.ios_fingerprint_hash }}
wait_for_in_progress: true
get_android_build:
name: Find matching Android app build
needs: [fingerprint]
type: get-build
params:
platform: android
profile: testerarmy-android-apk
fingerprint_hash: ${{ needs.fingerprint.outputs.android_fingerprint_hash }}
wait_for_in_progress: true
repack_ios:
name: Repack iOS app
needs: [get_ios_build]
if: ${{ needs.get_ios_build.outputs.build_id }}
type: repack
params:
build_id: ${{ needs.get_ios_build.outputs.build_id }}
repack_android:
name: Repack Android app
needs: [get_android_build]
if: ${{ needs.get_android_build.outputs.build_id }}
type: repack
params:
build_id: ${{ needs.get_android_build.outputs.build_id }}
build_ios:
name: Build iOS Simulator app
needs: [get_ios_build]
if: ${{ !needs.get_ios_build.outputs.build_id }}
type: build
environment: preview
params:
platform: ios
profile: testerarmy-ios-simulator
build_android:
name: Build Android app
needs: [get_android_build]
if: ${{ !needs.get_android_build.outputs.build_id }}
type: build
environment: preview
params:
platform: android
profile: testerarmy-android-apk
upload_ios_app:
name: Upload iOS app to TesterArmy
after: [repack_ios, build_ios]
if: ${{ after.repack_ios.outputs.build_id || after.build_ios.outputs.build_id }}
environment: preview
outputs:
app_id: ${{ fromJSON(steps.upload_app.outputs.upload_result).uploadedAppId }}
steps:
- uses: eas/checkout
- uses: eas/download_build
id: download_build
with:
build_id: ${{ after.repack_ios.outputs.build_id || after.build_ios.outputs.build_id }}
extensions:
- app
- name: Upload app
id: upload_app
run: |
set -euo pipefail
APP_PATH="${{ steps.download_build.outputs.artifact_path }}"
mkdir -p .testerarmy
npx --yes testerarmy@latest upload-app \
--app-path "$APP_PATH" \
--project "$TESTERARMY_PROJECT_ID" \
--remove-after 86400 \
--output .testerarmy/upload.json
set-output upload_result "$(tr -d '\n' < .testerarmy/upload.json)"
upload_android_app:
name: Upload Android app to TesterArmy
after: [repack_android, build_android]
if: ${{ after.repack_android.outputs.build_id || after.build_android.outputs.build_id }}
environment: preview
outputs:
app_id: ${{ fromJSON(steps.upload_app.outputs.upload_result).uploadedAppId }}
steps:
- uses: eas/checkout
- uses: eas/download_build
id: download_build
with:
build_id: ${{ after.repack_android.outputs.build_id || after.build_android.outputs.build_id }}
extensions:
- apk
- name: Upload app
id: upload_app
run: |
set -euo pipefail
APP_PATH="${{ steps.download_build.outputs.artifact_path }}"
mkdir -p .testerarmy
npx --yes testerarmy@latest upload-app \
--app-path "$APP_PATH" \
--project "$TESTERARMY_PROJECT_ID" \
--remove-after 86400 \
--output .testerarmy/upload.json
set-output upload_result "$(tr -d '\n' < .testerarmy/upload.json)"
run_ios_tests:
name: Run iOS TesterArmy tests
needs: [upload_ios_app]
environment: preview
steps:
- uses: eas/checkout
- name: Run tests
run: |
set -euo pipefail
APP_ID="${{ needs.upload_ios_app.outputs.app_id }}"
COMMIT_SHA="${{ github.sha }}"
EVENT_NAME="${{ github.event_name }}"
PR_NUMBER="${{ github.event.pull_request.number || '' }}"
TIMEOUT_MS="1800000"
POLL_INTERVAL_SECONDS="10"
mkdir -p .testerarmy
args=(
ci
--group "$TESTERARMY_GROUP_ID"
--project "$TESTERARMY_PROJECT_ID"
--platform ios
--app-id "$APP_ID"
--commit-sha "$COMMIT_SHA"
--timeout "$TIMEOUT_MS"
--poll-interval-seconds "$POLL_INTERVAL_SECONDS"
--output .testerarmy/ci-result.json
--delete-app-after-run
)
if [ "$EVENT_NAME" = "pull_request" ] && [ -n "$PR_NUMBER" ]; then
args+=(--pr-number "$PR_NUMBER")
fi
npx --yes testerarmy@latest "${args[@]}"
run_android_tests:
name: Run Android TesterArmy tests
needs: [upload_android_app]
environment: preview
steps:
- uses: eas/checkout
- name: Run tests
run: |
set -euo pipefail
APP_ID="${{ needs.upload_android_app.outputs.app_id }}"
COMMIT_SHA="${{ github.sha }}"
EVENT_NAME="${{ github.event_name }}"
PR_NUMBER="${{ github.event.pull_request.number || '' }}"
TIMEOUT_MS="1800000"
POLL_INTERVAL_SECONDS="10"
mkdir -p .testerarmy
args=(
ci
--group "$TESTERARMY_GROUP_ID"
--project "$TESTERARMY_PROJECT_ID"
--platform android
--app-id "$APP_ID"
--commit-sha "$COMMIT_SHA"
--timeout "$TIMEOUT_MS"
--poll-interval-seconds "$POLL_INTERVAL_SECONDS"
--output .testerarmy/ci-result.json
--delete-app-after-run
)
if [ "$EVENT_NAME" = "pull_request" ] && [ -n "$PR_NUMBER" ]; then
args+=(--pr-number "$PR_NUMBER")
fi
npx --yes testerarmy@latest "${args[@]}"
This version runs the saved test group and, on pull requests, also runs the exploration agent, which explores what the pull request changes instead of running a fixed group. Both run against a single shared upload.The workflow:
- Builds the iOS Simulator app and Android APK
- Uploads each app to TesterArmy once
- Runs the saved test group once for iOS and once for Android
- On pull requests, runs the exploration agent for each platform against the same upload
.eas/workflows/testerarmy-mobile-tests.yml
name: TesterArmy Mobile Tests
on:
push:
branches:
- main
pull_request:
branches:
- main
workflow_dispatch: {}
jobs:
build_ios:
name: Build iOS Simulator app
type: build
environment: preview
params:
platform: ios
profile: testerarmy-ios-simulator
build_android:
name: Build Android app
type: build
environment: preview
params:
platform: android
profile: testerarmy-android-apk
upload_ios_app:
name: Upload iOS app to TesterArmy
needs: [build_ios]
environment: preview
outputs:
app_id: ${{ fromJSON(steps.upload_app.outputs.upload_result).uploadedAppId }}
steps:
- uses: eas/checkout
- uses: eas/download_build
id: download_build
with:
build_id: ${{ needs.build_ios.outputs.build_id }}
extensions:
- app
- name: Upload app
id: upload_app
run: |
set -euo pipefail
APP_PATH="${{ steps.download_build.outputs.artifact_path }}"
mkdir -p .testerarmy
npx --yes testerarmy@latest upload-app \
--app-path "$APP_PATH" \
--project "$TESTERARMY_PROJECT_ID" \
--remove-after 86400 \
--output .testerarmy/upload.json
set-output upload_result "$(tr -d '\n' < .testerarmy/upload.json)"
upload_android_app:
name: Upload Android app to TesterArmy
needs: [build_android]
environment: preview
outputs:
app_id: ${{ fromJSON(steps.upload_app.outputs.upload_result).uploadedAppId }}
steps:
- uses: eas/checkout
- uses: eas/download_build
id: download_build
with:
build_id: ${{ needs.build_android.outputs.build_id }}
extensions:
- apk
- name: Upload app
id: upload_app
run: |
set -euo pipefail
APP_PATH="${{ steps.download_build.outputs.artifact_path }}"
mkdir -p .testerarmy
npx --yes testerarmy@latest upload-app \
--app-path "$APP_PATH" \
--project "$TESTERARMY_PROJECT_ID" \
--remove-after 86400 \
--output .testerarmy/upload.json
set-output upload_result "$(tr -d '\n' < .testerarmy/upload.json)"
run_ios_tests:
name: Run iOS TesterArmy tests
needs: [upload_ios_app]
environment: preview
steps:
- uses: eas/checkout
- name: Run tests
run: |
set -euo pipefail
APP_ID="${{ needs.upload_ios_app.outputs.app_id }}"
COMMIT_SHA="${{ github.sha }}"
EVENT_NAME="${{ github.event_name }}"
PR_NUMBER="${{ github.event.pull_request.number || '' }}"
TIMEOUT_MS="1800000"
POLL_INTERVAL_SECONDS="10"
mkdir -p .testerarmy
args=(
ci
--group "$TESTERARMY_GROUP_ID"
--project "$TESTERARMY_PROJECT_ID"
--platform ios
--app-id "$APP_ID"
--commit-sha "$COMMIT_SHA"
--timeout "$TIMEOUT_MS"
--poll-interval-seconds "$POLL_INTERVAL_SECONDS"
--output .testerarmy/ci-result.json
)
if [ "$EVENT_NAME" = "pull_request" ] && [ -n "$PR_NUMBER" ]; then
args+=(--pr-number "$PR_NUMBER")
fi
npx --yes testerarmy@latest "${args[@]}"
run_android_tests:
name: Run Android TesterArmy tests
needs: [upload_android_app]
environment: preview
steps:
- uses: eas/checkout
- name: Run tests
run: |
set -euo pipefail
APP_ID="${{ needs.upload_android_app.outputs.app_id }}"
COMMIT_SHA="${{ github.sha }}"
EVENT_NAME="${{ github.event_name }}"
PR_NUMBER="${{ github.event.pull_request.number || '' }}"
TIMEOUT_MS="1800000"
POLL_INTERVAL_SECONDS="10"
mkdir -p .testerarmy
args=(
ci
--group "$TESTERARMY_GROUP_ID"
--project "$TESTERARMY_PROJECT_ID"
--platform android
--app-id "$APP_ID"
--commit-sha "$COMMIT_SHA"
--timeout "$TIMEOUT_MS"
--poll-interval-seconds "$POLL_INTERVAL_SECONDS"
--output .testerarmy/ci-result.json
)
if [ "$EVENT_NAME" = "pull_request" ] && [ -n "$PR_NUMBER" ]; then
args+=(--pr-number "$PR_NUMBER")
fi
npx --yes testerarmy@latest "${args[@]}"
run_ios_dynamic_agent:
name: Run iOS TesterArmy exploration agent
needs: [upload_ios_app]
if: ${{ github.event_name == 'pull_request' }}
environment: preview
env:
APP_ID: ${{ needs.upload_ios_app.outputs.app_id }}
COMMIT_SHA: ${{ github.sha }}
PR_NUMBER: ${{ github.event.pull_request.number || '' }}
PR_TITLE: ${{ github.event.pull_request.title || '' }}
PR_DESCRIPTION: ${{ github.event.pull_request.body || '' }}
BASE_BRANCH: ${{ github.event.pull_request.base.ref || '' }}
HEAD_BRANCH: ${{ github.event.pull_request.head.ref || '' }}
steps:
- uses: eas/checkout
- name: Run exploration agent
run: |
set -euo pipefail
if [ "${TESTERARMY_DYNAMIC_AGENT_ENABLED:-true}" = "false" ]; then
echo "TesterArmy exploration agent is disabled because TESTERARMY_DYNAMIC_AGENT_ENABLED=false."
exit 0
fi
if [ -z "$PR_NUMBER" ]; then
echo "No pull request number found; skipping TesterArmy exploration agent."
exit 0
fi
mkdir -p .testerarmy
rm -f .testerarmy/dynamic-result.json
dynamic_pr_title="$PR_TITLE"
if [ -z "$dynamic_pr_title" ]; then
dynamic_pr_title="Pull request #$PR_NUMBER"
fi
args=(
pr
run-dynamic
--project "$TESTERARMY_PROJECT_ID"
--platform ios
--app-id "$APP_ID"
--pr-number "$PR_NUMBER"
--pr-title "$dynamic_pr_title"
--commit-sha "$COMMIT_SHA"
--output .testerarmy/dynamic-result.json
)
if [ -n "$PR_DESCRIPTION" ]; then
args+=(--pr-description "$PR_DESCRIPTION")
fi
if [ -n "$BASE_BRANCH" ]; then
args+=(--base-branch "$BASE_BRANCH")
fi
if [ -n "$HEAD_BRANCH" ]; then
args+=(--head-branch "$HEAD_BRANCH")
fi
npx --yes testerarmy@latest "${args[@]}"
run_android_dynamic_agent:
name: Run Android TesterArmy exploration agent
needs: [upload_android_app]
if: ${{ github.event_name == 'pull_request' }}
environment: preview
env:
APP_ID: ${{ needs.upload_android_app.outputs.app_id }}
COMMIT_SHA: ${{ github.sha }}
PR_NUMBER: ${{ github.event.pull_request.number || '' }}
PR_TITLE: ${{ github.event.pull_request.title || '' }}
PR_DESCRIPTION: ${{ github.event.pull_request.body || '' }}
BASE_BRANCH: ${{ github.event.pull_request.base.ref || '' }}
HEAD_BRANCH: ${{ github.event.pull_request.head.ref || '' }}
steps:
- uses: eas/checkout
- name: Run exploration agent
run: |
set -euo pipefail
if [ "${TESTERARMY_DYNAMIC_AGENT_ENABLED:-true}" = "false" ]; then
echo "TesterArmy exploration agent is disabled because TESTERARMY_DYNAMIC_AGENT_ENABLED=false."
exit 0
fi
if [ -z "$PR_NUMBER" ]; then
echo "No pull request number found; skipping TesterArmy exploration agent."
exit 0
fi
mkdir -p .testerarmy
rm -f .testerarmy/dynamic-result.json
dynamic_pr_title="$PR_TITLE"
if [ -z "$dynamic_pr_title" ]; then
dynamic_pr_title="Pull request #$PR_NUMBER"
fi
args=(
pr
run-dynamic
--project "$TESTERARMY_PROJECT_ID"
--platform android
--app-id "$APP_ID"
--pr-number "$PR_NUMBER"
--pr-title "$dynamic_pr_title"
--commit-sha "$COMMIT_SHA"
--output .testerarmy/dynamic-result.json
)
if [ -n "$PR_DESCRIPTION" ]; then
args+=(--pr-description "$PR_DESCRIPTION")
fi
if [ -n "$BASE_BRANCH" ]; then
args+=(--base-branch "$BASE_BRANCH")
fi
if [ -n "$HEAD_BRANCH" ]; then
args+=(--head-branch "$HEAD_BRANCH")
fi
npx --yes testerarmy@latest "${args[@]}"
Exploration agent
On pull requests you can also run the exploration agent, which explores what a pull request changes instead of running a fixed test group. See the With exploration agent tab above for a complete workflow. The agent runs on the same app you already uploaded and only on pull requests. To turn it off, setTESTERARMY_DYNAMIC_AGENT_ENABLED=false.
If a pull request doesn’t change anything users can see in the app, the exploration run is skipped: the PR check shows a neutral “Tests skipped” status with the reason, the run is recorded as cancelled with that reason, and the CLI reports it as skipped and exits 0, so it never blocks the PR. Real failures and cancellations still fail the job.
GitHub PR comments and checks
When your workflow is triggered by a pull request, TesterArmy will create a GitHub check and comment on the PR. When workflow is triggered by a commit (eg. on push to main), TesterArmy will create a GitHub check on the commit. To enable this, you need to configure your TesterArmy project with your GitHub repository. You can do it in Integrations Tab in project.Example repository
You can find a working example repository here.Troubleshooting
The build has an unsupported platform or format
TesterArmy requires an iOS Simulator.app build or Android .apk build. If you are using an iOS physical-device build, build a simulator app first. If you are using Android, make sure the EAS profile produces an APK, not an .aab.
Prefer release builds for CI, especially for Android runs that execute without a Metro dev
server.