Skip to main content
Connect one GitHub.com repository to a TesterArmy project to run tests against pull request deployments and report results in GitHub. The native GitHub integration can:
  • Discover pull requests and deployment previews.
  • Read PR metadata and changed files for the exploration agent.
  • Run a saved regression group, the exploration agent, or both.
  • Create best-effort GitHub Check Runs and PR comments.
  • Filter automatic runs by the PR’s target branch.

​
1. Install the GitHub App

  1. Open your TesterArmy project.
  2. Go to Integrations → GitHub.
  3. Select Install GitHub App.
  4. Choose the GitHub organization or account and grant access to the repositories TesterArmy should test.
  5. Return to TesterArmy and select Refresh if the installation is not shown yet.
If TesterArmy asks you to reconnect your GitHub account, select Connect GitHub account before continuing.

​
2. Connect a repository

  1. Select the GitHub Installation.
  2. Select the Repository for this TesterArmy project.
  3. Optionally select a Target branch. Choose main, for example, to run automatic tests only for PRs targeting main.
  4. Select Save connection.
Each TesterArmy project connects to one GitHub repository. A repository connection cannot be shared by multiple TesterArmy projects.

​
3. Choose PR coverage

Open the project’s PR Testing tab and enable one or both modes:

​
4. Connect preview delivery

Choose how TesterArmy receives the deployed URL:

​
GitHub reporting behavior

GitHub reporting is best effort and does not block the TesterArmy run. Check Runs conclude success when all tests pass. By default, failed, cancelled, skipped, or mixed batches use a neutral conclusion with details in the check title and summary. Saved regression groups and exploration runs can maintain separate PR comments because they report different kinds of coverage.

​
Blocking merges on failed tests

GitHub counts a neutral conclusion as passing, so with the default reporting a failed test never blocks a merge. Enable Block merges on failed tests in the project’s PR Testing tab (or set prTestingBlockMerge through the API) to conclude failed tests as failure instead. Then mark the TesterArmy checks you care about - TesterArmy / Exploration Test and TesterArmy / <group name> - as required status checks in your branch protection or ruleset. When several projects share the repository, each project’s checks carry a (<project name>) suffix; require the suffixed names. Runs without a product verdict stay neutral even when the setting is on: environment- or setup-blocked runs, runs that crashed before reaching a verdict, cancellations, and PRs the exploration agent skipped. Because a check run only exists after a deployment TesterArmy accepted, a required check on a PR that never produces that deployment waits indefinitely - only mark a check required if every PR you want to merge triggers it.

​
Limitations

  • The native integration supports GitHub.com. GitHub Enterprise Server cannot install this GitHub App flow.
  • TesterArmy reads only the repository connected to the project.
  • Automatic PR testing requires a deployed URL that TesterArmy’s hosted runners can reach.
For a non-native or GitHub Enterprise Server pipeline, use an API or webhook trigger. Without an active supported GitHub connection, exploration planning uses the PR metadata supplied by the caller and cannot read repository changes or post GitHub results.

​
Troubleshooting

​
Repository is missing

Update the GitHub App installation to grant access to the repository, then return to TesterArmy and select Refresh.

​
GitHub permissions need reconnect

Select Connect GitHub account in the integration panel, complete authorization, and retry the connection.

​
No PR run appears

Confirm that PR testing is enabled, the deployment completed successfully, the preview provider is configured, and the PR targets the selected branch.