Automated end-to-end tests for korthout/backport-action.
To improve the reliability of the backport-action and to have confidence when improving it.
Users of the backport-action run it in different ways:
- triggered by different events:
pull_request,pull_request_target,issue_comment, etc. - using local or remote pull requests (whether or not the pull request comes from a forked repo)
- using one of the merge strategies:
merge-commit,rebase,squash
All of these scenarios should be accounted for in automated tests. Of course, there are unit and integration tests but these don't really show that the action works in the above cases.
This test suite covers the following cases:
| case. | event | pull request | merge strategy | # commits |
|---|---|---|---|---|
| 1. | pull_request |
local |
merge-commit |
1 |
| 2. | pull_request_target |
fork |
merge-commit |
1 |
| 3. | pull_request |
local |
squash |
2 |
| 4. | pull_request |
local |
rebase |
1 |
| 5. | pull_request |
local |
rebase |
2 |
The tests in this repository simulate user behavior: they commit changes, open pull requests and merge them. They also wait for the backport-action to create the relevant pull requests, and inspect that these contain the right changes. In other words, they seek to answer: could the backport-action successfully backport the changes in this scenario?
The Test workflow orchestrates the test case execution. Each case is represented by a job. They all checkout this repo (note, cases dealing with forked repos checkout a fork of this repo), and then run the related test case script. The test case script then commits changes, opens and merges the pull request and verifies the created backport pull request. The scripts also do their own clean-up, e.g. closing any left-over pull requests.
For each case, a specific backport-action workflow is defined,
e.g. backport-pr-closed.yml belongs to case 1.
To make sure they are triggered as a result of the Test workflow, each job provides a
PAT
instead of the GITHUB_TOKEN.
If you want to run the test against the backport-action, then:
- Navigate to the Test workflow
- Click
Run workflow - (Optionally) choose case: all, 1, 2, etc
If you want to run the tests against a specific version, then:
- Refer to your specific version or branch in all the
backport-pr-*.ymlworkflow files, e.g. .github/workflows/backport-pr-closed.yml. - Commit (e.g.
test: backport-action/pull/255) and push this tomain - Run the test as described above
- Don't forget to revert the commit you pushed to
main