Skip to content

Commit

Permalink
TT-1853 Bump flakeguard (#15306)
Browse files Browse the repository at this point in the history
* Bump flakeguard and save test results for all tests

* TO REVERT

* bump

* Revert "TO REVERT"

This reverts commit 4d5fd0b.

* Fix
  • Loading branch information
lukaszcl authored and cedric-cordenier committed Nov 20, 2024
1 parent 8c861ac commit 12021e8
Showing 1 changed file with 15 additions and 7 deletions.
22 changes: 15 additions & 7 deletions .github/workflows/find-new-flaky-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ jobs:

- name: Install flakeguard
shell: bash
run: go install github.com/smartcontractkit/chainlink-testing-framework/tools/flakeguard@9c9821d6013f4838eb26970c2eef594f4d25398b
run: go install github.com/smartcontractkit/chainlink-testing-framework/tools/flakeguard@8b02ed1703ef40755a4c46ff454cf4ff2e89275d

- name: Find new or updated test packages
if: ${{ inputs.runAllTests == false }}
Expand Down Expand Up @@ -259,7 +259,7 @@ jobs:

- name: Install flakeguard
shell: bash
run: go install github.com/smartcontractkit/chainlink-testing-framework/tools/flakeguard@9c9821d6013f4838eb26970c2eef594f4d25398b
run: go install github.com/smartcontractkit/chainlink-testing-framework/tools/flakeguard@8b02ed1703ef40755a4c46ff454cf4ff2e89275d

- name: Run tests with flakeguard
shell: bash
Expand Down Expand Up @@ -301,7 +301,7 @@ jobs:

- name: Install flakeguard
shell: bash
run: go install github.com/smartcontractkit/chainlink-testing-framework/tools/flakeguard@9c9821d6013f4838eb26970c2eef594f4d25398b
run: go install github.com/smartcontractkit/chainlink-testing-framework/tools/flakeguard@8b02ed1703ef40755a4c46ff454cf4ff2e89275d

- name: Set combined test results
id: set_test_results
Expand All @@ -316,16 +316,16 @@ jobs:
PATH=$PATH:$(go env GOPATH)/bin
export PATH
# Use flakeguard aggregate-all to aggregate test results
flakeguard aggregate-all --results-path . --output-results ../all_tests.json
# Use flakeguard to aggregate all test results
flakeguard aggregate-results --results-path . --output-results ../all_tests.json
# Count all tests
ALL_TESTS_COUNT=$(jq 'length' ../all_tests.json)
echo "All tests count: $ALL_TESTS_COUNT"
echo "all_tests_count=$ALL_TESTS_COUNT" >> "$GITHUB_OUTPUT"
# Use flakeguard aggregate-failed to filter and output failed tests based on PassRatio threshold
flakeguard aggregate-failed --threshold "${{ inputs.runThreshold }}" --min-pass-ratio=${{ env.MIN_PASS_RATIO }} --results-path . --output-results ../failed_tests.json --output-logs ../failed_test_logs.json
# Use flakeguard to filter and output failed tests based on PassRatio threshold
flakeguard aggregate-results --filter-failed=true --threshold "${{ inputs.runThreshold }}" --min-pass-ratio=${{ env.MIN_PASS_RATIO }} --results-path . --output-results ../failed_tests.json --output-logs ../failed_test_logs.json
# Count failed tests
if [ -f "../failed_tests.json" ]; then
Expand All @@ -347,6 +347,14 @@ jobs:
threshold_percentage=$(echo '${{ inputs.runThreshold }}' | awk '{printf "%.0f", $1 * 100}')
echo "threshold_percentage=$threshold_percentage" >> $GITHUB_OUTPUT
- name: Upload All Test Results as Artifact
if: ${{ fromJson(steps.set_test_results.outputs.all_tests_count) > 0 }}
uses: actions/[email protected]
with:
path: all_tests.json
name: all-test-results.json
retention-days: 7

- name: Upload Failed Test Results as Artifact
if: ${{ fromJson(steps.set_test_results.outputs.failed_tests_count) > 0 }}
uses: actions/[email protected]
Expand Down

0 comments on commit 12021e8

Please sign in to comment.