Skip to content

Commit

Permalink
add e2e-flakiness-detector GitHub action
Browse files Browse the repository at this point in the history
Every day at 1400 UTC, this action runs the e2e tests many times (10 times per test case) to detect intermittent flakiness proactively instead of having it be occasionally detected in unrelated PRs. This information will help us fix flaky e2e tests.
  • Loading branch information
sqs committed Mar 18, 2024
1 parent 5abac6a commit 516bbbf
Showing 1 changed file with 45 additions and 0 deletions.
45 changes: 45 additions & 0 deletions .github/workflows/e2e-flakiness-detector.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# Runs the e2e tests many times to detect intermittent flakiness proactively instead of having it be
# occasionally detected in unrelated PRs.
name: e2e-flakiness-detector

on:
schedule:
- cron: '0 14 * * *' # daily at 1400 UTC
workflow_dispatch:

jobs:
test-e2e:
strategy:
fail-fast: false
matrix:
runner: [ubuntu, windows]
runs-on: ${{ matrix.runner }}-latest
timeout-minutes: 15
permissions:
id-token: write
contents: read
steps:
- uses: actions/checkout@v3
- uses: actions/setup-node@v3
with:
node-version-file: .tool-versions
- uses: pnpm/action-setup@d882d12c64e032187b2edb46d3a0d003b7a43598 # SECURITY: pin third-party action hashes
- run: echo "STORE_PATH=$(pnpm store path)" >> $GITHUB_OUTPUT
shell: bash
id: pnpm-cache
- name: Cache pnpm store
uses: actions/cache@v3
with:
path: ${{ steps.pnpm-cache.outputs.STORE_PATH }}
key: ${{ runner.os }}-pnpm-store-${{ hashFiles('**/pnpm-lock.yaml') }}
restore-keys: ${{ runner.os }}-pnpm-store-
- run: pnpm install
- run: xvfb-run -a pnpm -C vscode run test:e2e --repeat-each 10 --retries 0
if: matrix.runner == 'ubuntu'
- run: pnpm -C vscode run test:e2e --repeat-each 10 --retries 0
if: matrix.runner != 'ubuntu'
- uses: actions/upload-artifact@v3
if: ${{ failure() }}
with:
name: playwright-recordings ${{ matrix.runner }}
path: playwright/**/*.webm

0 comments on commit 516bbbf

Please sign in to comment.