Restrict the number of E2E jobs that can run in parallel

[jeromy-cannon]

potential solution is using matrix strategy. See examples:

example solution from chatgpt:

To restructure this workflow so that all jobs starting with "e2e-" run in a matrix strategy with a max-parallel value of 5, follow these steps:

1. Consolidate the "e2e-" jobs into a matrix
   We will create a job matrix that dynamically runs the e2e- jobs while preserving the custom configuration for each job.

2. Define the matrix with max-parallel
   Use the max-parallel feature to control the maximum number of jobs running concurrently.

3. Update each e2e- job to work with matrix variables
   Here’s how you can restructure the workflow:

name: "Build Application"
on:
  workflow_dispatch:
    inputs:
      enable-unit-tests:
        description: "Unit Testing Enabled"
        type: boolean
        required: false
        default: true
      enable-e2e-tests:
        description: "E2E Testing Enabled"
        type: boolean
        required: false
        default: false
      enable-snyk-scan:
        description: "Snyk Scan Enabled"
        type: boolean
        required: false
        default: false
  push:
    branches:
      - main
      - 'release/*'

defaults:
  run:
    shell: bash

jobs:
  env-vars:
    name: Set Environment Variables
    uses: ./.github/workflows/zxc-env-vars.yaml
    with:
      custom-job-label: Set Environment Variables

  code-style:
    name: Code Style
    uses: ./.github/workflows/zxc-code-style.yaml
    with:
      custom-job-label: Standard

  unit-tests:
    name: Unit Tests
    uses: ./.github/workflows/zxc-unit-test.yaml
    if: ${{ github.event_name == 'push' || github.event.inputs.enable-unit-tests == 'true' }}
    needs:
      - code-style
    with:
      custom-job-label: Standard

  e2e-tests:
    name: E2E Tests Matrix
    if: ${{ github.event_name == 'push' || github.event.inputs.enable-e2e-tests == 'true' }}
    needs:
      - env-vars
      - code-style
    strategy:
      matrix:
        e2e-test-type: 
          - { name: "Standard", npm-test-script: "test-${{ needs.env-vars.outputs.e2e-standard-test-subdir }}", coverage-subdirectory: "${{ needs.env-vars.outputs.e2e-standard-test-subdir }}", coverage-report-name: "${{ needs.env-vars.outputs.e2e-standard-coverage-report }}" }
          - { name: "Mirror Node", npm-test-script: "test-${{ needs.env-vars.outputs.e2e-mirror-node-test-subdir }}", coverage-subdirectory: "${{ needs.env-vars.outputs.e2e-mirror-node-test-subdir }}", coverage-report-name: "${{ needs.env-vars.outputs.e2e-mirror-node-coverage-report }}" }
          - { name: "Node PEM Stop", npm-test-script: "test-${{ needs.env-vars.outputs.e2e-node-pem-stop-test-subdir }}", coverage-subdirectory: "${{ needs.env-vars.outputs.e2e-node-pem-stop-test-subdir }}", coverage-report-name: "${{ needs.env-vars.outputs.e2e-node-pem-stop-coverage-report }}" }
          - { name: "Node PEM Kill", npm-test-script: "test-${{ needs.env-vars.outputs.e2e-node-pem-kill-test-subdir }}", coverage-subdirectory: "${{ needs.env-vars.outputs.e2e-node-pem-kill-test-subdir }}", coverage-report-name: "${{ needs.env-vars.outputs.e2e-node-pem-kill-coverage-report }}" }
          - { name: "Node Local Build", npm-test-script: "test-${{ needs.env-vars.outputs.e2e-node-local-build-test-subdir }}", coverage-subdirectory: "${{ needs.env-vars.outputs.e2e-node-local-build-test-subdir }}", coverage-report-name: "${{ needs.env-vars.outputs.e2e-node-local-build-coverage-report }}" }
          - { name: "Node Add", npm-test-script: "test-${{ needs.env-vars.outputs.e2e-node-add-test-subdir }}", coverage-subdirectory: "${{ needs.env-vars.outputs.e2e-node-add-test-subdir }}", coverage-report-name: "${{ needs.env-vars.outputs.e2e-node-add-coverage-report }}" }
          - { name: "Node Add - Separate commands", npm-test-script: "test-${{ needs.env-vars.outputs.e2e-node-add-separate-commands-test-subdir }}", coverage-subdirectory: "${{ needs.env-vars.outputs.e2e-node-add-separate-commands-test-subdir }}", coverage-report-name: "${{ needs.env-vars.outputs.e2e-node-add-separate-commands-coverage-report }}" }
          - { name: "Node Update", npm-test-script: "test-${{ needs.env-vars.outputs.e2e-node-update-test-subdir }}", coverage-subdirectory: "${{ needs.env-vars.outputs.e2e-node-update-test-subdir }}", coverage-report-name: "${{ needs.env-vars.outputs.e2e-node-update-coverage-report }}" }
          - { name: "Node Delete", npm-test-script: "test-${{ needs.env-vars.outputs.e2e-node-delete-test-subdir }}", coverage-subdirectory: "${{ needs.env-vars.outputs.e2e-node-delete-test-subdir }}", coverage-report-name: "${{ needs.env-vars.outputs.e2e-node-delete-coverage-report }}" }
          - { name: "Relay", npm-test-script: "test-${{ needs.env-vars.outputs.e2e-relay-test-subdir }}", coverage-subdirectory: "${{ needs.env-vars.outputs.e2e-relay-test-subdir }}", coverage-report-name: "${{ needs.env-vars.outputs.e2e-relay-coverage-report }}" }
      max-parallel: 5
    runs-on: ubuntu-latest
    steps:
      - name: Run E2E Tests ${{ matrix.e2e-test-type.name }}
        uses: ./.github/workflows/zxc-e2e-test.yaml
        with:
          custom-job-label: ${{ matrix.e2e-test-type.name }}
          npm-test-script: ${{ matrix.e2e-test-type.npm-test-script }}
          coverage-subdirectory: ${{ matrix.e2e-test-type.coverage-subdirectory }}
          coverage-report-name: ${{ matrix.e2e-test-type.coverage-report-name }}

  analyze:
    name: Analyze
    uses: ./.github/workflows/zxc-code-analysis.yaml
    needs:
      - env-vars
      - unit-tests
      - e2e-tests
    if: ${{ (github.event_name == 'push' || github.event.inputs.enable-unit-tests == 'true' || github.event.inputs.enable-e2e-tests == 'true') && !failure() && !cancelled() }}
    with:
      custom-job-label: Source Code
      enable-codecov-analysis: true
      enable-codacy-coverage: true
      enable-e2e-coverage-report: ${{ github.event_name == 'push' || github.event.inputs.enable-e2e-tests == 'true' }}
    secrets:
      snyk-token: ${{ secrets.SNYK_TOKEN }}
      codecov-token: ${{ secrets.CODECOV_TOKEN }}
      codacy-project-token: ${{ secrets.CODACY_PROJECT_TOKEN }}

Key Changes:
Matrix Creation: All the e2e- jobs are grouped into a matrix under the e2e-tests job.
max-parallel: 5: This ensures that no more than 5 jobs will run concurrently.
Matrix Variables: Each test case (e.g., "Standard", "Mirror Node", etc.) is represented as an object in the matrix with custom properties (npm-test-script, coverage-subdirectory, and coverage-report-name).
Job Execution: The reusable workflow (zxc-e2e-test.yaml) is invoked using the matrix.e2e-test-type variables.
This setup preserves the structure of the individual E2E tests while controlling their parallelism.

example solution pulling data from JSON

https://github.com/nbusseneau/cilium/blob/7178a98ef9f11bea0e32c68f0bc941d7063ee854/.github/workflows/conformance-ginkgo.yaml#L225-L236

more...

https://github.com/search?q=path%3A.github%2F**%2F*.yaml%20strategy%20matrix%20max-parallel&type=code