Reorganised and grouped PR validation checks in fewer jobs + added pi…

…peline aws_infra typing check
alliance-genome · Apr 9, 2024 · 8950253 · 8950253
1 parent 2b4beba
commit 8950253
Show file tree

Hide file tree

Showing 2 changed files with 29 additions and 70 deletions.
diff --git a/.github/workflows/PR-validation.yml b/.github/workflows/PR-validation.yml
@@ -29,8 +29,8 @@ jobs:
         with:
           name: seq_retrieval_image
           path: /tmp/pavi_seq_retrieval_docker_image.tar
-  pipeline-seq-retrieval-python-typing-check:
-    name: pipeline/seq_retrieval python typing check
+  pipeline-seq-retrieval-code-checks:
+    name: pipeline/seq_retrieval code checks
     runs-on: ubuntu-22.04
     defaults:
       run:
@@ -49,43 +49,9 @@ jobs:
       - name: Check python typing
         run: |
           make run-python-type-check
-  pipeline-seq-retrieval-python-style-check:
-    name: pipeline/seq_retrieval python style check
-    runs-on: ubuntu-22.04
-    defaults:
-      run:
-        shell: bash
-        working-directory: ./pipeline/seq_retrieval/
-    steps:
-      - name: Check out repository code
-        uses: actions/checkout@v4
-        with:
-          fetch-depth: 0
-          sparse-checkout: |
-            pipeline/seq_retrieval/
-      - uses: actions/setup-python@v5
-        with:
-          python-version: '3.12'
       - name: Check python style
         run: |
           make run-python-style-check
-  pipeline-seq-retrieval-unit-tests:
-    name: pipeline/seq_retrieval unit tests
-    runs-on: ubuntu-22.04
-    defaults:
-      run:
-        shell: bash
-        working-directory: ./pipeline/seq_retrieval/
-    steps:
-      - name: Check out repository code
-        uses: actions/checkout@v4
-        with:
-          fetch-depth: 0
-          sparse-checkout: |
-            pipeline/seq_retrieval/
-      - uses: actions/setup-python@v5
-        with:
-          python-version: '3.12'
       - name: Run unit tests
         run: |
           make run-unit-tests
@@ -151,8 +117,8 @@ jobs:
       - name: Run integration test
         run: |
           make run-integration-test
-  pipeline-aws-infra-cdk-code-validation:
-    name: pipeline/aws_infra CDK code validation
+  pipeline-aws-infra-code-checks:
+    name: pipeline/aws_infra code checks
     permissions:
       id-token: write # This is required for requesting the JWT for gaining permissions to assume the IAM role to perform AWS actions
     runs-on: ubuntu-22.04
@@ -162,16 +128,26 @@ jobs:
     steps:
     - name: Check out repository code
       uses: actions/checkout@v4
-    - name: Setup node.js (CDK requirement)
-      uses: actions/setup-node@v4
       with:
-        node-version: "18"
-    - name: Install CDK
-      run: npm install -g aws-cdk
+        fetch-depth: 0
+        sparse-checkout: |
+          pipeline/aws_infra/
     - name: Setup Python
       uses: actions/setup-python@v5
       with:
         python-version: "3.12"
+    - name: CDK resource assertions (unit tests)
+      run: |
+        make run-unit-tests
+    - name: Python typing test
+      run: |
+        make run-python-type-check
+    - name: Setup node.js (CDK requirement)
+      uses: actions/setup-node@v4
+      with:
+        node-version: "18"
+    - name: Install CDK CLI
+      run: npm install -g aws-cdk
     - name: Install CDK stack dependencies
       run: pip install -r requirements.txt
     - name: AWS credentials configuration
@@ -180,25 +156,5 @@ jobs:
         role-to-assume: ${{secrets.GH_ACTIONS_AWS_ROLE}}
         role-session-name: gh-actions-${{github.run_id}}.${{github.run_number}}.${{github.run_attempt}}-cdk-test
         aws-region: us-east-1
-    - name: cdk diff
+    - name: Validate CDK stack code
       run: cdk diff
-  pipeline-aws-infra-cdk-assertions:
-    name: pipeline/aws_infra CDK resource assertions
-    runs-on: ubuntu-22.04
-    defaults:
-      run:
-        shell: bash
-        working-directory: ./pipeline/aws_infra/
-    steps:
-      - name: Check out repository code
-        uses: actions/checkout@v4
-        with:
-          fetch-depth: 0
-          sparse-checkout: |
-            pipeline/aws_infra/
-      - uses: actions/setup-python@v5
-        with:
-          python-version: '3.12'
-      - name: Run unit tests
-        run: |
-          make run-unit-tests
diff --git a/pipeline/seq_retrieval/README.md b/pipeline/seq_retrieval/README.md
@@ -122,8 +122,9 @@ make run-python-type-check-dev
 
 `mypy` checks are automatically run and enforced as part of the PR validation
 and all reported errors must be fixed to enable merging each PR into `main`.  
-If the `pipeline/seq_retrieval python typing check` status check fails on a PR in github,
-click the details link and inspect the failing step output for hints on what to fix.
+If the `pipeline/seq_retrieval code checks` status check fails on a PR in github,
+click the details link and check which step failed, and investigate the step output
+for hints on what to fix. The python type checks are run in the `Check python typing` step.
 
 #### Code documentation (docstrings)
 All modules, functions, classes and methods should have their input, attributes and output documented
@@ -143,8 +144,9 @@ which would otherwise result in inconsistent code style and reduced readability.
 
 These style checks are automatically run and enforced as part of the PR validation
 and all reported errors must be fixed to enable merging each PR into `main`.  
-If the `pipeline/seq_retrieval python style check` status check fails on a PR in github,
-click the details link and inspect the failing step output for hints on what to fix.
+If the `pipeline/seq_retrieval code checks` status check fails on a PR in github,
+click the details link and check which step failed, and investigate the step output
+for hints on what to fix. The python style checks are run in the `Check python style` step.
 
 #### Unit testing
 [Pytest](https://pytest.org/) is being used for unit testing.
@@ -159,8 +161,9 @@ testing before getting merged, which ensures the code is functional and won't br
 unnoticed in future development iterations.  
 All unit tests are automatically run and enforced as part of the PR validation
 and all reported errors must be fixed to enable merging each PR into `main`.  
-If the `pipeline/seq_retrieval unit tests` status check fails on a PR in github,
-click the details link and inspect the failing step output for hints on what to fix.
+If the `pipeline/seq_retrieval code checks` status check fails on a PR in github,
+click the details link and check which step failed, and investigate the step output
+for hints on what to fix. The unit tests are run in the `Run unit tests` step.
 
 ## Building
 To build a clean docker image (for production usage and troubleshooting):