feat: automated MACAW tests (#840)

* feat: first macaw test scripts

* fix: set python version to 3.10

* fix: path to human-GEM repo

* fix: correct path to macaw-tests.py

* feat: store macaw results

* fix: prepare output for comment post

* test: storing of output file in new commit

* chore: add macaw test result

* fix: reorganize macaw output file

* fix: write macaw results to correct location

* chore: add macaw test result

* fix: reset when different workflow are run

* chore: add macaw test result

* test: reset macaw_results to test workflow commit

* chore: add macaw test result

* fix: no need to specify action on PR, as push already has the same effect

* fix: but run macaw only in PR

as output is tracked with PR number, which would not exist with every push

* fix: remove link to macaw_results in comment

* fix: reference to PR # in README.md

---------

Co-authored-by: edkerk <[email protected]>

.github/workflows/check-metabolictasks.yml

@@ -1,6 +1,6 @@
 name: Test metabolic tasks
 
-on: [push, pull_request]
+on: [push]
 
 jobs:
   check-metabolictasks:

.github/workflows/commentsFromTests.md

@@ -0,0 +1,9 @@
+This PR has been [automatically tested with GH Actions](https://github.com/SysBioChalmers/Human-GEM/actions/runs/{GH_ACTION_RUN}). Here is the output of the macaw test:
+
+<pre>
+{TEST_RESULTS}
+</pre>
+
+A more detailed output from this test run is also committed to `data/macawResults/macaw_results.csv`.
+
+> _Note: In the case of multiple test runs, this post will be edited._

.github/workflows/macaw-tests.yml

@@ -0,0 +1,58 @@
+name: Run macaw tests
+
+on: [pull_request]
+
+jobs:
+  macaw-tests:
+    runs-on: ubuntu-latest
+    timeout-minutes: 60
+
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Test checkout
+        run: ls -la && cd code && ls -la
+
+      - name: Set up Python 3
+        uses: actions/setup-python@v4
+        with:
+          python-version: "3.10"
+
+      - name: Install macaw
+        run: pip install git+https://github.com/Devlin-Moyer/macaw.git@main numpy==1.26.4
+
+      - name: Run macaw
+        id: macaw-run
+        run: |
+          TEST_RESULTS=$(python code/test/macawTests.py)
+          echo $TEST_RESULTS
+          PARSED_RESULTS="${TEST_RESULTS//$'\n'/'<br>'}"
+          PARSED_RESULTS="${PARSED_RESULTS//$'\r'/'<br>'}"
+          echo $PARSED_RESULTS
+          echo "results=$PARSED_RESULTS" >> $GITHUB_OUTPUT
+
+      - name: Mention PR# in README.md
+        env:
+          PR_NUMBER: ${{ github.event.number }}
+        run: sed -i -e "s/#[[:digit:]]\{3,4\}/#$PR_NUMBER/g" data/macawResults/README.md
+
+      - name: Auto-commit results
+        uses: stefanzweifel/git-auto-commit-action@v4
+        with:
+          commit_user_name: memote-bot
+          commit_message: "chore: add macaw test result"
+          file_pattern: data/macawResults/*
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          PR_NUMBER: ${{ github.event.number }}
+
+      - name: Post comment
+        uses: NejcZdovc/comment-pr@v2
+        with:
+          file: "commentsFromTests.md"
+        env:
+          GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}}
+          TEST_RESULTS: ${{steps.macaw-run.outputs.results}}
+          GH_ACTION_RUN: ${{github.run_id}}
+

.github/workflows/yaml-conversion.yml

@@ -1,6 +1,6 @@
 name: Test YAML conversion
 
-on: [push, pull_request]
+on: [push]
 
 jobs:
   yaml-conversion:

.github/workflows/yaml-validation.yml

@@ -1,6 +1,6 @@
 name: YAML validation
 
-on: [push, pull_request]
+on: [push]
 
 jobs:
   yaml-validation:

code/test/macawTests.py

@@ -0,0 +1,8 @@
+import cobra
+from macaw.main import dead_end_test, duplicate_test
+
+model = cobra.io.load_yaml_model("model/Human-GEM.yml")
+(dead_end_results, dead_end_edges) = dead_end_test(model)
+(duplicate_results, duplicate_edges) = duplicate_test(model)
+output = dead_end_results.merge(duplicate_results)
+output.to_csv('data/macawResults/macaw_results.csv', index = False)

data/macawResults/README.md

@@ -0,0 +1,18 @@
+# MACAW results
+
+The file here contains results from the [MACAW](https://github.com/Devlin-Moyer/macaw) `dead_end_test` and `duplicate_test` tests.
+
+The test results shown here were obtained by the GitHub Actions run in **PR #840**, and will be updated by any subsequent PR. Summary results are shown as a comment in the corresponding PR.
+
+### `dead_end_test`
+Looks for metabolites in Human-GEM that can only be produced by all reactions they participate in or only consumed, then identifies all reactions that are prevented from sustaining steady-state fluxes because of each of these dead-end metabolites. The simplest case of a dead-end metabolite is one that only participates in a single reaction. Also flags all reversible reactions that can only carry fluxes in a single direction because one of their metabolites can either only be consumed or only be produced by all other reactions it participates in.
+
+### `duplicate_test`
+Identifies sets of reactions that may be duplicates of each other because they:
+
+- Involve exactly the same metabolites with exactly the same stoichiometric coefficients (but potentially different associated genes).
+- Involve exactly the same metabolites, but go in different directions and/or some are reversible and some are not.
+- Involve exactly the same metabolites, but with different stoichiometric coefficients.
+- Represent the oxidation and/or reduction of the same metabolite, but use different electron acceptors/donors from the given list of pairs of oxidized and reduced forms of various electron carriers (e.g. NAD(H), NADP(H), FAD(H2), ubiquinone/ubiquinol, cytochromes).
+
+It is possible for a single reaction to fit in multiple of the above categories. There are sometimes cases where sets of reactions that fall into one of the above categories are completely legitimate representations of real biochemistry (e.g. separate irreversible reactions for importing vs exporting the same metabolite because two different transporters encoded by different genes are each responsible for transporting that metabolite in only one direction, enzymes that can use NAD(H) or NADP(H) interchangeably to catalyze the same redox reaction), but reactions that meet these criteria are generally worth close examination to ensure that they should actually all exist as separate reactions.