Show 🎄

[qstokkink]

We have had extended unit tests run on validate since 4cf28ad, lovingly referred to as the "Christmas tree". 🎄

Initially, these tests were not added as a PR check, pending assessment of their stability.

By now, these tests are stable enough to detect issues (e.g., in #1257). However, they are not shown as a PR check, so the test results are hidden for the PR author.

We should modify .github/workflows/pr-comment-validate.yml to report its status on the corresponding PR.

[qstokkink]

After a lot of digging (through repos and tutorials), I found https://github.com/marketplace/actions/github-status-action-v2 to set commit statuses. This action seems to (1) still be maintained and (2) able to set statuses based on our triggering event, which is:

py-ipv8/.github/workflows/pr-comment-validate.yml

Lines 3 to 5 in d2d31ee

	on:
	issue_comment:
	types: [created]

However, the issue_comment event does not include the commit SHA to set the status for. Open problem: we have to somehow fetch that from the issue number.

Our current approach for the checkout action is to create a remote ref from the issue number. We should - probably - be able to dereference this (in a preceding step) and feed it to github-status-action-v2.

py-ipv8/.github/workflows/pr-comment-validate.yml

Line 19 in d2d31ee

ref: refs/pull/${{ github.event.issue.number }}/head

[qstokkink]

I can't find any action to grab a SHA from a PR ref. We'll probably need a custom command and write it to an env var.

The command itself should be something like the following:

git ls-remote origin refs/pull/${{ github.event.issue.number }}/head | grep -o "^\w*\b"

The origin may also need to be determined from the issue_comment. We can try to see if it works without that.

Whatever it may end up looking like: we can then set the env var and feed it to github-status-action-v2. This should be enough info to start the implementation.

[qstokkink]

Lesson learned: env vars can only be used within the steps of a single job. To communicate output of jobs between jobs we need to write to $GITHUB_OUTPUT instead of $GITHUB_ENV (https://docs.github.com/en/actions/using-jobs/defining-outputs-for-jobs).

[qstokkink]

🎉 The results are now visible in the PR, this issue is resolved.