Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

factor out common workflows #332

Merged
merged 11 commits into from
Feb 17, 2024
Merged

factor out common workflows #332

merged 11 commits into from
Feb 17, 2024

Conversation

lsf37
Copy link
Member

@lsf37 lsf37 commented Feb 10, 2024

With reusable GitHub workflows we can now factor out common workflow files and maintain them only once here in the ci-actions repo.

Should also make it easier to deploy additional PR label actions to many/all of the seL4 org repos.

Unfortunately they have to live in the somewhat hidden .github/workflows directory, which I find a bit awkward, but it's still better than having to replicate everything.

I have so far only tested the workflows that don't need secrets and trigger on pull_request_target, but those work. The display name changes, which mean we will have to update the names of required checks in all repos, but the benefit shoudl be worth the one-time effort.

@lsf37
Copy link
Member Author

lsf37 commented Feb 10, 2024

Have now also tested pull_request_target, and it works. The only thing that is missing is testing secrets which only really works if the caller workflow has been merged.

# PR. Ok on the workflow level here, because this workflow does not re-trigger
# on "labeled" events.
concurrency:
group: ${{ github.workflow }}-sel4sim-pr-${{ github.event.number }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Debugging gives me seL4Test-sel4sim-pr- here when a never job cancels an old run. Debugging says

  "event_name": "pull_request",
    "event": {
      "action": "synchronize",
      ...
      "number": 140,      
      ...

so I'm a but puzzled why the PR number is missing there.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems the whole github.event does not exist a this stage.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In which scenario is that happening? I.e. are you seeing this in the old job that is being cancelled or in the new job that is the reason for cancellation?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm seeing this in the old job, where github gives the cancellation reason. This is the only place where the group name is ever printed. It seems impossible to see the concurrency group for a running job anywhere. You can print it in a job/step, but the context there seems slightly different, because this runs on the runner. The workflow control and concurrency seems to run on an "action server", which does not see some dynamic parts of the contexts. The github documentation is quite thin here, but that is what I found in some github community postings.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, the docs do say that github, inputs, and vars are available at the workflow level, and additionally needs, strategy, and matrix on the job level. So I think we are fine with this expression, it's just hard to debug as usual on GitHub.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's see if I get some answers here: https://github.com/orgs/community/discussions/107552

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to put it into the job level for now, so that we can merge this. Would still be good to figure out what exactly is available, though.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The other option could be adding another callable workflow layer here. The the job level concurrency group there will cover everything. It could also filter the events and invoke this workflow with a simple bool parameter for enabling hw-test.

These can now be called from other workflows, reducing duplication.

Signed-off-by: Gerwin Klein <[email protected]>
@lsf37
Copy link
Member Author

lsf37 commented Feb 14, 2024

I think this is ready to merge now. We can move the concurrency group for sel4test-simp back up to the workflow level when we get a definitive answer in https://github.com/orgs/community/discussions/107552.

@lsf37 lsf37 requested a review from corlewis February 15, 2024 23:21
Copy link
Member

@axel-h axel-h left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's good to merge this in the current state. I've done quite a lot of experimenting now and in general this works. So no gain in blocking the merge.

lsf37 and others added 10 commits February 17, 2024 12:48
- use job concurrency instead of workflow concurrency, because on the
  workflow level a label trigger event will already call all current
  runs, whereas the job level is guarded by `if` conditions.

- disambiguate concurrency group names, because the workflow name is
  inherited from the caller.

Signed-off-by: Gerwin Klein <[email protected]>
The workflow is intended to only run for pull_request_target.

Signed-off-by: Gerwin Klein <[email protected]>
This makes sure all build jobs run on the same repo manifest state.

Signed-off-by: Gerwin Klein <[email protected]>
Signed-off-by: Gerwin Klein <[email protected]>
Add the strategy.job-index back into the concurrency key, now that
these are back at the job level.

Co-authored-by: Axel Heider <[email protected]>
Signed-off-by: Gerwin Klein <[email protected]>
It looks like ${{ github.event.number }} might not exist yet at the
workflow level when the workflow is called, so we move it to the job
level where we know that this info exists.

See also <https://github.com/orgs/community/discussions/107552>

Signed-off-by: Gerwin Klein <[email protected]>
Since multiple workflows can now be called from the same caller
workflow, these will all end up in the same UI, so we should make
sure the artifact names are not ambiguous.

Signed-off-by: Gerwin Klein <[email protected]>
@lsf37 lsf37 merged commit adb6b20 into master Feb 17, 2024
7 checks passed
@lsf37 lsf37 deleted the workflows branch February 17, 2024 01:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants