Skip machine tests if they don't need to be run

[edsantiago]

Followup to #13936 : add an exclusion to localmachine tests
so we can avoid running those on test- or doc-only PRs.
Reason: #22551, the machine-start-timeout flake, is causing
hours of wasted time.

Signed-off-by: Ed Santiago [email protected]

None

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: edsantiago

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [edsantiago]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[Luap99]

LGTM things like this are on my TODO list for the CI work.
You don't have to answer this here right now but I like to understand why this was implemented like this and not via skip/only_if in cirrus.yml. IMO this currently violates POLA as users wrongly assume the tests have been run even when they are not.

[edsantiago]

Oh, huh. I thought it had been explained in #13936, but no.

Basically: yes, in an ideal world this would be done in .cirrus.yml. In this world, that is impossible. See the _bail_etc function for details. Can you find a way to implement that in YAML?

Side note: Cirrus apparently has some sort of other scripting language, starlark, that can replace .cirrus.yml and that provides more flexibility. I've poked into it a couple of times but never in sufficient depth to see if we can even use it in our environment, let alone find out if it can do tricks like _bail_etc.

[Luap99]

Can you find a way to implement that in YAML?

I have not looked deeply into it but for _bail_if_test_can_be_skipped and system tests
only_if: changesInclude('test/system/**') || !changesIncludeOnly('test/**','docs/**')

Unless I am missing some conditional this does not look to complex to me

[edsantiago]

That's a good first approximation but it neglects the fact that most (all?) xxx_task rules already have an existing only_if that would have to be considered. Plus the tag thing, and the branch thing, and the cron thing (other different ways of running CI). It's possible, even likely that I'm mistaken in using the word "impossible"; it's just something that has been too daunting for me.

[TomSweeneyRedHat]

LGTM

[Luap99]

/lgtm