Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exclude E2E tests for documentation-only PRs #3248

Closed
dtzar opened this issue Mar 8, 2023 · 9 comments · Fixed by kubernetes/test-infra#29123
Closed

Exclude E2E tests for documentation-only PRs #3248

dtzar opened this issue Mar 8, 2023 · 9 comments · Fixed by kubernetes/test-infra#29123
Assignees
Labels
kind/feature Categorizes issue or PR as related to a new feature.
Milestone

Comments

@dtzar
Copy link
Contributor

dtzar commented Mar 8, 2023

/kind feature

Describe the solution you'd like
When there are only documentation changes in a PR, ignore / do not run any of the E2E tests since this is a waste of time and money.

Some example PRs:
#3116
#3161

@k8s-ci-robot k8s-ci-robot added the kind/feature Categorizes issue or PR as related to a new feature. label Mar 8, 2023
@CecileRobertMichon
Copy link
Contributor

This is our current regex: https://github.com/kubernetes/test-infra/blob/master/config/jobs/kubernetes-sigs/cluster-api-provider-azure/cluster-api-provider-azure-presubmits-main.yaml#L60

Looks like the first one didn't run the e2e job, as expected. The second did, I believe because the .codespellignore file changed. We could exclude that too.

@dtzar
Copy link
Contributor Author

dtzar commented Mar 9, 2023

@mboersma Did you have another job to provide as an example which unnecessarily ran e2e tests for just docs?

@mboersma
Copy link
Contributor

mboersma commented Mar 9, 2023

#2805 only touched README.md but for some reason we ran the base e2e test and unit tests. (It was last November, so maybe we updated the regex since then?)

Other than that, I'm not finding any doc-only PRs that run unnecessary e2e tests. But #3230 only changes the Tiltfile, which also shouldn't cause any real tests to run, but did.

@CecileRobertMichon CecileRobertMichon added this to the v1.9 milestone Mar 9, 2023
@CecileRobertMichon
Copy link
Contributor

Another PR type that shouldn't be running e2e tests but does: OWNER file updates #3208

@mboersma
Copy link
Contributor

#3268 only changes a markdown file, but ran all the tests.

@mboersma
Copy link
Contributor

mboersma commented Mar 23, 2023

#3293 is another markdown-only change that spawned all the tests.

@CecileRobertMichon
Copy link
Contributor

/assign

@CecileRobertMichon
Copy link
Contributor

#3368 successfully didn't run e2e tests 🎉

@mboersma
Copy link
Contributor

@CecileRobertMichon I also created #3366 and #3367 yesterday just to test that this is working. Looks good! Thank you for fixing this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/feature Categorizes issue or PR as related to a new feature.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants