Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release workflow doesn't run tests successfully but continues anyway #1633

Closed
tsibley opened this issue Sep 17, 2024 · 2 comments · Fixed by #1634
Closed

Release workflow doesn't run tests successfully but continues anyway #1633

tsibley opened this issue Sep 17, 2024 · 2 comments · Fixed by #1634
Labels
bug Something isn't working

Comments

@tsibley
Copy link
Member

tsibley commented Sep 17, 2024

The test step:

- run: devel/test

doesn't error on test failures because devel/test masks the exit status:

augur/devel/test

Lines 28 to 30 in 227f526

# Run unit and functional tests.
echo Running tests
./run_tests.sh || true

I noticed this with the 26.0.0 release just now, but it's been happening with past release workflow runs too.

While we can "fix" this by propagating the exit status in devel/test, I don't think devel/test is really appropriate for CI/CD, at least in its current developer-focused form. Our release process should ideally use a very similar test setup as CI does, or even be part of the CI/CD workflow.

@tsibley tsibley added the bug Something isn't working label Sep 17, 2024
@victorlin
Copy link
Member

How about something like #1634?

@tsibley
Copy link
Member Author

tsibley commented Sep 19, 2024

Left commentary there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants