Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IF: workflows fail because of newly added tests in hotstuff_integration branch #2369

Closed
greg7mdp opened this issue Apr 2, 2024 · 1 comment · Fixed by #2370
Closed

IF: workflows fail because of newly added tests in hotstuff_integration branch #2369

greg7mdp opened this issue Apr 2, 2024 · 1 comment · Fixed by #2370
Assignees

Comments

@greg7mdp
Copy link
Contributor

greg7mdp commented Apr 2, 2024

We determined that the addition of two new tests in #2335 caused a string of workflow failures, causing at least one failure for just about every commit. Failures are in NP, LR, ubsan, asan, asserton workflows.

Two workflow runs, using hotstuff_integration with just these tests removed, passed without problems. See https://github.com/AntelopeIO/leap/actions/runs/8523025337.

Since we need the new tests in order to run validations both with and without Savanna activated, we need to beef-up the instances the workflow VMs are running on.

We determined that, even before the systematic failures, the tests were running extremely slowly.

In order to provide reliable ci/cd workflows, and to provide some buffer so that the issue doesn't reoccur as soon as we add a few new tests, I feel that we should at least double the number of cores per instance (or the equivalent if each core is faster), provided the cost is not prohibitive.

This increase should apply to the following workflows: NP, LR, ubsan, asan, asserton

@enf-ci-bot enf-ci-bot moved this to Todo in Team Backlog Apr 2, 2024
@greg7mdp greg7mdp changed the title IF: workflow fail because of newly added tests in hotstuff_integration branch IF: workflows fail because of newly added tests in hotstuff_integration branch Apr 2, 2024
@greg7mdp greg7mdp removed the triage label Apr 2, 2024
@spoonincode spoonincode moved this from Todo to In Progress in Team Backlog Apr 2, 2024
@arhag
Copy link
Member

arhag commented Apr 3, 2024

Seems to be resolved by increasing performance of CI/CD VMs.

@arhag arhag closed this as completed Apr 3, 2024
@github-project-automation github-project-automation bot moved this from In Progress to Done in Team Backlog Apr 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

4 participants