Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flaky Test Tracking #4223

Open
theunrepentantgeek opened this issue Aug 28, 2024 · 2 comments
Open

Flaky Test Tracking #4223

theunrepentantgeek opened this issue Aug 28, 2024 · 2 comments
Labels
test Test refactoring related tasks

Comments

@theunrepentantgeek
Copy link
Member

theunrepentantgeek commented Aug 28, 2024

Describe the current behavior

Some of our tests are flaky; we've made improvements to our test infrastructure to improve things, but the tests need to be rerecorded for those improvements to kick in.

Describe the improvement

We should track tests that are flaky that haven't yet been updated, and treat them as a priority for rerecording

Test Flakes Observed
Test_Compute_VM_20220301_CRUD 2024-08-28
Test_Insights_v1api_CreationAndDeletion 2024-09-24
Test_Keyvault_v1api20230701_CreationAndDeletion 2024-10-02
Test_MariaDB_Server_CRUD 2024-09-16
Test_Multitenant_SingleOperator_PerResourceCredential_MatchSubscriptionWithOwner 2024-10-21
Test_Synapse_v1api_CreationAndDeletion 2024-10-01
2024-09-17
2024-09-05
2024-09-02

(Please update this table with new occurrences; list tests alphabetically, list newer occurrences first.)

Additional context
Add any other context about the suggested improvement.

@matthchr
Copy link
Member

matthchr commented Oct 1, 2024

Samples tests sometimes flake too:
Test_Samples_CreationAndDeletion/Test_Synapse_v1api_CreationAndDeletion I saw just now

@matthchr
Copy link
Member

matthchr commented Oct 1, 2024

Also seen:
Test_Samples_CreationAndDeletion/Test_Keyvault_v1api20230701_CreationAndDeletion

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
test Test refactoring related tasks
Projects
Development

No branches or pull requests

2 participants