-
Notifications
You must be signed in to change notification settings - Fork 218
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[CI] Idempotency tests are too flaky #3339
Comments
Thanks for starting the conversation on that. |
No 😞 , I just noticed it. I was guessing that this turns true (but I cannot prove it):
The uploaded artifacts from the actions are not that useful as they are bugged (overwriting files instead of keeping the separate folders). I don't have much time to dedicate to this now unfortunately. |
Fair enough, let's at least get to a place where we have good artifacts to investigate potential issues. Scheduled for 1.8 at the moment. Let's revisit once we make progress on other topics. |
update on this. With the recent improvements we've made, the only place I still see a flaky behaviour is for the twitter API at this point for the last month/6 weeks. (for the stable languages) |
Kind of ... looking at the most recent failures: yes Twitter is the one failing more often, should we remove it from the idempotency tests? |
What makes twitter different from the others? Anything specific in the description that we should be looking at? |
To Seb's point, before we remove it, I think we should try to identify the source. If it's caused by something that "is wrong in the description", I don't have any issue to remove it. If we uncover something though, I'd rather fix the code. |
As you prefer, of course fixing the issues is much better! So, removing the file, is like an "ignore for all" that will, at least, reduce the noise. |
@andreaTP while working on #3814 and with the added improvements of #3813 I think we indirectly addressed this issue.
|
Hey @andreaTP, it's been a week and we haven't see any idempotency issues coming up. We'll close for now, feel free to re-open if they start showing up again! |
I noticed this during the last couple of PRs, I think that this is causing some level of friction:
The text was updated successfully, but these errors were encountered: