Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failing test: API Integration Tests.test/api_integration/apis/home/sample_data·ts - apis home apis sample data apis install in the default space dates should load elasticsearch index containing sample data with dates relative to current time #166572

Closed
kibanamachine opened this issue Sep 17, 2023 · 9 comments · Fixed by #169638
Assignees
Labels
failed-test A test failure on a tracked branch, potentially flaky-test Team:SharedUX Team label for AppEx-SharedUX (formerly Global Experience)

Comments

@kibanamachine
Copy link
Contributor

kibanamachine commented Sep 17, 2023

A test failed on a tracked branch

Error: expected 3098747711 to be below 3024000000
    at Assertion.assert (expect.js:100:11)
    at Assertion.apply (expect.js:336:8)
    at Function.lessThan (expect.js:531:15)
    at Context.<anonymous> (sample_data.ts:86:33)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at Object.apply (wrap_function.js:73:16)

First failure: CI Build - main

@kibanamachine kibanamachine added the failed-test A test failure on a tracked branch, potentially flaky-test label Sep 17, 2023
@botelastic botelastic bot added the needs-team Issues missing a team label label Sep 17, 2023
@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@mistic mistic added the Team:SharedUX Team label for AppEx-SharedUX (formerly Global Experience) label Oct 15, 2023
@elasticmachine
Copy link
Contributor

Pinging @elastic/appex-sharedux (Team:SharedUX)

@botelastic botelastic bot removed the needs-team Issues missing a team label label Oct 15, 2023
mistic added a commit that referenced this issue Oct 15, 2023
@mistic
Copy link
Member

mistic commented Oct 15, 2023

Skipped.

main: fed5c52

@eokoneyo
Copy link
Contributor

eokoneyo commented Oct 26, 2023

I've spent some time looking at this; evaluating based off the logic here (#121051 (comment)) that explains the reason for the magic number 3024000000 I haven't been able to figure out what might be triggering the flaky behaviour, and for what it's worth running the test locally no errors are seen, also running a draft PR (#169638) in the flaky test runner with 100 runs passes 4 times yielding no failures, see here https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds?branch=refs%2Fpull%2F169638%2Fhead. I'd like to see if there's some sort of commonality in the environments of the test that failed next.

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - 8.11

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - 8.11

@eokoneyo eokoneyo linked a pull request Dec 8, 2023 that will close this issue
@eokoneyo
Copy link
Contributor

eokoneyo commented Dec 8, 2023

I have linked a PR that should resolve this issue, instead of capping our expectation to the number of weeks between the sample data we use the interval in milliseconds between the provided sample data, with this approach the value of our expectation exceeds any value that had resulted in the test suite failing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
failed-test A test failure on a tracked branch, potentially flaky-test Team:SharedUX Team label for AppEx-SharedUX (formerly Global Experience)
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants