Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix for flaky sample data test #199005

Merged
merged 2 commits into from
Nov 5, 2024
Merged

fix for flaky sample data test #199005

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
cc0dbe0
add fix for flaky sample data test
eokoneyo Nov 5, 2024
3f4547d
remove exclusive test marker
eokoneyo Nov 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 34 additions & 28 deletions test/functional/page_objects/home_page.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,40 +91,46 @@ export class HomePageObject extends FtrService {

async addSampleDataSet(id: string) {
await this.openSampleDataAccordion();
const isInstalled = await this.isSampleDataSetInstalled(id);
if (!isInstalled) {
await this.retry.waitFor('sample data to be installed', async () => {
// count for the edge case where some how installation completes just before the retry occurs
if (await this.isSampleDataSetInstalled(id)) {
return true;
}

this.log.debug(`Attempting to add sample data: ${id}`);
await this.retry.waitFor('sample data to be installed', async () => {
// Echoing the adjustments made to 'removeSampleDataSet', as we are seeing flaky test cases here as well
// https://github.com/elastic/kibana/issues/52714
await this.testSubjects.waitForEnabled(`addSampleDataSet${id}`);
await this.common.sleep(1010);
await this.testSubjects.click(`addSampleDataSet${id}`);
await this.common.sleep(1010);
await this._waitForSampleDataLoadingAction(id);
return await this.isSampleDataSetInstalled(id);
});
}

// Echoing the adjustments made to 'removeSampleDataSet', as we are seeing flaky test cases here as well
// https://github.com/elastic/kibana/issues/52714
await this.testSubjects.waitForEnabled(`addSampleDataSet${id}`);
await this.common.sleep(1010);
await this.testSubjects.click(`addSampleDataSet${id}`);
await this.common.sleep(1010);
await this._waitForSampleDataLoadingAction(id);
return await this.isSampleDataSetInstalled(id);
});
}
Comment on lines 92 to 111
Copy link
Contributor Author

@eokoneyo eokoneyo Nov 5, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed this logic, because it actually triggers the flakiness of the test, looking at the logs of a previous run, we see the sample data installation completes, however for some reason at the point whilst the installation status was checked a falsy value was returned which in turn recursively initiated the steps to complete an install that's no longer possible, hence the reason for adopting an early return once the desired state exists.


async removeSampleDataSet(id: string) {
await this.openSampleDataAccordion();
const isInstalled = await this.isSampleDataSetInstalled(id);
if (isInstalled) {
await this.retry.waitFor('sample data to be removed', async () => {
// account for the edge case where some how data is uninstalled just before the retry occurs
if (!(await this.isSampleDataSetInstalled(id))) {
return true;
}

this.log.debug(`Attempting to remove sample data: ${id}`);
await this.retry.waitFor('sample data to be removed', async () => {
// looks like overkill but we're hitting flaky cases where we click but it doesn't remove
await this.testSubjects.waitForEnabled(`removeSampleDataSet${id}`);
// https://github.com/elastic/kibana/issues/65949
// Even after waiting for the "Remove" button to be enabled we still have failures
// where it appears the click just didn't work.
await this.common.sleep(1010);
await this.testSubjects.click(`removeSampleDataSet${id}`);
await this.common.sleep(1010);
await this._waitForSampleDataLoadingAction(id);
return !(await this.isSampleDataSetInstalled(id));
});
}

// looks like overkill but we're hitting flaky cases where we click but it doesn't remove
await this.testSubjects.waitForEnabled(`removeSampleDataSet${id}`);
// https://github.com/elastic/kibana/issues/65949
// Even after waiting for the "Remove" button to be enabled we still have failures
// where it appears the click just didn't work.
await this.common.sleep(1010);
await this.testSubjects.click(`removeSampleDataSet${id}`);
await this.common.sleep(1010);
await this._waitForSampleDataLoadingAction(id);
return !(await this.isSampleDataSetInstalled(id));
});
}

// loading action is either uninstall and install
Expand Down