Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

failsOnMKI: serverless security search_index_details and start_elasticsearch test suitea #196981

Closed
pheyos opened this issue Oct 19, 2024 · 1 comment · Fixed by #197264
Closed
Assignees
Labels
failed-test A test failure on a tracked branch, potentially flaky-test Team:EnterpriseSearch

Comments

@pheyos
Copy link
Member

pheyos commented Oct 19, 2024

The serverless search UI tests suites search_index_details and start_elasticsearch started failing consistently when running on MKI. Combining them in one issue here as it seems to be the same root cause: assertions are expecting a mocked domain, which only works locally.

Failure details

search_index_details

  • serverless search UI Search index detail page index details page overview should have connection details*

Failure message:

Error: expected 'https://bk-serverless-ftr-3117-e10daaf3f19b-c7e3a5.es.eu-west-1.aws.qa.elastic.cloud:443' to equal 'https://fakeprojectid.es.fake-domain.cld.elstc.co:443'
    at Assertion.assert (expect.js:100:11)
    at Assertion.apply (expect.js:227:8)
    at Assertion.be (expect.js:69:22)
    at Object.expectConnectionDetails (svl_search_index_detail_page.ts:30:96)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at Context. (search_index_detail.ts:54:9)
    at Object.apply (wrap_function.js:74:16)

Failure screenshot:
Image

start_elasticsearch

serverless search UI Elasticsearch Start [Onboarding Empty State] developer should have o11y links

Failure message:

Error: expected 'https://console.qa.cld.elstc.co/projects/create/observability/start' to equal 'https://fake-cloud.elastic.co/projects/create/observability/start'
    at Assertion.assert (expect.js:100:11)
    at Assertion.equal (expect.js:227:8)
    at Object.expectO11yTrialLink (svl_search_elasticsearch_start_page.ts:88:76)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at Context. (elasticsearch_start.ts:157:9)
    at Object.apply (wrap_function.js:74:16)

Failure screenshot:
Image

@pheyos pheyos added failed-test A test failure on a tracked branch, potentially flaky-test Team:EnterpriseSearch labels Oct 19, 2024
pheyos added a commit that referenced this issue Oct 19, 2024
…uites for MKI (#196982)

## Summary

This PR skips the serverless search test suites `start_elasticsearch`
and `search_index_details` in MKI runs.
See #196981
@pheyos
Copy link
Member Author

pheyos commented Oct 19, 2024

Tests skipped for MKI runs in #196982.

@TattdCodeMonkey TattdCodeMonkey self-assigned this Oct 22, 2024
TattdCodeMonkey added a commit that referenced this issue Oct 22, 2024
## Summary

Closes #196981

- Updated Search Index Details with data test suite to check for index
documents first, this should help reduce flakey tests in MKI since we
will ensure the documents are available before moving forward
- Updated URL checks to use regex instead `.equal` with the mocked
value. This should ensure the tests can pass in MKI and with integration
testing
- Removed the navigation check for the Maps page, this was causing
intermittent failures due to a modal on the page.
- Updated the getting started test suite to use `developer` role instead
of `admin`

### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
failed-test A test failure on a tracked branch, potentially flaky-test Team:EnterpriseSearch
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants