-
Notifications
You must be signed in to change notification settings - Fork 8.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Observability] add custom threshold functional test #184602
[Observability] add custom threshold functional test #184602
Conversation
/ci |
🤖 GitHub commentsExpand to view the GitHub comments
Just comment with:
|
.send({ | ||
page: 1, | ||
per_page: 10, | ||
filter: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure if this is needed by I copied the request body on the alerts page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we do something like this?
response = await supertest.get(
`${getUrlPrefix(
Spaces.space1.id
)}/internal/alerting/rules/_find?filter=alert.attributes.params.risk_score:40`
);
}, | ||
], | ||
groupBy: ['docker.container.name'], | ||
// searchConfiguration: { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One thing I'm not sure how to test is the data view configuration. I didn't see anywhere in the rule definition that directly referenced the data view? Or is this index id below someone a uuid referencing the data view?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I know this searchConfiguration is passed to the executor a bit differently but I didn't find a way to see how we fetch it in the edit flyout, I will check more and will comment here if I find out something.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about creating a data view similar to this and validate the related data view id is correct?
Btw, the value of the optional filter should be saved in the query.query
. Can you please also add the expected filter there?
await testSubjects.setValue('ruleKqlFilterSearchField', 'service.name : "opbeans-node"'); | ||
await testSubjects.click('o11yClosablePopoverTitleButton'); | ||
|
||
// set custom equation |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I couldn't figure out how to test the custom equation field, at least not quickly figure it out.
I encountered a element not intractable error. If anyone is a wizard in debugging those, let me know.
/ci |
…f github.com:dominiqueclarke/kibana into chore/observability-custom-threshold-functional-test
/ci |
1 similar comment
/ci |
/ci |
x-pack/test/observability_functional/apps/observability/pages/alerts/custom_threshold.ts
Outdated
Show resolved
Hide resolved
await kibanaServer.savedObjects.cleanStandardList(); | ||
}); | ||
|
||
it('is filtered to only show "all" alerts by default', async () => { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What does this step do?
.send({ | ||
page: 1, | ||
per_page: 10, | ||
filter: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we do something like this?
response = await supertest.get(
`${getUrlPrefix(
Spaces.space1.id
)}/internal/alerting/rules/_find?filter=alert.attributes.params.risk_score:40`
);
await testSubjects.click('observability.rules.custom_threshold-SelectOption'); | ||
}; | ||
|
||
const fillCustomThresholdRule = async (name: string) => { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am wondering if it would be better to move this logic to the actual test. Are you planning to allow passing parameters to this function to create different custom threshold rules?
Maybe we can have a separate service for the custom threshold rule creation, the rules page gives the impression that it is related to all rules, not the creation of one specific rule.
}, | ||
], | ||
groupBy: ['docker.container.name'], | ||
// searchConfiguration: { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I know this searchConfiguration is passed to the executor a bit differently but I didn't find a way to see how we fetch it in the edit flyout, I will check more and will comment here if I find out something.
…bservability-custom-threshold-functional-test
…bservability-custom-threshold-functional-test
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Data Discovery changes in tests LGTM 👍
Pinging @elastic/obs-ux-management-team (Team:obs-ux-management) |
const category = await categories.findByCssSelector(`.euiFacetButton[title="Observability"]`); | ||
await category.click(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit
const category = await categories.findByCssSelector(`.euiFacetButton[title="Observability"]`); | |
await category.click(); | |
await category.clickByCssSelector(`.euiFacetButton[title="Observability"]`); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I get the following error when using clickByCssSelector
:
fail: ObservabilityApp Custom threshold rule shows the custom threshold rule in the observability section
│ TypeError: categories.clickByCssSelector is not a function
Flaky Test Runner Stats🟠 Some tests failed. - kibana-flaky-test-suite-runner#6939[❌] x-pack/test/observability_functional/with_rac_write.config.ts: 0/25 tests passed. |
Flaky Test Runner Stats🟠 Some tests failed. - kibana-flaky-test-suite-runner#6945[❌] x-pack/test/observability_functional/with_rac_write.config.ts: 18/50 tests passed. |
Flaky Test Runner Stats🟠 Some tests failed. - kibana-flaky-test-suite-runner#6951[❌] x-pack/test/observability_functional/with_rac_write.config.ts: 0/25 tests passed. |
💚 Build Succeeded
Metrics [docs]Async chunks
History
To update your PR or re-run it, just comment with: |
Flaky Test Runner Stats🎉 All tests passed! - kibana-flaky-test-suite-runner#6964[✅] x-pack/test/observability_functional/with_rac_write.config.ts: 100/100 tests passed. |
Flaky Test Runner Stats🟠 Some tests failed. - kibana-flaky-test-suite-runner#6967[❌] x-pack/test/observability_functional/with_rac_write.config.ts: 81/100 tests passed. |
The last failure is due to failure in another test (observability security feature controls) and is not related to the test that we added to this PR. So, I will merge this PR. |
const aggregationNameA = await testSubjects.find('aggregationNameA'); | ||
return (await aggregationNameA.getVisibleText()) === 'AVERAGE\nmetricset.rtt'; | ||
}); | ||
await new Promise((r) => setTimeout(r, 1000)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For future reference, we added these timeouts to ensure the selected data is passed to the alerting framework and the new rule params are available in the rule flyout before changing the next one; otherwise, an old state will be passed when updating the next rule param.
Closes elastic#175301 ## Summary Resolves https://github.com/orgs/elastic/projects/1375/views/1?pane=issue&itemId=50879015 To run ``` cd x-pack && node ../scripts/functional_tests_server --config=test/observability_functional/with_rac_write.config.ts or node scripts/functional_tests_server.js --config x-pack/test/observability_functional/with_rac_write.config.ts ``` In a different terminal ``` node ../scripts/functional_test_runner --config=test/observability_functional/with_rac_write.config.ts --grep "Custom threshold rule" or node scripts/functional_test_runner --config=x-pack/test/observability_functional/with_rac_write.config.ts --grep "Custom threshold rule" ``` --------- Co-authored-by: Maryam Saeidi <[email protected]> (cherry picked from commit 0241cf6)
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
…193248) # Backport This will backport the following commits from `main` to `8.x`: - [[Observability] add custom threshold functional test (#184602)](#184602) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Dominique Clarke","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-09-18T07:56:58Z","message":"[Observability] add custom threshold functional test (#184602)\n\nCloses #175301\r\n\r\n## Summary\r\n\r\nResolves\r\nhttps://github.com/orgs/elastic/projects/1375/views/1?pane=issue&itemId=50879015\r\n\r\nTo run\r\n```\r\ncd x-pack && node ../scripts/functional_tests_server --config=test/observability_functional/with_rac_write.config.ts\r\n\r\nor\r\n\r\nnode scripts/functional_tests_server.js --config x-pack/test/observability_functional/with_rac_write.config.ts\r\n```\r\nIn a different terminal\r\n```\r\nnode ../scripts/functional_test_runner --config=test/observability_functional/with_rac_write.config.ts --grep \"Custom threshold rule\"\r\n\r\nor\r\n\r\nnode scripts/functional_test_runner --config=x-pack/test/observability_functional/with_rac_write.config.ts --grep \"Custom threshold rule\"\r\n```\r\n\r\n---------\r\n\r\nCo-authored-by: Maryam Saeidi <[email protected]>","sha":"0241cf690c631d4865b9c412240ab49357646a26","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["chore","release_note:skip","v9.0.0","ci:project-deploy-observability","Team:obs-ux-management","v8.16.0"],"title":"[Observability] add custom threshold functional test","number":184602,"url":"https://github.com/elastic/kibana/pull/184602","mergeCommit":{"message":"[Observability] add custom threshold functional test (#184602)\n\nCloses #175301\r\n\r\n## Summary\r\n\r\nResolves\r\nhttps://github.com/orgs/elastic/projects/1375/views/1?pane=issue&itemId=50879015\r\n\r\nTo run\r\n```\r\ncd x-pack && node ../scripts/functional_tests_server --config=test/observability_functional/with_rac_write.config.ts\r\n\r\nor\r\n\r\nnode scripts/functional_tests_server.js --config x-pack/test/observability_functional/with_rac_write.config.ts\r\n```\r\nIn a different terminal\r\n```\r\nnode ../scripts/functional_test_runner --config=test/observability_functional/with_rac_write.config.ts --grep \"Custom threshold rule\"\r\n\r\nor\r\n\r\nnode scripts/functional_test_runner --config=x-pack/test/observability_functional/with_rac_write.config.ts --grep \"Custom threshold rule\"\r\n```\r\n\r\n---------\r\n\r\nCo-authored-by: Maryam Saeidi <[email protected]>","sha":"0241cf690c631d4865b9c412240ab49357646a26"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/184602","number":184602,"mergeCommit":{"message":"[Observability] add custom threshold functional test (#184602)\n\nCloses #175301\r\n\r\n## Summary\r\n\r\nResolves\r\nhttps://github.com/orgs/elastic/projects/1375/views/1?pane=issue&itemId=50879015\r\n\r\nTo run\r\n```\r\ncd x-pack && node ../scripts/functional_tests_server --config=test/observability_functional/with_rac_write.config.ts\r\n\r\nor\r\n\r\nnode scripts/functional_tests_server.js --config x-pack/test/observability_functional/with_rac_write.config.ts\r\n```\r\nIn a different terminal\r\n```\r\nnode ../scripts/functional_test_runner --config=test/observability_functional/with_rac_write.config.ts --grep \"Custom threshold rule\"\r\n\r\nor\r\n\r\nnode scripts/functional_test_runner --config=x-pack/test/observability_functional/with_rac_write.config.ts --grep \"Custom threshold rule\"\r\n```\r\n\r\n---------\r\n\r\nCo-authored-by: Maryam Saeidi <[email protected]>","sha":"0241cf690c631d4865b9c412240ab49357646a26"}},{"branch":"8.x","label":"v8.16.0","branchLabelMappingKey":"^v8.16.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Dominique Clarke <[email protected]>
Closes #175301
Summary
Resolves https://github.com/orgs/elastic/projects/1375/views/1?pane=issue&itemId=50879015
To run
In a different terminal