Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Observability] add custom threshold functional test #184602

Conversation

dominiqueclarke
Copy link
Contributor

@dominiqueclarke dominiqueclarke commented May 31, 2024

Closes #175301

Summary

Resolves https://github.com/orgs/elastic/projects/1375/views/1?pane=issue&itemId=50879015

To run

cd x-pack && node ../scripts/functional_tests_server --config=test/observability_functional/with_rac_write.config.ts

or

node scripts/functional_tests_server.js --config x-pack/test/observability_functional/with_rac_write.config.ts

In a different terminal

node ../scripts/functional_test_runner --config=test/observability_functional/with_rac_write.config.ts --grep "Custom threshold rule"

or

node scripts/functional_test_runner --config=x-pack/test/observability_functional/with_rac_write.config.ts  --grep "Custom threshold rule"

@dominiqueclarke
Copy link
Contributor Author

/ci

@apmmachine
Copy link
Contributor

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

  • /oblt-deploy : Deploy a Kibana instance using the Observability test environments.
  • run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

.send({
page: 1,
per_page: 10,
filter:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if this is needed by I copied the request body on the alerts page.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we do something like this?

response = await supertest.get(
        `${getUrlPrefix(
          Spaces.space1.id
        )}/internal/alerting/rules/_find?filter=alert.attributes.params.risk_score:40`
      );

},
],
groupBy: ['docker.container.name'],
// searchConfiguration: {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One thing I'm not sure how to test is the data view configuration. I didn't see anywhere in the rule definition that directly referenced the data view? Or is this index id below someone a uuid referencing the data view?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this searchConfiguration is passed to the executor a bit differently but I didn't find a way to see how we fetch it in the edit flyout, I will check more and will comment here if I find out something.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about creating a data view similar to this and validate the related data view id is correct?

Btw, the value of the optional filter should be saved in the query.query. Can you please also add the expected filter there?
image

await testSubjects.setValue('ruleKqlFilterSearchField', 'service.name : "opbeans-node"');
await testSubjects.click('o11yClosablePopoverTitleButton');

// set custom equation
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I couldn't figure out how to test the custom equation field, at least not quickly figure it out.

I encountered a element not intractable error. If anyone is a wizard in debugging those, let me know.

@dominiqueclarke
Copy link
Contributor Author

/ci

…f github.com:dominiqueclarke/kibana into chore/observability-custom-threshold-functional-test
@dominiqueclarke
Copy link
Contributor Author

/ci

1 similar comment
@dominiqueclarke
Copy link
Contributor Author

/ci

@dominiqueclarke
Copy link
Contributor Author

/ci

await kibanaServer.savedObjects.cleanStandardList();
});

it('is filtered to only show "all" alerts by default', async () => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does this step do?

.send({
page: 1,
per_page: 10,
filter:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we do something like this?

response = await supertest.get(
        `${getUrlPrefix(
          Spaces.space1.id
        )}/internal/alerting/rules/_find?filter=alert.attributes.params.risk_score:40`
      );

await testSubjects.click('observability.rules.custom_threshold-SelectOption');
};

const fillCustomThresholdRule = async (name: string) => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am wondering if it would be better to move this logic to the actual test. Are you planning to allow passing parameters to this function to create different custom threshold rules?

Maybe we can have a separate service for the custom threshold rule creation, the rules page gives the impression that it is related to all rules, not the creation of one specific rule.

},
],
groupBy: ['docker.container.name'],
// searchConfiguration: {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this searchConfiguration is passed to the executor a bit differently but I didn't find a way to see how we fetch it in the edit flyout, I will check more and will comment here if I find out something.

@dominiqueclarke dominiqueclarke changed the title observability - add custom threshold functional test [Observability] add custom threshold functional test Jul 17, 2024
@dominiqueclarke dominiqueclarke added chore v8.16.0 release_note:skip Skip the PR/issue when compiling release notes labels Jul 17, 2024
@dominiqueclarke dominiqueclarke marked this pull request as ready for review July 17, 2024 13:27
@dominiqueclarke dominiqueclarke requested review from a team as code owners July 17, 2024 13:27
Copy link
Contributor

@jughosta jughosta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Data Discovery changes in tests LGTM 👍

@botelastic botelastic bot added ci:project-deploy-observability Create an Observability project Team:obs-ux-management Observability Management User Experience Team labels Jul 19, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/obs-ux-management-team (Team:obs-ux-management)

@dominiqueclarke dominiqueclarke requested a review from a team as a code owner July 30, 2024 02:46
Comment on lines +35 to +36
const category = await categories.findByCssSelector(`.euiFacetButton[title="Observability"]`);
await category.click();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit

Suggested change
const category = await categories.findByCssSelector(`.euiFacetButton[title="Observability"]`);
await category.click();
await category.clickByCssSelector(`.euiFacetButton[title="Observability"]`);

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get the following error when using clickByCssSelector:

fail: ObservabilityApp Custom threshold rule shows the custom threshold rule in the observability section
       │      TypeError: categories.clickByCssSelector is not a function

@kibanamachine
Copy link
Contributor

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#6939

[❌] x-pack/test/observability_functional/with_rac_write.config.ts: 0/25 tests passed.

see run history

@kibanamachine
Copy link
Contributor

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#6945

[❌] x-pack/test/observability_functional/with_rac_write.config.ts: 18/50 tests passed.

see run history

@kibanamachine
Copy link
Contributor

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#6951

[❌] x-pack/test/observability_functional/with_rac_write.config.ts: 0/25 tests passed.

see run history

@kibana-ci
Copy link
Collaborator

kibana-ci commented Sep 17, 2024

💚 Build Succeeded

  • Buildkite Build
  • Commit: 0778b1a
  • Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-184602-0778b1aeb5d5

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
observability 463.7KB 463.8KB +45.0B
triggersActionsUi 1.6MB 1.6MB +227.0B
total +272.0B

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @maryam-saeidi

@kibanamachine
Copy link
Contributor

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#6964

[✅] x-pack/test/observability_functional/with_rac_write.config.ts: 100/100 tests passed.

see run history

@kibanamachine
Copy link
Contributor

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#6967

[❌] x-pack/test/observability_functional/with_rac_write.config.ts: 81/100 tests passed.

see run history

@maryam-saeidi
Copy link
Member

The last failure is due to failure in another test (observability security feature controls) and is not related to the test that we added to this PR. So, I will merge this PR.

const aggregationNameA = await testSubjects.find('aggregationNameA');
return (await aggregationNameA.getVisibleText()) === 'AVERAGE\nmetricset.rtt';
});
await new Promise((r) => setTimeout(r, 1000));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For future reference, we added these timeouts to ensure the selected data is passed to the alerting framework and the new rule params are available in the rule flyout before changing the next one; otherwise, an old state will be passed when updating the next rule param.

@maryam-saeidi maryam-saeidi merged commit 0241cf6 into elastic:main Sep 18, 2024
22 checks passed
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Sep 18, 2024
Closes elastic#175301

## Summary

Resolves
https://github.com/orgs/elastic/projects/1375/views/1?pane=issue&itemId=50879015

To run
```
cd x-pack && node ../scripts/functional_tests_server --config=test/observability_functional/with_rac_write.config.ts

or

node scripts/functional_tests_server.js --config x-pack/test/observability_functional/with_rac_write.config.ts
```
In a different terminal
```
node ../scripts/functional_test_runner --config=test/observability_functional/with_rac_write.config.ts --grep "Custom threshold rule"

or

node scripts/functional_test_runner --config=x-pack/test/observability_functional/with_rac_write.config.ts  --grep "Custom threshold rule"
```

---------

Co-authored-by: Maryam Saeidi <[email protected]>
(cherry picked from commit 0241cf6)
@kibanamachine
Copy link
Contributor

💚 All backports created successfully

Status Branch Result
8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

kibanamachine added a commit that referenced this pull request Sep 18, 2024
…193248)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[Observability] add custom threshold functional test
(#184602)](#184602)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Dominique
Clarke","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-09-18T07:56:58Z","message":"[Observability]
add custom threshold functional test (#184602)\n\nCloses
#175301\r\n\r\n##
Summary\r\n\r\nResolves\r\nhttps://github.com/orgs/elastic/projects/1375/views/1?pane=issue&itemId=50879015\r\n\r\nTo
run\r\n```\r\ncd x-pack && node ../scripts/functional_tests_server
--config=test/observability_functional/with_rac_write.config.ts\r\n\r\nor\r\n\r\nnode
scripts/functional_tests_server.js --config
x-pack/test/observability_functional/with_rac_write.config.ts\r\n```\r\nIn
a different terminal\r\n```\r\nnode ../scripts/functional_test_runner
--config=test/observability_functional/with_rac_write.config.ts --grep
\"Custom threshold rule\"\r\n\r\nor\r\n\r\nnode
scripts/functional_test_runner
--config=x-pack/test/observability_functional/with_rac_write.config.ts
--grep \"Custom threshold
rule\"\r\n```\r\n\r\n---------\r\n\r\nCo-authored-by: Maryam Saeidi
<[email protected]>","sha":"0241cf690c631d4865b9c412240ab49357646a26","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["chore","release_note:skip","v9.0.0","ci:project-deploy-observability","Team:obs-ux-management","v8.16.0"],"title":"[Observability]
add custom threshold functional
test","number":184602,"url":"https://github.com/elastic/kibana/pull/184602","mergeCommit":{"message":"[Observability]
add custom threshold functional test (#184602)\n\nCloses
#175301\r\n\r\n##
Summary\r\n\r\nResolves\r\nhttps://github.com/orgs/elastic/projects/1375/views/1?pane=issue&itemId=50879015\r\n\r\nTo
run\r\n```\r\ncd x-pack && node ../scripts/functional_tests_server
--config=test/observability_functional/with_rac_write.config.ts\r\n\r\nor\r\n\r\nnode
scripts/functional_tests_server.js --config
x-pack/test/observability_functional/with_rac_write.config.ts\r\n```\r\nIn
a different terminal\r\n```\r\nnode ../scripts/functional_test_runner
--config=test/observability_functional/with_rac_write.config.ts --grep
\"Custom threshold rule\"\r\n\r\nor\r\n\r\nnode
scripts/functional_test_runner
--config=x-pack/test/observability_functional/with_rac_write.config.ts
--grep \"Custom threshold
rule\"\r\n```\r\n\r\n---------\r\n\r\nCo-authored-by: Maryam Saeidi
<[email protected]>","sha":"0241cf690c631d4865b9c412240ab49357646a26"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/184602","number":184602,"mergeCommit":{"message":"[Observability]
add custom threshold functional test (#184602)\n\nCloses
#175301\r\n\r\n##
Summary\r\n\r\nResolves\r\nhttps://github.com/orgs/elastic/projects/1375/views/1?pane=issue&itemId=50879015\r\n\r\nTo
run\r\n```\r\ncd x-pack && node ../scripts/functional_tests_server
--config=test/observability_functional/with_rac_write.config.ts\r\n\r\nor\r\n\r\nnode
scripts/functional_tests_server.js --config
x-pack/test/observability_functional/with_rac_write.config.ts\r\n```\r\nIn
a different terminal\r\n```\r\nnode ../scripts/functional_test_runner
--config=test/observability_functional/with_rac_write.config.ts --grep
\"Custom threshold rule\"\r\n\r\nor\r\n\r\nnode
scripts/functional_test_runner
--config=x-pack/test/observability_functional/with_rac_write.config.ts
--grep \"Custom threshold
rule\"\r\n```\r\n\r\n---------\r\n\r\nCo-authored-by: Maryam Saeidi
<[email protected]>","sha":"0241cf690c631d4865b9c412240ab49357646a26"}},{"branch":"8.x","label":"v8.16.0","branchLabelMappingKey":"^v8.16.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Dominique Clarke <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
chore ci:project-deploy-observability Create an Observability project release_note:skip Skip the PR/issue when compiling release notes Team:obs-ux-management Observability Management User Experience Team v8.16.0 v9.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Custom threshold] Add a functional test to test rule create/edit form
10 participants