Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[eem] top_value metadata aggregation #188243

Merged
merged 15 commits into from
Sep 16, 2024

Conversation

klacabane
Copy link
Contributor

@klacabane klacabane commented Jul 12, 2024

Metadata fields currently supports terms aggregation only to retrieve a list of values.
This change updates the metadata schema so one can define different aggregations and also implements a top_value agg that returns a single value according to a sort criteria.

The top_value aggregation is limited to the single top value since transforms do not support multi values for top_metrics

Usage:

{
  ...
  metadata: [
    { source: 'host.name', aggregation: { type: 'top_value', lookbackPeriod: '24h', sort: { '@timestamp': 'desc' } } }
  ]
  ...
}

@obltmachine
Copy link

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

  • /oblt-deploy : Deploy a Kibana instance using the Observability test environments.
  • run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

@klacabane klacabane added the Feature:EEM Elastic Entity Model label Jul 12, 2024
@klacabane
Copy link
Contributor Author

/ci

@klacabane klacabane changed the title [eem] top_metrics metadata agg [eem] top_value metadata aggregation Sep 11, 2024
@klacabane
Copy link
Contributor Author

/ci

@klacabane klacabane marked this pull request as ready for review September 11, 2024 11:10
@klacabane klacabane requested a review from a team as a code owner September 11, 2024 11:10
@klacabane klacabane self-assigned this Sep 11, 2024
@klacabane klacabane added release_note:skip Skip the PR/issue when compiling release notes Team:obs-entities Observability Entities Team labels Sep 11, 2024
@botelastic botelastic bot added the ci:project-deploy-observability Create an Observability project label Sep 11, 2024
@klacabane
Copy link
Contributor Author

@jaredburgettelastic @machadoum I've added a lookbackPeriod parameter to the aggregation - I'll merge now lmk if there's any follow up you'd like to add

@kibana-ci
Copy link
Collaborator

kibana-ci commented Sep 15, 2024

💛 Build succeeded, but was flaky

Failed CI Steps

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
@kbn/entities-schema 42 44 +2
Unknown metric groups

API count

id before after diff
@kbn/entities-schema 42 44 +2

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @klacabane

@klacabane klacabane merged commit 1e6b13c into elastic:main Sep 16, 2024
32 checks passed
@kibanamachine kibanamachine added v9.0.0 backport:skip This commit does not require backporting labels Sep 16, 2024
@jbudz
Copy link
Member

jbudz commented Sep 20, 2024

@klacabane can you check if this need to be backported to 8.x?

@jbudz jbudz removed the v9.0.0 label Sep 20, 2024
@klacabane klacabane added backport:prev-major Backport to (8.x, 8.17, 8.16) the previous major branch and other branches in development and removed backport:skip This commit does not require backporting labels Sep 23, 2024
@klacabane klacabane added backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) and removed backport:prev-major Backport to (8.x, 8.17, 8.16) the previous major branch and other branches in development labels Sep 23, 2024
@kibanamachine
Copy link
Contributor

💔 Some backports could not be created

Status Branch Result
8.15 Backport failed because of merge conflicts
8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Manual backport

To create the backport manually run:

node scripts/backport --pr 188243

Questions ?

Please refer to the Backport tool documentation

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Sep 23, 2024
@kibanamachine
Copy link
Contributor

💚 All backports created successfully

Status Branch Result
8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

kibanamachine added a commit that referenced this pull request Sep 23, 2024
# Backport

This will backport the following commits from `main` to `8.x`:
- [[eem] top_value metadata aggregation
(#188243)](#188243)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Kevin
Lacabane","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-09-16T01:01:41Z","message":"[eem]
top_value metadata aggregation
(#188243)","sha":"1e6b13c8e0791d49a672e614d5117f3dee1b813c","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","backport:prev-major","ci:project-deploy-observability","Feature:EEM","team:obs-entities"],"title":"[eem]
top_value metadata
aggregation","number":188243,"url":"https://github.com/elastic/kibana/pull/188243","mergeCommit":{"message":"[eem]
top_value metadata aggregation
(#188243)","sha":"1e6b13c8e0791d49a672e614d5117f3dee1b813c"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/188243","number":188243,"mergeCommit":{"message":"[eem]
top_value metadata aggregation
(#188243)","sha":"1e6b13c8e0791d49a672e614d5117f3dee1b813c"}}]}]
BACKPORT-->

Co-authored-by: Kevin Lacabane <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) ci:project-deploy-observability Create an Observability project Feature:EEM Elastic Entity Model release_note:skip Skip the PR/issue when compiling release notes Team:obs-entities Observability Entities Team v8.16.0 v9.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants