Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[APM] Correlations: Update field candidates request. #186182

Conversation

walterra
Copy link
Contributor

@walterra walterra commented Jun 13, 2024

Summary

Part of #178606.
Fixes #185875.

Since we created the correlations feature for APM, some new options were added to the _field_caps API which allow us to improve the way we retrieve field candidates for the analysis.

Previously we used 2 queries to get field candidates: We fetched all fields via _field_caps, then searched for a random sample of 1000 docs to identify fields with values. Additional code would then filter the supported fields.

Now we can use additional _field_caps options to get rid of the random docs request and simplify some of the filtering code.

  • filters: '-metadata,-parent' will exclude metadata and object fields,
  • include_empty_fields: false will include populated fields only,
  • index_filter: ... allows to provide a range filter with start/end to limit the scope of indices,
  • types: ... allows us to get only fields of the type supported by the analysis (keyword, boolean, ip).

Checklist

@walterra walterra self-assigned this Jun 13, 2024
@walterra walterra force-pushed the ml-apm-correlation-use-updated-field-caps-options branch from a671ac1 to b5b7911 Compare June 18, 2024 07:59
@walterra walterra added release_note:skip Skip the PR/issue when compiling release notes v8.15.0 labels Jun 18, 2024
@walterra walterra marked this pull request as ready for review June 18, 2024 11:37
@walterra walterra requested a review from a team as a code owner June 18, 2024 11:37
@botelastic botelastic bot added ci:project-deploy-observability Create an Observability project Team:obs-ux-infra_services Observability Infrastructure & Services User Experience Team labels Jun 21, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/obs-ux-infra_services-team (Team:obs-ux-infra_services)

@kibana-ci
Copy link
Collaborator

kibana-ci commented Jul 2, 2024

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

  • [job] [logs] FTR Configs #25 / @skipInServerless endpoint @ess @serverless For each artifact list under management When on the Event Filters entries list should be able to add a new Event Filters entry

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @walterra

@walterra walterra merged commit 8fc3d3c into elastic:main Jul 2, 2024
23 checks passed
@kibanamachine kibanamachine added the backport:skip This commit does not require backporting label Jul 2, 2024
@walterra walterra deleted the ml-apm-correlation-use-updated-field-caps-options branch July 2, 2024 15:26
walterra added a commit that referenced this pull request Jul 3, 2024
…187444)

## Summary

Fixes #176544.
Fixes #187421.
Fixes #176119.
Fixes #176425.
Fixes #175855.
Fixes #175911.
Fixes #176780.

Follow up to #186182.

Reenables and stabilizes APM correlations API integration tests.

Review hint: View with the `w=1` flag to ignore whitespace changes:
https://github.com/elastic/kibana/pull/187444/files?w=1

### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [x] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
apm:review backport:skip This commit does not require backporting ci:project-deploy-observability Create an Observability project release_note:skip Skip the PR/issue when compiling release notes Team:obs-ux-infra_services Observability Infrastructure & Services User Experience Team v8.15.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[ML] APM Correlations: Field caps filters.
6 participants