Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[data.search] Stop sending entire request body in polling requests #186138

Open
lukasolson opened this issue Jun 12, 2024 · 3 comments
Open

[data.search] Stop sending entire request body in polling requests #186138

lukasolson opened this issue Jun 12, 2024 · 3 comments
Assignees
Labels
Feature:Search Querying infrastructure in Kibana impact:medium Addressing this issue will have a medium level of impact on the quality/strength of our product. loe:medium Medium Level of Effort performance Project:AsyncSearch Background search, partial results, async search services. Team:DataDiscovery Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL.

Comments

@lukasolson
Copy link
Member

lukasolson commented Jun 12, 2024

Summary:

Currently, when using the data.search services, we are sending the entire search request body to every polling request. This takes up a lot of data transfer and is unnecessary, since all we're using is the search request ID to poll on the ES _async_status endpoint.

We should check and make sure that other search strategies aren't negatively affected if we switch over to just sending the request ID, and we should also make sure that things like "other bucket" scenarios still work.

Consider, for example, these requests:

image

This shows three distinct async search requests. The final responses are indicated by the larger response_length. It is totally unnecessary to be sending such a large request body (indicated by request_length) in each of the polls leading up to the final response.

@lukasolson lukasolson added Feature:Search Querying infrastructure in Kibana performance Project:AsyncSearch Background search, partial results, async search services. Team:DataDiscovery Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL. labels Jun 12, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-data-discovery (Team:DataDiscovery)

@lukasolson lukasolson added loe:medium Medium Level of Effort impact:medium Addressing this issue will have a medium level of impact on the quality/strength of our product. labels Jul 8, 2024
@thomasneirynck
Copy link
Contributor

@lukasolson would this and #186138 be related? ie improvements to one can include the other?

@davismcphee
Copy link
Contributor

@thomasneirynck I'd say they're so closely related they might even be identical! 🙂 (I think you meant to share another link)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:Search Querying infrastructure in Kibana impact:medium Addressing this issue will have a medium level of impact on the quality/strength of our product. loe:medium Medium Level of Effort performance Project:AsyncSearch Background search, partial results, async search services. Team:DataDiscovery Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL.
Projects
None yet
Development

No branches or pull requests

4 participants