Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable doc_values based searching across different field types #11205

Closed
harshavamsi opened this issue Nov 15, 2023 · 2 comments · Fixed by #11650
Closed

Enable doc_values based searching across different field types #11205

harshavamsi opened this issue Nov 15, 2023 · 2 comments · Fixed by #11650
Assignees
Labels
enhancement Enhancement or improvement to existing feature or request Search:Performance v2.14.0

Comments

@harshavamsi
Copy link
Contributor

harshavamsi commented Nov 15, 2023

Is your feature request related to a problem? Please describe.
While working on #11097 i realized that MultiTerm queries over Boolean fields in OpenSearch don't support doc_values. I also noticed that the documentation is incorrect and that booleans have doc_values enabled by default.

This is prevalent across multiple field types, there is a fix going out in #11144 to fix this for IP fields, I would also like to do this for other fields:

Describe the solution you'd like
Since boolean fields type extends TermBasedFieldType we should probably override the termQuery and termsQuery functions and add support for doc_values queries. While we're at it, we should also add IndexOrDocValuesQuery to the range query. This will help use-cases where we have a range on a boolean field matching a lot of values.

Describe alternatives you've considered
Alternative is to do nothing. This is obviously bad since users will miss out on doc_values based queries that can improve performance.

Additional context
N/A

@harshavamsi harshavamsi added enhancement Enhancement or improvement to existing feature or request untriaged labels Nov 15, 2023
@harshavamsi harshavamsi changed the title Boolean MultiTerm queries over Boolean fields don't support reading doc_values Nov 15, 2023
@harshavamsi harshavamsi added Search:Performance v2.12.0 Issues and PRs related to version 2.12.0 and removed untriaged labels Nov 15, 2023
@getsaurabh02 getsaurabh02 moved this from Todo to Now (This Quarter) in Performance Roadmap Nov 15, 2023
@macohen macohen moved this from 🆕 New to 🏗 In progress in Search Project Board Nov 29, 2023
@harshavamsi harshavamsi changed the title MultiTerm queries over Boolean fields don't support reading doc_values Enable doc_values based searching across different field types Dec 8, 2023
@harshavamsi harshavamsi self-assigned this Dec 8, 2023
@getsaurabh02 getsaurabh02 moved this from Now (This Quarter) to In Progress in Performance Roadmap Dec 20, 2023
@kiranprakash154
Copy link
Contributor

Hi, are we on track for this to be released in 2.12 ?

@msfroh
Copy link
Collaborator

msfroh commented Jan 19, 2024

Note that for Boolean fields, we might not see much benefit from a performance standpoint for indexed fields (since we have an efficient advance implementation for term queries and multiterm query expansion isn't going to expand beyond two terms).

That said, for consistency, now that we're supporting DV-only queries on many field types (including keyword, where Boolean is essentially a very restricted subcase of a keyword field), I like the idea that any field with doc values can support a (maybe slow) DV-only query.

@getsaurabh02 getsaurabh02 added v2.13.0 Issues and PRs related to version 2.13.0 and removed v2.12.0 Issues and PRs related to version 2.12.0 labels Feb 5, 2024
@reta reta added v2.14.0 and removed v2.13.0 Issues and PRs related to version 2.13.0 labels Mar 20, 2024
@github-project-automation github-project-automation bot moved this from 🏗 In progress to ✅ Done in Search Project Board May 2, 2024
@github-project-automation github-project-automation bot moved this from In Progress to Done in Performance Roadmap May 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Enhancement or improvement to existing feature or request Search:Performance v2.14.0
Projects
Status: Done
Archived in project
Development

Successfully merging a pull request may close this issue.

5 participants