Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate how to handle the deleted document in fast filter optimization #12596

Open
bowenlan-amzn opened this issue Mar 11, 2024 · 1 comment

Comments

@bowenlan-amzn
Copy link
Member

bowenlan-amzn commented Mar 11, 2024

Follow up task for #9310

The filter rewrite optimization in date histogram doesn't account for the deleted documents. We would fall back to default aggregation execution path if deleted document exists. However, there are situations where deleted documents are hard to be merged away. e.g. the segment is large (~5gb) but the deleted documents counts are relatively small. So our optimization will never be able to kick in under such scenario.

@bowenlan-amzn
Copy link
Member Author

As discussed in an internal review meeting, it’s probably better to look into the other way — how to clean up deleted documents of big segments easily, instead of hanlding in our optimization path.

Currently this is not in plan.

@bowenlan-amzn bowenlan-amzn moved this from Untriaged to Not In Plan in Performance Roadmap Jun 18, 2024
@getsaurabh02 getsaurabh02 moved this from 🆕 New to Later (6 months plus) in Search Project Board Aug 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Not In Plan
Status: Later (6 months plus)
Development

No branches or pull requests

1 participant