Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Count queries scrolls thru all elasticsearch index #2496

Open
gfmellado opened this issue Oct 30, 2024 · 0 comments
Open

[Bug] Count queries scrolls thru all elasticsearch index #2496

gfmellado opened this issue Oct 30, 2024 · 0 comments
Labels
bug TheHive4 TheHive4 related issues

Comments

@gfmellado
Copy link

Hello everyone! We have an issue: when we filter by native fields (like date, source, severity, etc.) the count query response time takes a normal amount of time (fast), but when we filter by custom fields, we think it scrolls thru all index database (elasticsearch) and two possible scenarios occur:

  1. when the count number is greater than the query threshold (query.limitedCountThreshold: 1000) it stops when it reaches that limit.
  2. when the count number is lower than the query threshold (query.limitedCountThreshold: 1000) it ends up scrolling thru all index database in order to get the definitive number.

The second scenario takes a lot of time and really stresses the platform.

What can we do to improve this situation? If we do a reindex, does custom fields get reindexed?

Thanks in advance for any help.

@gfmellado gfmellado added bug TheHive4 TheHive4 related issues labels Oct 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug TheHive4 TheHive4 related issues
Projects
None yet
Development

No branches or pull requests

1 participant