-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Enable docValues on sort fields #180
Labels
Comments
6 tasks
rladdusaw
changed the title
Consider enabling docValues on sort fields
Enable docValues on sort fields
Jun 4, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
What maintenance needs to be done?
Enable docValues for sort fields like author_sort, title_sort, for the catalog
Level of urgency
Why is this maintenance needed?
Sort fields are being created at runtime and stored in the cache. We could create these during indexing and potentially save memory and improve performance by creating them during indexing
Acceptance criteria
Implementation notes, if any
docValues documentation: https://lucene.apache.org/solr/guide/7_7/docvalues.html
@tpendragon did some research into this, saying:
Some things I'm reading (https://risdenk.github.io/2017/12/18/ambari-infra-solr-ranger.html and https://risdenk.github.io/2018/10/21/apache-solr-out-of-memory-symptoms-and-solutions.html) seems to imply that making sure our fields have docvalues enabled will make the heap much smaller.
Each replica's field cache for the catalog is ~ 3 GB, so that's 12 GB Heap.
author_sort is 21 MB, title_sort is ~ 160 MB
If we enable docValues on author_sort/title_sort etc, it'll use those at index time instead of populating that huge FieldCache
Figgy's field cache is also 2.8 GB
Er, or not? It looks like these metrics are showing other core fields
The text was updated successfully, but these errors were encountered: