Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restrict queried fields to those that are indexed #7

Open
DavidACameron-USDA opened this issue Apr 25, 2017 · 3 comments
Open

Restrict queried fields to those that are indexed #7

DavidACameron-USDA opened this issue Apr 25, 2017 · 3 comments

Comments

@DavidACameron-USDA
Copy link
Owner

This is a performance enhancement. The list of fields that is returned by a Solr query needs to be restricted to those that have been added to the index. Because of the way that SearchApiSolrBackend::extractResults() is written, only those fields are available for display. Any other fields in the Solr output are wasted overhead, which could be highly detrimental to some sites depending on the data they have stored in the index. It could also be really bad for sites that have their index hosted by a third party. That unneeded data would do nothing but eat up their bandwidth.

drunken-monkey added a commit to drunken-monkey/search_api_solr_datasource that referenced this issue May 23, 2017
@drunken-monkey
Copy link

I've included a fix for this in #9.
However, seems like this could also be beneficial for the Solr module itself, so I've also opened an issue there: #2880674: Retrieve only the required fields from the server. Once that gets fixed there, it should also take care of the problem for this module (at least for Views searches, and all others that use the new query option – probably we should still keep our override for those cases, I guess).

@drunken-monkey
Copy link

Also, it would be possible to retrieve all indexed fields and actually make those available in the view, too. (At least I'm pretty sure it should be possible.) So maybe we could make that configurable?
On the other hand, just adding all fields you want to use to the index is a pretty clear guideline, and also keeps the Views UI clean (not, potentially, laden with a lot of irrelevant Solr fields.)

@DavidACameron-USDA
Copy link
Owner Author

I'd considered doing that when I started thinking about this issue. I thought maybe it might be beneficial to have a checkbox setting that would enable all fields to be available. I would want it to be clear what the potential performance impacts are, possibly even labeling the setting as "Development mode" or something similar. That's mostly because my recommendation would be that it only be enabled on a dev server while setting up views. Admins would need to know that if they have it enabled on production then it could cause major bandwidth issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants