Chore/solr9 upgrade

[blms]

No description provided.

[blms]

Running into an issue getting any results back from keyword queries, seems to be a problem with the query construction format, but I don’t see any documentation of a change or deprecation.

These all fail...

keyword_search_qf = "{!type=edismax qf=$keyword_qf pf=$keyword_pf v=$keyword_query}"

def keyword_search(self, search_term):
    search = self.search(self.keyword_search_qf).raw_query_parameters(
        keyword_query=self._search_term_cleanup(search_term)
    )

keyword_search_qf = "{!type=edismax qf=description_en_bigram pf=description_en_bigram v=$keyword_query}"

def keyword_search(self, search_term):
    search = self.search(self.keyword_search_qf).raw_query_parameters(
        keyword_query=self._search_term_cleanup(search_term)
    )

keyword_search_qf = "{!type=edismax v=$keyword_query}"

def keyword_search(self, search_term):
    self.search(self.keyword_search_qf).raw_query_parameters(
        keyword_query=self._search_term_cleanup(search_term),
        qf="$keyword_qf",
        pf="$keyword_pf",
    )

This succeeds:

keyword_search_qf = "{!type=edismax v=$keyword_query}"

def keyword_search(self, search_term):
    self.search(self.keyword_search_qf).raw_query_parameters(
        keyword_query=self._search_term_cleanup(search_term),
        qf="description_en_bigram",
        pf="description_en_bigram",
    )

In other words, it seems qf and pf have to be specified as raw query params, and it only accepts actual field names like description_en_bigram and not references to lists of boosted fields like $keyword_qf or $keyword_pf.

[codecov]

Codecov Report

Merging #1406 (b2224c0) into develop (cc70b6d) will decrease coverage by 0.01%.
The diff coverage is 100.00%.

@@             Coverage Diff             @@
##           develop    #1406      +/-   ##
===========================================
- Coverage    98.65%   98.65%   -0.01%     
===========================================
  Files          196      196              
  Lines        10751    10748       -3     
===========================================
- Hits         10606    10603       -3     
  Misses         145      145              

[blms]

@rlskoeser Pushed an update that fixes search, but it does require defining the list of fields and boosts for each search in the raw query parameters. Is that OK? If so, there are cleaner ways to manage them than what I've done here, like lists concatenated into strings, so I can make that change—just doesn't seem to work to define them in the XML.

[rlskoeser]

@blms I don't think defining the search fields in the query is a good solution.

Have you tried using the solr query analyzer to see what's going on?

[blms]

@rlskoeser What are the drawbacks to that? Harder to maintain?

Yes, I have used the solr query analyzer and debugger to try and debug. Tried it again today and I'll document here:

With the original code, we get an attempt to literally look up the string {!type=edismax qf=$keyword_qf pf=$keyword_pf v=$keyword_query} (and thus 0 results):

    "rawquerystring":"{!type=edismax qf=$keyword_qf pf=$keyword_pf v=$keyword_query}",
    "querystring":"{!type=edismax qf=$keyword_qf pf=$keyword_pf v=$keyword_query}",
    "parsedquery":"+(DisjunctionMaxQuery((_text_:type)) DisjunctionMaxQuery((_text_:edismax)) DisjunctionMaxQuery((_text_:qf)) DisjunctionMaxQuery((_text_:keyword_qf)) DisjunctionMaxQuery((_text_:pf)) DisjunctionMaxQuery((_text_:keyword_pf)) DisjunctionMaxQuery((_text_:v)) DisjunctionMaxQuery((_text_:keyword_query)))",
    "parsedquery_toString":"+((_text_:type) (_text_:edismax) (_text_:qf) (_text_:keyword_qf) (_text_:pf) (_text_:keyword_pf) (_text_:v) (_text_:keyword_query))",

I thought there might be a problem with that syntax, so moving the string params into raw query params, like so:

search = self.raw_query_parameters(
    type="edismax",
    v=self._search_term_cleanup(search_term),
    qf="$keyword_qf",
    pf="$keyword_pf",
)

gets us this 400 error:

{
    "metadata": [
        "error-class",
        "org.apache.solr.common.SolrException",
        "root-error-class",
        "org.apache.solr.common.SolrException"
    ],
    "msg": "org.apache.solr.search.SyntaxError: Query Field '$keyword_qf' is not a valid field name",
    "code": 400
}

and without the $ before the query fields:

{
    "metadata": [
        "error-class",
        "org.apache.solr.common.SolrException",
        "root-error-class",
        "org.apache.solr.common.SolrException"
    ],
    "msg": "org.apache.solr.search.SyntaxError: Query Field 'keyword_qf' is not a valid field name",
    "code": 400
}

I couldn't figure out a way for it to pick up those fields from the XML config.

[blms]

All working now thanks to your help @rlskoeser!

[rlskoeser]

🎉🚀