BM25 retrieval of documents where a specific field is constant

[Fatima-Haouari]

Hi,
I indexed a collection of documents (contents was indexed) but stored the raw json of documents where I have multiple fields. Is it possible to search for documents where a specific field is constant. Basically, searching a subset of this index where field x=constant.

I tried the query builder as shown below:

from pyserini.search import querybuilder
# First, create term queries for each individual query term:
term1 = querybuilder.get_term_query("1459636761924091909",field="userid")
term2 = querybuilder.get_term_query(term,field="contents")

# Then, assemble into a "bag of words" query:
should = querybuilder.JBooleanClauseOccur['should'].value

boolean_query_builder = querybuilder.get_boolean_query_builder()
boolean_query_builder.add(term1, must)
boolean_query_builder.add(term2, should)
query = boolean_query_builder.build()
print(query)
hits = searcher.search(query)

for i in range(0, 10):
    print(f'{i+1:2} {hits[i].docid:15} {hits[i].score:.5f}')

But I am getting all documents for that specific field if the term I am using for search does not occur in any of the documents. Kindly advise. How can retrieve documents that have terms only.

[Fatima-Haouari]

Solved.

[Fatima-Haouari]

Hi, it seems I still advise regarding this issue. Since I am using must for the userid I am getting users documents even if no relevant documents. Does using filter instead is a better option if I want only relevant documents. I tried it and I am getting documents with 0 or negative scores. Do I consider those irrelevant. @lintool kindly advise.