Pure semantic search w/ Haystack: where are the Readers?

[nlstrait]

Hi there!

I am very excited to have discovered Haystack, but I am running into one major issue for my use case: I am struggling to construct a pipeline for pure semantic search (rather than QA search) that still enables me to pull out segments with the help of a Reader. Put another way, I've been unable to find a Reader for the simple semantic search case. Currently, the only examples of pure semantic search abandon Readers altogether, but for my application, a simple Retriever is not enough.

I've looked through models on Hugging Face, but looking at the 'Tasks' tags, none of them seem to fit my specification. I am relatively new to the ML and NLP communities and could easily be missing some useful terminology or understanding... Do you have any suggestions for Readers for pure semantic search?

As an aside: from a new-comers perspective, Haystack appears to be overemphasizing QA while having the latent potential of being more general-purpose. This seems like more an issue with marketing/branding than functionality, though the scarcity of general-purpose, non-QA tutorials is functionally limiting in a sense.

Thanks in advance for any feedback, and thank you for helping support this awesome tool and community!

[lalitpagaria]

@tholor @brandenchan can you please help @nlstrait out

@PiffPaffM adding you for this part -

As an aside: from a new-comers perspective, Haystack appears to be overemphasizing QA while having the latent potential of being more general-purpose. This seems like more an issue with marketing/branding than functionality, though the scarcity of general-purpose, non-QA tutorials is functionally limiting in a sense.

[brandenchan]

Hi @nlstrait, happy to try and help out. Just want to align some terminology first because I'm not fully sure what you mean with regards to semantic search vs QA search.

The tasks that we are interested in are Open Domain Questions answering (ODQA), Document Retrieval and Document Reranking.

In ODQA, you extract a text answer span from a large collection of documents, generally using a Retriever + Reader. In Document Retrieval, you might only want the documents returned, not specific text answer spans. In Document Ranking, you get the candidate documents provided by the Retriever, and rerank them so that the most relevant answers appear at the top of your list.

Is Document Ranking what you are looking for?

[nlstrait]

Hi @brandenchan. Thanks for laying out some terminology.

No, pure Document Ranking is not what I'm looking for. Neither is pure Document Retrieval or ODQA.

In the Haystack model, the Reader enables ODQA. What I'd like is a Reader that can handle queries in any form (rather than strictly in the form of a question) and gives results in any form (rather than strictly in the form of an answer). ODQA seems to me a specialized version of semantic search, whereas I'm looking for the generalized or "simple" version. Like the Retriever analyses and returns whole documents for semantic relevance, I'm looking for a Reader that analyses and returns segments of documents for pure semantic relevance.

[brandenchan]

Sure, can you give an example of what kind of queries you are expecting? One distinction we make is between full text questions ("What is the capital of Germany?") vs keyword queries ("Germany capital"). Are you thinking of another kind of query?

I should also mention that Readers are able to return variable length segments as answers. The Natural Questions dataset makes models highlight either sentences or paragraphs if a question can't be answered by a single short phrase. Or perhaps you might be interested in using a retriever on a sentence or paragraph level.

[nlstrait]

I suppose I'm expecting keyword queries, but I'm not sure that covers it.

I should elaborate on my use case: I'm searching through transcripts of lectures and podcasts, most of which are one long string with no paragraphs or punctuation. Since these lectures and podcasts are often multiple hours long, I'd like to be able to find moments within them via semantic search of the transcripts. I could perform a simple keyword search, perhaps with the help of a thesaurus, but I've imagined there must be a better way.

That's good to know about Readers returning variable length segments as answers. I think using a Retriever on a paragraph level would be ideal, since it wouldn't be strictly expecting a question as the query; however, as previously mentioned, most of the text I'm searching is without punctuation or separation into paragraphs. I've thought about models for generating punctuation and separating paragraphs from the raw transcripts, but I feel this may be adding unnecessary complexity to my application...

Now that you know more about my use case, what do you think? Am I bringing to light an unsolved problem?

[brandenchan]

Hey thanks for sharing your use case! Really cool idea and I definitely think that Haystack can form the backbone of your project!

So I think you can achieve your goal still using Retrievers and Readers if we perform document splitting in a smart way. Even though you have no explicit punctuation in your transcripts, I would still recommend splitting them into smaller chunks (maybe around 100 or 200 words each to start). These chunks could even have some overlap so that you don't accidentally split a sentence into 2 and lose the context around it. We have a PreProcessor class that can handle all this document splitting for you (check out the tutorial here). This approach is also more computational efficient down the line as we point out in our latest article!

If you perform Document Retrieval on these split documents, you can essentially return 100 word blocks that best address your query (which can be either keywords or full text questions). If you choose to place a Reader after the Retriever, you are homing in even more and trying to narrow down on the very text span that best addresses your query.

I would expect that either of these setups (Retriever only or Retriever + Reader) could be a good way of finding the most relevant information in your transcripts as long as your transcripts are split into smaller chunks. This will save you the trouble of trying to build a model that generates punctuation or insert paragraph breaks.

What do you think about this? Very curious to hear how your project develops!

[nlstrait]

Thank you for the thorough response! This makes sense. I didn't realize a Retriever could be using a language model (as with Dense Passage Retrieval) and had assumed this power was reserved for the Readers. I will give this document splitting method a shot and let you know how it goes!

[brandenchan]

Thanks also for your feedback regarding Haystack's potential as a general NLP framework. You're right we focus a lot on ODQA, but that's only because this was our first big task. Thanks to community input and contributions, we now have a lot more components, including the Ranker, Summarizer, Translator. With pipelines you can piece these together as it suits you. Interested to hear what tasks and tutorials you feel are missing!

[nlstrait]

Hopefully my reply to your other response helps address the "tasks and tutorials [I] feel are missing."