Skip to content

Commit

Permalink
Apply fixes from Adam's review
Browse files Browse the repository at this point in the history
Co-authored-by: Adam Demjen <[email protected]>
  • Loading branch information
leemthompo and demjened authored Jul 17, 2024
1 parent fc888d2 commit 7402dc5
Show file tree
Hide file tree
Showing 3 changed files with 35 additions and 19 deletions.
42 changes: 30 additions & 12 deletions docs/reference/search/retriever.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ A <<knn-retriever, retriever>> that replaces the functionality of a <<search-api
A <<rrf-retriever, retriever>> that produces top documents from <<rrf, reciprocal rank fusion (RRF)>>.

`text_similarity_reranker`::
A <<text-similarity-reranker, retriever>> that enhances search results by re-ranking documents based on text similarity to a specified inference text, using a machine learning model.
A <<text-similarity-reranker, retriever>> that enhances search results by re-ranking documents based on semantic similarity to a specified inference text, using a machine learning model.

[[standard-retriever]]
==== Standard Retriever
Expand Down Expand Up @@ -204,10 +204,10 @@ GET /index/_search
----
// NOTCONSOLE

[[text-similarity-reranker]]
==== Text Similarity Re-ranker
[[text-similarity-reranker-retriever]]
==== Text Similarity Re-ranker Retriever

The `text_similarity_reranker` is a type of retriever that enhances search results by re-ranking documents based on text similarity to a specified inference text, using a machine learning model.
The `text_similarity_reranker` is a type of retriever that enhances search results by re-ranking documents based on semantic similarity to a specified inference text, using a machine learning model.

===== Prerequisites

Expand All @@ -222,33 +222,51 @@ Currently you can integrate directly with the Cohere Rerank endpoint using the <
+
The document field to be used for text similarity comparisons. This field should contain the text that will be evaluated against the `inferenceText`.

`inferenceId`::
`inference_id`::
(Required, `string`)
+
Unique identifier of the inference endpoint created using the {infer} API.

`inferenceText`::
`inference_text`::
(Required, `string`)
+
The text snippet used as the basis for similarity comparison.

`rankWindowSize`::
(Required, `int`)
`rank_window_size`::
(Optional, `int`)
+
The number of top documents to consider in the re-ranking process.
The number of top documents to consider in the re-ranking process. Defaults to `10`.

`minScore`::
`min_score`::
(Optional, `float`)
+
Sets a minimum threshold score for including documents in the re-ranked results. Documents with similarity scores below this threshold will be excluded. Note that score calculations vary depending on the model used.

===== Restrictions

//TODO
A text similarity re-ranker retriever is a compound retriever. Child retrievers may not use elements that are restricted by having a compound retriever as part of the retriever tree.

===== Example

//TODO
[source,js]
----
GET /index/_search
{
"retriever": {
"text_similarity_reranker": {
"retriever": {
"standard": { ... }
}
},
"field": "text",
"inference_id": "my-cohere-rerank-model-v3",
"inference_text": "Most famous landmark in Paris",
"rank_window_size": 100,
"min_score": 0.5
}
}
----
// NOTCONSOLE

==== Using `from` and `size` with a retriever tree

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ propagated to its sub retrievers.
+
Sub retrievers may not use elements that are restricted by having a compound retriever as part of the retriever tree.
See the <<rrf-using-multiple-standard-retrievers,RRF documentation>> for detailed examples and information on how to use the RRF retriever.
* *`text_similarity_reranker` Retriever*. Used for <<semantic-reranking,semantic reranking>>.
* <<text-similarity-reranker-retriever,*Text Similarity Re-ranker Retriever*>>. Used for <<semantic-reranking,semantic reranking>>.
Requires first creating a `rerank` task using the <<put-inference-api,{es} Inference API>>.

[discrete]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ Semantic reranking enables a variety of use cases:

* *Lexical (BM25) retrieval results reranking*
** Out-of-the-box semantic search by adding a simple API call to any lexical/BM25 retrieval pipeline.
** Add semantic search capabilities on top of existing indices without reindexing, perfect for quick improvements.
** Adds semantic search capabilities on top of existing indices without reindexing, perfect for quick improvements.
** Ideal for environments with complex existing indices.

* *Semantic retrieval results reranking*
Expand All @@ -54,7 +54,7 @@ NOTE: In this version, {es} *only supports cross-encoders* for semantic rerankin
* A *cross-encoder model* can be thought of as a more powerful, all-in-one solution, because it generates query-aware document representations.
It takes the query and document texts as a single, concatenated input.
* A *bi-encoder model* takes as input either document or query text.
Documents and query embeddeings are computed separately, so they aren't aware of each other.
Documents and query embeddings are computed separately, so they aren't aware of each other.
** To compute a ranking score, an external operation is required. This typically involves computing dot-product or cosine similarity between the query and document embeddings.

In brief, cross-encoders provide high accuracy but are more resource-intensive.
Expand All @@ -75,7 +75,7 @@ The following is a non-exhaustive list of considerations when choosing between c
For example, their ability to take word order into account can improve on dense or sparse embedding retrieval.
* When trained in tandem with specific retrievers (like lexical/BM25), cross-encoders can “correct” typical errors made by those retrievers.
* Cross-encoders output scores that are consistent across queries.
This means enables you to maintain high relevance in result sets, by setting a minimum score threshold for all queries.
This enables you to maintain high relevance in result sets, by setting a minimum score threshold for all queries.
For example, this is important when using results in a RAG workflow or if you're otherwise feeding results to LLMs.
Note that similarity scores from bi-encoders/embedding similarities are _query-dependent_, meaning you cannot set universal cut-offs.
* Bi-encoders rerank using embeddings. You can improve your reranking latency by creating embeddings at ingest-time. These embeddings can be stored for reranking without being indexed for retrieval, reducing your memory footprint.
Expand Down Expand Up @@ -121,9 +121,7 @@ POST _search
"rank_window_size": 100,
"min_score": 0.5
}
},
"_source": ["title", "text"],
"size": 10
}
}
----
// TEST[skip:TBD]
Expand Down

0 comments on commit 7402dc5

Please sign in to comment.