diff --git a/docs/reference/search/retriever.asciidoc b/docs/reference/search/retriever.asciidoc index 74c0ade5c8a0d..1bf0075569134 100644 --- a/docs/reference/search/retriever.asciidoc +++ b/docs/reference/search/retriever.asciidoc @@ -29,7 +29,7 @@ A <> that replaces the functionality of a <> that produces top documents from <>. `text_similarity_reranker`:: -A <> that enhances search results by re-ranking documents based on text similarity to a specified inference text, using a machine learning model. +A <> that enhances search results by re-ranking documents based on semantic similarity to a specified inference text, using a machine learning model. [[standard-retriever]] ==== Standard Retriever @@ -204,10 +204,10 @@ GET /index/_search ---- // NOTCONSOLE -[[text-similarity-reranker]] -==== Text Similarity Re-ranker +[[text-similarity-reranker-retriever]] +==== Text Similarity Re-ranker Retriever -The `text_similarity_reranker` is a type of retriever that enhances search results by re-ranking documents based on text similarity to a specified inference text, using a machine learning model. +The `text_similarity_reranker` is a type of retriever that enhances search results by re-ranking documents based on semantic similarity to a specified inference text, using a machine learning model. ===== Prerequisites @@ -222,33 +222,51 @@ Currently you can integrate directly with the Cohere Rerank endpoint using the < + The document field to be used for text similarity comparisons. This field should contain the text that will be evaluated against the `inferenceText`. -`inferenceId`:: +`inference_id`:: (Required, `string`) + Unique identifier of the inference endpoint created using the {infer} API. -`inferenceText`:: +`inference_text`:: (Required, `string`) + The text snippet used as the basis for similarity comparison. -`rankWindowSize`:: -(Required, `int`) +`rank_window_size`:: +(Optional, `int`) + -The number of top documents to consider in the re-ranking process. +The number of top documents to consider in the re-ranking process. Defaults to `10`. -`minScore`:: +`min_score`:: (Optional, `float`) + Sets a minimum threshold score for including documents in the re-ranked results. Documents with similarity scores below this threshold will be excluded. Note that score calculations vary depending on the model used. ===== Restrictions -//TODO +A text similarity re-ranker retriever is a compound retriever. Child retrievers may not use elements that are restricted by having a compound retriever as part of the retriever tree. ===== Example -//TODO +[source,js] +---- +GET /index/_search +{ + "retriever": { + "text_similarity_reranker": { + "retriever": { + "standard": { ... } + } + }, + "field": "text", + "inference_id": "my-cohere-rerank-model-v3", + "inference_text": "Most famous landmark in Paris", + "rank_window_size": 100, + "min_score": 0.5 + } +} +---- +// NOTCONSOLE ==== Using `from` and `size` with a retriever tree diff --git a/docs/reference/search/search-your-data/retrievers-reranking/retrievers-overview.asciidoc b/docs/reference/search/search-your-data/retrievers-reranking/retrievers-overview.asciidoc index 24579dfdf3adf..99659ae76e092 100644 --- a/docs/reference/search/search-your-data/retrievers-reranking/retrievers-overview.asciidoc +++ b/docs/reference/search/search-your-data/retrievers-reranking/retrievers-overview.asciidoc @@ -34,7 +34,7 @@ propagated to its sub retrievers. + Sub retrievers may not use elements that are restricted by having a compound retriever as part of the retriever tree. See the <> for detailed examples and information on how to use the RRF retriever. -* *`text_similarity_reranker` Retriever*. Used for <>. +* <>. Used for <>. Requires first creating a `rerank` task using the <>. [discrete] diff --git a/docs/reference/search/search-your-data/retrievers-reranking/semantic-reranking.asciidoc b/docs/reference/search/search-your-data/retrievers-reranking/semantic-reranking.asciidoc index b46676be3926e..0929167b52fb7 100644 --- a/docs/reference/search/search-your-data/retrievers-reranking/semantic-reranking.asciidoc +++ b/docs/reference/search/search-your-data/retrievers-reranking/semantic-reranking.asciidoc @@ -30,7 +30,7 @@ Semantic reranking enables a variety of use cases: * *Lexical (BM25) retrieval results reranking* ** Out-of-the-box semantic search by adding a simple API call to any lexical/BM25 retrieval pipeline. -** Add semantic search capabilities on top of existing indices without reindexing, perfect for quick improvements. +** Adds semantic search capabilities on top of existing indices without reindexing, perfect for quick improvements. ** Ideal for environments with complex existing indices. * *Semantic retrieval results reranking* @@ -54,7 +54,7 @@ NOTE: In this version, {es} *only supports cross-encoders* for semantic rerankin * A *cross-encoder model* can be thought of as a more powerful, all-in-one solution, because it generates query-aware document representations. It takes the query and document texts as a single, concatenated input. * A *bi-encoder model* takes as input either document or query text. -Documents and query embeddeings are computed separately, so they aren't aware of each other. +Documents and query embeddings are computed separately, so they aren't aware of each other. ** To compute a ranking score, an external operation is required. This typically involves computing dot-product or cosine similarity between the query and document embeddings. In brief, cross-encoders provide high accuracy but are more resource-intensive. @@ -75,7 +75,7 @@ The following is a non-exhaustive list of considerations when choosing between c For example, their ability to take word order into account can improve on dense or sparse embedding retrieval. * When trained in tandem with specific retrievers (like lexical/BM25), cross-encoders can “correct” typical errors made by those retrievers. * Cross-encoders output scores that are consistent across queries. -This means enables you to maintain high relevance in result sets, by setting a minimum score threshold for all queries. +This enables you to maintain high relevance in result sets, by setting a minimum score threshold for all queries. For example, this is important when using results in a RAG workflow or if you're otherwise feeding results to LLMs. Note that similarity scores from bi-encoders/embedding similarities are _query-dependent_, meaning you cannot set universal cut-offs. * Bi-encoders rerank using embeddings. You can improve your reranking latency by creating embeddings at ingest-time. These embeddings can be stored for reranking without being indexed for retrieval, reducing your memory footprint. @@ -121,9 +121,7 @@ POST _search "rank_window_size": 100, "min_score": 0.5 } - }, - "_source": ["title", "text"], - "size": 10 + } } ---- // TEST[skip:TBD]