Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request] When scores are already calculated with sorting applied & track_scores is set to true then do not recalculate scores in the Fetch phase #13610

Closed
vibrantvarun opened this issue May 8, 2024 · 1 comment
Labels
enhancement Enhancement or improvement to existing feature or request Search:Query Capabilities untriaged

Comments

@vibrantvarun
Copy link
Member

vibrantvarun commented May 8, 2024

Is your feature request related to a problem? Please describe

When Hybrid query with the sorting is applied and track_scores=true in the search request then do not recalculate the scores in fetch phase.
Currently, when sorting is applied and track_scores=true then instance of FetchScorePhase processor is created in the fetch phase to calculate the scores for the hits fetched in the fetch phase.
In the Hybrid query case scenario(sort is applied and track_scores is true), then normalized scores are calculated while executing phase results pipeline processor therefore we do not have to calculate scores in the fetch phase because normalized scores are calculated exclusively by normalization processor.

Describe the solution you'd like

When hybrid query with sorting applied and track scores is enabled then we can deduce that score is already calculated.
In other query type scenarios then score will be float.NaN in the scoredocs.

In FetchSearchPhase.java innerRun() method when we deduce the reduceQueryPhase from the topDocs then those TopDocs contains ScoreDocs. The score calculated in the Query phase is present in present in the ScoreDocs. Moreover, if sorting is applied then reduceQueryPhase has an object of sortedTopDocs which contains isSortedByField object.

Therefore we can deduce one condition that

boolean isSortingApplied=reducedQueryPhase.sortedTopDocs.isSortedByField;
ScoreDoc[] scoreDocs = reducedQueryPhase.sortedTopDocs.scoreDocs;

if(scoreDocs.length>0){
         if (scoreDocs[0].score!=Float.NaN && isSortingApplied){
               isScoreAlreadyCalculated=true;
         }
}

If we can deduce score is already calculated then do not calculate the scores in Fetch Phase by disabling the FetchScorePhase instantiation.

Also we need to add a new parameter isScoreAlreadyCalculated in the ShardFetchRequest which is propagated to FetchPhase.java and if isScoreAlreadyCalculated =true then do not instantiate FetchScorePhase processor.

Related component

Search:Query Capabilities

Additional context

Hybrid Scores are normalized using normalization processor which is a phase results processor. Normalization works as below during search

Multiple Shards Scenario the order of execution of phases looks like

Query Phase
Normalization Processor
Fetch Phase

Single Shard Scenario the order of execution of phases looks like

Query Phase 
Fetch Phase
Normalization Processor
@vibrantvarun vibrantvarun added enhancement Enhancement or improvement to existing feature or request untriaged labels May 8, 2024
@vibrantvarun vibrantvarun changed the title [Feature Request] When scores are already calculated & track_scores is set to true then do not recalculate scores in the Fetch phase [Feature Request] When scores are already calculated with sorting applied & track_scores is set to true then do not recalculate scores in the Fetch phase May 8, 2024
@github-project-automation github-project-automation bot moved this from 🆕 New to ✅ Done in Search Project Board May 8, 2024
@vibrantvarun
Copy link
Member Author

vibrantvarun commented May 8, 2024

Closing for now as reevaluation is on my end is needed for the use case

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Enhancement or improvement to existing feature or request Search:Query Capabilities untriaged
Projects
Archived in project
Development

No branches or pull requests

1 participant