Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change query clause name to neural_sparse #353

Closed
wants to merge 3 commits into from

Conversation

zane-neo
Copy link
Collaborator

Description

Change query clause name. This doesn't need to be backported to 2.x.

Issues Resolved

[List any issues this PR will resolve]

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed as per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@codecov
Copy link

codecov bot commented Sep 28, 2023

Codecov Report

Merging #353 (9aba394) into main (67ced0d) will not change coverage.
The diff coverage is 80.00%.

@@            Coverage Diff            @@
##               main     #353   +/-   ##
=========================================
  Coverage     84.56%   84.56%           
  Complexity      427      427           
=========================================
  Files            35       35           
  Lines          1289     1289           
  Branches        189      189           
=========================================
  Hits           1090     1090           
  Misses          118      118           
  Partials         81       81           
Files Coverage Δ
...rch/neuralsearch/processor/InferenceProcessor.java 92.71% <100.00%> (ø)
...euralsearch/processor/SparseEncodingProcessor.java 100.00% <ø> (ø)
...neuralsearch/processor/TextEmbeddingProcessor.java 100.00% <ø> (ø)
...h/neuralsearch/query/NeuralSparseQueryBuilder.java 66.37% <100.00%> (ø)
...g/opensearch/neuralsearch/plugin/NeuralSearch.java 73.68% <0.00%> (ø)

@@ -51,7 +51,7 @@ public abstract class NLPProcessor extends AbstractProcessor {

private final Environment environment;

public NLPProcessor(
public InferenceProcessor(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better naming would be TextInferenceProcessor as we dealing with Texts in these processors.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are several considerations:

  1. This is an abstract parent class and it's better to be generic.
  2. Most of the code in this class are to create corresponding data structure to ml-commons which is generic logic.
  3. ml-commons is supporting more inference inputs like image(base64), when neural search going to support this, we can create a new class to extend this since this is generic.

@@ -55,8 +55,8 @@
@Accessors(chain = true, fluent = true)
@NoArgsConstructor
@AllArgsConstructor
public class SparseEncodingQueryBuilder extends AbstractQueryBuilder<SparseEncodingQueryBuilder> {
public static final String NAME = "sparse_encoding";
public class NeuralSparseQueryBuilder extends AbstractQueryBuilder<NeuralSparseQueryBuilder> {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is neural about this QueryBuilder? I thought SparseEncodingQuery Builder was a good name

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is because we call the query clause neural_sparse.

public class SparseEncodingQueryBuilder extends AbstractQueryBuilder<SparseEncodingQueryBuilder> {
public static final String NAME = "sparse_encoding";
public class NeuralSparseQueryBuilder extends AbstractQueryBuilder<NeuralSparseQueryBuilder> {
public static final String NAME = "neural_sparse";
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is neural about this query clause?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We use a neural network model to expand the query text to token/weight, we also had an email to discuss this name and there's no objections so this should be the final decision.

@navneet1v
Copy link
Collaborator

@zane-neo lets remove the skip-changelog tag and provide add the info in changelog around what this change is about.

@navneet1v
Copy link
Collaborator

@zane-neo are you actively working on this PR?

@zane-neo
Copy link
Collaborator Author

This is merged in another PR, closing it now

@zane-neo zane-neo closed this Jan 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants