Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support to parse response if fields or _source are enabled #427

Merged
merged 3 commits into from
Dec 29, 2023

Conversation

VijayanB
Copy link
Member

Description

If custom field is used as unique field instead of _id, users can either enable _source to fetch those fields, or, use fields to include only return those fields with or without source. Hence, update parser to look for field name in fields first, and from _source at last.

Issues Resolved

Part of #103

Testing

  • New functionality includes testing

[Describe how this change was tested]

make test

tests/worker_coordinator/runner_test.py::QueryRunnerTests::test_query_vector_search_with_custom_id_field PASSED [ 64%]
tests/worker_coordinator/runner_test.py::QueryRunnerTests::test_query_vector_search_with_custom_id_field_inside_source PASSED [ 64%]


================= 1208 passed, 5 skipped, 3 warnings in 15.50s =================


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

if id_field in hit: # Will add to candidates if field value is present
candidates.append(hit[id_field])
field_value = _get_field_value(hit, id_field)
if field_value: # Will add to candidates if field value is present

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

have we tested this if the field_value is 0 ? will this condition work?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. Let me add test case

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated test case and added is None check.

If custom field is used as unique field instead of _id,
users can either enable _source to fetch those fields,
or, use fields to include only return those fields with or
without source. Hence, update parser to look for field name
in fields first, and from _source at last.

Signed-off-by: Vijayan Balasubramanian <[email protected]>
Maximum number of public method is reached, hence,
move vector serach query test methods into
separate class.

Signed-off-by: Vijayan Balasubramanian <[email protected]>
Will allow body params from workload except query parameter.
We will replace that with vector search query.
In future we will extend this query to support other
features like post filter, efficient filter, etc...

Signed-off-by: Vijayan Balasubramanian <[email protected]>
@VijayanB VijayanB requested a review from jmazanec15 December 27, 2023 21:01
@VijayanB
Copy link
Member Author

@IanHoang Can you take a look at this PR?

@IanHoang IanHoang merged commit b861175 into opensearch-project:main Dec 29, 2023
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants