Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix post_process_function bug on sort results for rerank_pipeline_with_bge-rerank-m3-v2_model_deployed_on_Sagemaker.md #3277

Conversation

tkykenmt
Copy link
Contributor

…n recieve empty result from a model.

Description

fix post_process_function bug on sort results for docs/tutorials/rerank/rerank_pipeline_with_bge-rerank-m3-v2_model_deployed_on_Sagemaker.md

Related Issues

Resolves #3247

Check List

  • New functionality includes testing.
  • New functionality has been documented.
  • API changes companion pull request created.
  • Commits are signed per the DCO using --signoff.
  • Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

…n recieve empty result from a model.

Signed-off-by: tkykenmt <[email protected]>
@tkykenmt tkykenmt force-pushed the patch.tutorial-rerank-bge-rerank-m3-v2 branch from 91abc83 to 093fed8 Compare December 15, 2024 14:21
@tkykenmt tkykenmt temporarily deployed to ml-commons-cicd-env-require-approval December 15, 2024 14:21 — with GitHub Actions Inactive
@tkykenmt tkykenmt temporarily deployed to ml-commons-cicd-env-require-approval December 15, 2024 14:21 — with GitHub Actions Inactive
@tkykenmt tkykenmt temporarily deployed to ml-commons-cicd-env-require-approval December 16, 2024 17:41 — with GitHub Actions Inactive
{
"index": 3,
"score": 0.00012148176
}
]
```

The connector `post_process_function` transforms the model's output into a format that the [Reranker processor](https://opensearch.org/docs/latest/search-plugins/search-pipelines/rerank-processor/) can interpret. This adapted format is as follows:
The connector `post_process_function` transforms the model's output into a format that the [Reranker processor](https://opensearch.org/docs/latest/search-plugins/search-pipelines/rerank-processor/) can interpretm, and order result by index. This adapted format is as follows:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit, interpretm should be interpret?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for review. I fixed it on fixed up commit.

@tkykenmt tkykenmt deployed to ml-commons-cicd-env-require-approval December 25, 2024 04:09 — with GitHub Actions Active
@tkykenmt tkykenmt temporarily deployed to ml-commons-cicd-env-require-approval December 25, 2024 04:09 — with GitHub Actions Inactive
@tkykenmt tkykenmt requested a deployment to ml-commons-cicd-env-require-approval December 25, 2024 06:28 — with GitHub Actions Waiting
akolarkunnu and others added 5 commits December 25, 2024 15:53
…-project#3260)

This PR addresses the first part of this enhancement "Validate if connector payload has all the required fields. If not provided, throw the illegal argument exception".
Validation of fields description, parameters, credential, and request_body are missing. That validations are added in this fix.
Added new test cases correspong to these validations and fixed all failing test cases because of these new validations.

Partially Resolves opensearch-project#1382

Signed-off-by: Abdul Muneer Kolarkunnu <[email protected]>
…oject#3282)

Modify getMemory(Conversation) to return the application_type parameter.
Include application_type in the ConversationMeta data model.
Update existing tests to validate the new parameter.

Signed-off-by: rithin-pullela-aws <[email protected]>
…ject#3283)

* Enchance Message and Memory API Validation and storage

Throw an error when an unknown field is provided in CreateConversation or CreateInteraction.
Skip saving empty fields in interactions and conversations to optimize storage usage.
Modify GET requests for interactions and conversations to return only non-null fields.
Throw an exception if all fields in a create interaction call are empty or null.
Add unit tests to cover the above cases.

Signed-off-by: rithin-pullela-aws <[email protected]>

* Update unit test to check for null instead of empty map

Signed-off-by: rithin-pullela-aws <[email protected]>

* Refactored userstr to Camel Case

Signed-off-by: rithin-pullela-aws <[email protected]>

* Addressing comments

Used assertThrows and added promptTemplate with empty string in test_ToXContent to ensure well rounded testing of expected functionality

Signed-off-by: rithin-pullela-aws <[email protected]>

* Undo: throw an error when an unknown field is provided in CreateConversation or CreateInteraction.

Signed-off-by: rithin-pullela-aws <[email protected]>

---------

Signed-off-by: rithin-pullela-aws <[email protected]>
…function recieve empty result from a model.

Signed-off-by: tkykenmt <[email protected]>
@tkykenmt tkykenmt requested a deployment to ml-commons-cicd-env-require-approval December 25, 2024 07:09 — with GitHub Actions Waiting
@tkykenmt tkykenmt requested a deployment to ml-commons-cicd-env-require-approval December 25, 2024 07:09 — with GitHub Actions Waiting
@tkykenmt tkykenmt closed this Dec 25, 2024
@tkykenmt tkykenmt deleted the patch.tutorial-rerank-bge-rerank-m3-v2 branch December 25, 2024 08:10
@tkykenmt
Copy link
Contributor Author

close to move to #3296

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] post_process_function on rerank_pipeline_with_bge-rerank-m3-v2_model_deployed_on_Sagemaker.md
4 participants