Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/embedding only support colang 2 #737

Merged
merged 4 commits into from
Sep 15, 2024

Conversation

Pouyanpi
Copy link
Collaborator

@Pouyanpi Pouyanpi commented Sep 9, 2024

  • Add test for embeddings only and fallback_intent in Colang 2
  • Fix a bug: fall back intent must be used when the embedding only branch does not return any intent.

@Pouyanpi Pouyanpi force-pushed the feat/embedding-only-support-colang-2 branch from 11ddfc7 to 2a31394 Compare September 9, 2024 07:50
@drazvan
Copy link
Collaborator

drazvan commented Sep 10, 2024

@Pouyanpi : I think there's a bug in both Colang 1 and 2 implementations. The threshold is meant to be for the top choice confidence. But if none is above that, and I don't have a fallback intent, then no examples are used either (because the search is performed only once with the threshold). Can you double-check? In the fallback with no intent, we should include the top 5 examples regardless of their similarity score.

@Pouyanpi
Copy link
Collaborator Author

Pouyanpi commented Sep 11, 2024

In the fallback with no intent, we should include the top 5 examples regardless of their similarity score.

@drazvan, let me clarify my understanding of this:

Scenario: We are using embeddings exclusively with a search threshold and no fallback intent.

  • If the search threshold filters out all results, leaving the result set empty, we ignore the threshold and return the top 5 results regardless of their similarity scores.
  • However, if there are any results that meet the search threshold (even if fewer than 5), we keep only those results.

@Pouyanpi Pouyanpi self-assigned this Sep 11, 2024
Copy link
Collaborator

@drazvan drazvan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good 👍.

@drazvan drazvan merged commit e163259 into develop Sep 15, 2024
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants