Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inconsistent Training / Faulty recognition of exact sentence #15

Open
Aran30 opened this issue Sep 19, 2023 · 0 comments
Open

Inconsistent Training / Faulty recognition of exact sentence #15

Aran30 opened this issue Sep 19, 2023 · 0 comments
Assignees
Labels
enhancement New feature or request question Further information is requested

Comments

@Aran30
Copy link
Collaborator

Aran30 commented Sep 19, 2023

While working on a bot model, I noticed two things.

First of all, each restart of the bot (and thus each re-training of the NLP model) causes the recognition scores to variate.

While not necessarily bad, in some examples this might create inconsistencies in how bots work.

Secondly, and linked to the first "problem", the wrong intent is recognized even though the exact word or sentence that is present in the training data for the correct intent is given as input.

The example in question:

stop_entity = bot.new_entity('stop_entity', entries=bus_stops)
city_entity = bot.new_entity('city_entity', entries=cities)
stop_intent = bot.new_intent('stop_intent',[
    'STOP'
])
city_intent = bot.new_intent('city_intent',[
    'CITY'
])

"cities" is a list of cities from Luxembourg and "bus_stops" is the list of bus stops in Luxembourg.
This might lead to "cities" containing an example like "Differdange" and "bus_stops" examples such as "Differdange, Schoul" or "Differdange, Fousbann".
While it is understandable that the NLP engine might not entirely recognize the "city_intent" with a 100% confidence, I feel that when writing the exact sentence "Differdange, Fousbann" that the "stop_intent" should be recognized. Yet, currently, depending on the restart, "Differdange, Fousbann" will be classified as "city_intent"

Any thought, correction or opinion is appreciated.

@Aran30 Aran30 added enhancement New feature or request question Further information is requested labels Sep 19, 2023
@mgv99 mgv99 added this to the Version 1.0 milestone Oct 20, 2023
@jcabot jcabot removed this from the Version 1.0 milestone Nov 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request question Further information is requested
Projects
None yet
Development

No branches or pull requests

3 participants