You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi Enrique !
I am thinking that run_check could use another metric for some tasks, which is Score (Not empty) (so accuracy, prediction, recall on those) .
This is something we have using eval full with classes scores, but I do think that could be useful to track over training.
Basically, in task declaration have an optional not null "empty_value": "_" new key (I am thinking about the dictionary with name in it. And compute accuracy, prediction and recall on filtered results as well. While it's important to know the accuracy of empty values as well (Tense should not be applied to Nouns), morphological features are incredibly skewed (as an exemple, there is definitely less Tense occurrences than Case for example).
Inter-categorical information is also important as well but we do have that in eval (ie. morphological features that are not empty are also skewed data: there is less subjunctive perfect than there is present, less 2nd person than there is 3rd ones...)
The text was updated successfully, but these errors were encountered:
Hi Enrique !
I am thinking that
run_check
could use another metric for some tasks, which isScore (Not empty)
(so accuracy, prediction, recall on those) .This is something we have using
eval
full with classes scores, but I do think that could be useful to track over training.Basically, in task declaration have an optional not null
"empty_value": "_"
new key (I am thinking about the dictionary withname
in it. And compute accuracy, prediction and recall on filtered results as well. While it's important to know the accuracy of empty values as well (Tense should not be applied to Nouns), morphological features are incredibly skewed (as an exemple, there is definitely less Tense occurrences than Case for example).Inter-categorical information is also important as well but we do have that in eval (ie. morphological features that are not empty are also skewed data: there is less subjunctive perfect than there is present, less 2nd person than there is 3rd ones...)
The text was updated successfully, but these errors were encountered: