Skip to content

Commit

Permalink
Update intro/Intro_Supervision.ipynb: -1 as ABSTAIN and metrics in LF…
Browse files Browse the repository at this point in the history
…Analysis
  • Loading branch information
Hiromu Hota authored and lukehsiao committed May 27, 2020
1 parent bf63236 commit 96717f9
Showing 1 changed file with 7 additions and 8 deletions.
15 changes: 7 additions & 8 deletions intro/Intro_Supervision.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@
"source": [
"In this tutorial, you will learn what a labeling function (LF) is and how to write them by leverage Fonduer's [data model utilities](https://fonduer.readthedocs.io/en/stable/user/data_model_utils.html).\n",
"\n",
"At a high level, a labeling function is a simple Python function that takes a candidate (a part and numerical value, in these intro tutorials) as input, and returns a label for the input candidate. Labels can be one of these values: {-1, 0, 1}. A label of -1 signifies that a candidate is False, 0 is a way to abstain from voting, and +1 labels the candidate as True."
"At a high level, a labeling function is a simple Python function that takes a candidate (a part and numerical value, in these intro tutorials) as input, and returns a label for the input candidate. Labels can be one of these values: {-1, 0, 1}. -1 is a way to abstain from voting, a label of 0 signifies that a candidate is False, and +1 labels the candidate as True."
]
},
{
Expand Down Expand Up @@ -360,13 +360,12 @@
"Next, we can view insights provided by Fonduer to better understand the quality and coverage of our labeling functions.\n",
"\n",
"In order to view statistics about the resulting label matrix, we provide several metrics to evaluate labelding functions:\n",
"* **Coverage** is the fraction of candidates that the labeling function emits a non-zero label for.\n",
"* **Overlap** is the fraction candidates that the labeling function emits a non-zero label for and that another labeling function emits a non-zero label for.\n",
"* **Conflict** is the fraction candidates that the labeling function emits a non-zero label for and that another labeling function emits a conflicting non-zero label for.\n",
"* **TP** is the number of True Positive candidates, or true candidates which were correctly labeled as True.\n",
"* **FP** is the number of False Positive candidates, or false candidates which were incorrectly labeled as True.\n",
"* **FN** is the number of False Negative candidates, or true candidates which were incorrectly labeled as False.\n",
"* **TN** is the number of True Negative candidates, or false candidates which were correctly labeled as False.\n",
"* **Coverage** is the fraction of candidates that the labeling function emits a non-abstain label for.\n",
"* **Overlaps** is the fraction candidates that the labeling function emits a non-abstain label for and that another labeling function emits a non-abstain label for.\n",
"* **Conflicts** is the fraction candidates that the labeling function emits a non-abstain label for and that another labeling function emits a conflicting non-abstain label for.\n",
"* **Correct** is the number of candidates that the labeling function labels correctly.\n",
"* **Incorrect** is the number of candidates that the labeling function labels incorrectly.\n",
"* **Empirical Accuracy** is the fraction of correctly labeled candidates.\n",
"\n",
"In addition, because we have already loaded the gold labels, we can view the emperical accuracy of these labeling functions when compared to our gold labels:"
]
Expand Down

0 comments on commit 96717f9

Please sign in to comment.