Skip to content

Commit

Permalink
Apply 1 suggestion(s) to 1 file(s)
Browse files Browse the repository at this point in the history
  • Loading branch information
trebedea authored and prasoonvarshney committed Nov 24, 2023
1 parent c42988c commit 9ed7b19
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion nemoguardrails/eval/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ Results on _banking_ dataset, metric used is accuracy.
In the Guardrails library, we provide two approaches out of the box for the fact-checking rail, these are colloquially referred to as AskLLM and AlignScore in the rest of this documentation. For more details read the [library guide](./../../docs/user_guides/guardrails-library.md).

#### AskLLM
In this approach, the fact-checking rail is implemented as an entailment prediction problem. Given an evidence passage and the predicted answer, we prompt an LLM to predict yes/no to whether the answer grounded in the evidence or not. This is the default approach.
In this approach, the fact-checking rail is implemented as an entailment prediction problem. Given an evidence passage and the predicted answer, we prompt an LLM to predict yes/no whether the answer is grounded in the evidence or not. This is the default approach.

#### AlignScore
This approach is based on the AlignScore model [Zha et al. 2023](https://aclanthology.org/2023.acl-long.634.pdf). Given an evidence passage and the predicted answer, the model is finetuned to predict that they are aligned when:
Expand Down

0 comments on commit 9ed7b19

Please sign in to comment.