Skip to content

Commit

Permalink
fix: Update CHANGELOG.md to include elo evaluation how to
Browse files Browse the repository at this point in the history
TASK: IL-502
  • Loading branch information
MerlinKallenbornAA committed May 21, 2024
1 parent b8376fc commit 61e8002
Showing 1 changed file with 3 additions and 1 deletion.
4 changes: 3 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,14 @@

### Breaking Changes
- Changed the behavior of `IncrementalEvaluator::do_evaluate` such that it now sends all `SuccessfulExampleOutput`s to `do_incremental_evaluate` instead of only the new `SuccessfulExampleOutput`s.
-

### New Features
- Add generic `EloEvaluationLogic` class for implementation of Elo evaluation use cases.
- Add `EloQaEvaluationLogic` for Elo evaluation of QA runs, with optional later addition of more runs to an existing evaluation.
- Add `EloAggregationAdapter` class to simplify using the `ComparisonEvaluationAggregationLogic` for different Elo use cases.
- Add `elo_qa_eval` tutorial notebook describing the use of an (incremental) Elo evaluation use case for QA models.
- Add `how_to_implement_elo_evaluations` how-to as skeleton for implementing Elo evaluation cases

### Fixes
- `ExpandChunks`-task is now fast even for very large documents

Expand Down

0 comments on commit 61e8002

Please sign in to comment.