Skip to content

Commit

Permalink
adjust readme
Browse files Browse the repository at this point in the history
  • Loading branch information
ArneBinder committed Sep 18, 2023
1 parent 110740d commit 9d23948
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1472,13 +1472,13 @@ This will evaluate serialized documents including predicted annotations (see [In
**Have a look into the [evaluate_documents.yaml](configs/evaluate_documents.yaml) config to see all available options**

```bash
python src/evaluate_documents.py metric=f1 metric.layer=entities +dataset.data_dir=PATH/TO/DIR/WITH/SPLITS +dataset.document_type=TYPE/OF/DOCUMENT/IN/JSONL/FILE
python src/evaluate_documents.py metric=f1 metric.layer=entities +dataset.data_dir=PATH/TO/DIR/WITH/SPLITS
```

Note: By default, this utilizes the dataset provided by the
[from_serialized_documents](configs/dataset/from_serialized_documents.yaml) configuration. This configuration is
designed to facilitate the loading of serialized documents, as generated during the [Inference](#inference) step. It
requires to set the parameters `document_type` and `data_dir` or `data_files`. If you want to use a different dataset,
requires to set the parameter `data_dir`. If you want to use a different dataset,
you can override the `dataset` parameter as usual with any existing dataset config, e.g `dataset=conll2003`. But
calculating the F1 score on the bare `conll2003` dataset does not make much sense, because it does not contain any
predictions. However, it could be used with statistical metrics such as
Expand Down

0 comments on commit 9d23948

Please sign in to comment.