diff --git a/README.md b/README.md index 9b89b2b..7ff9d0d 100644 --- a/README.md +++ b/README.md @@ -1472,13 +1472,13 @@ This will evaluate serialized documents including predicted annotations (see [In **Have a look into the [evaluate_documents.yaml](configs/evaluate_documents.yaml) config to see all available options** ```bash -python src/evaluate_documents.py metric=f1 metric.layer=entities +dataset.data_dir=PATH/TO/DIR/WITH/SPLITS +dataset.document_type=TYPE/OF/DOCUMENT/IN/JSONL/FILE +python src/evaluate_documents.py metric=f1 metric.layer=entities +dataset.data_dir=PATH/TO/DIR/WITH/SPLITS ``` Note: By default, this utilizes the dataset provided by the [from_serialized_documents](configs/dataset/from_serialized_documents.yaml) configuration. This configuration is designed to facilitate the loading of serialized documents, as generated during the [Inference](#inference) step. It -requires to set the parameters `document_type` and `data_dir` or `data_files`. If you want to use a different dataset, +requires to set the parameter `data_dir`. If you want to use a different dataset, you can override the `dataset` parameter as usual with any existing dataset config, e.g `dataset=conll2003`. But calculating the F1 score on the bare `conll2003` dataset does not make much sense, because it does not contain any predictions. However, it could be used with statistical metrics such as