Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

load adjudication recipe on local environment #75

Open
mnaydan opened this issue Aug 19, 2024 · 1 comment
Open

load adjudication recipe on local environment #75

mnaydan opened this issue Aug 19, 2024 · 1 comment
Assignees

Comments

@mnaydan
Copy link
Collaborator

mnaydan commented Aug 19, 2024

Recipe needs to allow for character-level text span annotations, which the original annotators for round 1 couldn't do because they did tokens.

If possible, we should use Prodigy's built-in review recipe: https://prodi.gy/docs/recipes#review

@mnaydan mnaydan added this to the Annotation work completed milestone Sep 10, 2024
@mnaydan mnaydan changed the title write a custom recipe for adjudication for first round to convert to character-level indexing write a custom recipe for adjudication for round 1 Sep 20, 2024
@mnaydan mnaydan changed the title write a custom recipe for adjudication for round 1 load adjudication recipe on second VM Oct 3, 2024
@mnaydan mnaydan changed the title load adjudication recipe on second VM load adjudication recipe on second VM or local environment Oct 9, 2024
@jerielizabeth jerielizabeth changed the title load adjudication recipe on second VM or local environment load adjudication recipe on local environment Oct 21, 2024
@laurejt
Copy link
Contributor

laurejt commented Oct 22, 2024

Unfortunately, the default review recipe does not work the way we want as is.

Below are several issues and the status for fixing/addressing these through a custom recipe.

  • Need character-level span selection
    • This can be fixed by setting ner_manual_highlight_chars in the config
  • The "default" review interface does not include the page image or page metadata (e.g., page id, volume title, etc.)
    • I expect that we can adopt the approach(es) used by our custom annotation recipe
  • Annotations are difficult to view and compare in the "default" review interface
    • I still need to investigate this further, but I suspect we can get around this by preloading the existing annotations which we give alternative label names.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Under Review
Development

No branches or pull requests

2 participants