adjust Prodigy recipe to include new configs

[mnaydan]

• character versus token level highlighting
• settings so that each page is annotated by two and only two annotators
• ignore button (tracked as update Python recipe to hide ignore button #60)

We'll have to decide what settings in the recipe versus environment.

For testing:

• verify that character highlighting works as desired
• verify that ignore button is no longer shown
• verify that a page is only shown to two annotators (sessions); if more occur this could be a case of work stealing which can be verified in the logs

[jerielizabeth]

Put in recipe for now and add comments noting that this is not long term.

Also of note: turning off feed overlap in Ansible main config.

[mnaydan]

The character-level highlighting and ignore button issues look good. But as far as I can tell from the test sessions, it's still serving out everything to everyone. Even after annotations are saved/submitted in two sessions, the other test sessions receive those same pages.

[mnaydan]

@laurejt thanks for loading it with the smaller test set so we can try to test the two-annotators behavior. I'm still running into weird behavior. I opened up 3 different test sessions and each session was annotating different pages (there was no overlap), and I got a "no more tasks available" message after just 1 or 2 pages.

[laurejt]

For task routing to work properly when we want more than one annotation per task (i.e. page), we need to specify the session names up front. The distribution of annotations will only be approximately even (routing relies on mods of hashes). Work stealing has the potential to make this more uneven, since tasks can be "stolen" from one session and reassigned to another based on the relative age of the assignment. While this can make the work distribution more uneven, it will let us finish faster. For more on work stealing, see the answer I received from the Prodigy support forum.

For the future annotations rounds and efforts, I'm investigating how we might guarantee 2 or more annotations per task without specifying session names. We can probably achieve this by writing a custom task router routine. That said, I've made a post asking Prodigy support on any recommendations for how to proceed.