Using INCEpTION with an ontolex wikibase

[reckart]

Originally here: https://groups.google.com/g/inception-users/c/tU7fAqDgnkQ

[reckart]

The property that connects lexemes to senses in a Wikibase is ontolex:sense, as you can see in https://unidive.wikibase.cloud/wiki/Special:EntityData/L129.ttl , line 54-56. Both LexicalEntry and LexicalSense have labels as rdfs:label; to display these in inception would be enough for the user to select the sense that applies.

In the unidive wikibase, it seems that there is no rdfs:label. For LexicalEntry, I can configure the lemma property as the label and for LexicalSense, I can use definition.

What we would like to do is to enable users to annotate lexemes in a corpus with Wikibase sense IDs, e.g. https://unidive.wikibase.cloud/entity/L129-S2 .

Like shown in the screenshots?

[dlindem]

This looks perfect, thanks a lot !!! We will certainly use INCEpTION for this annotation task, and keep you posted about progress and related presentations / publications.
As soon as we have lexemes and senses in other languages on our wikibase, it would be very good to restrict match suggestions to lexemes of the language of the corpus to annotate. You are already getting wiki language codes (like "en"); could such a restriction be set up?

[dlindem]

BTW, I think that semantic annotation using Wikidata lexeme senses and morphological annotation using Wikidata word forms should be interesting for a lot of people. Forms are, like senses, are "subentities" of lexemes (they have URI like https://www.wikidata.org/entity/L2957-F7).

[reckart]

BTW, I think that semantic annotation using Wikidata lexeme senses and morphological annotation using Wikidata word forms should be interesting for a lot of people. Forms are, like senses, are "subentities" of lexemes (they have URI like https://www.wikidata.org/entity/L2957-F7).

Using these settings:

• URL: https://query.wikidata.org/sparql
• Language: en
• Full text search mode: Wikidata
• IRI Schema: Custom
  • Class IRI: http://www.w3.org/ns/lemon/ontolex#LexicalEntry
  • Subclass IRI: http://www.w3.org/ns/lemon/ontolex#lexicalForm
  • Label IRI: http://www.w3.org/ns/lemon/ontolex#lexicalForm
  • the rest set as in IRI Scheme "RDF"

INCEpTION allows the configuration of a hierarchy mapping using the Subclass IRI mapping. However, that requires that statements expressing the relation are in the form:

<subject-iri> <subclassof-iri> <superclass-iri>
urn:red_wine rdfs:ubClassOf urn:wine

In your case, however, the statements are constructed as

<superclass-iri> <has-subclass-iri> <subject-iri>
urn:mouse has-form urn:mice

This is not supported by INCEpTION at the moment. If you could add statements that do the reverse mapping from forms to their respective lexical entries, then you could configure INCEpTION to list forms "under" their respective lexical entries. Currently you could only configure it to list lexemes under their forms.

Next problem is that the forms and lexical entries do not have a common label property. Lexical entries use the lemma property and forms use the representation property. INCEpTION currently expects that all concepts share a common label property. It would make the mapping configuration considerably more complicated to allow for different label properties depending on the type of a concept.

Next, INCEpTION expects that a knowledge base has one or more labels (synonyms) in potentially different languages. It does not really expect that it has to filter the knowledge-base according to some language property like dct:language.

So if I change the language of the knowledge base configuration in INCEpTION to fr for example, then I start being able to see the lemma in this example:

[dlindem]

Just adding this reference, where it is explained that rdfs:label indeed is not available for Lexeme, Sense and Form.

[dlindem]

We also would like to test annotating corpora using another wikibase instance. Could you maybe provide an example (a list of values that worked for unidive.wikibase.cloud for the SPARQL etc. settings) so we learn how to configure INCEpTION for that?

[reckart]

Here are the settings:

unidive.wikibase.cloud - Senses

• SPARQL URL: https://unidive.wikibase.cloud/query/sparql
• Language: en
• Full text search mode: Unknown
• IRI Schema: Custom
  • Class IRI: http://www.w3.org/ns/lemon/ontolex#LexicalSense
  • Label IRI: http://www.w3.org/2004/02/skos/core#definition
  • the rest set as in IRI Scheme "RDF"

unidive.wikibase.cloud - Lexical Entry

• SPARQL URL: https://unidive.wikibase.cloud/query/sparql
• Language: en
• Full text search mode: Unknown
• IRI Schema: Custom
  • Class IRI: http://www.w3.org/ns/lemon/ontolex#LexicalEntry
  • Label IRI: http://wikiba.se/ontology#lemma
  • Description IRI: http://schema.org/description
  • the rest set as in IRI Scheme "RDF"

By setting the Class IRI to http://www.w3.org/ns/lemon/ontolex#LexicalEntry and http://www.w3.org/ns/lemon/ontolex#LexicalSense in the two KB configurations, we effectively restrict each of them to list only concepts belonging to each of these classes respectively. It is a bit of an abuse of the mapping, but it seems to work.

[reckart]

If I put myself into the position of an annotator who wants to annotate words in text with lexical entries and senses, I would imagine maybe that I first select the lexical entry from a list and then from a second list, I select one of the senses associated with that lexical entry. So I imagine I want the senses that I can select to be constraint to those suitable for the selected lexical entry.

There is a way to model such a situation in INCEpTION, but it would require first converting the data from your knowledge bases into another format. Specifically, you could

• generate a tagset JSON file based on your lexical entries (cf. this tagset JSON example)
• generate a tagset JSON file based on your senses
• generate a constraint rules file expressing which senses are value for which lexical entries

A similar setup for frame-based annotation is available in this example project.

While this approach requires additional data conversion and does not link directly to your Wikibases, I believe it may produce the more convenient experience for the annotators.

[reckart]

With tagsets, there is no separation between the tag and the label. The tag is saved in the CAS and displayed in the UI. But you could generate tags that are not full but abbreviated URIs, e.g. Haus (L2957-F4). You could also "abuse" the tag description to store additional information in human-readable but also machine-extractable form.

[dlindem]

I have tested this, and produced a JSON tagset (looks good). When trying to load it, two issues:

1. A remote instance refuses to load it (does not react)
2. A local instance tries to load it, but then returns an error (see image). The file contains 16,106 tags. Is 255 the maximum?

[reckart]

You seem to try inserting a single tag that is longer than 255 characters. The number of tags per tagset is not really limited. We've had tagsets with thousands of tags already.

[reckart]

The local and remote instances should react the same. Do you see any error in the logs of the remote instance?

[dlindem]

Thanks for your super quick reply.
The remote instance is somewhere else, I'll ask what's the matter.
And yes, it was the too long sense description in the tag! Now the dictionary sense annotation layer looks like this, very nice:

[dlindem]

This is incredibly useful, many thanks!!

The lexeme search works well! Great!

We actually want to annotate historical text (stemming from here) with standard lemmata in Wikibase. A recommender for string-matching tokens also works, awesome!
The sense annotation having to match the sense description, however, is not that useful. But, if entering the lexeme URI in the sense search field, and adding "-S1", the first sense description of the correct entry shows up. If that value could be set as default value to search, perfect. Connected to that: Can't the search field be filled with the token form instead of starting from blank and type the form?

I think we should be able to feed the annotations to Wikibase, creating entities describing Tokens and Spans like these, using CAS format, then cassis. Since token identification is offset based, problems would arise if the source changes. Can't we somehow import a document (create a CAS) having already defined token URI (example), e.g. in the xml:id of the CAS tokens?

[dlindem]

...if in the sense search field to the lexeme uri r'-S[0-9]' could be added...

[reckart]

I think we should be able to feed the annotations to Wikibase, creating entities describing Tokens and Spans like these, using CAS format, then cassis. Since token identification is offset based, problems would arise if the source changes. Can't we somehow import a document (create a CAS) having already defined token URI (example), e.g. in the xml:id of the CAS tokens?

The xml:id field in the CAS is only used during (de)serialization to allow resolving references. It is not actually part of the data. If you want to give tokens an ID, you have to use the id feature of the Token layer. You can do that if you prepare your documents for import using cassis as part of the adding an initial token and sentence layers to your data. A cassis example that includes adding a sentence layer is here - added tokens would work in the same manner. Make sure though that tokens do not start/end with whitespace, that sentence boundaries are always aligned with token boundaries, and that there is no text in between sentences / tokens that is not covered by them.

[reckart]

The sense annotation having to match the sense description, however, is not that useful. But, if entering the lexeme URI in the sense search field, and adding "-S1", the first sense description of the correct entry shows up. If that value could be set as default value to search, perfect. Connected to that: Can't the search field be filled with the token form instead of starting from blank and type the form?

The search goes over the value of the label field (and over the additional search properties). So as long as you have some statement going

<subject> <label-iri> "L59231"

you could search over it. If you input a full IRI into the input field, the system will also be able to find the entry. However, that is basically a fall-back mechanism. Normally, you would want to use a full-text index to search over your concepts labels and the IRIs are typically not indexed by such a mechanism.

To avoid always having to write the full URL, it could be considered to extend the current behavior such that the base prefix configured in the knowledge base settings could be implicitly prepended. That would then lead to the user only having to enter e.g. L59231 instead of https://monumenta.wikibase.cloud/entity/L59383 if the base prefix would be configured accordingly. Note that this is not present behavior (33.2), but I believe it could easily be added to an upcoming version if it were found to be useful.

[reckart]

The sense annotation having to match the sense description, however, is not that useful. But, if entering the lexeme URI in the sense search field, and adding "-S1", the first sense description of the correct entry shows up. If that value could be set as default value to search, perfect.

If I understand it correctly, you want the value of one field to kind of pre-fill the value of another field. At first glance, it seems like a functionality that is very specific to your case and at the moment I don't see other use-cases that this could transfer to. So I am not sure right now if adding the complexity of the interaction between the fields and the configuration for such a mechanism justifies the impact.

Connected to that: Can't the search field be filled with the token form instead of starting from blank and type the form?

The search field is intentionally blank at the start. However, the text covered by the annotation is always implicitly included in the query. So what you do is, you just press the space bar in the field to trigger the search and the dropdown will open using the annotation text as the query term.

[dlindem]

yes! Space bar works for sending the token form as-is to the search! Thx!

[engelbrechtdesign]

Hm, I installed INCEpTION because I would like to find and mark/list all verbs in text-documents, that I import, and I want INCEpTION to attribute a sentimental value to each verb in those texts - let's say between +2 and -2. Then I would like INCEpTION to memorize, if I corrected some of these values, and complete the database by time.
So I tried already to create a layer, which at least marks all verbs in the imported text document (?) and to somehow connect it with an appropriate external (?) knowledge base (which lists verbs in German language) - But hey... next to impossible for me so far!!!!

[reckart]

Do you have a specific need for using an ontolex ontology for your task?

Otherwise, I'd suggest you use the Simple span and relation project template, delete the relation layer, change the tagset to include your labels (-2 to +2) and go ahead with your annotation.

[engelbrechtdesign]

Thank You very much for Your indications. Are You talking about a "universal Dependencies" project form? It comes with the following pre-defined layers: Dependencies, Lemma, Morphological Features, part of speech, surface form and - span. I could not find a "relation" layer. Is this meant with "dependencies"? And which exact tagset would I need? There is a multiple choice also... (in most cases: Dependency flavors, UD Universal dependencies, UD universal POS tags.
As far as I understand from your message, I do not need ANY supplementary knowledge base, because all the necessary knowledge for extracting verbs from a text for example, is already "built in" with INCEpTION? Best, Christine

[reckart]

I am not exactly sure what your annotation task is really meant to be, but here is what I meant.

demo.mp4

Please consider starting a new discussion if you have additional questions so we can leave this discussion thread here for ontolex stuff.

[engelbrechtdesign]

Whao, great!!! I really would like to open a new discussion thread about INCEpTION with some title like: "How setting up layers and basic functions on the linguistic program INCEpTION", meant to help average users for a better understanding - but guess what: I even didn't succeed in opening a new thread so far! I would really like to ask some more questions based on the demonstration video You posted though...

[engelbrechtdesign]

Maybe the reason is, that my account on GitHub is brand-new,

[reckart]

Try this

• Go to https://github.com/inception-project/inception
• Click on Discussions there, that should take you to discussions
• Click on New discussion
  
• Click on Get started in the topic row General (or another row if you find it more appropriate)