All
Language Understanding
Low-Resource Learning
diff --git a/index.json b/index.json
index f65e403..c23a299 100644
--- a/index.json
+++ b/index.json
@@ -1 +1 @@
-[{"authors":["roland-roller"],"categories":null,"content":"I am a senior researcher and project manager in the Speech and Language Technology group at the German Research Center for Artificial Intelligence (DFKI). I focus on natural language processing and machine learning topics with a high interest in medical use cases. My work spans classical information extraction, anonymization, clinical decision support, chatbots and LLM agents. I work on various projects related to different medical domains, such as nephrology, emergency medicine, tumor boards, and sexual medicine. Here is the list of current projects: KIBATIN, PRIMA-AI, Medinym, ADBoard, SmartNTx, and Veranda. Please contact me if you are interested in our research, collaboration/supervision or would like to visit us.\n","date":1729641600,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":1729641600,"objectID":"5a5a71e8444dfa2557b703df82e80e8d","permalink":"https://dfki-nlp.github.io/authors/roland-roller/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/authors/roland-roller/","section":"authors","summary":"I am a senior researcher and project manager in the Speech and Language Technology group at the German Research Center for Artificial Intelligence (DFKI). I focus on natural language processing and machine learning topics with a high interest in medical use cases.","tags":["Researchers"],"title":"Roland Roller","type":"authors"},{"authors":["nils-feldhus"],"categories":null,"content":"","date":1727339583,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":1727339583,"objectID":"ad47029271d18471d52333582f6f09d3","permalink":"https://dfki-nlp.github.io/authors/nils-feldhus/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/authors/nils-feldhus/","section":"authors","summary":"","tags":["PhD Candidates"],"title":"Nils Feldhus","type":"authors"},{"authors":["aleksandra-gabryszak"],"categories":null,"content":"","date":1726907583,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":1726907583,"objectID":"bbfb2073a6e0589384f60ea2fae79732","permalink":"https://dfki-nlp.github.io/authors/aleksandra-gabryszak/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/authors/aleksandra-gabryszak/","section":"authors","summary":"","tags":["PhD Candidates"],"title":"Aleksandra Gabryszak","type":"authors"},{"authors":["arne-binder"],"categories":null,"content":"","date":1726907583,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":1726907583,"objectID":"db472dfe584699b02c3fb0b542f4efba","permalink":"https://dfki-nlp.github.io/authors/arne-binder/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/authors/arne-binder/","section":"authors","summary":"","tags":["PhD Candidates"],"title":"Arne Binder","type":"authors"},{"authors":["leonhard-hennig"],"categories":null,"content":"I\u0026rsquo;m a senior researcher and project manager at the DFKI Speech \u0026amp; Language Technology Lab. I\u0026rsquo;m interested in applying machine learning techniques to computational linguistics problems, such as information extraction and summarization, and making these work on real-world, domain-specific, noisy data in low-resource settings, where little or no language resources are readily available. As a project lead, I\u0026rsquo;ve managed various national research projects, such as Smart Data Web, PLASS, DAYSTREAM, and the DFKI part in the Berlin Big Data Center, as well as industry-funded projects, e.g. for Deutsche Telekom and Lenovo.\n","date":1726907583,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":1726907583,"objectID":"76acb2a1dbfa728042427546fca4cab6","permalink":"https://dfki-nlp.github.io/authors/leonhard-hennig/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/authors/leonhard-hennig/","section":"authors","summary":"I\u0026rsquo;m a senior researcher and project manager at the DFKI Speech \u0026amp; Language Technology Lab. I\u0026rsquo;m interested in applying machine learning techniques to computational linguistics problems, such as information extraction and summarization, and making these work on real-world, domain-specific, noisy data in low-resource settings, where little or no language resources are readily available.","tags":["Researchers"],"title":"Leonhard Hennig","type":"authors"},{"authors":["ajaymadhavan-ravichandran"],"categories":null,"content":"","date":1720742400,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":1720742400,"objectID":"a8ef52c2af75d3f4d651189c9b598c0d","permalink":"https://dfki-nlp.github.io/authors/ajaymadhavan-ravichandran/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/authors/ajaymadhavan-ravichandran/","section":"authors","summary":"","tags":["PhD Candidates"],"title":"Ajay Madhavan Ravichandran","type":"authors"},{"authors":["david-harbecke"],"categories":null,"content":"","date":1720742400,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":1720742400,"objectID":"57423a933a38af75c718059324719b6e","permalink":"https://dfki-nlp.github.io/authors/david-harbecke/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/authors/david-harbecke/","section":"authors","summary":"","tags":["PhD Candidates"],"title":"David Harbecke","type":"authors"},{"authors":["lisa-raithel"],"categories":null,"content":"Hey!\nI am Lisa, a post-doc at Technische Universität Berlin at the Quality and Usability Lab and BIFOLD and a guest researcher at DFKI GmbH, working closely with Philippe Thomas and Roland Roller.\nI obtained my master\u0026rsquo;s degree at Universität Potsdam in Computational Linguistics (B.Sc. in Computational Linguistics, M.Sc. in Cognitive Systems). I then briefly worked as a software engineer before transitioning back to academia for a double degree PhD program (cotutelle) at TU Berlin and Université Paris-Saclay. I was supervised by Prof. Sebastian Möller and Pierre Zweigenbaum, Directeur de Recherche CNRS. My doctoral research focused on cross-lingual information extraction for the detection of adverse drug reactions. During that time, I spent one year at LISN in Orsay, France (2021 - 2022) and three months at the Social Computing Lab at NAIST in Nara, Japan (2023). In February 2024, I successfully defended my thesis at TU Berlin.\n","date":1720742400,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":1720742400,"objectID":"2bf26a380b092795d4a187e33b919ab1","permalink":"https://dfki-nlp.github.io/authors/lisa-raithel/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/authors/lisa-raithel/","section":"authors","summary":"Hey!\nI am Lisa, a post-doc at Technische Universität Berlin at the Quality and Usability Lab and BIFOLD and a guest researcher at DFKI GmbH, working closely with Philippe Thomas and Roland Roller.","tags":["PhD Candidates"],"title":"Lisa Raithel","type":"authors"},{"authors":["malte-ostendorff"],"categories":null,"content":"In my research work, I mainly focus on information retrieval, recommender systems, and natural language processing. In particular, techniques for the information extraction from unstructured data such as text and making information more accessible are of great interest for me. In my recent work I apply these techniques on content from the legal domain, e.g. laws, patents, case files. Moreover, I explore how recommender systems can assist users in finding relevant information to cope with today’s information overload. Due to the large amounts of available data, all my work requires the use of scalable and distributed computing. Generally speaking, all topics that are somehow related to the following fields can be considered as my research interest:\n Recommender Systems Natural Language Processing Text Mining Applied Machine Learning Scalable Data Processing (\u0026ldquo;Big Data\u0026rdquo;) Legal Tech Language Models Feel free to contact me if you have any questions regarding my work. I am always open for new ideas, projects and collaborations with other researchers and students.\n","date":1720742400,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":1720742400,"objectID":"b87fe2d0e7981b54435e6d6bf9861b08","permalink":"https://dfki-nlp.github.io/authors/malte-ostendorff/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/authors/malte-ostendorff/","section":"authors","summary":"In my research work, I mainly focus on information retrieval, recommender systems, and natural language processing. In particular, techniques for the information extraction from unstructured data such as text and making information more accessible are of great interest for me.","tags":["Alumni"],"title":"Malte Ostendorff","type":"authors"},{"authors":["philippe-thomas"],"categories":null,"content":"","date":1715817600,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":1715817600,"objectID":"e87ec0a5f91d896556892e387ce48cc6","permalink":"https://dfki-nlp.github.io/authors/philippe-thomas/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/authors/philippe-thomas/","section":"authors","summary":"","tags":["Researchers"],"title":"Philippe Thomas","type":"authors"},{"authors":["yuxuan-chen"],"categories":null,"content":"","date":1715817600,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":1715817600,"objectID":"12c49279a0e7bdb8a10aea6a1c50529d","permalink":"https://dfki-nlp.github.io/authors/yuxuan-chen/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/authors/yuxuan-chen/","section":"authors","summary":"","tags":["PhD Candidates"],"title":"Yuxuan Chen","type":"authors"},{"authors":["eleftherios-avramidis"],"categories":null,"content":"","date":1689897600,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":1689897600,"objectID":"143f99ce31d6112e2c40d7cec4733d66","permalink":"https://dfki-nlp.github.io/authors/eleftherios-avramidis/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/authors/eleftherios-avramidis/","section":"authors","summary":"","tags":["Researchers"],"title":"Eleftherios Avramidis","type":"authors"},{"authors":["robert-schwarzenberg"],"categories":null,"content":"I conducted my PhD research at the Speech and Language Technologies Lab of the German Research Center for Artificial Intelligence (DFKI).\nMy interests include\n (Neural) Explainability Methods and Explainable Models, NLP and NLU, some Image Processing on the side, and Graph Algorithms because, you see, everything seems to be part of some graph. ","date":1686787200,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":1686787200,"objectID":"9c4340557d33d0eef87e7e24354df0fe","permalink":"https://dfki-nlp.github.io/authors/robert-schwarzenberg/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/authors/robert-schwarzenberg/","section":"authors","summary":"I conducted my PhD research at the Speech and Language Technologies Lab of the German Research Center for Artificial Intelligence (DFKI).\nMy interests include\n (Neural) Explainability Methods and Explainable Models, NLP and NLU, some Image Processing on the side, and Graph Algorithms because, you see, everything seems to be part of some graph.","tags":["Alumni"],"title":"Robert Schwarzenberg","type":"authors"},{"authors":["nils-rethmeier"],"categories":null,"content":"","date":1683534783,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":1683534783,"objectID":"5fa9f4aadfb6bfa3e5e6ad75c3611835","permalink":"https://dfki-nlp.github.io/authors/nils-rethmeier/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/authors/nils-rethmeier/","section":"authors","summary":"","tags":["PhD Candidates"],"title":"Nils Rethmeier","type":"authors"},{"authors":["steffen-castle"],"categories":null,"content":"","date":1680344191,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":1680344191,"objectID":"ad183e9940c0547e2260bca696fd06ca","permalink":"https://dfki-nlp.github.io/authors/steffen-castle/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/authors/steffen-castle/","section":"authors","summary":"","tags":["PhD Candidates"],"title":"Steffen Castle","type":"authors"},{"authors":["he-wang"],"categories":null,"content":"","date":1661173944,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":1661173944,"objectID":"6b5ebf54ed7db6b1d510ed1b4ccd6c8d","permalink":"https://dfki-nlp.github.io/authors/he-wang/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/authors/he-wang/","section":"authors","summary":"","tags":["Software Engineers"],"title":"He Wang","type":"authors"},{"authors":null,"categories":null,"content":"Christoph is now a postdoc in the Machine Learning group at Humboldt University of Berlin.\n","date":1653523200,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":1653523200,"objectID":"55f7fc0a0becc469231bd11edf9d90c1","permalink":"https://dfki-nlp.github.io/authors/christoph-alt/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/authors/christoph-alt/","section":"authors","summary":"Christoph is now a postdoc in the Machine Learning group at Humboldt University of Berlin.","tags":null,"title":"Christoph Alt","type":"authors"},{"authors":["karolina-zaczynska"],"categories":null,"content":"","date":1592870400,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":1592870400,"objectID":"e7523b67ec5174035b12fb4d48d29306","permalink":"https://dfki-nlp.github.io/authors/karolina-zaczynska/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/authors/karolina-zaczynska/","section":"authors","summary":"","tags":["PhD Candidates"],"title":"Karolina Zaczynska","type":"authors"},{"authors":null,"categories":null,"content":"DFKI-NLP is a Natural Language Processing group of researchers, software engineers and students at the Berlin office of the German Research Center for Artificial Intelligence (DFKI) working on basic and applied research in areas covering, among others, information extraction, knowledge base population, dialogue, sentiment analysis, and summarization. We are particularly interested in core research on learning in low-resource settings, reasoning over larger contexts, and continual learning. We strive for a deeper understanding of human language and thinking, with the goal of developing novel methods for processing and generating human language text, speech, and knowledge. An important part of our work is the creation of corpora, the evaluation of NLP datasets and tasks, and the explainability of (neural) models.\nKey topics:\n Applied / domain-specific information extraction Learning in low-resource settings and over large contexts Construction and analysis of IE datasets, linguistic annotation Multilingual information extraction Evaluation methodology research Explainability Our group forms a part of DFKI\u0026rsquo;s Speech and Language Technology department led by Prof. Sebastian Möller, and closely collaborates with e.g. the Technische Universität Berlin, DFKI\u0026rsquo;s Language Technology and Multilinguality department and DFKI\u0026rsquo;s Intelligent Analytics for Massive Data group.\n","date":-62135596800,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":-62135596800,"objectID":"28cf2f802ba8f342b4dc1a3a2cf18f61","permalink":"https://dfki-nlp.github.io/authors/dfki-nlp/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/authors/dfki-nlp/","section":"authors","summary":"DFKI-NLP is a Natural Language Processing group of researchers, software engineers and students at the Berlin office of the German Research Center for Artificial Intelligence (DFKI) working on basic and applied research in areas covering, among others, information extraction, knowledge base population, dialogue, sentiment analysis, and summarization.","tags":null,"title":"DFKI-NLP","type":"authors"},{"authors":["marc-huebner"],"categories":null,"content":"","date":-62135596800,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":-62135596800,"objectID":"bb1a2f736e1bfeb288ac79f61fa27578","permalink":"https://dfki-nlp.github.io/authors/marc-huebner/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/authors/marc-huebner/","section":"authors","summary":"","tags":["Alumni"],"title":"Marc Hübner","type":"authors"},{"authors":["Akhila Abdulnazar","Roland Roller","Stefan Schulz","Markus Kreuzthaler"],"categories":[],"content":"","date":1729641600,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1729641600,"objectID":"559b9ef1595bbbf3993e55b24309cc00","permalink":"https://dfki-nlp.github.io/publication/sage2024/","publishdate":"2024-10-23T00:00:00Z","relpermalink":"/publication/sage2024/","section":"publication","summary":"Clinical narratives provide comprehensive patient information. Achieving interoperability involves mapping relevant details to standardized medical vocabularies. Typically, natural language processing divides this task into named entity recognition (NER) and medical concept normalization (MCN). State-of-the-art results require supervised setups with abundant training data. However, the limited availability of annotated data due to sensitivity and time constraints poses challenges. This study addressed the need for unsupervised medical concept annotation (MCA) to overcome these limitations and support the creation of annotated datasets.","tags":[],"title":"Unsupervised SapBERT-based bi-encoders for medical concept annotation of clinical narratives with SNOMED CT","type":"publication"},{"authors":["Qianli Wang","Tatiana Anikina","Nils Feldhus","Simon Ostermann","Sebastian Möller"],"categories":[],"content":"","date":1727339583,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1727339583,"objectID":"99efa56c1e8a67379f387296b904a08b","permalink":"https://dfki-nlp.github.io/publication/wang-etal-2024-coxql/","publishdate":"2024-09-26T10:33:03+02:00","relpermalink":"/publication/wang-etal-2024-coxql/","section":"publication","summary":"Conversational explainable artificial intelligence (ConvXAI) systems based on large language models (LLMs) have garnered significant interest from the research community in natural language processing (NLP) and human-computer interaction (HCI). Such systems can provide answers to user questions about explanations in dialogues, have the potential to enhance users' comprehension and offer more information about the decision-making and generation processes of LLMs. Currently available ConvXAI systems are based on intent recognition rather than free chat, as this has been found to be more precise and reliable in identifying users' intentions. However, the recognition of intents still presents a challenge in the case of ConvXAI, since little training data exist and the domain is highly specific, as there is a broad range of XAI methods to map requests onto. In order to bridge this gap, we present CoXQL, the first dataset for user intent recognition in ConvXAI, covering 31 intents, seven of which require filling multiple slots. Subsequently, we enhance an existing parsing approach by incorporating template validations, and conduct an evaluation of several LLMs on CoXQL using different parsing strategies. We conclude that the improved parsing approach (MP+) surpasses the performance of previous approaches. We also discover that intents with multiple slots remain highly challenging for LLMs.","tags":[],"title":"CoXQL: A Dataset for Parsing Explanation Requests in Conversational XAI Systems ","type":"publication"},{"authors":["Aleksandra Gabryszak","Daniel Röder","Arne Binder","Luca Sion","Leonhard Hennig"],"categories":[],"content":"","date":1726907583,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1726907583,"objectID":"78ee9149556e62dd13d0d9fa63804b9b","permalink":"https://dfki-nlp.github.io/publication/gabryszak-etal-2024-enhancing/","publishdate":"2024-09-21T10:33:03+02:00","relpermalink":"/publication/gabryszak-etal-2024-enhancing/","section":"publication","summary":"In this paper, we investigate the use of large language models (LLMs) to enhance the editorial process of rewriting customer help pages. We introduce a German-language dataset comprising Frequently Asked Question-Answer pairs, presenting both raw drafts and their revisions by professional editors. On this dataset, we evaluate the performance of four large language models (LLM) through diverse prompts tailored for the rewriting task. We conduct automatic evaluations of content and text quality using ROUGE, BERTScore, and ChatGPT. Furthermore, we let professional editors assess the helpfulness of automatically generated FAQ revisions for editorial enhancement. Our findings indicate that LLMs can produce FAQ reformulations beneficial to the editorial process. We observe minimal performance discrepancies among LLMs for this task, and our survey on helpfulness underscores the subjective nature of editors' perspectives on editorial refinement.","tags":[],"title":"Enhancing Editorial Tasks: A Case Study on Rewriting Customer Help Page Contents Using Large Language Models","type":"publication"},{"authors":null,"categories":null,"content":"","date":1726617600,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1726617600,"objectID":"18dcd7d0f0a813f4fcfa369cee2f0f78","permalink":"https://dfki-nlp.github.io/dataset/faq-rewrites/","publishdate":"2024-09-18T00:00:00Z","relpermalink":"/dataset/faq-rewrites/","section":"dataset","summary":"We introduce a German-language dataset comprising Frequently Asked Question-Answer pairs: raw FAQ drafts, their revisions by professional editors and LLM generated revisions. The data was used to investigate the use of large language models (LLMs) to enhance the editorial process of rewriting customer help pages. The corpus comprises 56 question-answer pairs addressing potential customer inquiries across various topics. For each FAQ pair, a raw input is provided by specialized departments, and a rewritten gold output is crafted by a professional editor of Deutsche Telekom. The final dataset also includes LLM generated FAQ-pairs. Please see our [paper](https://aclanthology.org/2024.inlg-main.13/) accepted at INLG 20204, Tokyo, Japan. You can find the Github repo containing the dataset here [https://github.com/DFKI-NLP/faq-rewrites-llms](https://github.com/DFKI-NLP/faq-rewrites-llms).","tags":null,"title":"LLM-based FAQ Rewrites","type":"dataset"},{"authors":["Maximilian Bleick","Nils Feldhus","Aljoscha Burchardt","Sebastian Möller"],"categories":[],"content":"","date":1725784383,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1725784383,"objectID":"623b830ca923eb72a8af042acb36a763","permalink":"https://dfki-nlp.github.io/publication/bleick-etal-2024-german/","publishdate":"2024-09-08T10:33:03+02:00","relpermalink":"/publication/bleick-etal-2024-german/","section":"publication","summary":"We investigate the impact of LLMs on political discourse with a particular focus on the influence of generated personas on model responses. We find an echo chamber effect from LLM chatbots when provided with German-language biographical information of politicians and voters in German politics, leading to sycophantic responses and the reinforcement of existing political biases. Findings reveal that personas of certain political party, such as those of the 'Alternative für Deutschland' party, exert a stronger influence on LLMs, potentially amplifying extremist views. Unlike prior studies, we cannot corroborate a tendency for larger models to exert stronger sycophantic behaviour. We propose that further development should aim at reducing sycophantic behaviour in LLMs across all sizes and diversifying language capabilities in LLMs to enhance inclusivity.","tags":[],"title":"German Voter Personas can Radicalize LLM Chatbots via the Echo Chamber Effect","type":"publication"},{"authors":[],"categories":[],"content":"Two papers from DFKI NLP researchers have been accepted at the 17th International Natural Language Generation Conference (INLG 2024) that will take place September 23-27 in Tokyo, Japan. One paper presents a case study on using large language models to produce customer-friendly help page contents from more technical text, and includes a text quality evaluation by experienced editors. The other paper analyzes echo chamber effects in LLM-based chatbots in political conversations.\n Aleksandra Gabryszak, Daniel Röder, Arne Binder, Luca Sion, Leonhard Hennig (2024). Enhancing Editorial Tasks: A Case Study on Rewriting Customer Help Page Contents Using Large Language Models. INLG 2024. PDF Cite Dataset Project Maximilian Bleick, Nils Feldhus, Aljoscha Burchardt, Sebastian Möller (2024). German Voter Personas can Radicalize LLM Chatbots via the Echo Chamber Effect. INLG 2024. PDF Cite Code Dataset Project DOI ","date":1724311441,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1724311441,"objectID":"d1cb844ddbb79c8c8a9844aaabdad93a","permalink":"https://dfki-nlp.github.io/post/inlg2024/","publishdate":"2024-08-22T09:24:01+02:00","relpermalink":"/post/inlg2024/","section":"post","summary":"Two papers from DFKI NLP researchers have been accepted at the 17th International Natural Language Generation Conference (INLG 2024) that will take place September 23-27 in Tokyo, Japan. One paper presents a case study on using large language models to produce customer-friendly help page contents from more technical text, and includes a text quality evaluation by experienced editors.","tags":[],"title":"Two papers accepted to INLG 2024","type":"post"},{"authors":["Arne Binder","Tatiana Anikina","Leonhard Hennig","Simon Ostermann"],"categories":[],"content":"","date":172368e4,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":172368e4,"objectID":"a65105095b7518cd6949b90ddc3f0f13","permalink":"https://dfki-nlp.github.io/publication/binder-etal-2024-dfki/","publishdate":"2024-08-15T00:00:00Z","relpermalink":"/publication/binder-etal-2024-dfki/","section":"publication","summary":"This paper presents the dfki-mlst submission for the DialAM shared task (Ruiz-Dolz et al., 2024) on identification of argumentative and illocutionary relations in dialogue. Our model achieves best results in the global setting: 48.25 F1 at the focused level when looking only at the related arguments/locutions and 67.05 F1 at the general level when evaluating the complete argument maps. We describe our implementation of the data pre-processing, relation encoding and classification, evaluating 11 different base models and performing experiments with, e.g., node text combination and data augmentation. Our source code is publicly available.","tags":[],"title":"DFKI-MLST at DialAM-2024 Shared Task: System Description","type":"publication"},{"authors":["Leonhard Hennig"],"categories":[],"content":"","date":1722507391,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1722507391,"objectID":"24c3bb0906e4acd5db28286d2b543dc3","permalink":"https://dfki-nlp.github.io/project/trails/","publishdate":"2024-08-01T11:16:31+01:00","relpermalink":"/project/trails/","section":"project","summary":"Natural language processing (NLP) has demonstrated impressive performance in some human tasks. To achieve such performance, current neural models need to be pre-trained on huge amounts of raw text data. This dependence on uncurated data has at least four indirect and unintended consequences: 1) Uncurated data tends to be linguistically and culturally non-diverse due to the statistical dominance of major languages and dialects in online texts (English vs. North Frisian, US English vs. UK English, etc.). 2) Pre-trained neural models such as the ubiquitous pre-trained language models (PLM) reproduce the features present in the data, including human biases. 3) Rare phenomena (or languages) in the 'long tail' are often not sufficiently taken into account in model evaluation, leading to an underestimation of model performance, especially in real-world application scenarios. 4) The focus on achieving state-of-the-art results through the use of transfer learning with giant PLMs such as GPT4 or mT5 often underestimates alternative methods that are more accessible, efficient and sustainable.\nAs inclusion and trust are undermined by these problems, in TRAILS we focus on three main research directions to address such problems: (i) inclusion of underrepresented languages and cultures through multilingual and culturally sensitive NLP, (ii) robustness and fairness with respect to long-tail phenomena and classes and 'trustworthy content', and (iii) robust and efficient NLP models that enable training and deployment of models for (i) and (ii). We also partially address economic inequality by aiming for more efficient models (objective (iii)), which directly translates into a lower resource/cost footprint.","tags":["Bias","Evaluation","Large Language Models"],"title":"TRAILS - Trustworthy and Inclusive Machines","type":"project"},{"authors":[],"categories":[],"content":"DFKI will have a strong presence at the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024), one of the top venues of language technology, that will take place August 11-16 in Bangkok, Thailand. Researchers from Speech and Language Technology department will present 6 papers. The papers appeared in the main conference (Findings) as well as 3 co-located events: The 11th Workshop on Argument Mining, the BioNLP 2024 and Shared Tasks Workshop, and the Towards Knowledgeable Language Models Workshop.\nThe participation in the conference was supported by current research projects, as presentation of their recent results. Some of these projects were: TRAILS (BMBF), KEEPHA (BMBF), and XAINES (BMBF).\nThe DFKI papers presented at the conference are the following:\n Yuxuan Chen, Daniel Röder, Justus-Jonas Erker, Leonhard Hennig, Philippe Thomas, Sebastian Möller, Roland Roller (2024). Retrieval-Augmented Knowledge Integration into Language Models: A Survey. KnowledgeLM2024. PDF Cite Arne Binder, Tatiana Anikina, Leonhard Hennig, Simon Ostermann (2024). DFKI-MLST at DialAM-2024 Shared Task: System Description. ArgMining 2024. PDF Cite Code Project Dorothea MacPhail, David Harbecke, Lisa Raithel, Sebastian Möller (2024). Evaluating the Robustness of Adverse Drug Event Classification Models Using Templates. BioNLP 2024. PDF Cite Code Project Martin Courtois, Malte Ostendorff, Leonhard Hennig, Georg Rehm (2024). Symmetric Dot-Product Attention for Efficient Training of BERT Language Models. Findings 2024. PDF Cite Code Project Faraz Maschhur, Klaus Netter, Sven Schmeier, Katrin Ostermann, Rimantas Palunis, Tobias Strapatsas, Roland Roller (2024). Towards ML-supported Triage Prediction in Real-World Emergency Room Scenarios. BioNLP 2024. Cite Ajay Madhavan Ravichandran, Julianna Grune, Nils Feldhus, Aljoscha Burchardt, Sebastian Möller, Roland Roller (2024). XAI for Better Exploitation of Text in Medical Decision Support. BioNLP 2024. Cite ","date":1720769041,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1720769041,"objectID":"69b8354c5da150ae592c474736581fa2","permalink":"https://dfki-nlp.github.io/post/acl2024/","publishdate":"2024-07-12T09:24:01+02:00","relpermalink":"/post/acl2024/","section":"post","summary":"DFKI will have a strong presence at the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024), one of the top venues of language technology, that will take place August 11-16 in Bangkok, Thailand.","tags":[],"title":"Multiple papers by DFKI authors accepted to ACL 2024 and co-located events","type":"post"},{"authors":["Dorothea MacPhail","David Harbecke","Lisa Raithel","Sebastian Möller"],"categories":[],"content":"","date":1720742400,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1720742400,"objectID":"fdc2da6b0dba224e9bd38917cfc8ea71","permalink":"https://dfki-nlp.github.io/publication/acl2024-evaluating-macphail/","publishdate":"2024-07-12T00:00:00Z","relpermalink":"/publication/acl2024-evaluating-macphail/","section":"publication","summary":"An adverse drug effect (ADE) is any harmful event resulting from medical drug treatment. Despite their importance, ADEs are often under-reported in official channels. Some research has therefore turned to detecting discussions of ADEs in social media. Impressive results have been achieved in various attempts to detect ADEs. In a high-stakes domain such as medicine, however, an in-depth evaluation of a model's abilities is crucial. We address the issue of thorough performance evaluation in English-language ADE detection with hand-crafted templates for four capabilities: Temporal order, negation, sentiment, and beneficial effect. We find that models with similar performance on held-out test sets have varying results on these capabilities.","tags":[],"title":"Evaluating the Robustness of Adverse Drug Event Classification Models Using Templates","type":"publication"},{"authors":["Martin Courtois","Malte Ostendorff","Leonhard Hennig","Georg Rehm"],"categories":[],"content":"","date":1720742400,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1720742400,"objectID":"5a3a2f0b2356655d730f7f29362947ae","permalink":"https://dfki-nlp.github.io/publication/acl2024-symmetric-courtois/","publishdate":"2024-07-12T00:00:00Z","relpermalink":"/publication/acl2024-symmetric-courtois/","section":"publication","summary":"Initially introduced as a machine translation model, the Transformer architecture has now become the foundation for modern deep learning architecture, with applications in a wide range of fields, from computer vision to natural language processing. Nowadays, to tackle increasingly more complex tasks, Transformer-based models are stretched to enormous sizes, requiring increasingly larger training datasets, and unsustainable amount of compute resources. The ubiquitous nature of the Transformer and its core component, the attention mechanism, are thus prime targets for efficiency research. In this work, we propose an alternative compatibility function for the self-attention mechanism introduced by the Transformer architecture. This compatibility function exploits an overlap in the learned representation of the traditional scaled dot-product attention, leading to a symmetric with pairwise coefficient dot-product attention. When applied to the pre-training of BERT-like models, this new symmetric attention mechanism reaches a score of 79.36 on the GLUE benchmark against 78.74 for the traditional implementation, leads to a reduction of 6% in the number of trainable parameters, and reduces the number of training steps required before convergence by half.","tags":[],"title":"Symmetric Dot-Product Attention for Efficient Training of BERT Language Models","type":"publication"},{"authors":["Faraz Maschhur","Klaus Netter","Sven Schmeier","Katrin Ostermann","Rimantas Palunis","Tobias Strapatsas","Roland Roller"],"categories":[],"content":"","date":1720742400,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1720742400,"objectID":"a8bbc17aa0331a8e581bd199f28c7539","permalink":"https://dfki-nlp.github.io/publication/acl2024-towards-maschhur/","publishdate":"2024-07-12T00:00:00Z","relpermalink":"/publication/acl2024-towards-maschhur/","section":"publication","summary":"","tags":[],"title":"Towards ML-supported Triage Prediction in Real-World Emergency Room Scenarios","type":"publication"},{"authors":["Ajay Madhavan Ravichandran","Julianna Grune","Nils Feldhus","Aljoscha Burchardt","Sebastian Moller","Roland Roller"],"categories":[],"content":"","date":1720742400,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1720742400,"objectID":"03090be3e7ed9f3e1284cb6e6630e817","permalink":"https://dfki-nlp.github.io/publication/acl2024-xai-ravichandran/","publishdate":"2024-07-12T00:00:00Z","relpermalink":"/publication/acl2024-xai-ravichandran/","section":"publication","summary":"","tags":[],"title":"XAI for Better Exploitation of Text in Medical Decision Support","type":"publication"},{"authors":[],"categories":[],"content":"One paper by researchers from the Speech and Language Technology department of DFKI will be presented at the 18th International Workshop on Semantic Evaluation, co-located with NAACL 2024:\n Bhuvanesh Verma, Lisa Raithel (2024). DFKI-NLP at SemEval-2024 Task 2: Towards Robust LLMs Using Data Perturbations and MinMax Training. SemEval 2024. PDF Cite ","date":1718177041,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1718177041,"objectID":"467a68aea8a6f4e685863007687839c3","permalink":"https://dfki-nlp.github.io/post/semeval2024/","publishdate":"2024-06-12T09:24:01+02:00","relpermalink":"/post/semeval2024/","section":"post","summary":"One paper by researchers from the Speech and Language Technology department of DFKI will be presented at the 18th International Workshop on Semantic Evaluation, co-located with NAACL 2024:\n Bhuvanesh Verma, Lisa Raithel (2024).","tags":[],"title":"One paper by DFKI authors accepted to SemEval-2024","type":"post"},{"authors":["Bhuvanesh Verma","Lisa Raithel"],"categories":[],"content":"","date":1717977600,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1717977600,"objectID":"f96c13727064b9a6a645174943d09bed","permalink":"https://dfki-nlp.github.io/publication/semeval2024-verma/","publishdate":"2024-06-10T00:00:00Z","relpermalink":"/publication/semeval2024-verma/","section":"publication","summary":"The NLI4CT task at SemEval-2024 emphasizes the development of robust models for Natural Language Inference on Clinical Trial Reports (CTRs) using large language models (LLMs). This edition introduces interventions specifically targeting the numerical, vocabulary, and semantic aspects of CTRs. Our proposed system harnesses the capabilities of the state-of-the-art Mistral model (Jiang et al., 2023), complemented by an auxiliary model, to focus on the intricate input space of the NLI4CT dataset. Through the incorporation of numerical and acronym-based perturbations to the data, we train a robust system capable of handling both semantic-altering and numerical contradiction interventions. Our analysis on the dataset sheds light on the challenging sections of the CTRs for reasoning.","tags":[],"title":"DFKI-NLP at SemEval-2024 Task 2: Towards Robust LLMs Using Data Perturbations and MinMax Training","type":"publication"},{"authors":["Qianli Wang","Tatiana Anikina","Nils Feldhus","Josef van Genabith","Leonhard Hennig","Sebastian Möller"],"categories":[],"content":"","date":1716193983,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1716193983,"objectID":"99bceffd936427b8b8e8bcf5f83d367c","permalink":"https://dfki-nlp.github.io/publication/hcinlp24-wang-llmcheckup/","publishdate":"2024-05-20T10:33:03+02:00","relpermalink":"/publication/hcinlp24-wang-llmcheckup/","section":"publication","summary":"Interpretability tools that offer explanations in the form of a dialogue have demonstrated their efficacy in enhancing users' understanding (Slack et al., 2023; Shen et al., 2023), as one-off explanations may fall short in providing sufficient information to the user. Current solutions for dialogue-based explanations, however, often require external tools and modules and are not easily transferable to tasks they were not designed for. With LLMCheckup, we present an easily accessible tool that allows users to chat with any state-of-the-art large language model (LLM) about its behavior. We enable LLMs to generate explanations and perform user intent recognition without fine-tuning, by connecting them with a broad spectrum of Explainable AI (XAI) methods, including white-box explainability tools such as feature attributions, and self-explanations (e.g., for rationale generation). LLM-based (self-)explanations are presented as an interactive dialogue that supports follow-up questions and generates suggestions. LLMCheckup provides tutorials for operations available in the system, catering to individuals with varying levels of expertise in XAI and supporting multiple input modalities. We introduce a new parsing strategy that substantially enhances the user intent recognition accuracy of the LLM. Finally, we showcase LLMCheckup for the tasks of fact checking and commonsense question answering.","tags":[],"title":"LLMCheckup: Conversational Examination of Large Language Models via Interpretability Tools and Self-Explanations","type":"publication"},{"authors":["Yuxuan Chen","Daniel Röder","Justus-Jonas Erker","Leonhard Hennig","Philippe Thomas","Sebastian Möller","Roland Roller"],"categories":[],"content":"","date":1715817600,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1715817600,"objectID":"3d4e499fd5ccd1a4522be502c4b78abd","permalink":"https://dfki-nlp.github.io/publication/acl2024-raki-chen/","publishdate":"2024-05-16T00:00:00Z","relpermalink":"/publication/acl2024-raki-chen/","section":"publication","summary":"","tags":[],"title":"Retrieval-Augmented Knowledge Integration into Language Models: A Survey","type":"publication"},{"authors":["Tomohiro Nishiyama","Lisa Raithel","Roland Roller","Pierre Zweigenbaum","Eiji Aramaki"],"categories":[],"content":"","date":1710923583,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1710923583,"objectID":"2908ce950332e2c1a90ff5f3c8245000","permalink":"https://dfki-nlp.github.io/publication/cald-eacl24-nishiyama-assessing/","publishdate":"2024-03-23T10:33:03+02:00","relpermalink":"/publication/cald-eacl24-nishiyama-assessing/","section":"publication","summary":"Since medical text cannot be shared easily due to privacy concerns, synthetic data bears much potential for natural language processing applications. In the context of social media and user-generated messages about drug intake and adverse drug effects, this work presents different methods to examine the authenticity of synthetic text. We conclude that the generated tweets are untraceable and show enough authenticity from the medical point of view to be used as a replacement for a real Twitter corpus. However, original data might still be the preferred choice as they contain much more diversity.","tags":[],"title":"Assessing Authenticity and Anonymity of Synthetic User-generated Content in the Medical Domain","type":"publication"},{"authors":["Nils Feldhus","Qianli Wang","Tatiana Anikina","Sahil Chopra","Cennet Oguz","Sebastian Möller"],"categories":[],"content":"","date":1701820800,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1701820800,"objectID":"ade58ad3c98ec74e683c702490dd0759","permalink":"https://dfki-nlp.github.io/publication/emnlp2023-feldhus-interrolang/","publishdate":"2023-10-15T00:00:00Z","relpermalink":"/publication/emnlp2023-feldhus-interrolang/","section":"publication","summary":"While recently developed NLP explainability methods let us open the black box in various ways (Madsen et al., 2022), a missing ingredient in this endeavor is an interactive tool offering a conversational interface. Such a dialogue system can help users explore datasets and models with explanations in a contextualized manner, e.g. via clarification or follow-up questions, and through a natural language interface. We adapt the conversational explanation framework TalkToModel (Slack et al., 2022) to the NLP domain, add new NLP-specific operations such as free-text rationalization, and illustrate its generalizability on three NLP tasks (dialogue act classification, question answering, hate speech detection). To recognize user queries for explanations, we evaluate fine-tuned and few-shot prompting models and implement a novel adapter-based approach. We then conduct two user studies on (1) the perceived correctness and helpfulness of the dialogues, and (2) the simulatability, i.e. how objectively helpful dialogical explanations are for humans in figuring out the model's predicted label when it's not shown. We found rationalization and feature attribution were helpful in explaining the model behavior. Moreover, users could more reliably predict the model outcome based on an explanation dialogue rather than one-off explanations.","tags":["Explainability"],"title":"InterroLang: Exploring NLP Models and Datasets through Dialogue-based Explanations","type":"publication"},{"authors":[],"categories":[],"content":"DFKI had a strong presence at the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP 2023), one of the top venues of language technology, that took place on December 6-10 in Singapore. The research center was represented by the departments MLT, SLT, and DRΧ with 11 papers, presented by 10 researchers. The papers appeared in the main conference as well as 3 co-located events: “Conference for Machine Translation” (WMT), “Workshop on analyzing and interpreting neural networks for NLP” (BlackboxNLP), and “Workshop on Computational Models of Reference, Anaphora and Coreference” (CRAC).\nAdditionally, DFKI researchers contributed to the organization of the workshops by participating in the organization committees in 3 shared tasks and particularly the ones on Sign Language Translation, Generic Machine Translation and Machine Translation Metrics. Noteworthy is also the participation of the DFKI researchers in the program committees, where one researcher was an Area Chair in the Semantics track of the main conference, and numerous others contributed with peer-reviewing of submitted papers. The participation in the conference was supported by current research projects, as presentation of their recent results. Some of these projects were: CORA4NLP (BMBF), IMPRESS (INRIA-DFKI), SFB 1102 “Information Density and Linguistic Encoding” (DFG), SocialWear (BMBF), TextQ (DFG) and XAINES (BMBF).\nThe DFKI papers presented at the conference are the following:\n Challenging the State-of-the-art Machine Translation Metrics from a Linguistic Perspective Find-2-Find: Multitask Learning for Anaphora Resolution and Object Localization Findings of the 2023 Conference on Machine Translation (WMT23): LLMs Are Here but Not Quite There Yet Findings of the Second WMT Shared Task on Sign Language Translation (WMT-SLT23) InterroLang: Exploring NLP Models and Datasets through Dialogue-based Explanations Investigating the Encoding of Words in BERT\u0026rsquo;s Neurons using Feature Textualization Linguistically Motivated Evaluation of the 2023 State-of-the-art Machine Translation: Can ChatGPT Outperform NMT? Multilingual Coarse Political Stance Classification of Media. The Editorial Line of a ChatGPT and Bard Newspaper Multilingual coreference resolution: Adapt and Generate Results of WMT23 Metrics Shared Task: Metrics Might Be Guilty but References Are Not Innocent Translating away Translationese without Parallel Data Where exactly does contextualization in a PLM happen? ","date":1697268241,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1697268241,"objectID":"d545350b6724f77c8c18f21a46167a22","permalink":"https://dfki-nlp.github.io/post/emnlp2023/","publishdate":"2023-10-14T09:24:01+02:00","relpermalink":"/post/emnlp2023/","section":"post","summary":"DFKI had a strong presence at the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP 2023), one of the top venues of language technology, that took place on December 6-10 in Singapore.","tags":[],"title":"Multiple papers by DFKI authors accepted to EMNLP 2023 and co-located events","type":"post"},{"authors":[],"categories":[],"content":"One paper from DFKI-NLP researchers has been accepted for publication at KONVENS 2023, the 19th German Conference on Natural Language Processing. The conference will take place in Ingolstadt, Germany, from Sep 18th to Sep 22nd, 2023. The paper presents an approach using machine translation to translate English data to German to train a transformer-based factuality detection model for clinical data, where supervised data is usually very scarce due to its sensitive nature and privacy concerns.\n Mohammed Bin Sumait, Aleksandra Gabryszak, Leonhard Hennig, Roland Roller (2023). Factuality Detection using Machine Translation - a Use Case for German Clinical Text. KONVENS 2023. Cite Project ","date":1692257041,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1692230400,"objectID":"5975d667dbf9d890f02cad51fa67c6ac","permalink":"https://dfki-nlp.github.io/post/konvens2023/","publishdate":"2023-08-17T09:24:01+02:00","relpermalink":"/post/konvens2023/","section":"post","summary":"One paper from DFKI-NLP researchers has been accepted for publication at KONVENS 2023, the 19th German Conference on Natural Language Processing. The conference will take place in Ingolstadt, Germany, from Sep 18th to Sep 22nd, 2023.","tags":[],"title":"1 paper to be presented at KONVENS 2023","type":"post"},{"authors":["Mohammed Bin Sumait","Aleksandra Gabryszak","Leonhard Hennig","Roland Roller"],"categories":[],"content":"","date":1692253983,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1692253983,"objectID":"e5b8f54c9a6e59a63b21d49103b50302","permalink":"https://dfki-nlp.github.io/publication/konvens2023-binsumait-etal-factuality/","publishdate":"2023-08-17T08:33:03+02:00","relpermalink":"/publication/konvens2023-binsumait-etal-factuality/","section":"publication","summary":"Factuality can play an important role when automatically processing clinical text, as it makes a difference if particular symptoms are explicitly not present, possibly present, not mentioned, or affirmed. In most cases, a sufficient number of examples is necessary to handle such phenomena in a supervised machine learning setting. However, as clinical text might contain sensitive information, data cannot be easily shared. In the context of factuality detection, this work presents a simple solution using machine translation to translate English data to German to train a transformer-based factuality detection model.","tags":[],"title":"Factuality Detection using Machine Translation - a Use Case for German Clinical Text","type":"publication"},{"authors":["Vincent Vandeghinste","Mirella De Sisto","Maria Kopf","Davy Van Landuyt Picron","Irene Murtagh","Eleftherios Avramidis","Mathieu De Coster"],"categories":[],"content":"","date":1689897600,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1689897600,"objectID":"e18a0f6dfe5bba3b5c9c9da0cb96bf7d","permalink":"https://dfki-nlp.github.io/publication/vandeghinste-2023-ele-sign/","publishdate":"2023-07-21T00:00:00Z","relpermalink":"/publication/vandeghinste-2023-ele-sign/","section":"publication","summary":"This report on Europe's Sign Languages is part of a series of language deliverables developed within the framework of the European Language Equality (ELE) project. The series seeks to not only delineate the current state of affairs for each European language, but to additionally identify the gaps and factors that hinder further development in research and technology. The survey presented here focuses on the condition of Language Technology (LT) with regard to Europe's Sign Languages, a set of languages often forgotten in the context of European Language Equality. With the rise of the deep learning paradigm in artificial intelligence, sign language technologies become technologically feasible, provided that enough data is available to feed this data-hungry paradigm. It is exactly the quality and quantity of data that is the main bottleneck in development of well performing and useful technologies. In the past, there have been several projects aimed at developing sign language technologies and methodologies that have been deemed of little value by the deaf communities. Co-creation and involvement of deaf communities throughout projects and development of technologies ensures that this does not happen again.","tags":["Machine Translation"],"title":"European Language Equality, Report on Europe's Sign Languages","type":"publication"},{"authors":["Gabriele Sarti","Nils Feldhus","Ludwig Sickert","Oskar van der Wal","Malvina Nissim","Arianna Bisazza"],"categories":[],"content":"","date":1686787200,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1686787200,"objectID":"4720cee2cc9dd56df3c79f707f31c6fd","permalink":"https://dfki-nlp.github.io/publication/acl2023-sarti-inseq/","publishdate":"2023-06-15T00:00:00Z","relpermalink":"/publication/acl2023-sarti-inseq/","section":"publication","summary":"Past work in natural language processing interpretability focused mainly on popular classification tasks while largely overlooking generation settings, partly due to a lack of dedicated tools. In this work, we introduce Inseq, a Python library to democratize access to interpretability analyses of sequence generation models. Inseq enables intuitive and optimized extraction of models' internal information and feature importance scores for popular decoder-only and encoder-decoder Transformers architectures. We showcase its potential by adopting it to highlight gender biases in machine translation models and locate factual knowledge inside GPT-2. Thanks to its extensible interface supporting cutting-edge techniques such as contrastive feature attribution, Inseq can drive future advances in explainable natural language generation, centralizing good practices and enabling fair and reproducible model evaluations.","tags":["Interpretability"],"title":"Inseq: An Interpretability Toolkit for Sequence Generation Models","type":"publication"},{"authors":["Dele Zhu","Vera Czehmann","Eleftherios Avramidis"],"categories":[],"content":"","date":1686787200,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1686787200,"objectID":"0f2680c7f042a0139cfa4dc09fcbe375","permalink":"https://dfki-nlp.github.io/publication/acl2023-zhu-sign/","publishdate":"2023-06-15T00:00:00Z","relpermalink":"/publication/acl2023-zhu-sign/","section":"publication","summary":"State-of-the-art techniques common to low resource Machine Translation (MT) are applied to improve MT of spoken language text to Sign Language (SL) glosses. In our experiments, we improve the performance of the transformer-based models via (1) data augmentation, (2) semi-supervised Neural Machine Translation (NMT), (3) transfer learning and (4) multilingual NMT. The proposed methods are implemented progressively on two German SL corpora containing gloss annotations. Multilingual NMT combined with data augmentation appear to be the most successful setting, yielding statistically significant improvements as measured by three automatic metrics (up to over 6 points BLEU), and confirmed via human evaluation. Our best setting outperforms all previous work that report on the same test-set and is also confirmed on a corpus of the American Sign Language (ASL).","tags":["Machine Translation"],"title":"Neural Machine Translation Methods for Translating Text to Sign Language Glosses","type":"publication"},{"authors":["Nils Feldhus","Leonhard Hennig","Maximilian Dustin Nasert","Christopher Ebert","Robert Schwarzenberg","Sebastian Möller"],"categories":[],"content":"","date":1686787200,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1686787200,"objectID":"86eedcf694f6a4c2143fe8c11145b0c7","permalink":"https://dfki-nlp.github.io/publication/acl2023-feldhus-smv/","publishdate":"2023-06-15T00:00:00Z","relpermalink":"/publication/acl2023-feldhus-smv/","section":"publication","summary":"Saliency maps can explain a neural model's predictions by identifying important input features. They are difficult to interpret for laypeople, especially for instances with many features. In order to make them more accessible, we formalize the underexplored task of translating saliency maps into natural language and compare methods that address two key challenges of this approach -- what and how to verbalize. In both automatic and human evaluation setups, using token-level attributions from text classification tasks, we compare two novel methods (search-based and instruction-based verbalizations) against conventional feature importance representations (heatmap visualizations and extractive rationales), measuring simulatability, faithfulness, helpfulness and ease of understanding. Instructing GPT-3.5 to generate saliency map verbalizations yields plausible explanations which include associations, abstractive summarization and commonsense reasoning, achieving by far the highest human ratings, but they are not faithfully capturing numeric information and are inconsistent in their interpretation of the task. In comparison, our search-based, model-free verbalization approach efficiently completes templated verbalizations, is faithful by design, but falls short in helpfulness and simulatability. Our results suggest that saliency map verbalization makes feature attribution explanations more comprehensible and less cognitively challenging to humans than conventional representations.","tags":["Interpretability"],"title":"Saliency Map Verbalization: Comparing Feature Importance Representations from Model-free and Instruction-based Methods","type":"publication"},{"authors":null,"categories":null,"content":"","date":1684886400,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1684886400,"objectID":"cabf17b70c41b00d24ce53389c9df8a5","permalink":"https://dfki-nlp.github.io/dataset/multitacred/","publishdate":"2023-05-24T00:00:00Z","relpermalink":"/dataset/multitacred/","section":"dataset","summary":"MultiTACRED is a multilingual version of the large-scale [TAC Relation Extraction Dataset](https://nlp.stanford.edu/projects/tacred). It covers 12 typologically diverse languages from 9 language families, and was created by machine-translating the instances of the original TACRED dataset and automatically projecting their entity annotations. For details of the original TACRED's data collection and annotation process, see the [Stanford paper](https://aclanthology.org/D17-1004/). Translations are syntactically validated by checking the correctness of the XML tag markup. Any translations with an invalid tag structure, e.g. missing or invalid head or tail tag pairs, are discarded (on average, 2.3% of the instances).\nLanguages covered are: Arabic, Chinese, Finnish, French, German, Hindi, Hungarian, Japanese, Polish, Russian, Spanish, Turkish. Intended use is supervised relation classification. Audience - researchers.\nThe dataset will be released via the LDC (link will follow).\nPlease see [our ACL paper](https://arxiv.org/abs/2305.04582) for full details. You can find the Github repo containing the translation and experiment code here [https://github.com/DFKI-NLP/MultiTACRED](https://github.com/DFKI-NLP/MultiTACRED).","tags":["Relation Extraction","Multilinguality","Transfer Learning"],"title":"The MultiTACRED dataset","type":"dataset"},{"authors":["Malte Ostendorff","Georg Rehm"],"categories":[],"content":"","date":1684222766,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1684222766,"objectID":"1a46f585e2b7b7c0c0685a90a91c5110","permalink":"https://dfki-nlp.github.io/publication/pml4dc-2023-ostendorff/","publishdate":"2023-05-16T09:39:26+02:00","relpermalink":"/publication/pml4dc-2023-ostendorff/","section":"publication","summary":"Most Transformer language models are primarily pretrained on English text, limiting their use for other languages. As the model sizes grow, the performance gap between English and other languages with fewer compute and data resources increases even further. Consequently, more resource-efficient training methods are needed to bridge the gap for languages with fewer resources available. To address this problem, we introduce a cross-lingual and progressive transfer learning approach, called CLP-Transfer, that transfers models from a source language, for which pretrained models are publicly available, like English, to a new target language. As opposed to prior work, which focused on the cross-lingual transfer between two languages, we extend the transfer to the model size. Given a pretrained model in a source language, we aim for a same-sized model in a target language. Instead of training a model from scratch, we exploit a smaller model that is in the target language but requires much fewer resources. Both small and source models are then used to initialize the token embeddings of the larger model based on the overlapping vocabulary of the source and target language. All remaining weights are reused from the model in the source language. This approach outperforms the sole cross-lingual transfer and can save up to 80% of the training steps compared to the random initialization. ","tags":[],"title":"Efficient Language Model Training through Cross-Lingual and Progressive Transfer Learning","type":"publication"},{"authors":["Leonhard Hennig","Philippe Thomas","Sebastian Möller"],"categories":[],"content":"","date":1683534783,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1683534783,"objectID":"b18b701ca39dc8fc113c349cb912ec16","permalink":"https://dfki-nlp.github.io/publication/acl2023-hennig-multitacred/","publishdate":"2023-05-08T10:33:03+02:00","relpermalink":"/publication/acl2023-hennig-multitacred/","section":"publication","summary":"Relation extraction (RE) is a fundamental task in information extraction, whose extension to multilingual settings has been hindered by the lack of supervised resources comparable in size to large English datasets such as TACRED (Zhang et al., 2017). To address this gap, we introduce the MultiTACRED dataset, covering 12 typologically diverse languages from 9 language families, which is created by machine-translating TACRED instances and automatically projecting their entity annotations. We analyze translation and annotation projection quality, identify error categories, and experimentally evaluate fine-tuned pretrained mono- and multilingual language models in common transfer learning scenarios. Our analyses show that machine translation is a viable strategy to transfer RE instances, with native speakers judging more than 83% of the translated instances to be linguistically and semantically acceptable. We find monolingual RE model performance to be comparable to the English original for many of the target languages, and that multilingual models trained on a combination of English and target language data can outperform their monolingual counterparts. However, we also observe a variety of translation and annotation projection errors, both due to the MT systems and linguistic features of the target languages, such as pronoun-dropping, compounding and inflection, that degrade dataset quality and RE model performance.","tags":[],"title":"MultiTACRED: A Multilingual Version of the TAC Relation Extraction Dataset","type":"publication"},{"authors":["Akhila Abdulnazar","Markus Kreuzthaler","Roland Roller","Stefan Schulz"],"categories":[],"content":"","date":1683534783,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1683534783,"objectID":"50063dc2283e29f4dcc33d54d6467958","permalink":"https://dfki-nlp.github.io/publication/shti2023-abdulnazar-sapbert/","publishdate":"2023-05-08T10:33:03+02:00","relpermalink":"/publication/shti2023-abdulnazar-sapbert/","section":"publication","summary":"Word vector representations, known as embeddings, are commonly used for natural language processing. Particularly, contextualized representations have been very successful recently. In this work, we analyze the impact of contextualized and non-contextualized embeddings for medical concept normalization, mapping clinical terms via a k-NN approach to SNOMED CT. The non-contextualized concept mapping resulted in a much better performance (F1-score = 0.853) than the contextualized representation (F1-score = 0.322).","tags":[],"title":"SapBERT-Based Medical Concept Normalization Using SNOMED CT","type":"publication"},{"authors":["Vageesh Saxena","Nils Rethmeier","Gijs Van Dijck","Gerasimos Spanakis"],"categories":[],"content":"","date":1683534783,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1683534783,"objectID":"162357eac1664c564ac3e6a725d23a9d","permalink":"https://dfki-nlp.github.io/publication/acl2023-rethmeier-vendorlink/","publishdate":"2023-05-08T10:33:03+02:00","relpermalink":"/publication/acl2023-rethmeier-vendorlink/","section":"publication","summary":"The anonymity on the Darknet allows vendors to stay undetected by using multiple vendor aliases or frequently migrating between markets. Consequently, illegal markets and their connections are challenging to uncover on the Darknet. To identify relationships between illegal markets and their vendors, we propose VendorLink, an NLP-based approach that examines writing patterns to verify, identify, and link unique vendor accounts across text advertisements (ads) on seven public Darknet markets. In contrast to existing literature, VendorLink utilizes the strength of supervised pretraining to perform closed-set vendor verification, open-set vendor identification, and low-resource market adaption tasks. Through VendorLink, we uncover (i) 15 migrants and 71 potential aliases in the Alphabay-Dreams-Silk dataset, (ii) 17 migrants and 3 potential aliases in the Valhalla-Berlusconi dataset, and (iii) 75 migrants and 10 potential aliases in the Traderoute-Agora dataset. Altogether, our approach can help Law Enforcement Agencies (LEA) make more informed decisions by verifying and identifying migrating vendors and their potential aliases on existing and Low-Resource (LR) emerging Darknet markets.","tags":[],"title":"VendorLink: An NLP approach for Identifying \u0026 Linking Vendor Migrants \u0026 Potential Aliases on Darknet Markets","type":"publication"},{"authors":[],"categories":[],"content":"Five papers from DFKI-NLP researchers have been accepted for publication at ACL 2023, the 61st Annual Meeting of the Association for Computational Linguistics. The conference is planned to be a hybrid meeting and will take place in Toronto, Canada, from Jul 9th through July 14th, 2023. The first one presents a multilingual version of the TAC relation extraction dataset that covers 12 additional languages beside the original English, the second examines writing patterns to verify, identify, and link unique vendor accounts across text advertisements on public Darknet markets, enabling Law Enforcement Agencies to make more informed decisions. The third is about neural machine translation methods for translating text to sign language glosses, and has received an outstanding paper award. The fourth is an interpretability toolkit for sequence generation models, while the fifth is a comparative study of feature attribution representations, including model-free and instruction-based methods for saliency map verbalization.\n Leonhard Hennig, Philippe Thomas, Sebastian Möller (2023). MultiTACRED: A Multilingual Version of the TAC Relation Extraction Dataset. ACL 2023. PDF Cite Code Dataset Project Project Project DOI Vageesh Saxena, Nils Rethmeier, Gijs Van Dijck, Gerasimos Spanakis (2023). VendorLink: An NLP approach for Identifying \u0026amp; Linking Vendor Migrants \u0026amp; Potential Aliases on Darknet Markets. ACL 2023. PDF Cite Code Project Dele Zhu, Vera Czehmann, Eleftherios Avramidis (2023). Neural Machine Translation Methods for Translating Text to Sign Language Glosses. ACL 2023. PDF Cite Gabriele Sarti, Nils Feldhus, Ludwig Sickert, Oskar van der Wal, Malvina Nissim, Arianna Bisazza (2023). Inseq: An Interpretability Toolkit for Sequence Generation Models. ACL 2023 Demos. PDF Cite Code Nils Feldhus, Leonhard Hennig, Maximilian Dustin Nasert, Christopher Ebert, Robert Schwarzenberg, Sebastian Möller (2023). Saliency Map Verbalization: Comparing Feature Importance Representations from Model-free and Instruction-based Methods. ACL 2023 NLRSE. PDF Cite Code ","date":1683530641,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1686787200,"objectID":"221570ef38422c5369b9e10cac109395","permalink":"https://dfki-nlp.github.io/post/acl2023/","publishdate":"2023-05-08T09:24:01+02:00","relpermalink":"/post/acl2023/","section":"post","summary":"Five papers from DFKI-NLP researchers have been accepted for publication at ACL 2023, the 61st Annual Meeting of the Association for Computational Linguistics. The conference is planned to be a hybrid meeting and will take place in Toronto, Canada, from Jul 9th through July 14th, 2023.","tags":[],"title":"5 papers to be presented at ACL 2023","type":"post"},{"authors":[" David Samhammer","Susanne Beck","Klemens Budde","Aljoscha Burchardt","Michelle Faber","Simon Gerndt","Sebastian Möller","Bilgin Osmanodja","Roland Roller","Peter Dabrock"],"categories":[],"content":"","date":1683016383,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1683016383,"objectID":"2d238f58e9f415a79574aaddca268d79","permalink":"https://dfki-nlp.github.io/publication/samhammer-2023-klinische/","publishdate":"2023-05-26T10:33:03+02:00","relpermalink":"/publication/samhammer-2023-klinische/","section":"publication","summary":"Dieses Open-Access-essential schafft Orientierung, wenn Künstliche Intelligenz im klinischen Alltag eingesetzt wird. Die Herausforderungen werden anhand zweier Beispiele aus dem Bereich der Nephrologie erläutert, die ethisch und rechtlich reflektiert werden. Ein umfangreicher Empfehlungsteil schließt diesen durchweg interdisziplinär erarbeiteten Band ab.","tags":[],"title":"Klinische Entscheidungsfindung mit Künstlicher Intelligenz: Ein interdisziplinärer Governance-Ansatz","type":"publication"},{"authors":["Roland Roller","Aljoscha Burchardt","David Samhammer","Simon Ronicke","Wiebke Duettmann","Sven Schmeier","Sebastian Möller","Peter Dabrock","Klemens Budde","Manuel Mayrdorfer","Bilgin Osmanodja"],"categories":[],"content":"","date":1682238783,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1682238783,"objectID":"fc18ffb4cb00dcf2c1f993205b2ae152","permalink":"https://dfki-nlp.github.io/publication/plosone2023-roller-performance/","publishdate":"2023-04-23T10:33:03+02:00","relpermalink":"/publication/plosone2023-roller-performance/","section":"publication","summary":"Scientific publications about the application of machine learning models in healthcare often focus on improving performance metrics. However, beyond often short-lived improvements, many additional aspects need to be taken into consideration to make sustainable progress. What does it take to implement a clinical decision support system, what makes it usable for the domain experts, and what brings it eventually into practical usage? So far, there has been little research to answer these questions. This work presents a multidisciplinary view of machine learning in medical decision support systems and covers information technology, medical, as well as ethical aspects. The target audience is computer scientists, who plan to do research in a clinical context. The paper starts from a relatively straightforward risk prediction system in the subspecialty nephrology that was evaluated on historic patient data both intrinsically and based on a reader study with medical doctors. Although the results were quite promising, the focus of this article is not on the model itself or potential performance improvements. Instead, we want to let other researchers participate in the lessons we have learned and the insights we have gained when implementing and evaluating our system in a clinical setting within a highly interdisciplinary pilot project in the cooperation of computer scientists, medical doctors, ethicists, and legal experts.","tags":[],"title":"When performance is not enough—A multidisciplinary view on clinical decision support","type":"publication"},{"authors":["Steffen Castle","Leonhard Hennig"],"categories":[],"content":"","date":1680344191,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1680344191,"objectID":"55ec654e3395f87ccf3f3d9f6afb8417","permalink":"https://dfki-nlp.github.io/project/data4transparency/","publishdate":"2023-04-01T11:16:31+01:00","relpermalink":"/project/data4transparency/","section":"project","summary":"According to the World Bank and the UN, some US$1tn is paid in bribes every year. Corrupt financial transactions divert funds from legitimate public services, as well as distort free markets—potentially thwarting economic development—and reduce trust in institutions. The Organized Crime and Corruption Reporting Project (OCCRP) is a global platform for investigative reporting, providing resources to journalists and media centres, enabling cost-effective collaboration between editors and offering tools to secure themselves against threats to independent media. Exposing previously-unknown connections between entities makes it possible for citizens, policymakers, activists and law enforcement agencies to act. As the number of such leaks and publications grows, there is an increasing need for effective, scalable and reproducible methods to discover any anomalies and evidence of malfeasance that might exist within them.","tags":["Information Extraction","Low-Resource Learning"],"title":"Data4Transparency","type":"project"},{"authors":["Davy Weissenbacher","Karen O’Connor","Siddharth Rawal","Yu Zhang","Richard Tzong-Han Tsai","Timothy Miller","Dongfang Xu","Carol Anderson","Bo Liu","Qing Han","Jinfeng Zhang","Igor Kulev","Berkay Köprü","Raul Rodriguez-Esteban","Elif Ozkirimli","Ammer Ayach","Roland Roller","Stephen Piccolo","Peijin Han","V G Vinod Vydiswaran","Ramya Tekumalla","Juan M Banda","Parsa Bagherzadeh","Sabine Bergler","João F Silva","Tiago Almeida","Paloma Martinez","Renzo Rivera-Zavala","Chen-Kai Wang","Hong-Jie Dai","Luis Alberto Robles Hernandez","Graciela Gonzalez-Hernandez"],"categories":[],"content":"","date":1676449983,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1676449983,"objectID":"73acdadd2ba1002e88ce137fddb2c895","permalink":"https://dfki-nlp.github.io/publication/database2023-roller-biocreative/","publishdate":"2023-04-23T10:33:03+02:00","relpermalink":"/publication/database2023-roller-biocreative/","section":"publication","summary":"This study presents the outcomes of the shared task competition BioCreative VII (Task 3) focusing on the extraction of medication names from a Twitter user’s publicly available tweets (the user’s ‘timeline’). In general, detecting health-related tweets is notoriously challenging for natural language processing tools. The main challenge, aside from the informality of the language used, is that people tweet about any and all topics, and most of their tweets are not related to health. Thus, finding those tweets in a user’s timeline that mention specific health-related concepts such as medications requires addressing extreme imbalance. Task 3 called for detecting tweets in a user’s timeline that mentions a medication name and, for each detected mention, extracting its span. The organizers made available a corpus consisting of 182 049 tweets publicly posted by 212 Twitter users with all medication mentions manually annotated. The corpus exhibits the natural distribution of positive tweets, with only 442 tweets (0.2\\%) mentioning a medication. This task was an opportunity for participants to evaluate methods that are robust to class imbalance beyond the simple lexical match. A total of 65 teams registered, and 16 teams submitted a system run. This study summarizes the corpus created by the organizers and the approaches taken by the participating teams for this challenge. The corpus is freely available at https://biocreative.bioinformatics.udel.edu/tasks/biocreative-vii/track-3/. The methods and the results of the competing systems are analyzed with a focus on the approaches taken for learning from class-imbalanced data.","tags":[],"title":"Automatic Extraction of Medication Mentions from Tweets—Overview of the BioCreative VII Shared Task 3 Competition","type":"publication"},{"authors":["David Samhammer","Roland Roller","Patrik Hummel","Bilgin Osmanodja","Aljoscha Burchardt","Manuel Mayrdorfer","Wiebke Duettmann","Peter Dabrock"],"categories":[],"content":"","date":1671525183,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1671525183,"objectID":"982508b5ca2ce30e639b39d3f2a2d4fb","permalink":"https://dfki-nlp.github.io/publication/frontiers2022-roller-nothing/","publishdate":"2023-04-23T10:33:03+02:00","relpermalink":"/publication/frontiers2022-roller-nothing/","section":"publication","summary":"Introduction: Artificial intelligence–driven decision support systems (AI–DSS) have the potential to help physicians analyze data and facilitate the search for a correct diagnosis or suitable intervention. The potential of such systems is often emphasized. However, implementation in clinical practice deserves continuous attention. This article aims to shed light on the needs and challenges arising from the use of AI-DSS from physicians' perspectives. Methods: The basis for this study is a qualitative content analysis of expert interviews with experienced nephrologists after testing an AI-DSS in a straightforward usage scenario. Results: The results provide insights on the basics of clinical decision-making, expected challenges when using AI-DSS as well as a reflection on the test run. Discussion: While we can confirm the somewhat expectable demand for better explainability and control, other insights highlight the need to uphold classical strengths of the medical profession when using AI-DSS as well as the importance of broadening the view of AI-related challenges to the clinical environment, especially during treatment. Our results stress the necessity for adjusting AI-DSS to shared decision-making. We conclude that explainability must be context-specific while fostering meaningful interaction with the systems available.","tags":[],"title":"''Nothing works without the doctor:'' Physicians' perception of clinical decision-making and artificial intelligence","type":"publication"},{"authors":["Mariana Neves","Antonio Jimeno Yepes","Amy Siu","Roland Roller","Philippe Thomas","Maika Vicente Navarro","Lana Yeganova","Dina Wiemann","Giorgio Maria Di Nunzio","Federica Vezzani","Christel Gerardin","Rachel Bawden","Darryl Johan Estrada","Salvador Lima-lopez","Eulalia Farre-maduel","Martin Krallinger","Cristian Grozea","Aurelie Neveol"],"categories":[],"content":"","date":1671525183,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1671525183,"objectID":"bb0a3f814cf91d3fafe1ed03a5c8b11e","permalink":"https://dfki-nlp.github.io/publication/wmt-emnlp2022-roller-findings/","publishdate":"2023-04-23T10:33:03+02:00","relpermalink":"/publication/wmt-emnlp2022-roller-findings/","section":"publication","summary":"In the seventh edition of the WMT Biomedical Task, we addressed a total of seven languagepairs, namely English/German, English/French, English/Spanish, English/Portuguese, English/Chinese, English/Russian, English/Italian. This year{'}s test sets covered three types of biomedical text genre. In addition to scientific abstracts and terminology items used in previous editions, we released test sets of clinical cases. The evaluation of clinical cases translations were given special attention by involving clinicians in the preparation of reference translations and manual evaluation. For the main MEDLINE test sets, we received a total of 609 submissions from 37 teams. For the ClinSpEn sub-task, we had the participation of five teams.","tags":[],"title":"Findings of the WMT 2022 Biomedical Translation Shared Task: Monolingual Clinical Case Reports","type":"publication"},{"authors":["Roland Roller","Manuel Mayrdorfer","Wiebke Duettmann","Marcel G. Naik","Danilo Schmidt","Fabian Halleck","Patrik Hummel","Aljoscha Burchardt","Sebastian Möller","Peter Dabrock","Bilgin Osmanodja","Klemens Budde"],"categories":[],"content":"","date":1666686783,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1666686783,"objectID":"b8e2adf7e4b9059716bd83bf49ca8994","permalink":"https://dfki-nlp.github.io/publication/frontiers2022-roller-evaluation/","publishdate":"2023-04-23T10:33:03+02:00","relpermalink":"/publication/frontiers2022-roller-evaluation/","section":"publication","summary":"Patient care after kidney transplantation requires integration of complex information to make informed decisions on risk constellations. Many machine learning models have been developed for detecting patient outcomes in the past years. However, performance metrics alone do not determine practical utility. We present a newly developed clinical decision support system (CDSS) for detection of patients at risk for rejection and death-censored graft failure. The CDSS is based on clinical routine data including 1,516 kidney transplant recipients and more than 100,000 data points. In a reader study we compare the performance of physicians at a nephrology department with and without the CDSS. Internal validation shows AUC-ROC scores of 0.83 for rejection, and 0.95 for graft failure. The reader study shows that predictions by physicians converge toward the CDSS. However, performance does not improve (AUC–ROC; 0.6413 vs. 0.6314 for rejection; 0.8072 vs. 0.7778 for graft failure). Finally, the study shows that the CDSS detects partially different patients at risk compared to physicians. This indicates that the combination of both, medical professionals and a CDSS might help detect more patients at risk for graft failure. However, the question of how to integrate such a system efficiently into clinical practice remains open.","tags":[],"title":"Evaluation of a clinical decision support system for detection of patients at risk after kidney transplantation","type":"publication"},{"authors":["Yuxuan Chen","David Harbecke","Leonhard Hennig"],"categories":[],"content":"","date":1666427583,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1666427583,"objectID":"4c041d8a26606010a045082e8f180e45","permalink":"https://dfki-nlp.github.io/publication/emnlp2022-chen-meffiprompt/","publishdate":"2022-10-22T10:33:03+02:00","relpermalink":"/publication/emnlp2022-chen-meffiprompt/","section":"publication","summary":"Prompting pre-trained language models has achieved impressive performance on various NLP tasks, especially in low data regimes. Despite the success of prompting in monolingual settings, applying prompt-based methods in multilingual scenarios has been limited to a narrow set of tasks, due to the high cost of handcrafting multilingual prompts. In this paper, we present the first work on prompt-based multilingual relation classification (RC), by introducing an efficient and effective method that constructs prompts from relation triples and involves only minimal translation for the class labels. We evaluate its performance in fully supervised, few-shot and zero-shot scenarios, and analyze its effectiveness across 14 languages, prompt variants, and English-task training in cross-lingual settings. We find that in both fully supervised and few-shot scenarios, our prompt method beats competitive baselines: fine-tuning XLM-R-EM and null prompts. It also outperforms the random baseline by a large margin in zero-shot experiments. Our method requires little in-language knowledge and can be used as a strong baseline for similar multilingual classification tasks.","tags":[],"title":"Multilingual Relation Classification via Efficient and Effective Prompting","type":"publication"},{"authors":["Arne Binder","Bhuvanesh Verma","Leonhard Hennig"],"categories":[],"content":"","date":1666310400,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1666310400,"objectID":"4bb3720e75628da71f90ed6d4ddf47f7","permalink":"https://dfki-nlp.github.io/publication/wiesp2022-binder-etal-full/","publishdate":"2022-10-21T00:00:00Z","relpermalink":"/publication/wiesp2022-binder-etal-full/","section":"publication","summary":"Scholarly Argumentation Mining (SAM) has recently gained attention due to its potential to help scholars with the rapid growth of published scientific literature. It comprises two subtasks: argumentative discourse unit recognition (ADUR) and argumentative relation extraction (ARE), both of which are challenging since they require e.g. the integration of domain knowledge, the detection of implicit statements, and the disambiguation of argument structure. While previous work focused on dataset construction and baseline methods for specific document sections, such as abstract or results, full-text scholarly argumentation mining has seen little progress. In this work, we introduce a sequential pipeline model combining ADUR and ARE for full-text SAM, and provide a first analysis of the performance of pretrained language models (PLMs) on both subtasks. We establish a new SotA for ADUR on the Sci-Arg corpus, outperforming the previous best reported result by a large margin (+7% F1). We also present the first results for ARE, and thus for the full AM pipeline, on this benchmark dataset. Our detailed error analysis reveals that non-contiguous ADUs as well as the interpretation of discourse connectors pose major challenges and that data annotation needs to be more consistent.","tags":[],"title":"Full-Text Argumentation Mining on Scientific Publications","type":"publication"},{"authors":["Malte Ostendorff","Nils Rethmeier","Isabelle Augenstein","Bela Gipp","Georg Rehm"],"categories":[],"content":"","date":1665736383,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1665736383,"objectID":"db6ae7985024e062c2e16cc599a3472b","permalink":"https://dfki-nlp.github.io/publication/emnlp2022-ostendorff/","publishdate":"2022-10-14T10:33:03+02:00","relpermalink":"/publication/emnlp2022-ostendorff/","section":"publication","summary":"Learning scientific document representations can be substantially improved through contrastive learning objectives, where the challenge lies in creating positive and negative training samples that encode the desired similarity semantics. Prior work relies on discrete citation relations to generate contrast samples. However, discrete citations enforce a hard cut-off to similarity. This is counter-intuitive to similarity-based learning, and ignores that scientific papers can be very similar despite lacking a direct citation - a core problem of finding related research. Instead, we use controlled nearest neighbor sampling over citation graph embeddings for contrastive learning. This control allows us to learn continuous similarity, to sample hard-to-learn negatives and positives, and also to avoid collisions between negative and positive samples by controlling the sampling margin between them. The resulting method SciNCL outperforms the state-of-the-art on the SciDocs benchmark. Furthermore, we demonstrate that it can train (or tune) models sample-efficiently, and that it can be combined with recent training-efficient methods. Perhaps surprisingly, even training a general-domain language model this way outperforms baselines pretrained in-domain. ","tags":[],"title":"Neighborhood Contrastive Learning for Scientific Document Representations with Citation Embeddings","type":"publication"},{"authors":[],"categories":[],"content":"Two papers from DFKI-NLP authors have been accepted for publication at EMNLP 2022, the 2022 Conference on Empirical Methods in Natural Language Processing. The conference is planned to be a hybrid meeting and will take place in Abu Dhabi, from Dec 7th to Dec 11th, 2022. The first paper introduces an efficient and effective method that constructs prompts from relation triples and involves only minimal translation for the class labels, in the case of in-language prompting. We evaluate its performance in fully supervised, few-shot and zero-shot scenarios across 14 languages, soft prompt variants, and English-task training in cross-lingual settings. The second paper proposes neighborhood contrastive learning for the representation learning of scientific document and achieves new state-of-the-art results on the SciDocs benchmark.\n Yuxuan Chen, David Harbecke, Leonhard Hennig (2022). Multilingual Relation Classification via Efficient and Effective Prompting. EMNLP 2022. PDF Cite Code Project DOI Malte Ostendorff, Nils Rethmeier, Isabelle Augenstein, Bela Gipp, Georg Rehm (2022). Neighborhood Contrastive Learning for Scientific Document Representations with Citation Embeddings. EMNLP 2022. PDF Cite Code DOI ","date":1665732241,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1665732241,"objectID":"3db51a86704299a94615128c8e63eba2","permalink":"https://dfki-nlp.github.io/post/emnlp2022/","publishdate":"2022-10-14T09:24:01+02:00","relpermalink":"/post/emnlp2022/","section":"post","summary":"Two papers from DFKI-NLP authors have been accepted for publication at EMNLP 2022, the 2022 Conference on Empirical Methods in Natural Language Processing. The conference is planned to be a hybrid meeting and will take place in Abu Dhabi, from Dec 7th to Dec 11th, 2022.","tags":[],"title":"2 papers to be presented at EMNLP 2022","type":"post"},{"authors":[],"categories":[],"content":"One paper from DFKI-NLP authors has been accepted for publication at the Workshop on Information Extraction from Scientific Publications (WIESP). The workshop will be held at AACL-IJCNLP 2022, the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing, which will take place as an online-only event from Nov. 20 to Nov. 23, 2022. The paper proposes a new method for argument mining in full-text scientific documents by combining argumentative discourse unit recognition with relation extraction.\n Arne Binder, Bhuvanesh Verma, Leonhard Hennig (2022). Full-Text Argumentation Mining on Scientific Publications. WIESP 2022. PDF Cite Code Project ","date":1665728641,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1665728641,"objectID":"5ce70b7b52cd23896270f5442726b45e","permalink":"https://dfki-nlp.github.io/post/aacl-ijcnlp2022/","publishdate":"2022-10-14T08:24:01+02:00","relpermalink":"/post/aacl-ijcnlp2022/","section":"post","summary":"One paper from DFKI-NLP authors has been accepted for publication at the Workshop on Information Extraction from Scientific Publications (WIESP). The workshop will be held at AACL-IJCNLP 2022, the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing, which will take place as an online-only event from Nov.","tags":[],"title":"1 paper to be presented at AACL-IJCNLP 2022","type":"post"},{"authors":["Vivien Macketanz","Eleftherios Avramidis","Aljoscha Burchardt","He Wang","Renlong Ai","Shushen Manakhimova","Ursula Strohriegel","Sebastian Möller","Hans Uszkoreit"],"categories":[],"content":"","date":1661173944,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1661173944,"objectID":"fb56f0308045a6d58b32cc99c3de31e1","permalink":"https://dfki-nlp.github.io/publication/lrec2022-macketanz-testsuite/","publishdate":"2022-08-22T15:12:24+02:00","relpermalink":"/publication/lrec2022-macketanz-testsuite/","section":"publication","summary":"This paper presents a fine-grained test suite for the language pair German–English. The test suite is based on a number of linguistically motivated categories and phenomena and the semi-automatic evaluation is carried out with regular expressions. We describe the creation and implementation of the test suite in detail, providing a full list of all categories and phenomena. Furthermore, we present various exemplary applications of our test suite that have been implemented in the past years, like contributions to the Conference of Machine Translation, the usage of the test suite and MT outputs for quality estimation, and the expansion of the test suite to the language pair Portuguese–English. We describe how we tracked the development of the performance of various systems MT systems over the years with the help of the test suite and which categories and phenomena are prone to resulting in MT errors. For the first time, we also make a large part of our test suite publicly available to the research community.","tags":["machine translation","linguistic test suite"],"title":"A Linguistically Motivated Test Suite to Semi-Automatically Evaluate German–English Machine Translation Output","type":"publication"},{"authors":["Roland Roller","Aljoscha Burchardt","Nils Feldhus","Laura Seiffe","Klemens Budde","Simon Ronicke","Bilgin Osmanodja"],"categories":[],"content":"","date":1660309944,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1660309944,"objectID":"5eff43066a7b70ed84b0d7bf37aec2f6","permalink":"https://dfki-nlp.github.io/publication/lrec2022-roller/","publishdate":"2022-08-12T15:12:24+02:00","relpermalink":"/publication/lrec2022-roller/","section":"publication","summary":"In recent years, machine learning for clinical decision support has gained more and more attention. In order to introduce such applications into clinical practice, a good performance might be essential, however, the aspect of trust should not be underestimated. For the treating physician using such a system and being (legally) responsible for the decision made, it is particularly important to understand the system’s recommendation. To provide insights into a model’s decision, various techniques from the field of explainability (XAI) have been proposed whose output is often enough not targeted to the domain experts that want to use the model. To close this gap, in this work, we explore how explanations could possibly look like in future. To this end, this work presents a dataset of textual explanations in context of decision support. Within a reader study, human physicians estimated the likelihood of possible negative patient outcomes in the near future and justified each decision with a few sentences. Using those sentences, we created a novel corpus, annotated with different semantic layers. Moreover, we provide an analysis of how those explanations are constructed, and how they change depending on physician, on the estimated risk and also in comparison to an automatic clinical decision support system with feature importance.","tags":[],"title":"An Annotated Corpus of Textual Explanations for Clinical Decision Support","type":"publication"},{"authors":["Lisa Raithel","Philippe Thomas","Roland Roller","Oliver Sapina","Sebastian Möller","Pierre Zweigenbaum"],"categories":[],"content":"","date":1660309944,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1660309944,"objectID":"94c08d7d9fb865bdd286f9022b8ca973","permalink":"https://dfki-nlp.github.io/publication/lrec2022-raithel-cross-adr/","publishdate":"2022-08-12T15:12:24+02:00","relpermalink":"/publication/lrec2022-raithel-cross-adr/","section":"publication","summary":"In this work, we present the first corpus for German Adverse Drug Reaction (ADR) detection in patient-generated content. The data consists of 4,169 binary annotated documents from a German patient forum, where users talk about health issues and get advice from medical doctors. As is common in social media data in this domain, the class labels of the corpus are very imbalanced. This and a high topic imbalance make it a very challenging dataset, since often, the same symptom can have several causes and is not always related to a medication intake. We aim to encourage further multi-lingual efforts in the domain of ADR detection and provide preliminary experiments for binary classification using different methods of zero- and few-shot learning based on a multi-lingual model. When fine-tuning XLM-RoBERTa first on English patient forum data and then on the new German data, we achieve an F1-score of 37.52 for the positive class. We make the dataset and models publicly available for the community.","tags":["pharmacovigilance","text classification","adverse drug reactions"],"title":"Cross-lingual Approaches for the Detection of Adverse Drug Reactions in German from a Patient's Perspective","type":"publication"},{"authors":["Rémi Calizzano","Malte Ostendorff","Qian Ruan","Georg Rehm"],"categories":[],"content":"","date":1660309944,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1660309944,"objectID":"dc2a59df634a50a111d996aee581b263","permalink":"https://dfki-nlp.github.io/publication/lrec2022-calizzano/","publishdate":"2022-08-12T15:12:24+02:00","relpermalink":"/publication/lrec2022-calizzano/","section":"publication","summary":"Almost all summarisation methods and datasets focus on a single language and short summaries. We introduce a new dataset called WikinewsSum for English, German, French, Spanish, Portuguese, Polish, and Italian summarisation tailored for extended summaries of approx. 11 sentences. The dataset comprises 39,626 summaries which are news articles from Wikinews and their sources. We compare three multilingual transformer models on the extractive summarisation task and three training scenarios on which we fine-tune mT5 to perform abstractive summarisation. This results in strong baselines for both extractive and abstractive summarisation on WikinewsSum. We also show how the combination of an extractive model with an abstractive one can be used to create extended abstractive summaries from long input documents. Finally, our results show that fine-tuning mT5 on all the languages combined significantly improves the summarisation performance on low-resource languages.","tags":["summarization","wikinews"],"title":"Generating Extended and Multilingual Summaries with Pre-trained Transformers","type":"publication"},{"authors":["Aleksandra Gabryszak","Philippe Thomas"],"categories":[],"content":"","date":1660309944,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1660309944,"objectID":"340f60f8fc97aa804d5c6630207d3989","permalink":"https://dfki-nlp.github.io/publication/lrec2022-gabryszak/","publishdate":"2022-08-12T15:12:24+02:00","relpermalink":"/publication/lrec2022-gabryszak/","section":"publication","summary":"In this paper we show how aspect-based sentiment analysis might help public transport companies to improve their social responsibility for accessible travel. We present MobASA: a novel German-language corpus of tweets annotated with their relevance for public transportation, and with sentiment towards aspects related to barrier-free travel. We identified and labeled topics important for passengers limited in their mobility due to disability, age, or when travelling with young children. The data can be used to identify hurdles and improve travel planning for vulnerable passengers, as well as to monitor a perception of transportation businesses regarding the social inclusion of all passengers. The data is publicly available under: https://github.com/DFKI-NLP/sim3s-corpus","tags":["sentiment analysis","barrier-free travel"],"title":"MobASA: Corpus for Aspect-based Sentiment Analysis and Social Inclusion in the Mobility Domain","type":"publication"},{"authors":["Laura Seiffe","Fares Kallel","Sebastian Möller","Babak Naderi","Roland Roller"],"categories":[],"content":"","date":1660309944,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1660309944,"objectID":"038039f57d8b41bc4751f6245d315b22","permalink":"https://dfki-nlp.github.io/publication/lrec2022-seiffe-subjective/","publishdate":"2022-08-12T15:12:24+02:00","relpermalink":"/publication/lrec2022-seiffe-subjective/","section":"publication","summary":"For different reasons, text can be difficult to read and understand for many people, especially if the text’s language is too complex. In order to provide suitable text for the target audience, it is necessary to measure its complexity. In this paper we describe subjective experiments to assess the readability of German text. We compile a new corpus of sentences provided by a German IT service provider. The sentences are annotated with the subjective complexity ratings by two groups of participants, namely experts and non-experts for that text domain. We then extract an extensive set of linguistically motivated features that are supposedly interacting with complexity perception. We show that a linear regression model with a subset of these features can be a very good predictor of text complexity.","tags":[],"title":"Subjective Text Complexity Assessment for German","type":"publication"},{"authors":["Niklas Dehio","Malte Ostendorff","Georg Rehm"],"categories":[],"content":"","date":1657093069,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1657093069,"objectID":"95ceb26564e8c45d2ce1e79d6f6f04cd","permalink":"https://dfki-nlp.github.io/publication/lrec2022-dehio/","publishdate":"2022-07-06T09:37:49+02:00","relpermalink":"/publication/lrec2022-dehio/","section":"publication","summary":"To cope with the COVID-19 pandemic, many jurisdictions have introduced new or altered existing legislation. Even though these new rules are often communicated to the public in news articles, it remains challenging for laypersons to learn about what is currently allowed or forbidden since news articles typically do not reference underlying laws. We investigate an automated approach to extract legal claims from news articles and to match the claims with their corresponding applicable laws. We examine the feasibility of the two tasks concerning claims about COVID-19-related laws from Berlin, Germany. For both tasks, we create and make publicly available the data sets and report the results of initial experiments. We obtain promising results with Transformer-based models that achieve 46.7 F1 for claim extraction and 91.4 F1 for law matching, albeit with some conceptual limitations. Furthermore, we discuss challenges of current machine learning approaches for legal language processing and their ability for complex legal reasoning tasks.","tags":[],"title":"Claim Extraction and Law Matching for COVID-19-related Legislation","type":"publication"},{"authors":["Malte Ostendorff","Till Blume","Terry Ruas","Bela Gipp","Georg Rehm"],"categories":[],"content":"","date":1657092783,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1657092783,"objectID":"b2f475ebc5afd89e76b904be46a724d9","permalink":"https://dfki-nlp.github.io/publication/jcdl2022-ostendorff/","publishdate":"2022-07-06T09:33:03+02:00","relpermalink":"/publication/jcdl2022-ostendorff/","section":"publication","summary":"Document embeddings and similarity measures underpin content-based recommender systems, whereby a document is commonly represented as a single generic embedding. However, similarity computed on single vector representations provides only one perspective on document similarity that ignores which aspects make two documents alike. To address this limitation, aspect-based similarity measures have been developed using document segmentation or pairwise multi-class document classification. While segmentation harms the document coherence, the pairwise classification approach scales poorly to large scale corpora. In this paper, we treat aspect-based similarity as a classical vector similarity problem in aspect-specific embedding spaces. We represent a document not as a single generic embedding but as multiple specialized embeddings. Our approach avoids document segmentation and scales linearly w.r.t. the corpus size. In an empirical study, we use the Papers with Code corpus containing 157, 606 research papers and consider the task, method, and dataset of the respective research papers as their aspects. We compare and analyze three generic document embeddings, six specialized document embeddings and a pairwise classification baseline in the context of research paper recommendations. As generic document embeddings, we consider FastText, SciBERT, and SPECTER. To compute the specialized document embeddings, we compare three alternative methods inspired by retrofitting, fine-tuning, and Siamese networks. In our experiments, Siamese SciBERT achieved the highest scores. Additional analyses indicate an implicit bias of the generic document embeddings towards the dataset aspect and against the method aspect of each research paper. Our approach of aspect-based document embeddings mitigates potential risks arising from implicit biases by making them explicit. This can, for example, be used for more diverse and explainable recommendations.","tags":[],"title":"Specialized Document Embeddings for Aspect-based Similarity of Research Papers","type":"publication"},{"authors":["Qian Ruan","Malte Ostendorff","Georg Rehm"],"categories":[],"content":"","date":1657092324,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1657092324,"objectID":"6e990907eaa367fdde36268f7946a4be","permalink":"https://dfki-nlp.github.io/publication/acl2022-ruan/","publishdate":"2022-07-06T09:25:24+02:00","relpermalink":"/publication/acl2022-ruan/","section":"publication","summary":"Transformer-based language models usually treat texts as linear sequences. However, most texts also have an inherent hierarchical structure, i.e., parts of a text can be identified using their position in this hierarchy. In addition, section titles usually indicate the common topic of their respective sentences. We propose a novel approach to formulate, extract, encode and inject hierarchical structure information explicitly into an extractive summarization model based on a pre-trained, encoder-only Transformer language model (HiStruct+ model), which improves SOTA ROUGEs for extractive summarization on PubMed and arXiv substantially. Using various experimental settings on three datasets (i.e., CNN/DailyMail, PubMed and arXiv), our HiStruct+ model outperforms a strong baseline collectively, which differs from our model only in that the hierarchical structure information is not injected. It is also observed that the more conspicuous hierarchical structure the dataset has, the larger improvements our method gains. The ablation study demonstrates that the hierarchical position information is the main contributor to our model’s SOTA performance.","tags":[],"title":"HiStruct+: Improving Extractive Text Summarization with Hierarchical Structure Information","type":"publication"},{"authors":["Vivien Macketanz","Babak Naderi","Steven Schmidt","Sebastian Möller"],"categories":[],"content":"","date":1655039544,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1655039544,"objectID":"d1da1db8452d93f183ca43f6a295799d","permalink":"https://dfki-nlp.github.io/publication/acl2022-humeval-macketanz-perceptual/","publishdate":"2022-06-12T15:12:24+02:00","relpermalink":"/publication/acl2022-humeval-macketanz-perceptual/","section":"publication","summary":"The quality of machine-generated text is a complex construct consisting of various aspects and dimensions. We present a study that aims to uncover relevant perceptual quality dimensions for one type of machine-generated text, that is, Machine Translation. We conducted a crowdsourcing survey in the style of a Semantic Differential to collect attribute ratings for German MT outputs. An Exploratory Factor Analysis revealed the underlying perceptual dimensions. As a result, we extracted four factors that operate as relevant dimensions for the Quality of Experience of MT outputs: precision, complexity, grammaticality, and transparency.","tags":["text quality","semantic differential","machine-generated text"],"title":"Perceptual Quality Dimensions of Machine-Generated Text with a Focus on Machine Translation","type":"publication"},{"authors":["David Harbecke","Leonhard Hennig"],"categories":[],"content":"","date":1654078591,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1654078591,"objectID":"35c99541fbe06feac9d7dae7dcf7004d","permalink":"https://dfki-nlp.github.io/project/text2tech/","publishdate":"2022-06-01T11:16:31+01:00","relpermalink":"/project/text2tech/","section":"project","summary":"The goal of the Text2Tech project is the research and development of automated methods for information extraction from unstructured text sources in order to be able to provide companies with decision-relevant knowledge about technological developments quickly and efficiently. AI-based methods for information extraction (IE) already make it possible to extract selected information, e.g. B. to people, companies and places automatically from text sources. In the Text2Tech project, such approaches are to be further developed in order to extract machine-readable knowledge about technologies, technology categories, companies and their relationships with each other from German and English-language, domain-specific text sources, using the example of the automotive industry. The most important research goals are the modeling and filling of domain-specific knowledge graphs (Knowledge Base Population), the development of methods for cross-lingual proper name recognition and linking (Named Entity Recognition or Entity Linking), relation extraction (Relation Extraction), as well as the development of Model compression methods so that models run efficiently even on small hardware.","tags":["Information Extraction","Low-Resource Learning"],"title":"Text2Tech","type":"project"},{"authors":[],"categories":[],"content":"One paper from DFKI-NLP authors has been accepted for publication at JCDL 2022, the 22nd ACM/IEEE Joint Conference on Digital Libraries. The conference is planned to be a hybrid meeting and will take place in Cologne, Germany, from June 20th through June 24th, 2022. The paper proposes to replace generic document embeddings with specialized, per section document embeddings, and evaluates this approach on the task of aspect-based similarity computation for research papers.\n Malte Ostendorff, Till Blume, Terry Ruas, Bela Gipp, Georg Rehm (2022). Specialized Document Embeddings for Aspect-based Similarity of Research Papers. JCDL 2022. PDF Cite Code DOI ","date":1653981841,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1653981841,"objectID":"0be1d1e6f24e387dce50c1b6ba119577","permalink":"https://dfki-nlp.github.io/post/jcdl2022/","publishdate":"2022-05-31T09:24:01+02:00","relpermalink":"/post/jcdl2022/","section":"post","summary":"One paper from DFKI-NLP authors has been accepted for publication at JCDL 2022, the 22nd ACM/IEEE Joint Conference on Digital Libraries. The conference is planned to be a hybrid meeting and will take place in Cologne, Germany, from June 20th through June 24th, 2022.","tags":[],"title":"1 paper to be presented at JCDL 2022","type":"post"},{"authors":[],"categories":[],"content":"Six papers from DFKI-NLP authors have been accepted for publication at LREC 2022, the 13th Language Resources and Evaluation Conference. The conference is planned to be a hybrid meeting and will take place in Marseille, France, from June 20th through June 25th, 2022. The paper by Dehio et al. is on claim extraction and matching in COVID-19-related Legislation, the one by Raither et al. presents a novel corpus for German-language Adverse Drug Reaction (ADR) detection in patient-generated content. The paper by Gabryszak et al. also presents a corpus, in this case of German-language tweets annotated with their relevance for public transportation, and with sentiment towards aspects related to barrier-free travel. The fourth paper by Macketanz et al. presents a fine-grained machine translation test suite for the language pair German-English. The test suite is based on a number of linguistically motivated categories and phenomena and the semi-automatic evaluation is carried out with regular expressions. The fSeiffe et al.\u0026rsquo;s paper presents a work on how to model subjective text complexity, by constructing and analyzing a German text corpus labeled with expert and non-expert complexity ratings. The final paper by Calizzano et al. introduces a new dataset called WikinewsSum for English, German, French, Spanish, Portuguese, Polish, and Italian summarisation tailored for extended summaries of approx. 11 sentences, and compares three multilingual transformer models on the extractive summarisation task and three training scenarios on which we fine-tune mT5 to perform abstractive summarisation.\n Lisa Raithel, Philippe Thomas, Roland Roller, Oliver Sapina, Sebastian Möller, Pierre Zweigenbaum (2022). Cross-lingual Approaches for the Detection of Adverse Drug Reactions in German from a Patient\u0026#39;s Perspective. LREC 2022. PDF Cite Niklas Dehio, Malte Ostendorff, Georg Rehm (2022). Claim Extraction and Law Matching for COVID-19-related Legislation. LREC 2022. PDF Cite Code Aleksandra Gabryszak, Philippe Thomas (2022). MobASA: Corpus for Aspect-based Sentiment Analysis and Social Inclusion in the Mobility Domain. CSR @ LREC 2022. PDF Cite Project Vivien Macketanz, Eleftherios Avramidis, Aljoscha Burchardt, He Wang, Renlong Ai, Shushen Manakhimova, Ursula Strohriegel, Sebastian Möller, Hans Uszkoreit (2022). A Linguistically Motivated Test Suite to Semi-Automatically Evaluate German–English Machine Translation Output. LREC 2022. PDF Cite Laura Seiffe, Fares Kallel, Sebastian Möller, Babak Naderi, Roland Roller (2022). Subjective Text Complexity Assessment for German. LREC 2022. PDF Cite Rémi Calizzano, Malte Ostendorff, Qian Ruan, Georg Rehm (2022). Generating Extended and Multilingual Summaries with Pre-trained Transformers. LREC 2022. PDF Cite ","date":1653981841,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1653981841,"objectID":"b36478abac142030ef17c1c844a3ed47","permalink":"https://dfki-nlp.github.io/post/lrec2022/","publishdate":"2022-05-31T09:24:01+02:00","relpermalink":"/post/lrec2022/","section":"post","summary":"Six papers from DFKI-NLP authors have been accepted for publication at LREC 2022, the 13th Language Resources and Evaluation Conference. The conference is planned to be a hybrid meeting and will take place in Marseille, France, from June 20th through June 25th, 2022.","tags":[],"title":"6 papers to be presented at LREC 2022","type":"post"},{"authors":["Yuxuan Chen","Jonas Mikkelsen","Arne Binder","Christoph Alt","Leonhard Hennig"],"categories":[],"content":"","date":1653523200,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1653523200,"objectID":"71450b988e01035d8dda07fd4d7aebf8","permalink":"https://dfki-nlp.github.io/publication/acl2022-repl4nlp-chen-fewie/","publishdate":"2022-03-28T00:00:00Z","relpermalink":"/publication/acl2022-repl4nlp-chen-fewie/","section":"publication","summary":"Pre-trained language models (PLM) are effective components of few-shot named entity recognition (NER) approaches when augmented with continued pre-training on task-specific out-of-domain data or fine-tuning on in-domain data. However, their performance in low-resource scenarios, where such data is not available, remains an open question. We introduce an encoder evaluation framework, and use it to systematically compare the performance of state-of-the-art pre-trained representations on the task of low-resource NER. We analyze a wide range of encoders pre-trained with different strategies, model architectures, intermediate-task fine-tuning, and contrastive learning. Our experimental results across ten benchmark NER datasets in English and German show that encoder performance varies significantly, suggesting that the choice of encoder for a specific low-resource scenario needs to be carefully evaluated.","tags":[],"title":"A Comparative Study of Pre-trained Encoders for Low-Resource Named Entity Recognition","type":"publication"},{"authors":["David Harbecke","Yuxuan Chen","Leonhard Hennig","Christoph Alt"],"categories":[],"content":"","date":1653523200,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1653523200,"objectID":"9cd7d48423cca6a139bf79bcc5d90221","permalink":"https://dfki-nlp.github.io/publication/acl2022-nlppower-harbecke-f1/","publishdate":"2022-03-28T00:00:00Z","relpermalink":"/publication/acl2022-nlppower-harbecke-f1/","section":"publication","summary":"Relation classification models are conventionally evaluated using only a single measure, e.g., micro-F1, macro-F1 or AUC. In this work, we analyze weighting schemes, such as micro and macro, for imbalanced datasets. We introduce a framework for weighting schemes, where existing schemes are extremes, and two new intermediate schemes. We show that reporting results of different weighting schemes better highlights strengths and weaknesses of a model.","tags":[],"title":"Why only Micro-$F_1$? Class Weighting of Measures for Relation Classification","type":"publication"},{"authors":[],"categories":[],"content":"Four papers from DFKI-NLP authors have been accepted for publication at ACL 2022, the 60th Annual Meeting of the Association for Computational Linguistics. The conference is planned to be a hybrid meeting and will take place in Dublin, Ireland, from May 22nd through May 27th, 2022. One paper is on evaluating pre-trained encoders on the task of low-resource NER across several English and German datasets, the other analyzes relation classification evaluation and suggests that using F1 weightings other than micro-F1 tells us much more about model performance, e.g. on imbalanced datasets. The third paper proposes a novel approach to encode and inject hierarchical structure information explicitly into an extractive, transformer-based summarization model. The final paper present a study that aims to uncover relevant perceptual quality dimensions for one type of machine-generated text, that is, Machine Translation. We conducted a crowd-sourcing survey in the style of a Semantic Differential to collect attribute ratings for German MT outputs.\n Qian Ruan, Malte Ostendorff, Georg Rehm (2022). HiStruct\u0026#43;: Improving Extractive Text Summarization with Hierarchical Structure Information. ACL 2022 Findings. PDF Cite Code DOI Yuxuan Chen, Jonas Mikkelsen, Arne Binder, Christoph Alt, Leonhard Hennig (2022). A Comparative Study of Pre-trained Encoders for Low-Resource Named Entity Recognition. ACL-REPL4NLP 2022. PDF Cite Code Project Project David Harbecke, Yuxuan Chen, Leonhard Hennig, Christoph Alt (2022). Why only Micro-$F_1$? Class Weighting of Measures for Relation Classification. ACL-NLPPower 2022. Cite Project Project Vivien Macketanz, Babak Naderi, Steven Schmidt, Sebastian Möller (2022). Perceptual Quality Dimensions of Machine-Generated Text with a Focus on Machine Translation. HumEval @ ACL 2022. PDF Cite ","date":1648711441,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1648711441,"objectID":"011bca29552463724be0a68ec689af6a","permalink":"https://dfki-nlp.github.io/post/acl2022/","publishdate":"2022-03-31T09:24:01+02:00","relpermalink":"/post/acl2022/","section":"post","summary":"Four papers from DFKI-NLP authors have been accepted for publication at ACL 2022, the 60th Annual Meeting of the Association for Computational Linguistics. The conference is planned to be a hybrid meeting and will take place in Dublin, Ireland, from May 22nd through May 27th, 2022.","tags":[],"title":"4 papers to be presented at ACL 2022","type":"post"},{"authors":["Philippe Thomas","Leonhard Hennig"],"categories":[],"content":"","date":1642673791,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1642673791,"objectID":"427ce0221fcd0c6d0f6a0b3418cf8e0d","permalink":"https://dfki-nlp.github.io/project/bifold/","publishdate":"2022-01-20T11:16:31+01:00","relpermalink":"/project/bifold/","section":"project","summary":"BIFOLD conducts foundational research in big data management and machine learning, as well as its intersection, to educate future talent, and create high-impact knowledge exchange. The Berlin Institute for the Foundations of Learning and Data (BIFOLD), has evolved in 2019 from the merger of two national Artificial Intelligence Competence Centers: the Berlin Big Data Center (BBDC) and the Berlin Center for Machine Learning (BZML). Embedded in the vibrant Berlin metropolitan area, BIFOLD provides an outstanding scientific environment and numerous collaboration opportunities for national and international researchers. BIFOLD offers a broad range of research topics as well as a platform for interdisciplinary research and knowledge exchange with the sciences and humanities, industry, startups and society. Within BIFOLD, DFKI SLT conducts research in Clinical AI, specifically addressing the task of Pharmacovigilance. Pharmacovigilance is concerned with the assessment and prevention of adverse drug reactions (ADR) in pharmaceutical products. As the level of medication is generally raising all over the world, the potential risk of unwanted side effects, such as ADRs, is constantly increasing. Patients exchange views in their own language as 'experts in their own right,' in social media and disease-specific forums. Our project addresses the detection and extraction of ADR from medical forums and social media across different languages using cross-lingual transfer learning in combination with external knowledge sources.","tags":["Information Extraction"],"title":"BIFOLD","type":"project"},{"authors":["Steffen Castle","Robert Schwarzenberg","Mohsen Pourvali"],"categories":[],"content":"","date":1634256e3,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1634256e3,"objectID":"0909afe8749d563f9e6c0ad01633bcb5","permalink":"https://dfki-nlp.github.io/publication/nlpcc-castle/","publishdate":"2021-11-22T11:20:32+01:00","relpermalink":"/publication/nlpcc-castle/","section":"publication","summary":"Detecting when there is a domain drift between training and inference data is important for any model evaluated on data collected in real time. Many current data drift detection methods only utilize input features to detect domain drift. While effective, these methods disregard the model’s evaluation of the data, which may be a significant source of information about the data domain. We propose to use information from the model in the form of explanations, specifically gradient times input, in order to utilize this information. Following the framework of Rabanser et al. [11], we combine these explanations with two-sample tests in order to detect a shift in distribution between training and evaluation data. Promising initial experiments show that explanations provide useful information for detecting shift, which potentially improves upon the current state-of-the-art.","tags":[],"title":"Detecting Covariate Drift with Explanations","type":"publication"},{"authors":["Leonhard Hennig","Phuc Tran Truong","Aleksandra Gabryszak"],"categories":[],"content":"","date":1630972800,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1630972800,"objectID":"b8425af8dc50ee6b2cd49c19ffc86d1e","permalink":"https://dfki-nlp.github.io/publication/konvens2021-hennig-mobie/","publishdate":"2021-08-11T00:00:00Z","relpermalink":"/publication/konvens2021-hennig-mobie/","section":"publication","summary":"We present MobIE, a German-language dataset, which is human-annotated with 20 coarse- and fine-grained entity types and entity linking information for geographically linkable entities. The dataset consists of 3,232 social media texts and traffic reports with 91K tokens, and contains 20.5K annotated entities, 13.1K of which are linked to a knowledge base. A subset of the dataset is human-annotated with seven mobility-related, n-ary relation types, while the remaining documents are annotated using a weakly-supervised labeling approach implemented with the Snorkel framework. To the best of our knowledge, this is the first German-language dataset that combines annotations for NER, EL and RE, and thus can be used for joint and multi-task learning of these fundamental information extraction tasks. We make MobIE public at https://github.com/dfki-nlp/mobie.","tags":[],"title":"MobIE: A German Dataset for Named Entity Recognition, Entity Linking and Relation Extraction in the Mobility Domain","type":"publication"},{"authors":["Malte Ostendorff","Elliott Ash","Terry Ruas","Bela Gipp","Julian Moreno-Schneider","Georg Rehm"],"categories":[],"content":"","date":1622110683,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1622110683,"objectID":"ab6a46956823e641886d1cbdd6c30ba3","permalink":"https://dfki-nlp.github.io/publication/ostendorff2021/","publishdate":"2021-05-27T12:18:03+02:00","relpermalink":"/publication/ostendorff2021/","section":"publication","summary":"Recommender systems assist legal professionals in finding relevant literature for supporting their case. Despite its importance for the profession, legal applications do not reflect the latest advances in recommender systems and representation learning research. Simultaneously, legal recommender systems are typically evaluated in small-scale user study without any public available benchmark datasets. Thus, these studies have limited reproducibility. To address the gap between research and practice, we explore a set of state-of-the-art document representation methods for the task of retrieving semantically related US case law. We evaluate text-based (e.g., fastText, Transformers), citation-based (e.g., DeepWalk, Poincaré), and hybrid methods. We compare in total 27 methods using two silver standards with annotations for 2,964 documents. The silver standards are newly created from Open Case Book and Wikisource and can be reused under an open license facilitating reproducibility. Our experiments show that document representations from averaged fastText word vectors (trained on legal corpora) yield the best results, closely followed by Poincaré citation embeddings. Combining fastText and Poincaré in a hybrid manner further improves the overall result. Besides the overall performance, we analyze the methods depending on document length, citation count, and the coverage of their recommendations. We make our source code, models, and datasets publicly available at this https URL. ","tags":[],"title":"Evaluating Document Representations for Content-based Legal Literature Recommendations","type":"publication"},{"authors":["Leonhard Hennig","Christoph Alt"],"categories":[],"content":"","date":1614075782,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1614075782,"objectID":"a8921d49553043a8c438729c3f6507f8","permalink":"https://dfki-nlp.github.io/project/cora4nlp/","publishdate":"2021-02-23T11:23:02+01:00","relpermalink":"/project/cora4nlp/","section":"project","summary":"Language is implicit - it omits information. Filling this information gap requires contextual inference, background- and commonsense knowledge, and reasoning over situational context. Language also evolves, i.e., it specializes and changes over time. For example, many different languages and domains exist, new domains arise, and both evolve constantly. Thus, language understanding also requires continuous and efficient adaptation to new languages and domains, and transfer to, and between, both. Current language understanding technology, however, focuses on high resource languages and domains, uses little to no context, and assumes static data, task, and target distributions. The research in Cora4NLP aims to address these challenges. It builds on the expertise and results of the predecessor project DEEPLEE and is carried out jointly between the language technology research departments in Berlin and Saarbrücken. ","tags":["Information Extraction","Language Understanding"],"title":"Cora4NLP","type":"project"},{"authors":["Leonhard Hennig"],"categories":[],"content":"","date":1614075391,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1614075391,"objectID":"cbc4ff3659230581c9808dc981535447","permalink":"https://dfki-nlp.github.io/project/bbdc2/","publishdate":"2021-02-23T11:16:31+01:00","relpermalink":"/project/bbdc2/","section":"project","summary":"In order to optimally prepare industry, science and the society in Germany and Europe for the global Big Data trend, highly coordinated activities in research, teaching, and technology transfer regarding the integration of data analysis methods and scalable data processing are required. To achieve this, the Berlin Big Data Center is pursuing the following seven objectives: 1) Pooling expertise in scalable data management, data analytics, and big data application 2) Conducting fundamental research to develop novel and automatically scalable technologies capable of performing 'Deep Analysis' of 'Big Data'. 3) Developing an integrated, declarative, highly scalable open-source system that enables the specification, automatic optimization, parallelization and hardware adaptation, and fault-tolerant, efficient execution of advanced data analysis problems, using varying methods (e.g., drawn from machine learning, linear algebra, statistics and probability theory, computational linguistics, or signal processing), leveraging our work on Apache Flink 4) Transfering technology and know-how to support innovation in companies and startups. 5) Educating data scientists with respect to the five big data dimensions (i.e., applications, economic, legal, social, and technological) via leading educational programs. 6) Empowering people to leverage 'Smart Data', i.e., to discover newfound information based on their massive data sets. 7)Enabling the general public to conduct sound data-driven decision-making.","tags":["Information Extraction"],"title":"BBDC2","type":"project"},{"authors":["Sven Schmeier","Christoph Alt","Robert Schwarzenberg"],"categories":[],"content":"","date":1614075391,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1614075391,"objectID":"c389619c2f6d4f10d79bd3a7581e3344","permalink":"https://dfki-nlp.github.io/project/deeplee/","publishdate":"2021-02-23T11:16:31+01:00","relpermalink":"/project/deeplee/","section":"project","summary":"The research work in DEEPLEE, which is carried out in the Language Technology research departments in Saabrücken and Berlin, builds on DFKI's expertise in the areas of deep learning (DL) and language technology (LT) and develops it further. They aim for profound improvements of DL approaches in LT by focusing on four central, open research topics: Modularity in DNN architectures, Use of external knowledge, DNNs with explanation functionality, Machine Teaching Strategies for DNNs","tags":["Information Extraction","Language Understanding"],"title":"DEEPLEE","type":"project"},{"authors":["Leonhard Hennig","Christoph Alt"],"categories":[],"content":"","date":1614075391,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1614075391,"objectID":"cfeff35232a191ef8a348b8c96979ab5","permalink":"https://dfki-nlp.github.io/project/plass/","publishdate":"2021-02-23T11:16:31+01:00","relpermalink":"/project/plass/","section":"project","summary":"The aim of the PLASS project is to develop a prototypical B2B platform for AI-based decision support for supply chain management. The focus is on the automatic recognition of decision-relevant information and the acquisition of structured knowledge from global and multilingual text sources. These sources provide a large database for SCM information, especially for the early detection of critical events and risks, but also of opportunities, e.g. through new technologies, at suppliers and supply chains. PLASS enables SMEs and large companies to continuously monitor their suppliers and supply chains, and supports supply chain managers in risk assessment and decision-making.","tags":["Information Extraction","Low-Resource Learning"],"title":"PLASS","type":"project"},{"authors":["Philippe Thomas"],"categories":[],"content":"","date":1614075391,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1614075391,"objectID":"c25cf95b8ddec88b64c3b4a1a19063f6","permalink":"https://dfki-nlp.github.io/project/sim3s/","publishdate":"2021-02-23T11:16:31+01:00","relpermalink":"/project/sim3s/","section":"project","summary":"In the SIM3S project, data from the BMVI data offerings mCloud and MDM will be linked, refined and jointly analysed with other open data, user-generated content and data from individual modes of transport and other mobility-relevant companies in order to remove barriers and barriers to discrimination in everyday mobility. For the implementation of the project, state-of-the-art technologies and methods from the areas of Big Data Intelligent Analysis of mass data and artificial intelligence, in particular Natural Language Processing (NLP), are used.","tags":["Information Extraction"],"title":"SIM3S","type":"project"},{"authors":["Malte Ostendorff","Terry Ruas","Till Blume","Bela Gipp","Georg Rehm"],"categories":[],"content":"","date":1609064252,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1609064252,"objectID":"9e79e1aa43e10a8c99ed53fd1f129c58","permalink":"https://dfki-nlp.github.io/publication/ostendorff2020c/","publishdate":"2020-12-27T12:17:32+02:00","relpermalink":"/publication/ostendorff2020c/","section":"publication","summary":"Traditional document similarity measures provide a coarse-grained distinction between similar and dissimilar documents. Typically, they do not consider in what aspects two documents are similar. This limits the granularity of applications like recommender systems that rely on document similarity. In this paper, we extend similarity with aspect information by performing a pairwise document classification task. We evaluate our aspect-based document similarity for research papers. Paper citations indicate the aspect-based similarity, i.e., the section title in which a citation occurs acts as a label for the pair of citing and cited paper. We apply a series of Transformer models such as RoBERTa, ELECTRA, XLNet, and BERT variations and compare them to an LSTM baseline. We perform our experiments on two newly constructed datasets of 172,073 research paper pairs from the ACL Anthology and CORD-19 corpus. Our results show SciBERT as the best performing system. A qualitative examination validates our quantitative results. Our findings motivate future research of aspect-based document similarity and the development of a recommender system based on the evaluated techniques. We make our datasets, code, and trained models publicly available. ","tags":[],"title":"Aspect-based Document Similarity for Research Papers","type":"publication"},{"authors":["Marc Hübner","Christoph Alt","Robert Schwarzenberg","Leonhard Hennig"],"categories":[],"content":"Definition Extraction systems are a valuable knowledge source for both humans and algorithms. In this paper we describe our submissions to the DeftEval shared task (SemEval-2020 Task 6), which is evaluated on an English textbook corpus. We provide a detailed explanation of our system for the joint extraction of definition concepts and the relations among them. Furthermore we provide an ablation study of our model variations and describe the results of an error analysis.\n","date":1606780800,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1606780800,"objectID":"325726696f4edad34e20c5582f171a49","permalink":"https://dfki-nlp.github.io/publication/semeval2020-huebner-defx/","publishdate":"2020-12-01T14:36:10+02:00","relpermalink":"/publication/semeval2020-huebner-defx/","section":"publication","summary":"We describe our submissions to the DeftEval shared task (SemEval-2020 Task 6)","tags":[],"title":"Defx at SemEval-2020 Task 6: Joint Extraction of Concepts and Relations for Definition Extraction","type":"publication"},{"authors":["Malte Ostendorff","Terry Ruas","Moritz Schubotz","Georg Rehm","Bela Gipp"],"categories":[],"content":"","date":1598523462,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1598523462,"objectID":"4397f1dd4f22b53f0586d6677062f408","permalink":"https://dfki-nlp.github.io/publication/ostendorff2020/","publishdate":"2020-08-22T12:17:42+02:00","relpermalink":"/publication/ostendorff2020/","section":"publication","summary":"Many digital libraries recommend literature to their users considering the similarity between a query document and their repository. However, they often fail to distinguish what is the relationship that makes two documents alike. In this paper, we model the problem of finding the relationship between two documents as a pairwise document classification task. To find the semantic relation between documents, we apply a series of techniques, such as GloVe, Paragraph-Vectors, BERT, and XLNet under different configurations (e.g., sequence length, vector concatenation scheme), including a Siamese architecture for the Transformer-based systems. We perform our experiments on a newly proposed dataset of 32,168 Wikipedia article pairs and Wikidata properties that define the semantic document relations. Our results show vanilla BERT as the best performing system with an F1-score of 0.93, which we manually examine to better understand its applicability to other domains. Our findings suggest that classifying semantic relations between documents is a solvable task and motivates the development of recommender systems based on the evaluated techniques. The discussions in this paper serve as first steps in the exploration of documents through SPARQL-like queries such that one could find documents that are similar in one aspect but dissimilar in another. ","tags":[],"title":"Pairwise Multi-Class Document Classification for Semantic Relations between Wikipedia Articles","type":"publication"},{"authors":["Robert Schwarzenberg","Steffen Castle"],"categories":[],"content":"","date":1595980800,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1595980800,"objectID":"ca2fe3ec7f7c5501639381082cb399a5","permalink":"https://dfki-nlp.github.io/publication/icml-schwarzenberg-castle/","publishdate":"2020-08-26T14:09:16+02:00","relpermalink":"/publication/icml-schwarzenberg-castle/","section":"publication","summary":"Integrated Gradients (IG) and PatternAttribution (PA) are two established explainability methods for neural networks. Both methods are theoretically well-founded. However, they were designed to overcome different challenges. In this work, we combine the two methods into a new method, Pattern-Guided Integrated Gradients (PGIG). PGIG inherits important properties from both parent methods and passes stress tests that the originals fail. In addition, we benchmark PGIG against nine alternative explainability approaches (including its parent methods) in a large-scale image degradation experiment and find that it outperforms all of them.","tags":[],"title":"Pattern-Guided Integrated Gradients","type":"publication"},{"authors":["Hanchu Zhang","Leonhard Hennig","Christoph Alt","Changjian Hu","Yao Meng","Chao Wang"],"categories":[],"content":"","date":1594339200,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1594339200,"objectID":"d04a7113f73244eb9fc10ce17aa6eb39","permalink":"https://dfki-nlp.github.io/publication/acl-ecnlp2020-zhang-bootstrapping/","publishdate":"2020-07-10T00:00:00Z","relpermalink":"/publication/acl-ecnlp2020-zhang-bootstrapping/","section":"publication","summary":"In this work, we introduce a bootstrapped, iterative NER model that integrates a PU learning algorithm for recognizing named entities in a low-resource setting. Our approach combines dictionary-based labeling with syntactically-informed label expansion to efficiently enrich the seed dictionaries. Experimental results on a dataset of manually annotated e-commerce product descriptions demonstrate the effectiveness of the proposed framework.","tags":[],"title":"Bootstrapping Named Entity Recognition in E-Commerce with Positive Unlabeled Learning","type":"publication"},{"authors":["David Harbecke","Christoph Alt"],"categories":[],"content":"","date":1594339200,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1594339200,"objectID":"015c496995d3ee9b3de39d1f0d271c69","permalink":"https://dfki-nlp.github.io/publication/acl-srw2020-harbecke-considering/","publishdate":"2020-07-10T00:00:00Z","relpermalink":"/publication/acl-srw2020-harbecke-considering/","section":"publication","summary":"Recently, state-of-the-art NLP models gained an increasing syntactic and semantic understanding of language, and explanation methods are crucial to understand their decisions. Occlusion is a well established method that provides explanations on discrete language data, e.g. by removing a language unit from an input and measuring the impact on a model's decision. We argue that current occlusion-based methods often produce invalid or syntactically incorrect language data, neglecting the improved abilities of recent NLP models. Furthermore, gradient-based explanation methods disregard the discrete distribution of data in NLP. Thus, we propose OLM: a novel explanation method that combines occlusion and language models to sample valid and syntactically correct replacements with high likelihood, given the context of the original input. We lay out a theoretical foundation that alleviates these weaknesses of other explanation methods in NLP and provide results that underline the importance of considering data likelihood in occlusion-based explanation.","tags":["Explainability"],"title":"Considering Likelihood in NLP Classification Explanations with Occlusion and Language Modeling","type":"publication"},{"authors":["Christoph Alt","Aleksandra Gabryszak","Leonhard Hennig"],"categories":[],"content":"","date":1594339200,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1594339200,"objectID":"8c459d2cf0ab9b1905c893ec91b8871c","permalink":"https://dfki-nlp.github.io/publication/acl2020-alt-probing/","publishdate":"2020-07-10T00:00:00Z","relpermalink":"/publication/acl2020-alt-probing/","section":"publication","summary":"Despite the recent progress, little is known about the features captured by state-of-the-art neural relation extraction (RE) models. Common methods encode the source sentence, conditioned on the entity mentions, before classifying the relation. However, the complexity of the task makes it difficult to understand how encoder architecture and supporting linguistic knowledge affect the features learned by the encoder. We introduce 14 probing tasks targeting linguistic properties relevant to RE, and we use them to study representations learned by more than 40 different encoder architecture and linguistic feature combinations trained on two datasets, TACRED and SemEval 2010 Task 8. We find that the bias induced by the architecture and the inclusion of linguistic features are clearly expressed in the probing task performance. For example, adding contextualized word representations greatly increases performance on probing tasks with a focus on named entity and part-of-speech information, and yields better results in RE. In contrast, entity masking improves RE, but considerably lowers performance on entity type related probing tasks.","tags":[],"title":"Probing Linguistic Features of Sentence-Level Representations in Neural Relation Extraction","type":"publication"},{"authors":["Christoph Alt","Aleksandra Gabryszak","Leonhard Hennig"],"categories":[],"content":"","date":1594339200,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1594339200,"objectID":"4b6a4a0f754ff65730e25ac13406e529","permalink":"https://dfki-nlp.github.io/publication/acl2020-alt-tacred-revisited/","publishdate":"2020-07-10T00:00:00Z","relpermalink":"/publication/acl2020-alt-tacred-revisited/","section":"publication","summary":"TACRED is one of the largest, most widely used crowdsourced datasets in Relation Extraction (RE). But, even with recent advances in unsupervised pre-training and knowledge enhanced neural RE, models still show a high error rate. In this paper, we investigate the questions: Have we reached a performance ceiling or is there still room for improvement? And how do crowd annotations, dataset, and models contribute to this error rate? To answer these questions, we first validate the most challenging 5K examples in the development and test sets using trained annotators. We find that label errors account for 8% absolute F1 test error, and that more than 50% of the examples need to be relabeled. On the relabeled test set the average F1 score of a large baseline model set improves from 62.1 to 70.1. After validation, we analyze misclassifications on the challenging instances, categorize them into linguistically motivated error groups, and verify the resulting error hypotheses on three state-of-the-art RE models. We show that two groups of ambiguous relations are responsible for most of the remaining errors and that models may adopt shallow heuristics on the dataset when entities are not masked.","tags":[],"title":"TACRED Revisited: A Thorough Evaluation of the TACRED Relation Extraction Task","type":"publication"},{"authors":null,"categories":null,"content":"","date":1593561600,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1593561600,"objectID":"f8b2dc590dfded5b66e7919211387415","permalink":"https://dfki-nlp.github.io/dataset/ex4cds/","publishdate":"2020-07-01T00:00:00Z","relpermalink":"/dataset/ex4cds/","section":"dataset","summary":"Ex4CDS are explanations (or more precisely justifications) of physicians in the context of clinical decision support. In the course of a larger study, physicians estimated the probability of different clinical outcomes in nephology, namely rejection, graft loss and infections, within the next 90 days. Each estimation had to be justified within a short text - these are our explanations. The explanations were provided in German and have strong similarities to general clinical notes. You can find a description and the data here: https://github.com/DFKI-NLP/Ex4CDS","tags":["Language Understanding","Explainable AI","Clinical Decision Support"],"title":"Ex4CDS - Textual Explanations for Clinical Decision Support","type":"dataset"},{"authors":null,"categories":null,"content":"","date":1593561600,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1593561600,"objectID":"182a0843518c2c6c1a4676a87bddfa9f","permalink":"https://dfki-nlp.github.io/dataset/german-patient-adr/","publishdate":"2020-07-01T00:00:00Z","relpermalink":"/dataset/german-patient-adr/","section":"dataset","summary":"In this work, we present the first corpus for German Adverse Drug Reaction (ADR) detection in patient-generated content. The data consists of 4,169 binary annotated documents from a German patient forum, where users talk about health issues and get advice from medical doctors. As is common in social media data in this domain, the class labels of the corpus are very imbalanced. This and a high topic imbalance make it a very challenging dataset, since often, the same symptom can have several causes and is not always related to a medication intake. We aim to encourage further multi-lingual efforts in the domain of ADR detection. More info: https://aclanthology.org/2022.lrec-1.388/","tags":["Language Understanding","Information Extraction","Multilinguality"],"title":"German Adverse Drug Reaction (ADR) detection in patient-generated content","type":"dataset"},{"authors":null,"categories":null,"content":"","date":1593561600,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1593561600,"objectID":"025ca2ede65a805c2716900ab9fae712","permalink":"https://dfki-nlp.github.io/dataset/dfki-mobasa-corpus/","publishdate":"2020-07-01T00:00:00Z","relpermalink":"/dataset/dfki-mobasa-corpus/","section":"dataset","summary":"This repository contains corpus called MobASA: a novel German-language corpus of tweets annotated with their relevance for public transportation, and with sentiment towards aspects related to barrier-free travel. We identified and labeled topics important for passengers limited in their mobility due to disability, age, or when travelling with young children.\nThe data can be used for as a training or test corpus for aspect-oriented sentiment analysis. Moreover, the corpus can benefit building inclusive public transportation systems. You can find the corpus here: https://github.com/DFKI-NLP/sim3s-corpus, and the description of the corpus here: https://aclanthology.org/2022.csrnlp-1.5.pdf","tags":["Language Understanding","Information Extraction","Sentiment Analysis","Mobility"],"title":"MobASA Corpus","type":"dataset"},{"authors":null,"categories":null,"content":"","date":1593561600,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1593561600,"objectID":"55804a630a9c47f344ed7ef49cca65d6","permalink":"https://dfki-nlp.github.io/dataset/dfki-mobie-corpus/","publishdate":"2020-07-01T00:00:00Z","relpermalink":"/dataset/dfki-mobie-corpus/","section":"dataset","summary":"This repository contains the DFKI MobIE Corpus (formerly \"DAYSTREAM Corpus\"), a dataset of 3,232 German-language documents collected between May 2015 - Apr 2019 that have been annotated with fine-grained geo-entities, such as location-street, location-stop and location-route, as well as standard named entity types (organization, date, number, etc). All location-related entities have been linked to either Open Street Map identifiers or database ids of Deutsche Bahn / Rhein-Main-Verkehrsverbund. The corpus has also been annotated with a set of 7 traffic-related n-ary relations and events, such as Accidents, Traffic jams, and Canceled Routes. It consists of Twitter messages, and traffic reports from e.g. radio stations, police and public transport providers. It allows for training and evaluating both named entity recognition algorithms that aim for fine-grained typing of geo-entities, entity linking of these entities, as well as n-ary relation extraction systems. You can find the description of the corpus here: https://www.dfki.de/web/forschung/projekte-publikationen/publikationen-uebersicht/publikation/11741/","tags":["Language Understanding","Information Extraction","Mobility"],"title":"MobIE Corpus","type":"dataset"},{"authors":["Karolina Zaczynska","Nils Feldhus","Robert Schwarzenberg","Aleksandra Gabryszak","Sebastian Möller"],"categories":[],"content":"","date":1592870400,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1592870400,"objectID":"69575402fe2a9bf17cf5ea06cff1b67c","permalink":"https://dfki-nlp.github.io/publication/swisstext2020-zaczynska-evaluating/","publishdate":"2020-06-23T00:00:00Z","relpermalink":"/publication/swisstext2020-zaczynska-evaluating/","section":"publication","summary":"Pre-trained transformer language models (TLMs) have recently refashioned natural language processing (NLP): Most stateof-the-art NLP models now operate on top of TLMs to benefit from contextualization and knowledge induction. To explain their success, the scientific community conducted numerous analyses. Besides other methods, syntactic agreement tests were utilized to analyse TLMs. Most of the studies were conducted for the English language, however. In this work, we analyse German TLMs. To this end, we design numerous agreement tasks, some of which consider peculiarities of the German language. Our experimental results show that state-of-the-art German TLMs generally perform well on agreement tasks, but we also identify and discuss syntactic structures that push them to their limits.","tags":[],"title":"Evaluating German Transformer Language Models with Syntactic Agreement Tests","type":"publication"},{"authors":["Dmitrii Aksenov","Julian Moreno Schneider","Peter Bourgonje","Robert Schwarzenberg","Leonhard Hennig","Georg Rehm"],"categories":[],"content":"","date":1589587200,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1589587200,"objectID":"691b01834034e508889d3d916be8218c","permalink":"https://dfki-nlp.github.io/publication/lrec2020-aksenov-abstractive/","publishdate":"2020-05-16T00:00:00Z","relpermalink":"/publication/lrec2020-aksenov-abstractive/","section":"publication","summary":"We explore to what extent knowledge about the pre-trained language model that is used is beneficial for the task of abstractive summarization. To this end, we experiment with conditioning the encoder and decoder of a Transformer-based neural model on the BERT language model. In addition, we propose a new method of BERT-windowing, which allows chunk-wise processing of texts longer than the BERT window size. We also explore how locality modeling, i.e., the explicit restriction of calculations to the local context, can affect the summarization ability of the Transformer. This is done by introducing 2-dimensional convolutional self-attention into the first layers of the encoder. The results of our models are compared to a baseline and the state-of-the-art models on the CNN/Daily Mail dataset. We additionally train our model on the SwissText dataset to demonstrate usability on German. Both models outperform the baseline in ROUGE scores on two datasets and show its superiority in a manual qualitative analysis.","tags":[],"title":"Abstractive Text Summarization based on Language Model Conditioning and Locality Modeling","type":"publication"},{"authors":null,"categories":null,"content":"","date":1576508108,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1576508108,"objectID":"2fd1a9eee61a268e57a10ee4b5e09ea7","permalink":"https://dfki-nlp.github.io/dataset/dfki-product-corpus/","publishdate":"2019-12-16T15:55:08+01:00","relpermalink":"/dataset/dfki-product-corpus/","section":"dataset","summary":"The Product Corpus is a dataset of 174 English web pages and social media posts annotated for product and company named entities, and the relation CompanyProvidesProduct. The goal is to make extraction of non-standard, B2B products and relations from unstructured text easier and more reliable. The corpus is also annotated for coreference chains of companies and products.","tags":["Language Understanding","Information Extraction"],"title":"Product Corpus","type":"dataset"},{"authors":null,"categories":null,"content":"","date":1576508108,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1576508108,"objectID":"88423b8faaa8c52b9ff3f68c29045e86","permalink":"https://dfki-nlp.github.io/dataset/dfki-smartdata-corpus/","publishdate":"2019-12-16T15:55:08+01:00","relpermalink":"/dataset/dfki-smartdata-corpus/","section":"dataset","summary":"The SmartData Corpus is a dataset of 2598 German-language documents which has been annotated with fine-grained geo-entities, such as streets, stops and routes, as well as standard named entity types. It has also been annotated with a set of 15 traffic- and industry-related n-ary relations and events, such as Accidents, Traffic jams, Acquisitions, and Strikes. The corpus consists of newswire texts, Twitter messages, and traffic reports from radio stations, police and railway companies. It allows for training and evaluating both named entity recognition algorithms that aim for fine-grained typing of geo-entities, as well as n-ary relation extraction systems.","tags":["Language Understanding","Information Extraction"],"title":"SmartData Corpus","type":"dataset"},{"authors":["Lisa Raithel","Robert Schwarzenberg"],"categories":[],"content":"","date":1569863398,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1569863398,"objectID":"e8e32af7af385deca2d3740c85dea084","permalink":"https://dfki-nlp.github.io/publication/cl-nvc/","publishdate":"2019-09-30T18:09:58+01:00","relpermalink":"/publication/cl-nvc/","section":"publication","summary":"Recently, Neural Vector Conceptualization (NVC) was proposed as a means to interpret samples from a word vector space. For NVC, a neural model activates higher order concepts it recognizes in a word vector instance. To this end, the model first needs to be trained with a sufficiently large instance-to-concept ground truth, which only exists for a few languages. In this work, we tackle this lack of resources with word vector space alignment techniques: We train the NVC model on a high resource language and test it with vectors from an aligned word vector space of another language, without retraining or fine-tuning. A quantitative and qualitative analysis shows that the NVC model indeed activates meaningful concepts for unseen vectors from the aligned vector space. NVC thus becomes available for low resource languages for which no appropriate concept ground truth exists.","tags":[],"title":"Cross-lingual Neural Vector Conceptualization","type":"publication"},{"authors":["Robert Schwarzenberg","Marc Hübner","David Harbecke","Christoph Alt","Leonhard Hennig"],"categories":[],"content":"","date":1569476323,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1569476323,"objectID":"a473d0f5238b0e0755c80265a795c54a","permalink":"https://dfki-nlp.github.io/publication/layerwise-relevance-visualization-in-convolutional-text-graph-classifiers/","publishdate":"2019-09-26T07:38:43+02:00","relpermalink":"/publication/layerwise-relevance-visualization-in-convolutional-text-graph-classifiers/","section":"publication","summary":"Representations in the hidden layers of Deep Neural Networks (DNN) are often hard to interpret since it is difficult to project them into an interpretable domain. Graph Convolutional Networks (GCN) allow this projection, but existing explainability methods do not exploit this fact, i.e. do not focus their explanations on intermediate states. In this work, we present a novel method that traces and visualizes features that contribute to a classification decision in the visible and hidden layers of a GCN. Our method exposes hidden cross-layer dynamics in the input graph structure. We experimentally demonstrate that it yields meaningful layerwise explanations for a GCN sentence classifier.","tags":[],"title":"Layerwise Relevance Visualization in Convolutional Text Graph Classifiers","type":"publication"},{"authors":["Christoph Alt","Marc Hübner","Leonhard Hennig"],"categories":[],"content":"","date":1564358400,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1564358400,"objectID":"584cb9534c1efd44745dff10b6121dc1","permalink":"https://dfki-nlp.github.io/publication/acl2019-alt-finetuning/","publishdate":"2019-08-26T14:09:16+02:00","relpermalink":"/publication/acl2019-alt-finetuning/","section":"publication","summary":"We show that generative language model pre-training combined with selective attention improves recall for long-tail relations in distantly supervised neural relation extraction.","tags":[],"title":"Fine-Tuning Pre-Trained Transformer Language Models to Distantly Supervised Relation Extraction","type":"publication"},{"authors":["Neslihan Iskender","Aleksandra Gabryszak","Tim Polzehl","Leonhard Hennig","Sebastian Möller"],"categories":[],"content":"","date":1561334400,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1561334400,"objectID":"c481a30f36434bd26ba43c2e41347fd9","permalink":"https://dfki-nlp.github.io/publication/qomex2019-iskender-crowd/","publishdate":"2019-06-24T14:09:16+02:00","relpermalink":"/publication/qomex2019-iskender-crowd/","section":"publication","summary":"We analyze the feasibility and appropriateness of micro-task crowdsourcing for evaluation of different summary quality characteristics and report an ongoing work on the crowdsourced evaluation of query-based extractive text summaries","tags":[],"title":"A Crowdsourcing Approach to Evaluate the Quality of Query-based Extractive Text Summaries","type":"publication"},{"authors":["Malte Ostendorff","Peter Bourgonje","Maria Berger","Julian Moreno Schneider","Georg Rehm","Bela Gipp"],"categories":[],"content":"","date":1558953138,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1558953138,"objectID":"a37577f877550a6974c6b3374f48e5ea","permalink":"https://dfki-nlp.github.io/publication/ostendorff2019/","publishdate":"2019-05-27T12:32:18+02:00","relpermalink":"/publication/ostendorff2019/","section":"publication","summary":" In this paper, we focus on the classification of books using short descriptive texts (cover blurbs) and additional metadata. Building upon BERT, a deep neural language model, we demonstrate how to combine text representations with metadata and knowledge graph embeddings, which encode author information. Compared to the standard BERT approach we achieve considerably better results for the classification task. For a more coarse-grained classification using eight labels we achieve an F1- score of 87.20, while a detailed classification using 343 labels yields an F1-score of 64.70. We make the source code and trained models of our experiments publicly available ","tags":[],"title":"Enriching BERT with Knowledge Graph Embedding for Document Classification","type":"publication"},{"authors":["Christoph Alt","Marc Hübner","Leonhard Hennig"],"categories":[],"content":"","date":1558310400,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1558310400,"objectID":"d6c8a5f82560901c80529e3d2c9e706a","permalink":"https://dfki-nlp.github.io/publication/akbc2019-alt-improving/","publishdate":"2019-08-26T14:36:10+02:00","relpermalink":"/publication/akbc2019-alt-improving/","section":"publication","summary":"We show that transfer learning through generative language model pre-training improves supervised neural relation extraction, achieving new state-of-the-art performance on TACRED and SemEval 2010 Task 8.","tags":[],"title":"Improving Relation Extraction by Pre-Trained Language Representations","type":"publication"},{"authors":["Robert Schwarzenberg","Lisa Raithel","David Harbecke"],"categories":[],"content":"","date":1554224121,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1554224121,"objectID":"2e97d4e5206e642fb4fe1c042c0171cc","permalink":"https://dfki-nlp.github.io/publication/nvc/","publishdate":"2019-04-02T17:55:21+01:00","relpermalink":"/publication/nvc/","section":"publication","summary":"Distributed word vector spaces are considered hard to interpret which hinders the understanding of natural language processing (NLP) models. In this work, we introduce a new method to interpret arbitrary samples from a word vector space. To this end, we train a neural model to conceptualize word vectors, which means that it activates higher order concepts it recognizes in a given vector. Contrary to prior approaches, our model operates in the original vector space and is capable of learning non-linear relations between word vectors and concepts. Furthermore, we show that it produces considerably less entropic concept activation profiles than the popular cosine similarity.","tags":[],"title":"Neural Vector Conceptualization for Word Vector Space Interpretation","type":"publication"},{"authors":["Robert Schwarzenberg","David Harbecke","Vivien Macketanz","Eleftherios Avramidis","Sebastian Möller"],"categories":[],"content":"","date":1553791245,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1553791245,"objectID":"704fbe147dc60525fe2463f016efdd9d","permalink":"https://dfki-nlp.github.io/publication/diamat/","publishdate":"2019-03-28T17:40:45+01:00","relpermalink":"/publication/diamat/","section":"publication","summary":"Evaluating translation models is a trade-off between effort and detail. On the one end of the spectrum there are automatic count-based methods such as BLEU, on the other end linguistic evaluations by humans, which arguably are more informative but also require a disproportionately high effort. To narrow the spectrum, we propose a general approach on how to automatically expose systematic differences between human and machine translations to human experts. Inspired by adversarial settings, we train a neural text classifier to distinguish human from machine translations. A classifier that performs and generalizes well after training should recognize systematic differences between the two classes, which we uncover with neural explainability methods. Our proof-of-concept implementation, DiaMaT, is open source. Applied to a dataset translated by a state-of-the-art neural Transformer model, DiaMaT achieves a classification accuracy of 75% and exposes meaningful differences between humans and the Transformer, amidst the current discussion about human parity.","tags":[],"title":"Train, Sort, Explain: Learning to Diagnose Translation Models","type":"publication"},{"authors":["Roland Roller","Christoph Alt","Laura Seiffe","He Wang"],"categories":[],"content":"","date":1543622400,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1543622400,"objectID":"433a17498e14b803ea57862a8b6aadd1","permalink":"https://dfki-nlp.github.io/publication/mex-an-information-extraction-platform-for-german-medical-text/","publishdate":"2019-08-26T14:36:19+02:00","relpermalink":"/publication/mex-an-information-extraction-platform-for-german-medical-text/","section":"publication","summary":"","tags":[],"title":"mEx - an Information Extraction Platform for German Medical Text","type":"publication"},{"authors":["David Harbecke","Robert Schwarzenberg","Christoph Alt"],"categories":[],"content":"","date":1541116800,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1541116800,"objectID":"68f68d704af9c42e6dc03edd211ae360","permalink":"https://dfki-nlp.github.io/publication/learning-explanations-from-language-data/","publishdate":"2019-08-26T14:36:16+02:00","relpermalink":"/publication/learning-explanations-from-language-data/","section":"publication","summary":"PatternAttribution is a recent method, introduced in the vision domain, that explains classifications of deep neural networks. We demonstrate that it also generates meaningful interpretations in the language domain.","tags":[],"title":"Learning Explanations From Language Data","type":"publication"},{"authors":null,"categories":null,"content":"","date":-62135596800,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":-62135596800,"objectID":"f26b5133c34eec1aa0a09390a36c2ade","permalink":"https://dfki-nlp.github.io/admin/config.yml","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/admin/config.yml","section":"","summary":"","tags":null,"title":"","type":"wowchemycms"},{"authors":null,"categories":null,"content":"","date":-62135596800,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":-62135596800,"objectID":"6d99026b9e19e4fa43d5aadf147c7176","permalink":"https://dfki-nlp.github.io/contact/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/contact/","section":"","summary":"","tags":null,"title":"","type":"widget_page"},{"authors":null,"categories":null,"content":"","date":-62135596800,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":-62135596800,"objectID":"c1d17ff2b20dca0ad6653a3161942b64","permalink":"https://dfki-nlp.github.io/people/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/people/","section":"","summary":"","tags":null,"title":"","type":"widget_page"},{"authors":null,"categories":null,"content":"The German Research Center for Artificial Intelligence (Deutsches Forschungszentrum für Künstliche Intelligenz GmbH (DFKI)) and its staff are committed to goal- and risk-oriented information privacy and the fundamental right to the protection of personal data. In this data protection policy we inform you about the processing of your personal data when visiting and using our web site.\nResponsible service provider German Research Center for Artificial Intelligence (DFKI)\nPhone: +49 (0)631 / 205 75-0, Email: info@dfki.de\nData Protection Officer Phone: +49 (0)631 / 205 75-0\nEmail: datenschutz@dfki.de\nIntended use Provision of the information offering in the course of the public communication of the DFKI Establishment of contact and correspondence with visitors and users Anonymous and protected use Visit and usage of our web site are anonymous. At our web site personal data are only collected to the technically necessary extent. The processed data will not be transmitted to any third parties or otherwise disclosed, except on the basis of concrete lawful obligations. Within our information offering we do not embed information or service offerings of third party providers.\nWhile using our web site the data transmission in the internet is being protected by a generally accepted secure encryption procedure and hence cannot easily be eavesdropped or tampered.\nAccess data Every time our website is accessed, usage, traffic and connection data are collected, temporarily stored in a log file and generally deleted after 90 days.\nThe following data is stored for each access/retrieval:\n IP address Transmitted user agent information (in particular type/version of web browser, operating system, etc.) Transmitted referrer information (URL of the referring page) Date and time of access/retrieval Submitted Access Method/Function Transmitted input values (search terms, etc.) Accessed page or retrieved file Amount of data transferred Access/retrieval processing status The processing of the access data is lawful because it is necessary for the purposes of the legitimate interests pursued by DFKI. The legitimate interests pursued by DFKI are the adaptation and optimisation of the information offering and the investigation, detection and prosecution of illegal activities in connection with the usage of our web site.\nThe stored data records can be statistically evaluated in order to adapt and optimize our web site to the needs of our visitors. Any techniques that offer the possibility to retrace the access characteristics of users (tracking) will not be applied. The creation of user profiles and automated decision-making based on it is precluded.\nThe stored data records are not attributable to specific persons. They are generally not being combined with other data sources. However, the stored data can be analysed and combined with other data sources, if we become aware of concrete indications of any illegal usage.\nCookies We use so-called cookies on our web site. Cookies are small files that are being stored by your web browser. The cookies used on our web site do not harm your computer and do not contain any malicious software. They offer a user-friendly and effective usage of our web site. We do not use cookies for marketing purposes.\nWe transmit so-called session cookies to your web browser. They are valid only for the duration of your visit on our web site and they do not have any meaning outside of our web site. The session cookies are needed in order to identify your session with a unique number during your visit and to transmit our contents in your preferred language. At the end of your visit the session cookies will be automatically deleted upon termination of your web browser.\nWe also transmit permanent cookies to your web browser with a validity period of at most 365 days. We are exclusively using these cookies in order to respect your settings for the type of presentation (normal, inverted) and for the font size. Furthermore, it will be recorded whether you’ve taken notice of the information about the usage of cookies in your web browser.\nYou can adjust your web browser such that you will be informed on setting cookies and allow cookies on an individual basis resp. exclude the acceptance of cookies for specific cases or generally. You also can adjust the automatic deletion of cookies upon termination of your web browser. Upon deactivation of cookies the functionality of our web site can be limited. In any case, our information offering is available to its full extent.\nSocial media We do not embed social media plug-ins on our web site. When you are visiting our web site no data are transmitted to social media services. Profiling by any third parties hence is precluded. You though have the option to change over to our information offerings on Facebook, Twitter and YouTube. For the usage of these services we refer to the data protection policies of the respective service providers. We are processing your personal data within the social networks insofar as you post contributions, send messages or otherwise communicate with us.\nCorrespondence You have the option to contact us by e-mail. We will use your e-mail address and other personal contact data for the correspondence with you. Due to lawful obligation every e-mail correspondence will be archived. Subject to our legitimate interests your e-mail address and other personal contact data can be stored in our contact data base. In this case you will receive a corresponding information on the processing of your contact data.\nAccess and Intervention Besides the information in this data protection policy you have the right of access to your personal data. To ensure fair data processing, you have the following rights:\n The right to rectification and completion of your personal data The right to erasure of your personal data The right to restriction of the processing of your personal data The right to object to the processing of your personal data on grounds related to your particular situation To exercise these rights, please contact our data protection officer.\nRight to lodge a complaint You have the right to lodge a complaint with a supervisory authority if you consider that the processing of your personal data infringes statutory data protection regulations.\n","date":-62135596800,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":-62135596800,"objectID":"18d05a63a1c8d7ed973cc51838494e41","permalink":"https://dfki-nlp.github.io/privacy/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/privacy/","section":"","summary":"The German Research Center for Artificial Intelligence (Deutsches Forschungszentrum für Künstliche Intelligenz GmbH (DFKI)) and its staff are committed to goal- and risk-oriented information privacy and the fundamental right to the protection of personal data.","tags":null,"title":"Data Protection Notice","type":"page"},{"authors":null,"categories":null,"content":"Responsible service provider Responsible for the content of the domain dfki-nlp.github.io from the point of view of § 5 TMG:\nDeutsches Forschungszentrum für Künstliche Intelligenz GmbH (DFKI) Management:\nProf. Dr. Antonio Krüger\nHelmut Ditzer\nTrippstadter Str. 122\n67663 Kaiserslautern\nGermany\nPhone: +49 631 20575 0\nFax: +49 631 20575 5030\nEmail: info@dfki.de\nRegister Court: Amtsgericht Kaiserslautern\nRegister Number: HRB 2313\nID-Number: DE 148 646 973\nThe person responsible for the editorial content of the domain cora4nlp.github.io of the German Research Center for Artificial Intelligence GmbH within the meaning of § 18 para. 2 MStV is:\nDr. Leonhard Hennig, Senior Researcher\nDFKI Lab Berlin\nAlt-Moabit 91c\nD-10559 Berlin\nTel: +49 (0)30 / 238 95-0\nEmail: leonhard.hennig@dfki.de\nWebsite URL: www.dfki.de\nLegal notice concerning liability for proprietary content As a content provider in accordance with Section 7 (1) of the German Telemedia Act (Telemediengesetz), the Deutsches Forschungszentrum für Künstliche Intelligenz GmbH (DFKI) is responsible for its own content that is used pursuant to the general laws. The DFKI endeavors to ensure that the information provided on this website is accurate and current. Nevertheless, errors and uncertainties cannot be entirely ruled out. For this reason, the DFKI undertakes no liability for ensuring that the provided information is current, accurate or complete, and is not responsible for its quality. The DFKI is not liable for material or immaterial damages caused directly or indirectly by the use or non-use of the offered information, or by the use of erroneous and incomplete information, unless willful or grossly negligent fault can be demonstrated. This also applies with respect to software or data provided for download. The DFKI reserves the right to modify, expand or delete parts of the website or the entire website without separate announcement, or to cease publication temporarily or definitively.\nLegal notices for third party content and references to external websites As a service provider, we are responsible for our content on these pages in accordance with general law (§ 7 (1) TMG). According to § 8 – 10 TMG, as a service provider, we are not obliged to monior third-party information that is transmitted or stored, or to investigate circumstances that indicate illegal activity. Obligations to remove or block the use of information according to general laws remain unaffected. However, liability in this regard is only possible form the point in time at which we become aware of a specific legal violation. If we become aware of legal violations, we will remove this content immediately. Cross- references (“links”) to the content providers are to be distinguished from own content. Our offer contains links to external third party website. The respective provider is always responsible for the content of the linked external pages. We cannot accept any liability for the content for the content of the linked pages. After this external content was checked by the DFKI when the link was recenty set, it was checked whether there were any preliminary legal violations. At the time oft he review, no legal violations were apparent. However, it cannot be ruled out that the content may be changed afterwards by teh resprective providers. A permanent control oft he content of the linked pages is not reasonable without ecidence of a violation of the law. The DFKI does not constantly check the content to which it refers in ist offer for changes that could justify a new responsibility. If you are oft he option that the linked external pages violate applicable law or have otherwie inappropriate content, please inform us directly: info@dfki.de. Should the DFKI discover or receive a message that an external offer to whicht is has linked triggers civil or criminal liability, the DFKI will remove the link to this offer.\nLegal notice concerning copyright The layout of the homepage, the graphics used and other content on the DFKI website are protected by copyright. The reproduction, processing, distribution and any type of use outside the boundaries of copyright law require the written approval of the DFKI (in writing). Insofar as any content on this page was not prepared by the DFKI, the copyrights of third parties will be observed. If you become aware of a copyright breach nevertheless, please inform us accordingly. Upon becoming aware of relevant legal breaches, the DFKI will remove such content immediately.\n","date":-62135596800,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":-62135596800,"objectID":"9b10c1f64082d3869fd4cb1f85809430","permalink":"https://dfki-nlp.github.io/terms/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/terms/","section":"","summary":"Responsible service provider Responsible for the content of the domain dfki-nlp.github.io from the point of view of § 5 TMG:\nDeutsches Forschungszentrum für Künstliche Intelligenz GmbH (DFKI) Management:\nProf. Dr. Antonio Krüger","tags":null,"title":"Legal Information","type":"page"}]
\ No newline at end of file
+[{"authors":["roland-roller"],"categories":null,"content":"I am a senior researcher and project manager in the Speech and Language Technology group at the German Research Center for Artificial Intelligence (DFKI). I focus on natural language processing and machine learning topics with a high interest in medical use cases. My work spans classical information extraction, anonymization, clinical decision support, chatbots and LLM agents. I work on various projects related to different medical domains, such as nephrology, emergency medicine, tumor boards, and sexual medicine. Here is the list of current projects: KIBATIN, PRIMA-AI, Medinym, ADBoard, SmartNTx, and Veranda. Please contact me if you are interested in our research, collaboration/supervision or would like to visit us.\n","date":1729641600,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":1729641600,"objectID":"5a5a71e8444dfa2557b703df82e80e8d","permalink":"https://dfki-nlp.github.io/authors/roland-roller/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/authors/roland-roller/","section":"authors","summary":"I am a senior researcher and project manager in the Speech and Language Technology group at the German Research Center for Artificial Intelligence (DFKI). I focus on natural language processing and machine learning topics with a high interest in medical use cases.","tags":["Researchers"],"title":"Roland Roller","type":"authors"},{"authors":["nils-feldhus"],"categories":null,"content":"","date":1727339583,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":1727339583,"objectID":"ad47029271d18471d52333582f6f09d3","permalink":"https://dfki-nlp.github.io/authors/nils-feldhus/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/authors/nils-feldhus/","section":"authors","summary":"","tags":["PhD Candidates"],"title":"Nils Feldhus","type":"authors"},{"authors":["aleksandra-gabryszak"],"categories":null,"content":"","date":1726907583,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":1726907583,"objectID":"bbfb2073a6e0589384f60ea2fae79732","permalink":"https://dfki-nlp.github.io/authors/aleksandra-gabryszak/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/authors/aleksandra-gabryszak/","section":"authors","summary":"","tags":["PhD Candidates"],"title":"Aleksandra Gabryszak","type":"authors"},{"authors":["arne-binder"],"categories":null,"content":"","date":1726907583,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":1726907583,"objectID":"db472dfe584699b02c3fb0b542f4efba","permalink":"https://dfki-nlp.github.io/authors/arne-binder/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/authors/arne-binder/","section":"authors","summary":"","tags":["PhD Candidates"],"title":"Arne Binder","type":"authors"},{"authors":["leonhard-hennig"],"categories":null,"content":"I\u0026rsquo;m a senior researcher and project manager at the DFKI Speech \u0026amp; Language Technology Lab. I\u0026rsquo;m interested in applying machine learning techniques to computational linguistics problems, such as information extraction and summarization, and making these work on real-world, domain-specific, noisy data in low-resource settings, where little or no language resources are readily available. As a project lead, I\u0026rsquo;ve managed various national research projects, such as Smart Data Web, PLASS, DAYSTREAM, and the DFKI part in the Berlin Big Data Center, as well as industry-funded projects, e.g. for Deutsche Telekom and Lenovo.\n","date":1726907583,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":1726907583,"objectID":"76acb2a1dbfa728042427546fca4cab6","permalink":"https://dfki-nlp.github.io/authors/leonhard-hennig/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/authors/leonhard-hennig/","section":"authors","summary":"I\u0026rsquo;m a senior researcher and project manager at the DFKI Speech \u0026amp; Language Technology Lab. I\u0026rsquo;m interested in applying machine learning techniques to computational linguistics problems, such as information extraction and summarization, and making these work on real-world, domain-specific, noisy data in low-resource settings, where little or no language resources are readily available.","tags":["Researchers"],"title":"Leonhard Hennig","type":"authors"},{"authors":["ajaymadhavan-ravichandran"],"categories":null,"content":"","date":1720742400,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":1720742400,"objectID":"a8ef52c2af75d3f4d651189c9b598c0d","permalink":"https://dfki-nlp.github.io/authors/ajaymadhavan-ravichandran/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/authors/ajaymadhavan-ravichandran/","section":"authors","summary":"","tags":["PhD Candidates"],"title":"Ajay Madhavan Ravichandran","type":"authors"},{"authors":["david-harbecke"],"categories":null,"content":"","date":1720742400,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":1720742400,"objectID":"57423a933a38af75c718059324719b6e","permalink":"https://dfki-nlp.github.io/authors/david-harbecke/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/authors/david-harbecke/","section":"authors","summary":"","tags":["PhD Candidates"],"title":"David Harbecke","type":"authors"},{"authors":["lisa-raithel"],"categories":null,"content":"Hey!\nI am Lisa, a post-doc at Technische Universität Berlin at the Quality and Usability Lab and BIFOLD and a guest researcher at DFKI GmbH, working closely with Philippe Thomas and Roland Roller.\nI obtained my master\u0026rsquo;s degree at Universität Potsdam in Computational Linguistics (B.Sc. in Computational Linguistics, M.Sc. in Cognitive Systems). I then briefly worked as a software engineer before transitioning back to academia for a double degree PhD program (cotutelle) at TU Berlin and Université Paris-Saclay. I was supervised by Prof. Sebastian Möller and Pierre Zweigenbaum, Directeur de Recherche CNRS. My doctoral research focused on cross-lingual information extraction for the detection of adverse drug reactions. During that time, I spent one year at LISN in Orsay, France (2021 - 2022) and three months at the Social Computing Lab at NAIST in Nara, Japan (2023). In February 2024, I successfully defended my thesis at TU Berlin.\n","date":1720742400,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":1720742400,"objectID":"2bf26a380b092795d4a187e33b919ab1","permalink":"https://dfki-nlp.github.io/authors/lisa-raithel/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/authors/lisa-raithel/","section":"authors","summary":"Hey!\nI am Lisa, a post-doc at Technische Universität Berlin at the Quality and Usability Lab and BIFOLD and a guest researcher at DFKI GmbH, working closely with Philippe Thomas and Roland Roller.","tags":["PhD Candidates"],"title":"Lisa Raithel","type":"authors"},{"authors":["malte-ostendorff"],"categories":null,"content":"In my research work, I mainly focus on information retrieval, recommender systems, and natural language processing. In particular, techniques for the information extraction from unstructured data such as text and making information more accessible are of great interest for me. In my recent work I apply these techniques on content from the legal domain, e.g. laws, patents, case files. Moreover, I explore how recommender systems can assist users in finding relevant information to cope with today’s information overload. Due to the large amounts of available data, all my work requires the use of scalable and distributed computing. Generally speaking, all topics that are somehow related to the following fields can be considered as my research interest:\n Recommender Systems Natural Language Processing Text Mining Applied Machine Learning Scalable Data Processing (\u0026ldquo;Big Data\u0026rdquo;) Legal Tech Language Models Feel free to contact me if you have any questions regarding my work. I am always open for new ideas, projects and collaborations with other researchers and students.\n","date":1720742400,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":1720742400,"objectID":"b87fe2d0e7981b54435e6d6bf9861b08","permalink":"https://dfki-nlp.github.io/authors/malte-ostendorff/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/authors/malte-ostendorff/","section":"authors","summary":"In my research work, I mainly focus on information retrieval, recommender systems, and natural language processing. In particular, techniques for the information extraction from unstructured data such as text and making information more accessible are of great interest for me.","tags":["Alumni"],"title":"Malte Ostendorff","type":"authors"},{"authors":["philippe-thomas"],"categories":null,"content":"","date":1715817600,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":1715817600,"objectID":"e87ec0a5f91d896556892e387ce48cc6","permalink":"https://dfki-nlp.github.io/authors/philippe-thomas/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/authors/philippe-thomas/","section":"authors","summary":"","tags":["Researchers"],"title":"Philippe Thomas","type":"authors"},{"authors":["yuxuan-chen"],"categories":null,"content":"","date":1715817600,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":1715817600,"objectID":"12c49279a0e7bdb8a10aea6a1c50529d","permalink":"https://dfki-nlp.github.io/authors/yuxuan-chen/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/authors/yuxuan-chen/","section":"authors","summary":"","tags":["PhD Candidates"],"title":"Yuxuan Chen","type":"authors"},{"authors":["eleftherios-avramidis"],"categories":null,"content":"","date":1689897600,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":1689897600,"objectID":"143f99ce31d6112e2c40d7cec4733d66","permalink":"https://dfki-nlp.github.io/authors/eleftherios-avramidis/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/authors/eleftherios-avramidis/","section":"authors","summary":"","tags":["Researchers"],"title":"Eleftherios Avramidis","type":"authors"},{"authors":["robert-schwarzenberg"],"categories":null,"content":"I conducted my PhD research at the Speech and Language Technologies Lab of the German Research Center for Artificial Intelligence (DFKI).\nMy interests include\n (Neural) Explainability Methods and Explainable Models, NLP and NLU, some Image Processing on the side, and Graph Algorithms because, you see, everything seems to be part of some graph. ","date":1686787200,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":1686787200,"objectID":"9c4340557d33d0eef87e7e24354df0fe","permalink":"https://dfki-nlp.github.io/authors/robert-schwarzenberg/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/authors/robert-schwarzenberg/","section":"authors","summary":"I conducted my PhD research at the Speech and Language Technologies Lab of the German Research Center for Artificial Intelligence (DFKI).\nMy interests include\n (Neural) Explainability Methods and Explainable Models, NLP and NLU, some Image Processing on the side, and Graph Algorithms because, you see, everything seems to be part of some graph.","tags":["Alumni"],"title":"Robert Schwarzenberg","type":"authors"},{"authors":["nils-rethmeier"],"categories":null,"content":"","date":1683534783,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":1683534783,"objectID":"5fa9f4aadfb6bfa3e5e6ad75c3611835","permalink":"https://dfki-nlp.github.io/authors/nils-rethmeier/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/authors/nils-rethmeier/","section":"authors","summary":"","tags":["PhD Candidates"],"title":"Nils Rethmeier","type":"authors"},{"authors":["steffen-castle"],"categories":null,"content":"","date":1680344191,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":1680344191,"objectID":"ad183e9940c0547e2260bca696fd06ca","permalink":"https://dfki-nlp.github.io/authors/steffen-castle/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/authors/steffen-castle/","section":"authors","summary":"","tags":["PhD Candidates"],"title":"Steffen Castle","type":"authors"},{"authors":["he-wang"],"categories":null,"content":"","date":1661173944,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":1661173944,"objectID":"6b5ebf54ed7db6b1d510ed1b4ccd6c8d","permalink":"https://dfki-nlp.github.io/authors/he-wang/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/authors/he-wang/","section":"authors","summary":"","tags":["Software Engineers"],"title":"He Wang","type":"authors"},{"authors":null,"categories":null,"content":"Christoph is now a postdoc in the Machine Learning group at Humboldt University of Berlin.\n","date":1653523200,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":1653523200,"objectID":"55f7fc0a0becc469231bd11edf9d90c1","permalink":"https://dfki-nlp.github.io/authors/christoph-alt/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/authors/christoph-alt/","section":"authors","summary":"Christoph is now a postdoc in the Machine Learning group at Humboldt University of Berlin.","tags":null,"title":"Christoph Alt","type":"authors"},{"authors":["karolina-zaczynska"],"categories":null,"content":"","date":1592870400,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":1592870400,"objectID":"e7523b67ec5174035b12fb4d48d29306","permalink":"https://dfki-nlp.github.io/authors/karolina-zaczynska/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/authors/karolina-zaczynska/","section":"authors","summary":"","tags":["PhD Candidates"],"title":"Karolina Zaczynska","type":"authors"},{"authors":null,"categories":null,"content":"DFKI-NLP is a Natural Language Processing group of researchers, software engineers and students at the Berlin office of the German Research Center for Artificial Intelligence (DFKI) working on basic and applied research in areas covering, among others, information extraction, knowledge base population, dialogue, sentiment analysis, and summarization. We are particularly interested in core research on learning in low-resource settings, reasoning over larger contexts, and continual learning. We strive for a deeper understanding of human language and thinking, with the goal of developing novel methods for processing and generating human language text, speech, and knowledge. An important part of our work is the creation of corpora, the evaluation of NLP datasets and tasks, and the explainability of (neural) models.\nKey topics:\n Applied / domain-specific information extraction Learning in low-resource settings and over large contexts Construction and analysis of IE datasets, linguistic annotation Multilingual information extraction Evaluation methodology research Explainability Our group forms a part of DFKI\u0026rsquo;s Speech and Language Technology department led by Prof. Sebastian Möller, and closely collaborates with e.g. the Technische Universität Berlin, DFKI\u0026rsquo;s Language Technology and Multilinguality department and DFKI\u0026rsquo;s Intelligent Analytics for Massive Data group.\n","date":-62135596800,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":-62135596800,"objectID":"28cf2f802ba8f342b4dc1a3a2cf18f61","permalink":"https://dfki-nlp.github.io/authors/dfki-nlp/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/authors/dfki-nlp/","section":"authors","summary":"DFKI-NLP is a Natural Language Processing group of researchers, software engineers and students at the Berlin office of the German Research Center for Artificial Intelligence (DFKI) working on basic and applied research in areas covering, among others, information extraction, knowledge base population, dialogue, sentiment analysis, and summarization.","tags":null,"title":"DFKI-NLP","type":"authors"},{"authors":["marc-huebner"],"categories":null,"content":"","date":-62135596800,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":-62135596800,"objectID":"bb1a2f736e1bfeb288ac79f61fa27578","permalink":"https://dfki-nlp.github.io/authors/marc-huebner/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/authors/marc-huebner/","section":"authors","summary":"","tags":["Alumni"],"title":"Marc Hübner","type":"authors"},{"authors":["Akhila Abdulnazar","Roland Roller","Stefan Schulz","Markus Kreuzthaler"],"categories":[],"content":"","date":1729641600,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1729641600,"objectID":"559b9ef1595bbbf3993e55b24309cc00","permalink":"https://dfki-nlp.github.io/publication/sage2024/","publishdate":"2024-10-23T00:00:00Z","relpermalink":"/publication/sage2024/","section":"publication","summary":"Clinical narratives provide comprehensive patient information. Achieving interoperability involves mapping relevant details to standardized medical vocabularies. Typically, natural language processing divides this task into named entity recognition (NER) and medical concept normalization (MCN). State-of-the-art results require supervised setups with abundant training data. However, the limited availability of annotated data due to sensitivity and time constraints poses challenges. This study addressed the need for unsupervised medical concept annotation (MCA) to overcome these limitations and support the creation of annotated datasets.","tags":[],"title":"Unsupervised SapBERT-based bi-encoders for medical concept annotation of clinical narratives with SNOMED CT","type":"publication"},{"authors":["Qianli Wang","Tatiana Anikina","Nils Feldhus","Simon Ostermann","Sebastian Möller"],"categories":[],"content":"","date":1727339583,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1727339583,"objectID":"99efa56c1e8a67379f387296b904a08b","permalink":"https://dfki-nlp.github.io/publication/wang-etal-2024-coxql/","publishdate":"2024-09-26T10:33:03+02:00","relpermalink":"/publication/wang-etal-2024-coxql/","section":"publication","summary":"Conversational explainable artificial intelligence (ConvXAI) systems based on large language models (LLMs) have garnered significant interest from the research community in natural language processing (NLP) and human-computer interaction (HCI). Such systems can provide answers to user questions about explanations in dialogues, have the potential to enhance users' comprehension and offer more information about the decision-making and generation processes of LLMs. Currently available ConvXAI systems are based on intent recognition rather than free chat, as this has been found to be more precise and reliable in identifying users' intentions. However, the recognition of intents still presents a challenge in the case of ConvXAI, since little training data exist and the domain is highly specific, as there is a broad range of XAI methods to map requests onto. In order to bridge this gap, we present CoXQL, the first dataset for user intent recognition in ConvXAI, covering 31 intents, seven of which require filling multiple slots. Subsequently, we enhance an existing parsing approach by incorporating template validations, and conduct an evaluation of several LLMs on CoXQL using different parsing strategies. We conclude that the improved parsing approach (MP+) surpasses the performance of previous approaches. We also discover that intents with multiple slots remain highly challenging for LLMs.","tags":[],"title":"CoXQL: A Dataset for Parsing Explanation Requests in Conversational XAI Systems ","type":"publication"},{"authors":["Akhila Abdulnazar","Roland Roller","Stefan Schulz","Markus Kreuzthaler"],"categories":[],"content":"","date":1727222400,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1727222400,"objectID":"a446fec1fa9c0379fa540bc10ef6d621","permalink":"https://dfki-nlp.github.io/publication/ieee2024/","publishdate":"2024-09-25T00:00:00Z","relpermalink":"/publication/ieee2024/","section":"publication","summary":"Most clinical information is only available as free text. Large language models (LLMs) are increasingly applied to clinical data to streamline communication, enhance the accuracy of clinical documentation, and ultimately improve healthcare delivery. This study focuses on a corpus of anonymized clinical narratives in German. On the one hand it evaluates the use of ChatGPT for text cleaning, i.e., the automatic rephrasing of raw text into a more readable and standardized form, and on the other hand for retrieval-augmented generation (RAG). In both tasks, the final goal was medical concept normalization (MCN), i.e., the annotation of text segments with codes from a controlled vocabulary using natural language processing.We found that ChatGPT (GPT-4) significantly improves precision and recall compared to simple dictionary matching. For all scenarios, the importance of the underlying terminological basis was also demonstrated. Maximum F1 scores of 0.607, 0.735 and 0.754 (i.e, for top 1, 5 and 10 matches) were achieved through a pipeline including document cleaning, bi-encoder-based term matching based on a large domain dictionary linked to SNOMED CT, and finally re-ranking using RAG.","tags":[],"title":"Large Language Models for Clinical Text Cleansing Enhance Medical Concept Normalization","type":"publication"},{"authors":["Aleksandra Gabryszak","Daniel Röder","Arne Binder","Luca Sion","Leonhard Hennig"],"categories":[],"content":"","date":1726907583,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1726907583,"objectID":"78ee9149556e62dd13d0d9fa63804b9b","permalink":"https://dfki-nlp.github.io/publication/gabryszak-etal-2024-enhancing/","publishdate":"2024-09-21T10:33:03+02:00","relpermalink":"/publication/gabryszak-etal-2024-enhancing/","section":"publication","summary":"In this paper, we investigate the use of large language models (LLMs) to enhance the editorial process of rewriting customer help pages. We introduce a German-language dataset comprising Frequently Asked Question-Answer pairs, presenting both raw drafts and their revisions by professional editors. On this dataset, we evaluate the performance of four large language models (LLM) through diverse prompts tailored for the rewriting task. We conduct automatic evaluations of content and text quality using ROUGE, BERTScore, and ChatGPT. Furthermore, we let professional editors assess the helpfulness of automatically generated FAQ revisions for editorial enhancement. Our findings indicate that LLMs can produce FAQ reformulations beneficial to the editorial process. We observe minimal performance discrepancies among LLMs for this task, and our survey on helpfulness underscores the subjective nature of editors' perspectives on editorial refinement.","tags":[],"title":"Enhancing Editorial Tasks: A Case Study on Rewriting Customer Help Page Contents Using Large Language Models","type":"publication"},{"authors":null,"categories":null,"content":"","date":1726617600,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1726617600,"objectID":"18dcd7d0f0a813f4fcfa369cee2f0f78","permalink":"https://dfki-nlp.github.io/dataset/faq-rewrites/","publishdate":"2024-09-18T00:00:00Z","relpermalink":"/dataset/faq-rewrites/","section":"dataset","summary":"We introduce a German-language dataset comprising Frequently Asked Question-Answer pairs: raw FAQ drafts, their revisions by professional editors and LLM generated revisions. The data was used to investigate the use of large language models (LLMs) to enhance the editorial process of rewriting customer help pages. The corpus comprises 56 question-answer pairs addressing potential customer inquiries across various topics. For each FAQ pair, a raw input is provided by specialized departments, and a rewritten gold output is crafted by a professional editor of Deutsche Telekom. The final dataset also includes LLM generated FAQ-pairs. Please see our [paper](https://aclanthology.org/2024.inlg-main.13/) accepted at INLG 20204, Tokyo, Japan. You can find the Github repo containing the dataset here [https://github.com/DFKI-NLP/faq-rewrites-llms](https://github.com/DFKI-NLP/faq-rewrites-llms).","tags":null,"title":"LLM-based FAQ Rewrites","type":"dataset"},{"authors":["Maximilian Bleick","Nils Feldhus","Aljoscha Burchardt","Sebastian Möller"],"categories":[],"content":"","date":1725784383,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1725784383,"objectID":"623b830ca923eb72a8af042acb36a763","permalink":"https://dfki-nlp.github.io/publication/bleick-etal-2024-german/","publishdate":"2024-09-08T10:33:03+02:00","relpermalink":"/publication/bleick-etal-2024-german/","section":"publication","summary":"We investigate the impact of LLMs on political discourse with a particular focus on the influence of generated personas on model responses. We find an echo chamber effect from LLM chatbots when provided with German-language biographical information of politicians and voters in German politics, leading to sycophantic responses and the reinforcement of existing political biases. Findings reveal that personas of certain political party, such as those of the 'Alternative für Deutschland' party, exert a stronger influence on LLMs, potentially amplifying extremist views. Unlike prior studies, we cannot corroborate a tendency for larger models to exert stronger sycophantic behaviour. We propose that further development should aim at reducing sycophantic behaviour in LLMs across all sizes and diversifying language capabilities in LLMs to enhance inclusivity.","tags":[],"title":"German Voter Personas can Radicalize LLM Chatbots via the Echo Chamber Effect","type":"publication"},{"authors":[],"categories":[],"content":"Two papers from DFKI NLP researchers have been accepted at the 17th International Natural Language Generation Conference (INLG 2024) that will take place September 23-27 in Tokyo, Japan. One paper presents a case study on using large language models to produce customer-friendly help page contents from more technical text, and includes a text quality evaluation by experienced editors. The other paper analyzes echo chamber effects in LLM-based chatbots in political conversations.\n Aleksandra Gabryszak, Daniel Röder, Arne Binder, Luca Sion, Leonhard Hennig (2024). Enhancing Editorial Tasks: A Case Study on Rewriting Customer Help Page Contents Using Large Language Models. INLG 2024. PDF Cite Dataset Project Maximilian Bleick, Nils Feldhus, Aljoscha Burchardt, Sebastian Möller (2024). German Voter Personas can Radicalize LLM Chatbots via the Echo Chamber Effect. INLG 2024. PDF Cite Code Dataset Project DOI ","date":1724311441,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1724311441,"objectID":"d1cb844ddbb79c8c8a9844aaabdad93a","permalink":"https://dfki-nlp.github.io/post/inlg2024/","publishdate":"2024-08-22T09:24:01+02:00","relpermalink":"/post/inlg2024/","section":"post","summary":"Two papers from DFKI NLP researchers have been accepted at the 17th International Natural Language Generation Conference (INLG 2024) that will take place September 23-27 in Tokyo, Japan. One paper presents a case study on using large language models to produce customer-friendly help page contents from more technical text, and includes a text quality evaluation by experienced editors.","tags":[],"title":"Two papers accepted to INLG 2024","type":"post"},{"authors":["Arne Binder","Tatiana Anikina","Leonhard Hennig","Simon Ostermann"],"categories":[],"content":"","date":172368e4,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":172368e4,"objectID":"a65105095b7518cd6949b90ddc3f0f13","permalink":"https://dfki-nlp.github.io/publication/binder-etal-2024-dfki/","publishdate":"2024-08-15T00:00:00Z","relpermalink":"/publication/binder-etal-2024-dfki/","section":"publication","summary":"This paper presents the dfki-mlst submission for the DialAM shared task (Ruiz-Dolz et al., 2024) on identification of argumentative and illocutionary relations in dialogue. Our model achieves best results in the global setting: 48.25 F1 at the focused level when looking only at the related arguments/locutions and 67.05 F1 at the general level when evaluating the complete argument maps. We describe our implementation of the data pre-processing, relation encoding and classification, evaluating 11 different base models and performing experiments with, e.g., node text combination and data augmentation. Our source code is publicly available.","tags":[],"title":"DFKI-MLST at DialAM-2024 Shared Task: System Description","type":"publication"},{"authors":["Leonhard Hennig"],"categories":[],"content":"","date":1722507391,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1722507391,"objectID":"24c3bb0906e4acd5db28286d2b543dc3","permalink":"https://dfki-nlp.github.io/project/trails/","publishdate":"2024-08-01T11:16:31+01:00","relpermalink":"/project/trails/","section":"project","summary":"Natural language processing (NLP) has demonstrated impressive performance in some human tasks. To achieve such performance, current neural models need to be pre-trained on huge amounts of raw text data. This dependence on uncurated data has at least four indirect and unintended consequences: 1) Uncurated data tends to be linguistically and culturally non-diverse due to the statistical dominance of major languages and dialects in online texts (English vs. North Frisian, US English vs. UK English, etc.). 2) Pre-trained neural models such as the ubiquitous pre-trained language models (PLM) reproduce the features present in the data, including human biases. 3) Rare phenomena (or languages) in the 'long tail' are often not sufficiently taken into account in model evaluation, leading to an underestimation of model performance, especially in real-world application scenarios. 4) The focus on achieving state-of-the-art results through the use of transfer learning with giant PLMs such as GPT4 or mT5 often underestimates alternative methods that are more accessible, efficient and sustainable.\nAs inclusion and trust are undermined by these problems, in TRAILS we focus on three main research directions to address such problems: (i) inclusion of underrepresented languages and cultures through multilingual and culturally sensitive NLP, (ii) robustness and fairness with respect to long-tail phenomena and classes and 'trustworthy content', and (iii) robust and efficient NLP models that enable training and deployment of models for (i) and (ii). We also partially address economic inequality by aiming for more efficient models (objective (iii)), which directly translates into a lower resource/cost footprint.","tags":["Bias","Evaluation","Large Language Models"],"title":"TRAILS - Trustworthy and Inclusive Machines","type":"project"},{"authors":[],"categories":[],"content":"DFKI will have a strong presence at the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024), one of the top venues of language technology, that will take place August 11-16 in Bangkok, Thailand. Researchers from Speech and Language Technology department will present 6 papers. The papers appeared in the main conference (Findings) as well as 3 co-located events: The 11th Workshop on Argument Mining, the BioNLP 2024 and Shared Tasks Workshop, and the Towards Knowledgeable Language Models Workshop.\nThe participation in the conference was supported by current research projects, as presentation of their recent results. Some of these projects were: TRAILS (BMBF), KEEPHA (BMBF), and XAINES (BMBF).\nThe DFKI papers presented at the conference are the following:\n Yuxuan Chen, Daniel Röder, Justus-Jonas Erker, Leonhard Hennig, Philippe Thomas, Sebastian Möller, Roland Roller (2024). Retrieval-Augmented Knowledge Integration into Language Models: A Survey. KnowledgeLM2024. PDF Cite Arne Binder, Tatiana Anikina, Leonhard Hennig, Simon Ostermann (2024). DFKI-MLST at DialAM-2024 Shared Task: System Description. ArgMining 2024. PDF Cite Code Project Dorothea MacPhail, David Harbecke, Lisa Raithel, Sebastian Möller (2024). Evaluating the Robustness of Adverse Drug Event Classification Models Using Templates. BioNLP 2024. PDF Cite Code Project Martin Courtois, Malte Ostendorff, Leonhard Hennig, Georg Rehm (2024). Symmetric Dot-Product Attention for Efficient Training of BERT Language Models. Findings 2024. PDF Cite Code Project Faraz Maschhur, Klaus Netter, Sven Schmeier, Katrin Ostermann, Rimantas Palunis, Tobias Strapatsas, Roland Roller (2024). Towards ML-supported Triage Prediction in Real-World Emergency Room Scenarios. BioNLP 2024. Cite Ajay Madhavan Ravichandran, Julianna Grune, Nils Feldhus, Aljoscha Burchardt, Sebastian Möller, Roland Roller (2024). XAI for Better Exploitation of Text in Medical Decision Support. BioNLP 2024. Cite ","date":1720769041,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1720769041,"objectID":"69b8354c5da150ae592c474736581fa2","permalink":"https://dfki-nlp.github.io/post/acl2024/","publishdate":"2024-07-12T09:24:01+02:00","relpermalink":"/post/acl2024/","section":"post","summary":"DFKI will have a strong presence at the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024), one of the top venues of language technology, that will take place August 11-16 in Bangkok, Thailand.","tags":[],"title":"Multiple papers by DFKI authors accepted to ACL 2024 and co-located events","type":"post"},{"authors":["Dorothea MacPhail","David Harbecke","Lisa Raithel","Sebastian Möller"],"categories":[],"content":"","date":1720742400,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1720742400,"objectID":"fdc2da6b0dba224e9bd38917cfc8ea71","permalink":"https://dfki-nlp.github.io/publication/acl2024-evaluating-macphail/","publishdate":"2024-07-12T00:00:00Z","relpermalink":"/publication/acl2024-evaluating-macphail/","section":"publication","summary":"An adverse drug effect (ADE) is any harmful event resulting from medical drug treatment. Despite their importance, ADEs are often under-reported in official channels. Some research has therefore turned to detecting discussions of ADEs in social media. Impressive results have been achieved in various attempts to detect ADEs. In a high-stakes domain such as medicine, however, an in-depth evaluation of a model's abilities is crucial. We address the issue of thorough performance evaluation in English-language ADE detection with hand-crafted templates for four capabilities: Temporal order, negation, sentiment, and beneficial effect. We find that models with similar performance on held-out test sets have varying results on these capabilities.","tags":[],"title":"Evaluating the Robustness of Adverse Drug Event Classification Models Using Templates","type":"publication"},{"authors":["Martin Courtois","Malte Ostendorff","Leonhard Hennig","Georg Rehm"],"categories":[],"content":"","date":1720742400,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1720742400,"objectID":"5a3a2f0b2356655d730f7f29362947ae","permalink":"https://dfki-nlp.github.io/publication/acl2024-symmetric-courtois/","publishdate":"2024-07-12T00:00:00Z","relpermalink":"/publication/acl2024-symmetric-courtois/","section":"publication","summary":"Initially introduced as a machine translation model, the Transformer architecture has now become the foundation for modern deep learning architecture, with applications in a wide range of fields, from computer vision to natural language processing. Nowadays, to tackle increasingly more complex tasks, Transformer-based models are stretched to enormous sizes, requiring increasingly larger training datasets, and unsustainable amount of compute resources. The ubiquitous nature of the Transformer and its core component, the attention mechanism, are thus prime targets for efficiency research. In this work, we propose an alternative compatibility function for the self-attention mechanism introduced by the Transformer architecture. This compatibility function exploits an overlap in the learned representation of the traditional scaled dot-product attention, leading to a symmetric with pairwise coefficient dot-product attention. When applied to the pre-training of BERT-like models, this new symmetric attention mechanism reaches a score of 79.36 on the GLUE benchmark against 78.74 for the traditional implementation, leads to a reduction of 6% in the number of trainable parameters, and reduces the number of training steps required before convergence by half.","tags":[],"title":"Symmetric Dot-Product Attention for Efficient Training of BERT Language Models","type":"publication"},{"authors":["Faraz Maschhur","Klaus Netter","Sven Schmeier","Katrin Ostermann","Rimantas Palunis","Tobias Strapatsas","Roland Roller"],"categories":[],"content":"","date":1720742400,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1720742400,"objectID":"a8bbc17aa0331a8e581bd199f28c7539","permalink":"https://dfki-nlp.github.io/publication/acl2024-towards-maschhur/","publishdate":"2024-07-12T00:00:00Z","relpermalink":"/publication/acl2024-towards-maschhur/","section":"publication","summary":"","tags":[],"title":"Towards ML-supported Triage Prediction in Real-World Emergency Room Scenarios","type":"publication"},{"authors":["Ajay Madhavan Ravichandran","Julianna Grune","Nils Feldhus","Aljoscha Burchardt","Sebastian Moller","Roland Roller"],"categories":[],"content":"","date":1720742400,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1720742400,"objectID":"03090be3e7ed9f3e1284cb6e6630e817","permalink":"https://dfki-nlp.github.io/publication/acl2024-xai-ravichandran/","publishdate":"2024-07-12T00:00:00Z","relpermalink":"/publication/acl2024-xai-ravichandran/","section":"publication","summary":"","tags":[],"title":"XAI for Better Exploitation of Text in Medical Decision Support","type":"publication"},{"authors":[],"categories":[],"content":"One paper by researchers from the Speech and Language Technology department of DFKI will be presented at the 18th International Workshop on Semantic Evaluation, co-located with NAACL 2024:\n Bhuvanesh Verma, Lisa Raithel (2024). DFKI-NLP at SemEval-2024 Task 2: Towards Robust LLMs Using Data Perturbations and MinMax Training. SemEval 2024. PDF Cite ","date":1718177041,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1718177041,"objectID":"467a68aea8a6f4e685863007687839c3","permalink":"https://dfki-nlp.github.io/post/semeval2024/","publishdate":"2024-06-12T09:24:01+02:00","relpermalink":"/post/semeval2024/","section":"post","summary":"One paper by researchers from the Speech and Language Technology department of DFKI will be presented at the 18th International Workshop on Semantic Evaluation, co-located with NAACL 2024:\n Bhuvanesh Verma, Lisa Raithel (2024).","tags":[],"title":"One paper by DFKI authors accepted to SemEval-2024","type":"post"},{"authors":["Bhuvanesh Verma","Lisa Raithel"],"categories":[],"content":"","date":1717977600,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1717977600,"objectID":"f96c13727064b9a6a645174943d09bed","permalink":"https://dfki-nlp.github.io/publication/semeval2024-verma/","publishdate":"2024-06-10T00:00:00Z","relpermalink":"/publication/semeval2024-verma/","section":"publication","summary":"The NLI4CT task at SemEval-2024 emphasizes the development of robust models for Natural Language Inference on Clinical Trial Reports (CTRs) using large language models (LLMs). This edition introduces interventions specifically targeting the numerical, vocabulary, and semantic aspects of CTRs. Our proposed system harnesses the capabilities of the state-of-the-art Mistral model (Jiang et al., 2023), complemented by an auxiliary model, to focus on the intricate input space of the NLI4CT dataset. Through the incorporation of numerical and acronym-based perturbations to the data, we train a robust system capable of handling both semantic-altering and numerical contradiction interventions. Our analysis on the dataset sheds light on the challenging sections of the CTRs for reasoning.","tags":[],"title":"DFKI-NLP at SemEval-2024 Task 2: Towards Robust LLMs Using Data Perturbations and MinMax Training","type":"publication"},{"authors":["Qianli Wang","Tatiana Anikina","Nils Feldhus","Josef van Genabith","Leonhard Hennig","Sebastian Möller"],"categories":[],"content":"","date":1716193983,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1716193983,"objectID":"99bceffd936427b8b8e8bcf5f83d367c","permalink":"https://dfki-nlp.github.io/publication/hcinlp24-wang-llmcheckup/","publishdate":"2024-05-20T10:33:03+02:00","relpermalink":"/publication/hcinlp24-wang-llmcheckup/","section":"publication","summary":"Interpretability tools that offer explanations in the form of a dialogue have demonstrated their efficacy in enhancing users' understanding (Slack et al., 2023; Shen et al., 2023), as one-off explanations may fall short in providing sufficient information to the user. Current solutions for dialogue-based explanations, however, often require external tools and modules and are not easily transferable to tasks they were not designed for. With LLMCheckup, we present an easily accessible tool that allows users to chat with any state-of-the-art large language model (LLM) about its behavior. We enable LLMs to generate explanations and perform user intent recognition without fine-tuning, by connecting them with a broad spectrum of Explainable AI (XAI) methods, including white-box explainability tools such as feature attributions, and self-explanations (e.g., for rationale generation). LLM-based (self-)explanations are presented as an interactive dialogue that supports follow-up questions and generates suggestions. LLMCheckup provides tutorials for operations available in the system, catering to individuals with varying levels of expertise in XAI and supporting multiple input modalities. We introduce a new parsing strategy that substantially enhances the user intent recognition accuracy of the LLM. Finally, we showcase LLMCheckup for the tasks of fact checking and commonsense question answering.","tags":[],"title":"LLMCheckup: Conversational Examination of Large Language Models via Interpretability Tools and Self-Explanations","type":"publication"},{"authors":["Yuxuan Chen","Daniel Röder","Justus-Jonas Erker","Leonhard Hennig","Philippe Thomas","Sebastian Möller","Roland Roller"],"categories":[],"content":"","date":1715817600,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1715817600,"objectID":"3d4e499fd5ccd1a4522be502c4b78abd","permalink":"https://dfki-nlp.github.io/publication/acl2024-raki-chen/","publishdate":"2024-05-16T00:00:00Z","relpermalink":"/publication/acl2024-raki-chen/","section":"publication","summary":"","tags":[],"title":"Retrieval-Augmented Knowledge Integration into Language Models: A Survey","type":"publication"},{"authors":["Tomohiro Nishiyama","Lisa Raithel","Roland Roller","Pierre Zweigenbaum","Eiji Aramaki"],"categories":[],"content":"","date":1710923583,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1710923583,"objectID":"2908ce950332e2c1a90ff5f3c8245000","permalink":"https://dfki-nlp.github.io/publication/cald-eacl24-nishiyama-assessing/","publishdate":"2024-03-23T10:33:03+02:00","relpermalink":"/publication/cald-eacl24-nishiyama-assessing/","section":"publication","summary":"Since medical text cannot be shared easily due to privacy concerns, synthetic data bears much potential for natural language processing applications. In the context of social media and user-generated messages about drug intake and adverse drug effects, this work presents different methods to examine the authenticity of synthetic text. We conclude that the generated tweets are untraceable and show enough authenticity from the medical point of view to be used as a replacement for a real Twitter corpus. However, original data might still be the preferred choice as they contain much more diversity.","tags":[],"title":"Assessing Authenticity and Anonymity of Synthetic User-generated Content in the Medical Domain","type":"publication"},{"authors":["Nils Feldhus","Qianli Wang","Tatiana Anikina","Sahil Chopra","Cennet Oguz","Sebastian Möller"],"categories":[],"content":"","date":1701820800,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1701820800,"objectID":"ade58ad3c98ec74e683c702490dd0759","permalink":"https://dfki-nlp.github.io/publication/emnlp2023-feldhus-interrolang/","publishdate":"2023-10-15T00:00:00Z","relpermalink":"/publication/emnlp2023-feldhus-interrolang/","section":"publication","summary":"While recently developed NLP explainability methods let us open the black box in various ways (Madsen et al., 2022), a missing ingredient in this endeavor is an interactive tool offering a conversational interface. Such a dialogue system can help users explore datasets and models with explanations in a contextualized manner, e.g. via clarification or follow-up questions, and through a natural language interface. We adapt the conversational explanation framework TalkToModel (Slack et al., 2022) to the NLP domain, add new NLP-specific operations such as free-text rationalization, and illustrate its generalizability on three NLP tasks (dialogue act classification, question answering, hate speech detection). To recognize user queries for explanations, we evaluate fine-tuned and few-shot prompting models and implement a novel adapter-based approach. We then conduct two user studies on (1) the perceived correctness and helpfulness of the dialogues, and (2) the simulatability, i.e. how objectively helpful dialogical explanations are for humans in figuring out the model's predicted label when it's not shown. We found rationalization and feature attribution were helpful in explaining the model behavior. Moreover, users could more reliably predict the model outcome based on an explanation dialogue rather than one-off explanations.","tags":["Explainability"],"title":"InterroLang: Exploring NLP Models and Datasets through Dialogue-based Explanations","type":"publication"},{"authors":[],"categories":[],"content":"DFKI had a strong presence at the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP 2023), one of the top venues of language technology, that took place on December 6-10 in Singapore. The research center was represented by the departments MLT, SLT, and DRΧ with 11 papers, presented by 10 researchers. The papers appeared in the main conference as well as 3 co-located events: “Conference for Machine Translation” (WMT), “Workshop on analyzing and interpreting neural networks for NLP” (BlackboxNLP), and “Workshop on Computational Models of Reference, Anaphora and Coreference” (CRAC).\nAdditionally, DFKI researchers contributed to the organization of the workshops by participating in the organization committees in 3 shared tasks and particularly the ones on Sign Language Translation, Generic Machine Translation and Machine Translation Metrics. Noteworthy is also the participation of the DFKI researchers in the program committees, where one researcher was an Area Chair in the Semantics track of the main conference, and numerous others contributed with peer-reviewing of submitted papers. The participation in the conference was supported by current research projects, as presentation of their recent results. Some of these projects were: CORA4NLP (BMBF), IMPRESS (INRIA-DFKI), SFB 1102 “Information Density and Linguistic Encoding” (DFG), SocialWear (BMBF), TextQ (DFG) and XAINES (BMBF).\nThe DFKI papers presented at the conference are the following:\n Challenging the State-of-the-art Machine Translation Metrics from a Linguistic Perspective Find-2-Find: Multitask Learning for Anaphora Resolution and Object Localization Findings of the 2023 Conference on Machine Translation (WMT23): LLMs Are Here but Not Quite There Yet Findings of the Second WMT Shared Task on Sign Language Translation (WMT-SLT23) InterroLang: Exploring NLP Models and Datasets through Dialogue-based Explanations Investigating the Encoding of Words in BERT\u0026rsquo;s Neurons using Feature Textualization Linguistically Motivated Evaluation of the 2023 State-of-the-art Machine Translation: Can ChatGPT Outperform NMT? Multilingual Coarse Political Stance Classification of Media. The Editorial Line of a ChatGPT and Bard Newspaper Multilingual coreference resolution: Adapt and Generate Results of WMT23 Metrics Shared Task: Metrics Might Be Guilty but References Are Not Innocent Translating away Translationese without Parallel Data Where exactly does contextualization in a PLM happen? ","date":1697268241,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1697268241,"objectID":"d545350b6724f77c8c18f21a46167a22","permalink":"https://dfki-nlp.github.io/post/emnlp2023/","publishdate":"2023-10-14T09:24:01+02:00","relpermalink":"/post/emnlp2023/","section":"post","summary":"DFKI had a strong presence at the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP 2023), one of the top venues of language technology, that took place on December 6-10 in Singapore.","tags":[],"title":"Multiple papers by DFKI authors accepted to EMNLP 2023 and co-located events","type":"post"},{"authors":[],"categories":[],"content":"One paper from DFKI-NLP researchers has been accepted for publication at KONVENS 2023, the 19th German Conference on Natural Language Processing. The conference will take place in Ingolstadt, Germany, from Sep 18th to Sep 22nd, 2023. The paper presents an approach using machine translation to translate English data to German to train a transformer-based factuality detection model for clinical data, where supervised data is usually very scarce due to its sensitive nature and privacy concerns.\n Mohammed Bin Sumait, Aleksandra Gabryszak, Leonhard Hennig, Roland Roller (2023). Factuality Detection using Machine Translation - a Use Case for German Clinical Text. KONVENS 2023. Cite Project ","date":1692257041,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1692230400,"objectID":"5975d667dbf9d890f02cad51fa67c6ac","permalink":"https://dfki-nlp.github.io/post/konvens2023/","publishdate":"2023-08-17T09:24:01+02:00","relpermalink":"/post/konvens2023/","section":"post","summary":"One paper from DFKI-NLP researchers has been accepted for publication at KONVENS 2023, the 19th German Conference on Natural Language Processing. The conference will take place in Ingolstadt, Germany, from Sep 18th to Sep 22nd, 2023.","tags":[],"title":"1 paper to be presented at KONVENS 2023","type":"post"},{"authors":["Mohammed Bin Sumait","Aleksandra Gabryszak","Leonhard Hennig","Roland Roller"],"categories":[],"content":"","date":1692253983,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1692253983,"objectID":"e5b8f54c9a6e59a63b21d49103b50302","permalink":"https://dfki-nlp.github.io/publication/konvens2023-binsumait-etal-factuality/","publishdate":"2023-08-17T08:33:03+02:00","relpermalink":"/publication/konvens2023-binsumait-etal-factuality/","section":"publication","summary":"Factuality can play an important role when automatically processing clinical text, as it makes a difference if particular symptoms are explicitly not present, possibly present, not mentioned, or affirmed. In most cases, a sufficient number of examples is necessary to handle such phenomena in a supervised machine learning setting. However, as clinical text might contain sensitive information, data cannot be easily shared. In the context of factuality detection, this work presents a simple solution using machine translation to translate English data to German to train a transformer-based factuality detection model.","tags":[],"title":"Factuality Detection using Machine Translation - a Use Case for German Clinical Text","type":"publication"},{"authors":["Vincent Vandeghinste","Mirella De Sisto","Maria Kopf","Davy Van Landuyt Picron","Irene Murtagh","Eleftherios Avramidis","Mathieu De Coster"],"categories":[],"content":"","date":1689897600,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1689897600,"objectID":"e18a0f6dfe5bba3b5c9c9da0cb96bf7d","permalink":"https://dfki-nlp.github.io/publication/vandeghinste-2023-ele-sign/","publishdate":"2023-07-21T00:00:00Z","relpermalink":"/publication/vandeghinste-2023-ele-sign/","section":"publication","summary":"This report on Europe's Sign Languages is part of a series of language deliverables developed within the framework of the European Language Equality (ELE) project. The series seeks to not only delineate the current state of affairs for each European language, but to additionally identify the gaps and factors that hinder further development in research and technology. The survey presented here focuses on the condition of Language Technology (LT) with regard to Europe's Sign Languages, a set of languages often forgotten in the context of European Language Equality. With the rise of the deep learning paradigm in artificial intelligence, sign language technologies become technologically feasible, provided that enough data is available to feed this data-hungry paradigm. It is exactly the quality and quantity of data that is the main bottleneck in development of well performing and useful technologies. In the past, there have been several projects aimed at developing sign language technologies and methodologies that have been deemed of little value by the deaf communities. Co-creation and involvement of deaf communities throughout projects and development of technologies ensures that this does not happen again.","tags":["Machine Translation"],"title":"European Language Equality, Report on Europe's Sign Languages","type":"publication"},{"authors":["Gabriele Sarti","Nils Feldhus","Ludwig Sickert","Oskar van der Wal","Malvina Nissim","Arianna Bisazza"],"categories":[],"content":"","date":1686787200,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1686787200,"objectID":"4720cee2cc9dd56df3c79f707f31c6fd","permalink":"https://dfki-nlp.github.io/publication/acl2023-sarti-inseq/","publishdate":"2023-06-15T00:00:00Z","relpermalink":"/publication/acl2023-sarti-inseq/","section":"publication","summary":"Past work in natural language processing interpretability focused mainly on popular classification tasks while largely overlooking generation settings, partly due to a lack of dedicated tools. In this work, we introduce Inseq, a Python library to democratize access to interpretability analyses of sequence generation models. Inseq enables intuitive and optimized extraction of models' internal information and feature importance scores for popular decoder-only and encoder-decoder Transformers architectures. We showcase its potential by adopting it to highlight gender biases in machine translation models and locate factual knowledge inside GPT-2. Thanks to its extensible interface supporting cutting-edge techniques such as contrastive feature attribution, Inseq can drive future advances in explainable natural language generation, centralizing good practices and enabling fair and reproducible model evaluations.","tags":["Interpretability"],"title":"Inseq: An Interpretability Toolkit for Sequence Generation Models","type":"publication"},{"authors":["Dele Zhu","Vera Czehmann","Eleftherios Avramidis"],"categories":[],"content":"","date":1686787200,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1686787200,"objectID":"0f2680c7f042a0139cfa4dc09fcbe375","permalink":"https://dfki-nlp.github.io/publication/acl2023-zhu-sign/","publishdate":"2023-06-15T00:00:00Z","relpermalink":"/publication/acl2023-zhu-sign/","section":"publication","summary":"State-of-the-art techniques common to low resource Machine Translation (MT) are applied to improve MT of spoken language text to Sign Language (SL) glosses. In our experiments, we improve the performance of the transformer-based models via (1) data augmentation, (2) semi-supervised Neural Machine Translation (NMT), (3) transfer learning and (4) multilingual NMT. The proposed methods are implemented progressively on two German SL corpora containing gloss annotations. Multilingual NMT combined with data augmentation appear to be the most successful setting, yielding statistically significant improvements as measured by three automatic metrics (up to over 6 points BLEU), and confirmed via human evaluation. Our best setting outperforms all previous work that report on the same test-set and is also confirmed on a corpus of the American Sign Language (ASL).","tags":["Machine Translation"],"title":"Neural Machine Translation Methods for Translating Text to Sign Language Glosses","type":"publication"},{"authors":["Nils Feldhus","Leonhard Hennig","Maximilian Dustin Nasert","Christopher Ebert","Robert Schwarzenberg","Sebastian Möller"],"categories":[],"content":"","date":1686787200,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1686787200,"objectID":"86eedcf694f6a4c2143fe8c11145b0c7","permalink":"https://dfki-nlp.github.io/publication/acl2023-feldhus-smv/","publishdate":"2023-06-15T00:00:00Z","relpermalink":"/publication/acl2023-feldhus-smv/","section":"publication","summary":"Saliency maps can explain a neural model's predictions by identifying important input features. They are difficult to interpret for laypeople, especially for instances with many features. In order to make them more accessible, we formalize the underexplored task of translating saliency maps into natural language and compare methods that address two key challenges of this approach -- what and how to verbalize. In both automatic and human evaluation setups, using token-level attributions from text classification tasks, we compare two novel methods (search-based and instruction-based verbalizations) against conventional feature importance representations (heatmap visualizations and extractive rationales), measuring simulatability, faithfulness, helpfulness and ease of understanding. Instructing GPT-3.5 to generate saliency map verbalizations yields plausible explanations which include associations, abstractive summarization and commonsense reasoning, achieving by far the highest human ratings, but they are not faithfully capturing numeric information and are inconsistent in their interpretation of the task. In comparison, our search-based, model-free verbalization approach efficiently completes templated verbalizations, is faithful by design, but falls short in helpfulness and simulatability. Our results suggest that saliency map verbalization makes feature attribution explanations more comprehensible and less cognitively challenging to humans than conventional representations.","tags":["Interpretability"],"title":"Saliency Map Verbalization: Comparing Feature Importance Representations from Model-free and Instruction-based Methods","type":"publication"},{"authors":null,"categories":null,"content":"","date":1684886400,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1684886400,"objectID":"cabf17b70c41b00d24ce53389c9df8a5","permalink":"https://dfki-nlp.github.io/dataset/multitacred/","publishdate":"2023-05-24T00:00:00Z","relpermalink":"/dataset/multitacred/","section":"dataset","summary":"MultiTACRED is a multilingual version of the large-scale [TAC Relation Extraction Dataset](https://nlp.stanford.edu/projects/tacred). It covers 12 typologically diverse languages from 9 language families, and was created by machine-translating the instances of the original TACRED dataset and automatically projecting their entity annotations. For details of the original TACRED's data collection and annotation process, see the [Stanford paper](https://aclanthology.org/D17-1004/). Translations are syntactically validated by checking the correctness of the XML tag markup. Any translations with an invalid tag structure, e.g. missing or invalid head or tail tag pairs, are discarded (on average, 2.3% of the instances).\nLanguages covered are: Arabic, Chinese, Finnish, French, German, Hindi, Hungarian, Japanese, Polish, Russian, Spanish, Turkish. Intended use is supervised relation classification. Audience - researchers.\nThe dataset will be released via the LDC (link will follow).\nPlease see [our ACL paper](https://arxiv.org/abs/2305.04582) for full details. You can find the Github repo containing the translation and experiment code here [https://github.com/DFKI-NLP/MultiTACRED](https://github.com/DFKI-NLP/MultiTACRED).","tags":["Relation Extraction","Multilinguality","Transfer Learning"],"title":"The MultiTACRED dataset","type":"dataset"},{"authors":["Malte Ostendorff","Georg Rehm"],"categories":[],"content":"","date":1684222766,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1684222766,"objectID":"1a46f585e2b7b7c0c0685a90a91c5110","permalink":"https://dfki-nlp.github.io/publication/pml4dc-2023-ostendorff/","publishdate":"2023-05-16T09:39:26+02:00","relpermalink":"/publication/pml4dc-2023-ostendorff/","section":"publication","summary":"Most Transformer language models are primarily pretrained on English text, limiting their use for other languages. As the model sizes grow, the performance gap between English and other languages with fewer compute and data resources increases even further. Consequently, more resource-efficient training methods are needed to bridge the gap for languages with fewer resources available. To address this problem, we introduce a cross-lingual and progressive transfer learning approach, called CLP-Transfer, that transfers models from a source language, for which pretrained models are publicly available, like English, to a new target language. As opposed to prior work, which focused on the cross-lingual transfer between two languages, we extend the transfer to the model size. Given a pretrained model in a source language, we aim for a same-sized model in a target language. Instead of training a model from scratch, we exploit a smaller model that is in the target language but requires much fewer resources. Both small and source models are then used to initialize the token embeddings of the larger model based on the overlapping vocabulary of the source and target language. All remaining weights are reused from the model in the source language. This approach outperforms the sole cross-lingual transfer and can save up to 80% of the training steps compared to the random initialization. ","tags":[],"title":"Efficient Language Model Training through Cross-Lingual and Progressive Transfer Learning","type":"publication"},{"authors":["Leonhard Hennig","Philippe Thomas","Sebastian Möller"],"categories":[],"content":"","date":1683534783,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1683534783,"objectID":"b18b701ca39dc8fc113c349cb912ec16","permalink":"https://dfki-nlp.github.io/publication/acl2023-hennig-multitacred/","publishdate":"2023-05-08T10:33:03+02:00","relpermalink":"/publication/acl2023-hennig-multitacred/","section":"publication","summary":"Relation extraction (RE) is a fundamental task in information extraction, whose extension to multilingual settings has been hindered by the lack of supervised resources comparable in size to large English datasets such as TACRED (Zhang et al., 2017). To address this gap, we introduce the MultiTACRED dataset, covering 12 typologically diverse languages from 9 language families, which is created by machine-translating TACRED instances and automatically projecting their entity annotations. We analyze translation and annotation projection quality, identify error categories, and experimentally evaluate fine-tuned pretrained mono- and multilingual language models in common transfer learning scenarios. Our analyses show that machine translation is a viable strategy to transfer RE instances, with native speakers judging more than 83% of the translated instances to be linguistically and semantically acceptable. We find monolingual RE model performance to be comparable to the English original for many of the target languages, and that multilingual models trained on a combination of English and target language data can outperform their monolingual counterparts. However, we also observe a variety of translation and annotation projection errors, both due to the MT systems and linguistic features of the target languages, such as pronoun-dropping, compounding and inflection, that degrade dataset quality and RE model performance.","tags":[],"title":"MultiTACRED: A Multilingual Version of the TAC Relation Extraction Dataset","type":"publication"},{"authors":["Akhila Abdulnazar","Markus Kreuzthaler","Roland Roller","Stefan Schulz"],"categories":[],"content":"","date":1683534783,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1683534783,"objectID":"50063dc2283e29f4dcc33d54d6467958","permalink":"https://dfki-nlp.github.io/publication/shti2023-abdulnazar-sapbert/","publishdate":"2023-05-08T10:33:03+02:00","relpermalink":"/publication/shti2023-abdulnazar-sapbert/","section":"publication","summary":"Word vector representations, known as embeddings, are commonly used for natural language processing. Particularly, contextualized representations have been very successful recently. In this work, we analyze the impact of contextualized and non-contextualized embeddings for medical concept normalization, mapping clinical terms via a k-NN approach to SNOMED CT. The non-contextualized concept mapping resulted in a much better performance (F1-score = 0.853) than the contextualized representation (F1-score = 0.322).","tags":[],"title":"SapBERT-Based Medical Concept Normalization Using SNOMED CT","type":"publication"},{"authors":["Vageesh Saxena","Nils Rethmeier","Gijs Van Dijck","Gerasimos Spanakis"],"categories":[],"content":"","date":1683534783,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1683534783,"objectID":"162357eac1664c564ac3e6a725d23a9d","permalink":"https://dfki-nlp.github.io/publication/acl2023-rethmeier-vendorlink/","publishdate":"2023-05-08T10:33:03+02:00","relpermalink":"/publication/acl2023-rethmeier-vendorlink/","section":"publication","summary":"The anonymity on the Darknet allows vendors to stay undetected by using multiple vendor aliases or frequently migrating between markets. Consequently, illegal markets and their connections are challenging to uncover on the Darknet. To identify relationships between illegal markets and their vendors, we propose VendorLink, an NLP-based approach that examines writing patterns to verify, identify, and link unique vendor accounts across text advertisements (ads) on seven public Darknet markets. In contrast to existing literature, VendorLink utilizes the strength of supervised pretraining to perform closed-set vendor verification, open-set vendor identification, and low-resource market adaption tasks. Through VendorLink, we uncover (i) 15 migrants and 71 potential aliases in the Alphabay-Dreams-Silk dataset, (ii) 17 migrants and 3 potential aliases in the Valhalla-Berlusconi dataset, and (iii) 75 migrants and 10 potential aliases in the Traderoute-Agora dataset. Altogether, our approach can help Law Enforcement Agencies (LEA) make more informed decisions by verifying and identifying migrating vendors and their potential aliases on existing and Low-Resource (LR) emerging Darknet markets.","tags":[],"title":"VendorLink: An NLP approach for Identifying \u0026 Linking Vendor Migrants \u0026 Potential Aliases on Darknet Markets","type":"publication"},{"authors":[],"categories":[],"content":"Five papers from DFKI-NLP researchers have been accepted for publication at ACL 2023, the 61st Annual Meeting of the Association for Computational Linguistics. The conference is planned to be a hybrid meeting and will take place in Toronto, Canada, from Jul 9th through July 14th, 2023. The first one presents a multilingual version of the TAC relation extraction dataset that covers 12 additional languages beside the original English, the second examines writing patterns to verify, identify, and link unique vendor accounts across text advertisements on public Darknet markets, enabling Law Enforcement Agencies to make more informed decisions. The third is about neural machine translation methods for translating text to sign language glosses, and has received an outstanding paper award. The fourth is an interpretability toolkit for sequence generation models, while the fifth is a comparative study of feature attribution representations, including model-free and instruction-based methods for saliency map verbalization.\n Leonhard Hennig, Philippe Thomas, Sebastian Möller (2023). MultiTACRED: A Multilingual Version of the TAC Relation Extraction Dataset. ACL 2023. PDF Cite Code Dataset Project Project Project DOI Vageesh Saxena, Nils Rethmeier, Gijs Van Dijck, Gerasimos Spanakis (2023). VendorLink: An NLP approach for Identifying \u0026amp; Linking Vendor Migrants \u0026amp; Potential Aliases on Darknet Markets. ACL 2023. PDF Cite Code Project Dele Zhu, Vera Czehmann, Eleftherios Avramidis (2023). Neural Machine Translation Methods for Translating Text to Sign Language Glosses. ACL 2023. PDF Cite Gabriele Sarti, Nils Feldhus, Ludwig Sickert, Oskar van der Wal, Malvina Nissim, Arianna Bisazza (2023). Inseq: An Interpretability Toolkit for Sequence Generation Models. ACL 2023 Demos. PDF Cite Code Nils Feldhus, Leonhard Hennig, Maximilian Dustin Nasert, Christopher Ebert, Robert Schwarzenberg, Sebastian Möller (2023). Saliency Map Verbalization: Comparing Feature Importance Representations from Model-free and Instruction-based Methods. ACL 2023 NLRSE. PDF Cite Code ","date":1683530641,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1686787200,"objectID":"221570ef38422c5369b9e10cac109395","permalink":"https://dfki-nlp.github.io/post/acl2023/","publishdate":"2023-05-08T09:24:01+02:00","relpermalink":"/post/acl2023/","section":"post","summary":"Five papers from DFKI-NLP researchers have been accepted for publication at ACL 2023, the 61st Annual Meeting of the Association for Computational Linguistics. The conference is planned to be a hybrid meeting and will take place in Toronto, Canada, from Jul 9th through July 14th, 2023.","tags":[],"title":"5 papers to be presented at ACL 2023","type":"post"},{"authors":[" David Samhammer","Susanne Beck","Klemens Budde","Aljoscha Burchardt","Michelle Faber","Simon Gerndt","Sebastian Möller","Bilgin Osmanodja","Roland Roller","Peter Dabrock"],"categories":[],"content":"","date":1683016383,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1683016383,"objectID":"2d238f58e9f415a79574aaddca268d79","permalink":"https://dfki-nlp.github.io/publication/samhammer-2023-klinische/","publishdate":"2023-05-26T10:33:03+02:00","relpermalink":"/publication/samhammer-2023-klinische/","section":"publication","summary":"Dieses Open-Access-essential schafft Orientierung, wenn Künstliche Intelligenz im klinischen Alltag eingesetzt wird. Die Herausforderungen werden anhand zweier Beispiele aus dem Bereich der Nephrologie erläutert, die ethisch und rechtlich reflektiert werden. Ein umfangreicher Empfehlungsteil schließt diesen durchweg interdisziplinär erarbeiteten Band ab.","tags":[],"title":"Klinische Entscheidungsfindung mit Künstlicher Intelligenz: Ein interdisziplinärer Governance-Ansatz","type":"publication"},{"authors":["Roland Roller","Aljoscha Burchardt","David Samhammer","Simon Ronicke","Wiebke Duettmann","Sven Schmeier","Sebastian Möller","Peter Dabrock","Klemens Budde","Manuel Mayrdorfer","Bilgin Osmanodja"],"categories":[],"content":"","date":1682238783,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1682238783,"objectID":"fc18ffb4cb00dcf2c1f993205b2ae152","permalink":"https://dfki-nlp.github.io/publication/plosone2023-roller-performance/","publishdate":"2023-04-23T10:33:03+02:00","relpermalink":"/publication/plosone2023-roller-performance/","section":"publication","summary":"Scientific publications about the application of machine learning models in healthcare often focus on improving performance metrics. However, beyond often short-lived improvements, many additional aspects need to be taken into consideration to make sustainable progress. What does it take to implement a clinical decision support system, what makes it usable for the domain experts, and what brings it eventually into practical usage? So far, there has been little research to answer these questions. This work presents a multidisciplinary view of machine learning in medical decision support systems and covers information technology, medical, as well as ethical aspects. The target audience is computer scientists, who plan to do research in a clinical context. The paper starts from a relatively straightforward risk prediction system in the subspecialty nephrology that was evaluated on historic patient data both intrinsically and based on a reader study with medical doctors. Although the results were quite promising, the focus of this article is not on the model itself or potential performance improvements. Instead, we want to let other researchers participate in the lessons we have learned and the insights we have gained when implementing and evaluating our system in a clinical setting within a highly interdisciplinary pilot project in the cooperation of computer scientists, medical doctors, ethicists, and legal experts.","tags":[],"title":"When performance is not enough—A multidisciplinary view on clinical decision support","type":"publication"},{"authors":["Steffen Castle","Leonhard Hennig"],"categories":[],"content":"","date":1680344191,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1680344191,"objectID":"55ec654e3395f87ccf3f3d9f6afb8417","permalink":"https://dfki-nlp.github.io/project/data4transparency/","publishdate":"2023-04-01T11:16:31+01:00","relpermalink":"/project/data4transparency/","section":"project","summary":"According to the World Bank and the UN, some US$1tn is paid in bribes every year. Corrupt financial transactions divert funds from legitimate public services, as well as distort free markets—potentially thwarting economic development—and reduce trust in institutions. The Organized Crime and Corruption Reporting Project (OCCRP) is a global platform for investigative reporting, providing resources to journalists and media centres, enabling cost-effective collaboration between editors and offering tools to secure themselves against threats to independent media. Exposing previously-unknown connections between entities makes it possible for citizens, policymakers, activists and law enforcement agencies to act. As the number of such leaks and publications grows, there is an increasing need for effective, scalable and reproducible methods to discover any anomalies and evidence of malfeasance that might exist within them.","tags":["Information Extraction","Low-Resource Learning"],"title":"Data4Transparency","type":"project"},{"authors":["Davy Weissenbacher","Karen O’Connor","Siddharth Rawal","Yu Zhang","Richard Tzong-Han Tsai","Timothy Miller","Dongfang Xu","Carol Anderson","Bo Liu","Qing Han","Jinfeng Zhang","Igor Kulev","Berkay Köprü","Raul Rodriguez-Esteban","Elif Ozkirimli","Ammer Ayach","Roland Roller","Stephen Piccolo","Peijin Han","V G Vinod Vydiswaran","Ramya Tekumalla","Juan M Banda","Parsa Bagherzadeh","Sabine Bergler","João F Silva","Tiago Almeida","Paloma Martinez","Renzo Rivera-Zavala","Chen-Kai Wang","Hong-Jie Dai","Luis Alberto Robles Hernandez","Graciela Gonzalez-Hernandez"],"categories":[],"content":"","date":1676449983,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1676449983,"objectID":"73acdadd2ba1002e88ce137fddb2c895","permalink":"https://dfki-nlp.github.io/publication/database2023-roller-biocreative/","publishdate":"2023-04-23T10:33:03+02:00","relpermalink":"/publication/database2023-roller-biocreative/","section":"publication","summary":"This study presents the outcomes of the shared task competition BioCreative VII (Task 3) focusing on the extraction of medication names from a Twitter user’s publicly available tweets (the user’s ‘timeline’). In general, detecting health-related tweets is notoriously challenging for natural language processing tools. The main challenge, aside from the informality of the language used, is that people tweet about any and all topics, and most of their tweets are not related to health. Thus, finding those tweets in a user’s timeline that mention specific health-related concepts such as medications requires addressing extreme imbalance. Task 3 called for detecting tweets in a user’s timeline that mentions a medication name and, for each detected mention, extracting its span. The organizers made available a corpus consisting of 182 049 tweets publicly posted by 212 Twitter users with all medication mentions manually annotated. The corpus exhibits the natural distribution of positive tweets, with only 442 tweets (0.2\\%) mentioning a medication. This task was an opportunity for participants to evaluate methods that are robust to class imbalance beyond the simple lexical match. A total of 65 teams registered, and 16 teams submitted a system run. This study summarizes the corpus created by the organizers and the approaches taken by the participating teams for this challenge. The corpus is freely available at https://biocreative.bioinformatics.udel.edu/tasks/biocreative-vii/track-3/. The methods and the results of the competing systems are analyzed with a focus on the approaches taken for learning from class-imbalanced data.","tags":[],"title":"Automatic Extraction of Medication Mentions from Tweets—Overview of the BioCreative VII Shared Task 3 Competition","type":"publication"},{"authors":["David Samhammer","Roland Roller","Patrik Hummel","Bilgin Osmanodja","Aljoscha Burchardt","Manuel Mayrdorfer","Wiebke Duettmann","Peter Dabrock"],"categories":[],"content":"","date":1671525183,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1671525183,"objectID":"982508b5ca2ce30e639b39d3f2a2d4fb","permalink":"https://dfki-nlp.github.io/publication/frontiers2022-roller-nothing/","publishdate":"2023-04-23T10:33:03+02:00","relpermalink":"/publication/frontiers2022-roller-nothing/","section":"publication","summary":"Introduction: Artificial intelligence–driven decision support systems (AI–DSS) have the potential to help physicians analyze data and facilitate the search for a correct diagnosis or suitable intervention. The potential of such systems is often emphasized. However, implementation in clinical practice deserves continuous attention. This article aims to shed light on the needs and challenges arising from the use of AI-DSS from physicians' perspectives. Methods: The basis for this study is a qualitative content analysis of expert interviews with experienced nephrologists after testing an AI-DSS in a straightforward usage scenario. Results: The results provide insights on the basics of clinical decision-making, expected challenges when using AI-DSS as well as a reflection on the test run. Discussion: While we can confirm the somewhat expectable demand for better explainability and control, other insights highlight the need to uphold classical strengths of the medical profession when using AI-DSS as well as the importance of broadening the view of AI-related challenges to the clinical environment, especially during treatment. Our results stress the necessity for adjusting AI-DSS to shared decision-making. We conclude that explainability must be context-specific while fostering meaningful interaction with the systems available.","tags":[],"title":"''Nothing works without the doctor:'' Physicians' perception of clinical decision-making and artificial intelligence","type":"publication"},{"authors":["Mariana Neves","Antonio Jimeno Yepes","Amy Siu","Roland Roller","Philippe Thomas","Maika Vicente Navarro","Lana Yeganova","Dina Wiemann","Giorgio Maria Di Nunzio","Federica Vezzani","Christel Gerardin","Rachel Bawden","Darryl Johan Estrada","Salvador Lima-lopez","Eulalia Farre-maduel","Martin Krallinger","Cristian Grozea","Aurelie Neveol"],"categories":[],"content":"","date":1671525183,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1671525183,"objectID":"bb0a3f814cf91d3fafe1ed03a5c8b11e","permalink":"https://dfki-nlp.github.io/publication/wmt-emnlp2022-roller-findings/","publishdate":"2023-04-23T10:33:03+02:00","relpermalink":"/publication/wmt-emnlp2022-roller-findings/","section":"publication","summary":"In the seventh edition of the WMT Biomedical Task, we addressed a total of seven languagepairs, namely English/German, English/French, English/Spanish, English/Portuguese, English/Chinese, English/Russian, English/Italian. This year{'}s test sets covered three types of biomedical text genre. In addition to scientific abstracts and terminology items used in previous editions, we released test sets of clinical cases. The evaluation of clinical cases translations were given special attention by involving clinicians in the preparation of reference translations and manual evaluation. For the main MEDLINE test sets, we received a total of 609 submissions from 37 teams. For the ClinSpEn sub-task, we had the participation of five teams.","tags":[],"title":"Findings of the WMT 2022 Biomedical Translation Shared Task: Monolingual Clinical Case Reports","type":"publication"},{"authors":["Roland Roller","Manuel Mayrdorfer","Wiebke Duettmann","Marcel G. Naik","Danilo Schmidt","Fabian Halleck","Patrik Hummel","Aljoscha Burchardt","Sebastian Möller","Peter Dabrock","Bilgin Osmanodja","Klemens Budde"],"categories":[],"content":"","date":1666686783,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1666686783,"objectID":"b8e2adf7e4b9059716bd83bf49ca8994","permalink":"https://dfki-nlp.github.io/publication/frontiers2022-roller-evaluation/","publishdate":"2023-04-23T10:33:03+02:00","relpermalink":"/publication/frontiers2022-roller-evaluation/","section":"publication","summary":"Patient care after kidney transplantation requires integration of complex information to make informed decisions on risk constellations. Many machine learning models have been developed for detecting patient outcomes in the past years. However, performance metrics alone do not determine practical utility. We present a newly developed clinical decision support system (CDSS) for detection of patients at risk for rejection and death-censored graft failure. The CDSS is based on clinical routine data including 1,516 kidney transplant recipients and more than 100,000 data points. In a reader study we compare the performance of physicians at a nephrology department with and without the CDSS. Internal validation shows AUC-ROC scores of 0.83 for rejection, and 0.95 for graft failure. The reader study shows that predictions by physicians converge toward the CDSS. However, performance does not improve (AUC–ROC; 0.6413 vs. 0.6314 for rejection; 0.8072 vs. 0.7778 for graft failure). Finally, the study shows that the CDSS detects partially different patients at risk compared to physicians. This indicates that the combination of both, medical professionals and a CDSS might help detect more patients at risk for graft failure. However, the question of how to integrate such a system efficiently into clinical practice remains open.","tags":[],"title":"Evaluation of a clinical decision support system for detection of patients at risk after kidney transplantation","type":"publication"},{"authors":["Yuxuan Chen","David Harbecke","Leonhard Hennig"],"categories":[],"content":"","date":1666427583,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1666427583,"objectID":"4c041d8a26606010a045082e8f180e45","permalink":"https://dfki-nlp.github.io/publication/emnlp2022-chen-meffiprompt/","publishdate":"2022-10-22T10:33:03+02:00","relpermalink":"/publication/emnlp2022-chen-meffiprompt/","section":"publication","summary":"Prompting pre-trained language models has achieved impressive performance on various NLP tasks, especially in low data regimes. Despite the success of prompting in monolingual settings, applying prompt-based methods in multilingual scenarios has been limited to a narrow set of tasks, due to the high cost of handcrafting multilingual prompts. In this paper, we present the first work on prompt-based multilingual relation classification (RC), by introducing an efficient and effective method that constructs prompts from relation triples and involves only minimal translation for the class labels. We evaluate its performance in fully supervised, few-shot and zero-shot scenarios, and analyze its effectiveness across 14 languages, prompt variants, and English-task training in cross-lingual settings. We find that in both fully supervised and few-shot scenarios, our prompt method beats competitive baselines: fine-tuning XLM-R-EM and null prompts. It also outperforms the random baseline by a large margin in zero-shot experiments. Our method requires little in-language knowledge and can be used as a strong baseline for similar multilingual classification tasks.","tags":[],"title":"Multilingual Relation Classification via Efficient and Effective Prompting","type":"publication"},{"authors":["Arne Binder","Bhuvanesh Verma","Leonhard Hennig"],"categories":[],"content":"","date":1666310400,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1666310400,"objectID":"4bb3720e75628da71f90ed6d4ddf47f7","permalink":"https://dfki-nlp.github.io/publication/wiesp2022-binder-etal-full/","publishdate":"2022-10-21T00:00:00Z","relpermalink":"/publication/wiesp2022-binder-etal-full/","section":"publication","summary":"Scholarly Argumentation Mining (SAM) has recently gained attention due to its potential to help scholars with the rapid growth of published scientific literature. It comprises two subtasks: argumentative discourse unit recognition (ADUR) and argumentative relation extraction (ARE), both of which are challenging since they require e.g. the integration of domain knowledge, the detection of implicit statements, and the disambiguation of argument structure. While previous work focused on dataset construction and baseline methods for specific document sections, such as abstract or results, full-text scholarly argumentation mining has seen little progress. In this work, we introduce a sequential pipeline model combining ADUR and ARE for full-text SAM, and provide a first analysis of the performance of pretrained language models (PLMs) on both subtasks. We establish a new SotA for ADUR on the Sci-Arg corpus, outperforming the previous best reported result by a large margin (+7% F1). We also present the first results for ARE, and thus for the full AM pipeline, on this benchmark dataset. Our detailed error analysis reveals that non-contiguous ADUs as well as the interpretation of discourse connectors pose major challenges and that data annotation needs to be more consistent.","tags":[],"title":"Full-Text Argumentation Mining on Scientific Publications","type":"publication"},{"authors":["Malte Ostendorff","Nils Rethmeier","Isabelle Augenstein","Bela Gipp","Georg Rehm"],"categories":[],"content":"","date":1665736383,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1665736383,"objectID":"db6ae7985024e062c2e16cc599a3472b","permalink":"https://dfki-nlp.github.io/publication/emnlp2022-ostendorff/","publishdate":"2022-10-14T10:33:03+02:00","relpermalink":"/publication/emnlp2022-ostendorff/","section":"publication","summary":"Learning scientific document representations can be substantially improved through contrastive learning objectives, where the challenge lies in creating positive and negative training samples that encode the desired similarity semantics. Prior work relies on discrete citation relations to generate contrast samples. However, discrete citations enforce a hard cut-off to similarity. This is counter-intuitive to similarity-based learning, and ignores that scientific papers can be very similar despite lacking a direct citation - a core problem of finding related research. Instead, we use controlled nearest neighbor sampling over citation graph embeddings for contrastive learning. This control allows us to learn continuous similarity, to sample hard-to-learn negatives and positives, and also to avoid collisions between negative and positive samples by controlling the sampling margin between them. The resulting method SciNCL outperforms the state-of-the-art on the SciDocs benchmark. Furthermore, we demonstrate that it can train (or tune) models sample-efficiently, and that it can be combined with recent training-efficient methods. Perhaps surprisingly, even training a general-domain language model this way outperforms baselines pretrained in-domain. ","tags":[],"title":"Neighborhood Contrastive Learning for Scientific Document Representations with Citation Embeddings","type":"publication"},{"authors":[],"categories":[],"content":"Two papers from DFKI-NLP authors have been accepted for publication at EMNLP 2022, the 2022 Conference on Empirical Methods in Natural Language Processing. The conference is planned to be a hybrid meeting and will take place in Abu Dhabi, from Dec 7th to Dec 11th, 2022. The first paper introduces an efficient and effective method that constructs prompts from relation triples and involves only minimal translation for the class labels, in the case of in-language prompting. We evaluate its performance in fully supervised, few-shot and zero-shot scenarios across 14 languages, soft prompt variants, and English-task training in cross-lingual settings. The second paper proposes neighborhood contrastive learning for the representation learning of scientific document and achieves new state-of-the-art results on the SciDocs benchmark.\n Yuxuan Chen, David Harbecke, Leonhard Hennig (2022). Multilingual Relation Classification via Efficient and Effective Prompting. EMNLP 2022. PDF Cite Code Project DOI Malte Ostendorff, Nils Rethmeier, Isabelle Augenstein, Bela Gipp, Georg Rehm (2022). Neighborhood Contrastive Learning for Scientific Document Representations with Citation Embeddings. EMNLP 2022. PDF Cite Code DOI ","date":1665732241,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1665732241,"objectID":"3db51a86704299a94615128c8e63eba2","permalink":"https://dfki-nlp.github.io/post/emnlp2022/","publishdate":"2022-10-14T09:24:01+02:00","relpermalink":"/post/emnlp2022/","section":"post","summary":"Two papers from DFKI-NLP authors have been accepted for publication at EMNLP 2022, the 2022 Conference on Empirical Methods in Natural Language Processing. The conference is planned to be a hybrid meeting and will take place in Abu Dhabi, from Dec 7th to Dec 11th, 2022.","tags":[],"title":"2 papers to be presented at EMNLP 2022","type":"post"},{"authors":[],"categories":[],"content":"One paper from DFKI-NLP authors has been accepted for publication at the Workshop on Information Extraction from Scientific Publications (WIESP). The workshop will be held at AACL-IJCNLP 2022, the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing, which will take place as an online-only event from Nov. 20 to Nov. 23, 2022. The paper proposes a new method for argument mining in full-text scientific documents by combining argumentative discourse unit recognition with relation extraction.\n Arne Binder, Bhuvanesh Verma, Leonhard Hennig (2022). Full-Text Argumentation Mining on Scientific Publications. WIESP 2022. PDF Cite Code Project ","date":1665728641,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1665728641,"objectID":"5ce70b7b52cd23896270f5442726b45e","permalink":"https://dfki-nlp.github.io/post/aacl-ijcnlp2022/","publishdate":"2022-10-14T08:24:01+02:00","relpermalink":"/post/aacl-ijcnlp2022/","section":"post","summary":"One paper from DFKI-NLP authors has been accepted for publication at the Workshop on Information Extraction from Scientific Publications (WIESP). The workshop will be held at AACL-IJCNLP 2022, the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing, which will take place as an online-only event from Nov.","tags":[],"title":"1 paper to be presented at AACL-IJCNLP 2022","type":"post"},{"authors":["Vivien Macketanz","Eleftherios Avramidis","Aljoscha Burchardt","He Wang","Renlong Ai","Shushen Manakhimova","Ursula Strohriegel","Sebastian Möller","Hans Uszkoreit"],"categories":[],"content":"","date":1661173944,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1661173944,"objectID":"fb56f0308045a6d58b32cc99c3de31e1","permalink":"https://dfki-nlp.github.io/publication/lrec2022-macketanz-testsuite/","publishdate":"2022-08-22T15:12:24+02:00","relpermalink":"/publication/lrec2022-macketanz-testsuite/","section":"publication","summary":"This paper presents a fine-grained test suite for the language pair German–English. The test suite is based on a number of linguistically motivated categories and phenomena and the semi-automatic evaluation is carried out with regular expressions. We describe the creation and implementation of the test suite in detail, providing a full list of all categories and phenomena. Furthermore, we present various exemplary applications of our test suite that have been implemented in the past years, like contributions to the Conference of Machine Translation, the usage of the test suite and MT outputs for quality estimation, and the expansion of the test suite to the language pair Portuguese–English. We describe how we tracked the development of the performance of various systems MT systems over the years with the help of the test suite and which categories and phenomena are prone to resulting in MT errors. For the first time, we also make a large part of our test suite publicly available to the research community.","tags":["machine translation","linguistic test suite"],"title":"A Linguistically Motivated Test Suite to Semi-Automatically Evaluate German–English Machine Translation Output","type":"publication"},{"authors":["Roland Roller","Aljoscha Burchardt","Nils Feldhus","Laura Seiffe","Klemens Budde","Simon Ronicke","Bilgin Osmanodja"],"categories":[],"content":"","date":1660309944,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1660309944,"objectID":"5eff43066a7b70ed84b0d7bf37aec2f6","permalink":"https://dfki-nlp.github.io/publication/lrec2022-roller/","publishdate":"2022-08-12T15:12:24+02:00","relpermalink":"/publication/lrec2022-roller/","section":"publication","summary":"In recent years, machine learning for clinical decision support has gained more and more attention. In order to introduce such applications into clinical practice, a good performance might be essential, however, the aspect of trust should not be underestimated. For the treating physician using such a system and being (legally) responsible for the decision made, it is particularly important to understand the system’s recommendation. To provide insights into a model’s decision, various techniques from the field of explainability (XAI) have been proposed whose output is often enough not targeted to the domain experts that want to use the model. To close this gap, in this work, we explore how explanations could possibly look like in future. To this end, this work presents a dataset of textual explanations in context of decision support. Within a reader study, human physicians estimated the likelihood of possible negative patient outcomes in the near future and justified each decision with a few sentences. Using those sentences, we created a novel corpus, annotated with different semantic layers. Moreover, we provide an analysis of how those explanations are constructed, and how they change depending on physician, on the estimated risk and also in comparison to an automatic clinical decision support system with feature importance.","tags":[],"title":"An Annotated Corpus of Textual Explanations for Clinical Decision Support","type":"publication"},{"authors":["Lisa Raithel","Philippe Thomas","Roland Roller","Oliver Sapina","Sebastian Möller","Pierre Zweigenbaum"],"categories":[],"content":"","date":1660309944,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1660309944,"objectID":"94c08d7d9fb865bdd286f9022b8ca973","permalink":"https://dfki-nlp.github.io/publication/lrec2022-raithel-cross-adr/","publishdate":"2022-08-12T15:12:24+02:00","relpermalink":"/publication/lrec2022-raithel-cross-adr/","section":"publication","summary":"In this work, we present the first corpus for German Adverse Drug Reaction (ADR) detection in patient-generated content. The data consists of 4,169 binary annotated documents from a German patient forum, where users talk about health issues and get advice from medical doctors. As is common in social media data in this domain, the class labels of the corpus are very imbalanced. This and a high topic imbalance make it a very challenging dataset, since often, the same symptom can have several causes and is not always related to a medication intake. We aim to encourage further multi-lingual efforts in the domain of ADR detection and provide preliminary experiments for binary classification using different methods of zero- and few-shot learning based on a multi-lingual model. When fine-tuning XLM-RoBERTa first on English patient forum data and then on the new German data, we achieve an F1-score of 37.52 for the positive class. We make the dataset and models publicly available for the community.","tags":["pharmacovigilance","text classification","adverse drug reactions"],"title":"Cross-lingual Approaches for the Detection of Adverse Drug Reactions in German from a Patient's Perspective","type":"publication"},{"authors":["Rémi Calizzano","Malte Ostendorff","Qian Ruan","Georg Rehm"],"categories":[],"content":"","date":1660309944,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1660309944,"objectID":"dc2a59df634a50a111d996aee581b263","permalink":"https://dfki-nlp.github.io/publication/lrec2022-calizzano/","publishdate":"2022-08-12T15:12:24+02:00","relpermalink":"/publication/lrec2022-calizzano/","section":"publication","summary":"Almost all summarisation methods and datasets focus on a single language and short summaries. We introduce a new dataset called WikinewsSum for English, German, French, Spanish, Portuguese, Polish, and Italian summarisation tailored for extended summaries of approx. 11 sentences. The dataset comprises 39,626 summaries which are news articles from Wikinews and their sources. We compare three multilingual transformer models on the extractive summarisation task and three training scenarios on which we fine-tune mT5 to perform abstractive summarisation. This results in strong baselines for both extractive and abstractive summarisation on WikinewsSum. We also show how the combination of an extractive model with an abstractive one can be used to create extended abstractive summaries from long input documents. Finally, our results show that fine-tuning mT5 on all the languages combined significantly improves the summarisation performance on low-resource languages.","tags":["summarization","wikinews"],"title":"Generating Extended and Multilingual Summaries with Pre-trained Transformers","type":"publication"},{"authors":["Aleksandra Gabryszak","Philippe Thomas"],"categories":[],"content":"","date":1660309944,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1660309944,"objectID":"340f60f8fc97aa804d5c6630207d3989","permalink":"https://dfki-nlp.github.io/publication/lrec2022-gabryszak/","publishdate":"2022-08-12T15:12:24+02:00","relpermalink":"/publication/lrec2022-gabryszak/","section":"publication","summary":"In this paper we show how aspect-based sentiment analysis might help public transport companies to improve their social responsibility for accessible travel. We present MobASA: a novel German-language corpus of tweets annotated with their relevance for public transportation, and with sentiment towards aspects related to barrier-free travel. We identified and labeled topics important for passengers limited in their mobility due to disability, age, or when travelling with young children. The data can be used to identify hurdles and improve travel planning for vulnerable passengers, as well as to monitor a perception of transportation businesses regarding the social inclusion of all passengers. The data is publicly available under: https://github.com/DFKI-NLP/sim3s-corpus","tags":["sentiment analysis","barrier-free travel"],"title":"MobASA: Corpus for Aspect-based Sentiment Analysis and Social Inclusion in the Mobility Domain","type":"publication"},{"authors":["Laura Seiffe","Fares Kallel","Sebastian Möller","Babak Naderi","Roland Roller"],"categories":[],"content":"","date":1660309944,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1660309944,"objectID":"038039f57d8b41bc4751f6245d315b22","permalink":"https://dfki-nlp.github.io/publication/lrec2022-seiffe-subjective/","publishdate":"2022-08-12T15:12:24+02:00","relpermalink":"/publication/lrec2022-seiffe-subjective/","section":"publication","summary":"For different reasons, text can be difficult to read and understand for many people, especially if the text’s language is too complex. In order to provide suitable text for the target audience, it is necessary to measure its complexity. In this paper we describe subjective experiments to assess the readability of German text. We compile a new corpus of sentences provided by a German IT service provider. The sentences are annotated with the subjective complexity ratings by two groups of participants, namely experts and non-experts for that text domain. We then extract an extensive set of linguistically motivated features that are supposedly interacting with complexity perception. We show that a linear regression model with a subset of these features can be a very good predictor of text complexity.","tags":[],"title":"Subjective Text Complexity Assessment for German","type":"publication"},{"authors":["Niklas Dehio","Malte Ostendorff","Georg Rehm"],"categories":[],"content":"","date":1657093069,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1657093069,"objectID":"95ceb26564e8c45d2ce1e79d6f6f04cd","permalink":"https://dfki-nlp.github.io/publication/lrec2022-dehio/","publishdate":"2022-07-06T09:37:49+02:00","relpermalink":"/publication/lrec2022-dehio/","section":"publication","summary":"To cope with the COVID-19 pandemic, many jurisdictions have introduced new or altered existing legislation. Even though these new rules are often communicated to the public in news articles, it remains challenging for laypersons to learn about what is currently allowed or forbidden since news articles typically do not reference underlying laws. We investigate an automated approach to extract legal claims from news articles and to match the claims with their corresponding applicable laws. We examine the feasibility of the two tasks concerning claims about COVID-19-related laws from Berlin, Germany. For both tasks, we create and make publicly available the data sets and report the results of initial experiments. We obtain promising results with Transformer-based models that achieve 46.7 F1 for claim extraction and 91.4 F1 for law matching, albeit with some conceptual limitations. Furthermore, we discuss challenges of current machine learning approaches for legal language processing and their ability for complex legal reasoning tasks.","tags":[],"title":"Claim Extraction and Law Matching for COVID-19-related Legislation","type":"publication"},{"authors":["Malte Ostendorff","Till Blume","Terry Ruas","Bela Gipp","Georg Rehm"],"categories":[],"content":"","date":1657092783,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1657092783,"objectID":"b2f475ebc5afd89e76b904be46a724d9","permalink":"https://dfki-nlp.github.io/publication/jcdl2022-ostendorff/","publishdate":"2022-07-06T09:33:03+02:00","relpermalink":"/publication/jcdl2022-ostendorff/","section":"publication","summary":"Document embeddings and similarity measures underpin content-based recommender systems, whereby a document is commonly represented as a single generic embedding. However, similarity computed on single vector representations provides only one perspective on document similarity that ignores which aspects make two documents alike. To address this limitation, aspect-based similarity measures have been developed using document segmentation or pairwise multi-class document classification. While segmentation harms the document coherence, the pairwise classification approach scales poorly to large scale corpora. In this paper, we treat aspect-based similarity as a classical vector similarity problem in aspect-specific embedding spaces. We represent a document not as a single generic embedding but as multiple specialized embeddings. Our approach avoids document segmentation and scales linearly w.r.t. the corpus size. In an empirical study, we use the Papers with Code corpus containing 157, 606 research papers and consider the task, method, and dataset of the respective research papers as their aspects. We compare and analyze three generic document embeddings, six specialized document embeddings and a pairwise classification baseline in the context of research paper recommendations. As generic document embeddings, we consider FastText, SciBERT, and SPECTER. To compute the specialized document embeddings, we compare three alternative methods inspired by retrofitting, fine-tuning, and Siamese networks. In our experiments, Siamese SciBERT achieved the highest scores. Additional analyses indicate an implicit bias of the generic document embeddings towards the dataset aspect and against the method aspect of each research paper. Our approach of aspect-based document embeddings mitigates potential risks arising from implicit biases by making them explicit. This can, for example, be used for more diverse and explainable recommendations.","tags":[],"title":"Specialized Document Embeddings for Aspect-based Similarity of Research Papers","type":"publication"},{"authors":["Qian Ruan","Malte Ostendorff","Georg Rehm"],"categories":[],"content":"","date":1657092324,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1657092324,"objectID":"6e990907eaa367fdde36268f7946a4be","permalink":"https://dfki-nlp.github.io/publication/acl2022-ruan/","publishdate":"2022-07-06T09:25:24+02:00","relpermalink":"/publication/acl2022-ruan/","section":"publication","summary":"Transformer-based language models usually treat texts as linear sequences. However, most texts also have an inherent hierarchical structure, i.e., parts of a text can be identified using their position in this hierarchy. In addition, section titles usually indicate the common topic of their respective sentences. We propose a novel approach to formulate, extract, encode and inject hierarchical structure information explicitly into an extractive summarization model based on a pre-trained, encoder-only Transformer language model (HiStruct+ model), which improves SOTA ROUGEs for extractive summarization on PubMed and arXiv substantially. Using various experimental settings on three datasets (i.e., CNN/DailyMail, PubMed and arXiv), our HiStruct+ model outperforms a strong baseline collectively, which differs from our model only in that the hierarchical structure information is not injected. It is also observed that the more conspicuous hierarchical structure the dataset has, the larger improvements our method gains. The ablation study demonstrates that the hierarchical position information is the main contributor to our model’s SOTA performance.","tags":[],"title":"HiStruct+: Improving Extractive Text Summarization with Hierarchical Structure Information","type":"publication"},{"authors":["Vivien Macketanz","Babak Naderi","Steven Schmidt","Sebastian Möller"],"categories":[],"content":"","date":1655039544,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1655039544,"objectID":"d1da1db8452d93f183ca43f6a295799d","permalink":"https://dfki-nlp.github.io/publication/acl2022-humeval-macketanz-perceptual/","publishdate":"2022-06-12T15:12:24+02:00","relpermalink":"/publication/acl2022-humeval-macketanz-perceptual/","section":"publication","summary":"The quality of machine-generated text is a complex construct consisting of various aspects and dimensions. We present a study that aims to uncover relevant perceptual quality dimensions for one type of machine-generated text, that is, Machine Translation. We conducted a crowdsourcing survey in the style of a Semantic Differential to collect attribute ratings for German MT outputs. An Exploratory Factor Analysis revealed the underlying perceptual dimensions. As a result, we extracted four factors that operate as relevant dimensions for the Quality of Experience of MT outputs: precision, complexity, grammaticality, and transparency.","tags":["text quality","semantic differential","machine-generated text"],"title":"Perceptual Quality Dimensions of Machine-Generated Text with a Focus on Machine Translation","type":"publication"},{"authors":["David Harbecke","Leonhard Hennig"],"categories":[],"content":"","date":1654078591,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1654078591,"objectID":"35c99541fbe06feac9d7dae7dcf7004d","permalink":"https://dfki-nlp.github.io/project/text2tech/","publishdate":"2022-06-01T11:16:31+01:00","relpermalink":"/project/text2tech/","section":"project","summary":"The goal of the Text2Tech project is the research and development of automated methods for information extraction from unstructured text sources in order to be able to provide companies with decision-relevant knowledge about technological developments quickly and efficiently. AI-based methods for information extraction (IE) already make it possible to extract selected information, e.g. B. to people, companies and places automatically from text sources. In the Text2Tech project, such approaches are to be further developed in order to extract machine-readable knowledge about technologies, technology categories, companies and their relationships with each other from German and English-language, domain-specific text sources, using the example of the automotive industry. The most important research goals are the modeling and filling of domain-specific knowledge graphs (Knowledge Base Population), the development of methods for cross-lingual proper name recognition and linking (Named Entity Recognition or Entity Linking), relation extraction (Relation Extraction), as well as the development of Model compression methods so that models run efficiently even on small hardware.","tags":["Information Extraction","Low-Resource Learning"],"title":"Text2Tech","type":"project"},{"authors":[],"categories":[],"content":"One paper from DFKI-NLP authors has been accepted for publication at JCDL 2022, the 22nd ACM/IEEE Joint Conference on Digital Libraries. The conference is planned to be a hybrid meeting and will take place in Cologne, Germany, from June 20th through June 24th, 2022. The paper proposes to replace generic document embeddings with specialized, per section document embeddings, and evaluates this approach on the task of aspect-based similarity computation for research papers.\n Malte Ostendorff, Till Blume, Terry Ruas, Bela Gipp, Georg Rehm (2022). Specialized Document Embeddings for Aspect-based Similarity of Research Papers. JCDL 2022. PDF Cite Code DOI ","date":1653981841,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1653981841,"objectID":"0be1d1e6f24e387dce50c1b6ba119577","permalink":"https://dfki-nlp.github.io/post/jcdl2022/","publishdate":"2022-05-31T09:24:01+02:00","relpermalink":"/post/jcdl2022/","section":"post","summary":"One paper from DFKI-NLP authors has been accepted for publication at JCDL 2022, the 22nd ACM/IEEE Joint Conference on Digital Libraries. The conference is planned to be a hybrid meeting and will take place in Cologne, Germany, from June 20th through June 24th, 2022.","tags":[],"title":"1 paper to be presented at JCDL 2022","type":"post"},{"authors":[],"categories":[],"content":"Six papers from DFKI-NLP authors have been accepted for publication at LREC 2022, the 13th Language Resources and Evaluation Conference. The conference is planned to be a hybrid meeting and will take place in Marseille, France, from June 20th through June 25th, 2022. The paper by Dehio et al. is on claim extraction and matching in COVID-19-related Legislation, the one by Raither et al. presents a novel corpus for German-language Adverse Drug Reaction (ADR) detection in patient-generated content. The paper by Gabryszak et al. also presents a corpus, in this case of German-language tweets annotated with their relevance for public transportation, and with sentiment towards aspects related to barrier-free travel. The fourth paper by Macketanz et al. presents a fine-grained machine translation test suite for the language pair German-English. The test suite is based on a number of linguistically motivated categories and phenomena and the semi-automatic evaluation is carried out with regular expressions. The fSeiffe et al.\u0026rsquo;s paper presents a work on how to model subjective text complexity, by constructing and analyzing a German text corpus labeled with expert and non-expert complexity ratings. The final paper by Calizzano et al. introduces a new dataset called WikinewsSum for English, German, French, Spanish, Portuguese, Polish, and Italian summarisation tailored for extended summaries of approx. 11 sentences, and compares three multilingual transformer models on the extractive summarisation task and three training scenarios on which we fine-tune mT5 to perform abstractive summarisation.\n Lisa Raithel, Philippe Thomas, Roland Roller, Oliver Sapina, Sebastian Möller, Pierre Zweigenbaum (2022). Cross-lingual Approaches for the Detection of Adverse Drug Reactions in German from a Patient\u0026#39;s Perspective. LREC 2022. PDF Cite Niklas Dehio, Malte Ostendorff, Georg Rehm (2022). Claim Extraction and Law Matching for COVID-19-related Legislation. LREC 2022. PDF Cite Code Aleksandra Gabryszak, Philippe Thomas (2022). MobASA: Corpus for Aspect-based Sentiment Analysis and Social Inclusion in the Mobility Domain. CSR @ LREC 2022. PDF Cite Project Vivien Macketanz, Eleftherios Avramidis, Aljoscha Burchardt, He Wang, Renlong Ai, Shushen Manakhimova, Ursula Strohriegel, Sebastian Möller, Hans Uszkoreit (2022). A Linguistically Motivated Test Suite to Semi-Automatically Evaluate German–English Machine Translation Output. LREC 2022. PDF Cite Laura Seiffe, Fares Kallel, Sebastian Möller, Babak Naderi, Roland Roller (2022). Subjective Text Complexity Assessment for German. LREC 2022. PDF Cite Rémi Calizzano, Malte Ostendorff, Qian Ruan, Georg Rehm (2022). Generating Extended and Multilingual Summaries with Pre-trained Transformers. LREC 2022. PDF Cite ","date":1653981841,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1653981841,"objectID":"b36478abac142030ef17c1c844a3ed47","permalink":"https://dfki-nlp.github.io/post/lrec2022/","publishdate":"2022-05-31T09:24:01+02:00","relpermalink":"/post/lrec2022/","section":"post","summary":"Six papers from DFKI-NLP authors have been accepted for publication at LREC 2022, the 13th Language Resources and Evaluation Conference. The conference is planned to be a hybrid meeting and will take place in Marseille, France, from June 20th through June 25th, 2022.","tags":[],"title":"6 papers to be presented at LREC 2022","type":"post"},{"authors":["Yuxuan Chen","Jonas Mikkelsen","Arne Binder","Christoph Alt","Leonhard Hennig"],"categories":[],"content":"","date":1653523200,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1653523200,"objectID":"71450b988e01035d8dda07fd4d7aebf8","permalink":"https://dfki-nlp.github.io/publication/acl2022-repl4nlp-chen-fewie/","publishdate":"2022-03-28T00:00:00Z","relpermalink":"/publication/acl2022-repl4nlp-chen-fewie/","section":"publication","summary":"Pre-trained language models (PLM) are effective components of few-shot named entity recognition (NER) approaches when augmented with continued pre-training on task-specific out-of-domain data or fine-tuning on in-domain data. However, their performance in low-resource scenarios, where such data is not available, remains an open question. We introduce an encoder evaluation framework, and use it to systematically compare the performance of state-of-the-art pre-trained representations on the task of low-resource NER. We analyze a wide range of encoders pre-trained with different strategies, model architectures, intermediate-task fine-tuning, and contrastive learning. Our experimental results across ten benchmark NER datasets in English and German show that encoder performance varies significantly, suggesting that the choice of encoder for a specific low-resource scenario needs to be carefully evaluated.","tags":[],"title":"A Comparative Study of Pre-trained Encoders for Low-Resource Named Entity Recognition","type":"publication"},{"authors":["David Harbecke","Yuxuan Chen","Leonhard Hennig","Christoph Alt"],"categories":[],"content":"","date":1653523200,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1653523200,"objectID":"9cd7d48423cca6a139bf79bcc5d90221","permalink":"https://dfki-nlp.github.io/publication/acl2022-nlppower-harbecke-f1/","publishdate":"2022-03-28T00:00:00Z","relpermalink":"/publication/acl2022-nlppower-harbecke-f1/","section":"publication","summary":"Relation classification models are conventionally evaluated using only a single measure, e.g., micro-F1, macro-F1 or AUC. In this work, we analyze weighting schemes, such as micro and macro, for imbalanced datasets. We introduce a framework for weighting schemes, where existing schemes are extremes, and two new intermediate schemes. We show that reporting results of different weighting schemes better highlights strengths and weaknesses of a model.","tags":[],"title":"Why only Micro-$F_1$? Class Weighting of Measures for Relation Classification","type":"publication"},{"authors":[],"categories":[],"content":"Four papers from DFKI-NLP authors have been accepted for publication at ACL 2022, the 60th Annual Meeting of the Association for Computational Linguistics. The conference is planned to be a hybrid meeting and will take place in Dublin, Ireland, from May 22nd through May 27th, 2022. One paper is on evaluating pre-trained encoders on the task of low-resource NER across several English and German datasets, the other analyzes relation classification evaluation and suggests that using F1 weightings other than micro-F1 tells us much more about model performance, e.g. on imbalanced datasets. The third paper proposes a novel approach to encode and inject hierarchical structure information explicitly into an extractive, transformer-based summarization model. The final paper present a study that aims to uncover relevant perceptual quality dimensions for one type of machine-generated text, that is, Machine Translation. We conducted a crowd-sourcing survey in the style of a Semantic Differential to collect attribute ratings for German MT outputs.\n Qian Ruan, Malte Ostendorff, Georg Rehm (2022). HiStruct\u0026#43;: Improving Extractive Text Summarization with Hierarchical Structure Information. ACL 2022 Findings. PDF Cite Code DOI Yuxuan Chen, Jonas Mikkelsen, Arne Binder, Christoph Alt, Leonhard Hennig (2022). A Comparative Study of Pre-trained Encoders for Low-Resource Named Entity Recognition. ACL-REPL4NLP 2022. PDF Cite Code Project Project David Harbecke, Yuxuan Chen, Leonhard Hennig, Christoph Alt (2022). Why only Micro-$F_1$? Class Weighting of Measures for Relation Classification. ACL-NLPPower 2022. Cite Project Project Vivien Macketanz, Babak Naderi, Steven Schmidt, Sebastian Möller (2022). Perceptual Quality Dimensions of Machine-Generated Text with a Focus on Machine Translation. HumEval @ ACL 2022. PDF Cite ","date":1648711441,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1648711441,"objectID":"011bca29552463724be0a68ec689af6a","permalink":"https://dfki-nlp.github.io/post/acl2022/","publishdate":"2022-03-31T09:24:01+02:00","relpermalink":"/post/acl2022/","section":"post","summary":"Four papers from DFKI-NLP authors have been accepted for publication at ACL 2022, the 60th Annual Meeting of the Association for Computational Linguistics. The conference is planned to be a hybrid meeting and will take place in Dublin, Ireland, from May 22nd through May 27th, 2022.","tags":[],"title":"4 papers to be presented at ACL 2022","type":"post"},{"authors":["Philippe Thomas","Leonhard Hennig"],"categories":[],"content":"","date":1642673791,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1642673791,"objectID":"427ce0221fcd0c6d0f6a0b3418cf8e0d","permalink":"https://dfki-nlp.github.io/project/bifold/","publishdate":"2022-01-20T11:16:31+01:00","relpermalink":"/project/bifold/","section":"project","summary":"BIFOLD conducts foundational research in big data management and machine learning, as well as its intersection, to educate future talent, and create high-impact knowledge exchange. The Berlin Institute for the Foundations of Learning and Data (BIFOLD), has evolved in 2019 from the merger of two national Artificial Intelligence Competence Centers: the Berlin Big Data Center (BBDC) and the Berlin Center for Machine Learning (BZML). Embedded in the vibrant Berlin metropolitan area, BIFOLD provides an outstanding scientific environment and numerous collaboration opportunities for national and international researchers. BIFOLD offers a broad range of research topics as well as a platform for interdisciplinary research and knowledge exchange with the sciences and humanities, industry, startups and society. Within BIFOLD, DFKI SLT conducts research in Clinical AI, specifically addressing the task of Pharmacovigilance. Pharmacovigilance is concerned with the assessment and prevention of adverse drug reactions (ADR) in pharmaceutical products. As the level of medication is generally raising all over the world, the potential risk of unwanted side effects, such as ADRs, is constantly increasing. Patients exchange views in their own language as 'experts in their own right,' in social media and disease-specific forums. Our project addresses the detection and extraction of ADR from medical forums and social media across different languages using cross-lingual transfer learning in combination with external knowledge sources.","tags":["Information Extraction"],"title":"BIFOLD","type":"project"},{"authors":["Steffen Castle","Robert Schwarzenberg","Mohsen Pourvali"],"categories":[],"content":"","date":1634256e3,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1634256e3,"objectID":"0909afe8749d563f9e6c0ad01633bcb5","permalink":"https://dfki-nlp.github.io/publication/nlpcc-castle/","publishdate":"2021-11-22T11:20:32+01:00","relpermalink":"/publication/nlpcc-castle/","section":"publication","summary":"Detecting when there is a domain drift between training and inference data is important for any model evaluated on data collected in real time. Many current data drift detection methods only utilize input features to detect domain drift. While effective, these methods disregard the model’s evaluation of the data, which may be a significant source of information about the data domain. We propose to use information from the model in the form of explanations, specifically gradient times input, in order to utilize this information. Following the framework of Rabanser et al. [11], we combine these explanations with two-sample tests in order to detect a shift in distribution between training and evaluation data. Promising initial experiments show that explanations provide useful information for detecting shift, which potentially improves upon the current state-of-the-art.","tags":[],"title":"Detecting Covariate Drift with Explanations","type":"publication"},{"authors":["Leonhard Hennig","Phuc Tran Truong","Aleksandra Gabryszak"],"categories":[],"content":"","date":1630972800,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1630972800,"objectID":"b8425af8dc50ee6b2cd49c19ffc86d1e","permalink":"https://dfki-nlp.github.io/publication/konvens2021-hennig-mobie/","publishdate":"2021-08-11T00:00:00Z","relpermalink":"/publication/konvens2021-hennig-mobie/","section":"publication","summary":"We present MobIE, a German-language dataset, which is human-annotated with 20 coarse- and fine-grained entity types and entity linking information for geographically linkable entities. The dataset consists of 3,232 social media texts and traffic reports with 91K tokens, and contains 20.5K annotated entities, 13.1K of which are linked to a knowledge base. A subset of the dataset is human-annotated with seven mobility-related, n-ary relation types, while the remaining documents are annotated using a weakly-supervised labeling approach implemented with the Snorkel framework. To the best of our knowledge, this is the first German-language dataset that combines annotations for NER, EL and RE, and thus can be used for joint and multi-task learning of these fundamental information extraction tasks. We make MobIE public at https://github.com/dfki-nlp/mobie.","tags":[],"title":"MobIE: A German Dataset for Named Entity Recognition, Entity Linking and Relation Extraction in the Mobility Domain","type":"publication"},{"authors":["Malte Ostendorff","Elliott Ash","Terry Ruas","Bela Gipp","Julian Moreno-Schneider","Georg Rehm"],"categories":[],"content":"","date":1622110683,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1622110683,"objectID":"ab6a46956823e641886d1cbdd6c30ba3","permalink":"https://dfki-nlp.github.io/publication/ostendorff2021/","publishdate":"2021-05-27T12:18:03+02:00","relpermalink":"/publication/ostendorff2021/","section":"publication","summary":"Recommender systems assist legal professionals in finding relevant literature for supporting their case. Despite its importance for the profession, legal applications do not reflect the latest advances in recommender systems and representation learning research. Simultaneously, legal recommender systems are typically evaluated in small-scale user study without any public available benchmark datasets. Thus, these studies have limited reproducibility. To address the gap between research and practice, we explore a set of state-of-the-art document representation methods for the task of retrieving semantically related US case law. We evaluate text-based (e.g., fastText, Transformers), citation-based (e.g., DeepWalk, Poincaré), and hybrid methods. We compare in total 27 methods using two silver standards with annotations for 2,964 documents. The silver standards are newly created from Open Case Book and Wikisource and can be reused under an open license facilitating reproducibility. Our experiments show that document representations from averaged fastText word vectors (trained on legal corpora) yield the best results, closely followed by Poincaré citation embeddings. Combining fastText and Poincaré in a hybrid manner further improves the overall result. Besides the overall performance, we analyze the methods depending on document length, citation count, and the coverage of their recommendations. We make our source code, models, and datasets publicly available at this https URL. ","tags":[],"title":"Evaluating Document Representations for Content-based Legal Literature Recommendations","type":"publication"},{"authors":["Leonhard Hennig","Christoph Alt"],"categories":[],"content":"","date":1614075782,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1614075782,"objectID":"a8921d49553043a8c438729c3f6507f8","permalink":"https://dfki-nlp.github.io/project/cora4nlp/","publishdate":"2021-02-23T11:23:02+01:00","relpermalink":"/project/cora4nlp/","section":"project","summary":"Language is implicit - it omits information. Filling this information gap requires contextual inference, background- and commonsense knowledge, and reasoning over situational context. Language also evolves, i.e., it specializes and changes over time. For example, many different languages and domains exist, new domains arise, and both evolve constantly. Thus, language understanding also requires continuous and efficient adaptation to new languages and domains, and transfer to, and between, both. Current language understanding technology, however, focuses on high resource languages and domains, uses little to no context, and assumes static data, task, and target distributions. The research in Cora4NLP aims to address these challenges. It builds on the expertise and results of the predecessor project DEEPLEE and is carried out jointly between the language technology research departments in Berlin and Saarbrücken. ","tags":["Information Extraction","Language Understanding"],"title":"Cora4NLP","type":"project"},{"authors":["Leonhard Hennig"],"categories":[],"content":"","date":1614075391,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1614075391,"objectID":"cbc4ff3659230581c9808dc981535447","permalink":"https://dfki-nlp.github.io/project/bbdc2/","publishdate":"2021-02-23T11:16:31+01:00","relpermalink":"/project/bbdc2/","section":"project","summary":"In order to optimally prepare industry, science and the society in Germany and Europe for the global Big Data trend, highly coordinated activities in research, teaching, and technology transfer regarding the integration of data analysis methods and scalable data processing are required. To achieve this, the Berlin Big Data Center is pursuing the following seven objectives: 1) Pooling expertise in scalable data management, data analytics, and big data application 2) Conducting fundamental research to develop novel and automatically scalable technologies capable of performing 'Deep Analysis' of 'Big Data'. 3) Developing an integrated, declarative, highly scalable open-source system that enables the specification, automatic optimization, parallelization and hardware adaptation, and fault-tolerant, efficient execution of advanced data analysis problems, using varying methods (e.g., drawn from machine learning, linear algebra, statistics and probability theory, computational linguistics, or signal processing), leveraging our work on Apache Flink 4) Transfering technology and know-how to support innovation in companies and startups. 5) Educating data scientists with respect to the five big data dimensions (i.e., applications, economic, legal, social, and technological) via leading educational programs. 6) Empowering people to leverage 'Smart Data', i.e., to discover newfound information based on their massive data sets. 7)Enabling the general public to conduct sound data-driven decision-making.","tags":["Information Extraction"],"title":"BBDC2","type":"project"},{"authors":["Sven Schmeier","Christoph Alt","Robert Schwarzenberg"],"categories":[],"content":"","date":1614075391,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1614075391,"objectID":"c389619c2f6d4f10d79bd3a7581e3344","permalink":"https://dfki-nlp.github.io/project/deeplee/","publishdate":"2021-02-23T11:16:31+01:00","relpermalink":"/project/deeplee/","section":"project","summary":"The research work in DEEPLEE, which is carried out in the Language Technology research departments in Saabrücken and Berlin, builds on DFKI's expertise in the areas of deep learning (DL) and language technology (LT) and develops it further. They aim for profound improvements of DL approaches in LT by focusing on four central, open research topics: Modularity in DNN architectures, Use of external knowledge, DNNs with explanation functionality, Machine Teaching Strategies for DNNs","tags":["Information Extraction","Language Understanding"],"title":"DEEPLEE","type":"project"},{"authors":["Leonhard Hennig","Christoph Alt"],"categories":[],"content":"","date":1614075391,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1614075391,"objectID":"cfeff35232a191ef8a348b8c96979ab5","permalink":"https://dfki-nlp.github.io/project/plass/","publishdate":"2021-02-23T11:16:31+01:00","relpermalink":"/project/plass/","section":"project","summary":"The aim of the PLASS project is to develop a prototypical B2B platform for AI-based decision support for supply chain management. The focus is on the automatic recognition of decision-relevant information and the acquisition of structured knowledge from global and multilingual text sources. These sources provide a large database for SCM information, especially for the early detection of critical events and risks, but also of opportunities, e.g. through new technologies, at suppliers and supply chains. PLASS enables SMEs and large companies to continuously monitor their suppliers and supply chains, and supports supply chain managers in risk assessment and decision-making.","tags":["Information Extraction","Low-Resource Learning"],"title":"PLASS","type":"project"},{"authors":["Philippe Thomas"],"categories":[],"content":"","date":1614075391,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1614075391,"objectID":"c25cf95b8ddec88b64c3b4a1a19063f6","permalink":"https://dfki-nlp.github.io/project/sim3s/","publishdate":"2021-02-23T11:16:31+01:00","relpermalink":"/project/sim3s/","section":"project","summary":"In the SIM3S project, data from the BMVI data offerings mCloud and MDM will be linked, refined and jointly analysed with other open data, user-generated content and data from individual modes of transport and other mobility-relevant companies in order to remove barriers and barriers to discrimination in everyday mobility. For the implementation of the project, state-of-the-art technologies and methods from the areas of Big Data Intelligent Analysis of mass data and artificial intelligence, in particular Natural Language Processing (NLP), are used.","tags":["Information Extraction"],"title":"SIM3S","type":"project"},{"authors":["Malte Ostendorff","Terry Ruas","Till Blume","Bela Gipp","Georg Rehm"],"categories":[],"content":"","date":1609064252,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1609064252,"objectID":"9e79e1aa43e10a8c99ed53fd1f129c58","permalink":"https://dfki-nlp.github.io/publication/ostendorff2020c/","publishdate":"2020-12-27T12:17:32+02:00","relpermalink":"/publication/ostendorff2020c/","section":"publication","summary":"Traditional document similarity measures provide a coarse-grained distinction between similar and dissimilar documents. Typically, they do not consider in what aspects two documents are similar. This limits the granularity of applications like recommender systems that rely on document similarity. In this paper, we extend similarity with aspect information by performing a pairwise document classification task. We evaluate our aspect-based document similarity for research papers. Paper citations indicate the aspect-based similarity, i.e., the section title in which a citation occurs acts as a label for the pair of citing and cited paper. We apply a series of Transformer models such as RoBERTa, ELECTRA, XLNet, and BERT variations and compare them to an LSTM baseline. We perform our experiments on two newly constructed datasets of 172,073 research paper pairs from the ACL Anthology and CORD-19 corpus. Our results show SciBERT as the best performing system. A qualitative examination validates our quantitative results. Our findings motivate future research of aspect-based document similarity and the development of a recommender system based on the evaluated techniques. We make our datasets, code, and trained models publicly available. ","tags":[],"title":"Aspect-based Document Similarity for Research Papers","type":"publication"},{"authors":["Marc Hübner","Christoph Alt","Robert Schwarzenberg","Leonhard Hennig"],"categories":[],"content":"Definition Extraction systems are a valuable knowledge source for both humans and algorithms. In this paper we describe our submissions to the DeftEval shared task (SemEval-2020 Task 6), which is evaluated on an English textbook corpus. We provide a detailed explanation of our system for the joint extraction of definition concepts and the relations among them. Furthermore we provide an ablation study of our model variations and describe the results of an error analysis.\n","date":1606780800,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1606780800,"objectID":"325726696f4edad34e20c5582f171a49","permalink":"https://dfki-nlp.github.io/publication/semeval2020-huebner-defx/","publishdate":"2020-12-01T14:36:10+02:00","relpermalink":"/publication/semeval2020-huebner-defx/","section":"publication","summary":"We describe our submissions to the DeftEval shared task (SemEval-2020 Task 6)","tags":[],"title":"Defx at SemEval-2020 Task 6: Joint Extraction of Concepts and Relations for Definition Extraction","type":"publication"},{"authors":["Malte Ostendorff","Terry Ruas","Moritz Schubotz","Georg Rehm","Bela Gipp"],"categories":[],"content":"","date":1598523462,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1598523462,"objectID":"4397f1dd4f22b53f0586d6677062f408","permalink":"https://dfki-nlp.github.io/publication/ostendorff2020/","publishdate":"2020-08-22T12:17:42+02:00","relpermalink":"/publication/ostendorff2020/","section":"publication","summary":"Many digital libraries recommend literature to their users considering the similarity between a query document and their repository. However, they often fail to distinguish what is the relationship that makes two documents alike. In this paper, we model the problem of finding the relationship between two documents as a pairwise document classification task. To find the semantic relation between documents, we apply a series of techniques, such as GloVe, Paragraph-Vectors, BERT, and XLNet under different configurations (e.g., sequence length, vector concatenation scheme), including a Siamese architecture for the Transformer-based systems. We perform our experiments on a newly proposed dataset of 32,168 Wikipedia article pairs and Wikidata properties that define the semantic document relations. Our results show vanilla BERT as the best performing system with an F1-score of 0.93, which we manually examine to better understand its applicability to other domains. Our findings suggest that classifying semantic relations between documents is a solvable task and motivates the development of recommender systems based on the evaluated techniques. The discussions in this paper serve as first steps in the exploration of documents through SPARQL-like queries such that one could find documents that are similar in one aspect but dissimilar in another. ","tags":[],"title":"Pairwise Multi-Class Document Classification for Semantic Relations between Wikipedia Articles","type":"publication"},{"authors":["Robert Schwarzenberg","Steffen Castle"],"categories":[],"content":"","date":1595980800,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1595980800,"objectID":"ca2fe3ec7f7c5501639381082cb399a5","permalink":"https://dfki-nlp.github.io/publication/icml-schwarzenberg-castle/","publishdate":"2020-08-26T14:09:16+02:00","relpermalink":"/publication/icml-schwarzenberg-castle/","section":"publication","summary":"Integrated Gradients (IG) and PatternAttribution (PA) are two established explainability methods for neural networks. Both methods are theoretically well-founded. However, they were designed to overcome different challenges. In this work, we combine the two methods into a new method, Pattern-Guided Integrated Gradients (PGIG). PGIG inherits important properties from both parent methods and passes stress tests that the originals fail. In addition, we benchmark PGIG against nine alternative explainability approaches (including its parent methods) in a large-scale image degradation experiment and find that it outperforms all of them.","tags":[],"title":"Pattern-Guided Integrated Gradients","type":"publication"},{"authors":["Hanchu Zhang","Leonhard Hennig","Christoph Alt","Changjian Hu","Yao Meng","Chao Wang"],"categories":[],"content":"","date":1594339200,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1594339200,"objectID":"d04a7113f73244eb9fc10ce17aa6eb39","permalink":"https://dfki-nlp.github.io/publication/acl-ecnlp2020-zhang-bootstrapping/","publishdate":"2020-07-10T00:00:00Z","relpermalink":"/publication/acl-ecnlp2020-zhang-bootstrapping/","section":"publication","summary":"In this work, we introduce a bootstrapped, iterative NER model that integrates a PU learning algorithm for recognizing named entities in a low-resource setting. Our approach combines dictionary-based labeling with syntactically-informed label expansion to efficiently enrich the seed dictionaries. Experimental results on a dataset of manually annotated e-commerce product descriptions demonstrate the effectiveness of the proposed framework.","tags":[],"title":"Bootstrapping Named Entity Recognition in E-Commerce with Positive Unlabeled Learning","type":"publication"},{"authors":["David Harbecke","Christoph Alt"],"categories":[],"content":"","date":1594339200,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1594339200,"objectID":"015c496995d3ee9b3de39d1f0d271c69","permalink":"https://dfki-nlp.github.io/publication/acl-srw2020-harbecke-considering/","publishdate":"2020-07-10T00:00:00Z","relpermalink":"/publication/acl-srw2020-harbecke-considering/","section":"publication","summary":"Recently, state-of-the-art NLP models gained an increasing syntactic and semantic understanding of language, and explanation methods are crucial to understand their decisions. Occlusion is a well established method that provides explanations on discrete language data, e.g. by removing a language unit from an input and measuring the impact on a model's decision. We argue that current occlusion-based methods often produce invalid or syntactically incorrect language data, neglecting the improved abilities of recent NLP models. Furthermore, gradient-based explanation methods disregard the discrete distribution of data in NLP. Thus, we propose OLM: a novel explanation method that combines occlusion and language models to sample valid and syntactically correct replacements with high likelihood, given the context of the original input. We lay out a theoretical foundation that alleviates these weaknesses of other explanation methods in NLP and provide results that underline the importance of considering data likelihood in occlusion-based explanation.","tags":["Explainability"],"title":"Considering Likelihood in NLP Classification Explanations with Occlusion and Language Modeling","type":"publication"},{"authors":["Christoph Alt","Aleksandra Gabryszak","Leonhard Hennig"],"categories":[],"content":"","date":1594339200,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1594339200,"objectID":"8c459d2cf0ab9b1905c893ec91b8871c","permalink":"https://dfki-nlp.github.io/publication/acl2020-alt-probing/","publishdate":"2020-07-10T00:00:00Z","relpermalink":"/publication/acl2020-alt-probing/","section":"publication","summary":"Despite the recent progress, little is known about the features captured by state-of-the-art neural relation extraction (RE) models. Common methods encode the source sentence, conditioned on the entity mentions, before classifying the relation. However, the complexity of the task makes it difficult to understand how encoder architecture and supporting linguistic knowledge affect the features learned by the encoder. We introduce 14 probing tasks targeting linguistic properties relevant to RE, and we use them to study representations learned by more than 40 different encoder architecture and linguistic feature combinations trained on two datasets, TACRED and SemEval 2010 Task 8. We find that the bias induced by the architecture and the inclusion of linguistic features are clearly expressed in the probing task performance. For example, adding contextualized word representations greatly increases performance on probing tasks with a focus on named entity and part-of-speech information, and yields better results in RE. In contrast, entity masking improves RE, but considerably lowers performance on entity type related probing tasks.","tags":[],"title":"Probing Linguistic Features of Sentence-Level Representations in Neural Relation Extraction","type":"publication"},{"authors":["Christoph Alt","Aleksandra Gabryszak","Leonhard Hennig"],"categories":[],"content":"","date":1594339200,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1594339200,"objectID":"4b6a4a0f754ff65730e25ac13406e529","permalink":"https://dfki-nlp.github.io/publication/acl2020-alt-tacred-revisited/","publishdate":"2020-07-10T00:00:00Z","relpermalink":"/publication/acl2020-alt-tacred-revisited/","section":"publication","summary":"TACRED is one of the largest, most widely used crowdsourced datasets in Relation Extraction (RE). But, even with recent advances in unsupervised pre-training and knowledge enhanced neural RE, models still show a high error rate. In this paper, we investigate the questions: Have we reached a performance ceiling or is there still room for improvement? And how do crowd annotations, dataset, and models contribute to this error rate? To answer these questions, we first validate the most challenging 5K examples in the development and test sets using trained annotators. We find that label errors account for 8% absolute F1 test error, and that more than 50% of the examples need to be relabeled. On the relabeled test set the average F1 score of a large baseline model set improves from 62.1 to 70.1. After validation, we analyze misclassifications on the challenging instances, categorize them into linguistically motivated error groups, and verify the resulting error hypotheses on three state-of-the-art RE models. We show that two groups of ambiguous relations are responsible for most of the remaining errors and that models may adopt shallow heuristics on the dataset when entities are not masked.","tags":[],"title":"TACRED Revisited: A Thorough Evaluation of the TACRED Relation Extraction Task","type":"publication"},{"authors":null,"categories":null,"content":"","date":1593561600,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1593561600,"objectID":"f8b2dc590dfded5b66e7919211387415","permalink":"https://dfki-nlp.github.io/dataset/ex4cds/","publishdate":"2020-07-01T00:00:00Z","relpermalink":"/dataset/ex4cds/","section":"dataset","summary":"Ex4CDS are explanations (or more precisely justifications) of physicians in the context of clinical decision support. In the course of a larger study, physicians estimated the probability of different clinical outcomes in nephology, namely rejection, graft loss and infections, within the next 90 days. Each estimation had to be justified within a short text - these are our explanations. The explanations were provided in German and have strong similarities to general clinical notes. You can find a description and the data here: https://github.com/DFKI-NLP/Ex4CDS","tags":["Language Understanding","Explainable AI","Clinical Decision Support"],"title":"Ex4CDS - Textual Explanations for Clinical Decision Support","type":"dataset"},{"authors":null,"categories":null,"content":"","date":1593561600,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1593561600,"objectID":"182a0843518c2c6c1a4676a87bddfa9f","permalink":"https://dfki-nlp.github.io/dataset/german-patient-adr/","publishdate":"2020-07-01T00:00:00Z","relpermalink":"/dataset/german-patient-adr/","section":"dataset","summary":"In this work, we present the first corpus for German Adverse Drug Reaction (ADR) detection in patient-generated content. The data consists of 4,169 binary annotated documents from a German patient forum, where users talk about health issues and get advice from medical doctors. As is common in social media data in this domain, the class labels of the corpus are very imbalanced. This and a high topic imbalance make it a very challenging dataset, since often, the same symptom can have several causes and is not always related to a medication intake. We aim to encourage further multi-lingual efforts in the domain of ADR detection. More info: https://aclanthology.org/2022.lrec-1.388/","tags":["Language Understanding","Information Extraction","Multilinguality"],"title":"German Adverse Drug Reaction (ADR) detection in patient-generated content","type":"dataset"},{"authors":null,"categories":null,"content":"","date":1593561600,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1593561600,"objectID":"025ca2ede65a805c2716900ab9fae712","permalink":"https://dfki-nlp.github.io/dataset/dfki-mobasa-corpus/","publishdate":"2020-07-01T00:00:00Z","relpermalink":"/dataset/dfki-mobasa-corpus/","section":"dataset","summary":"This repository contains corpus called MobASA: a novel German-language corpus of tweets annotated with their relevance for public transportation, and with sentiment towards aspects related to barrier-free travel. We identified and labeled topics important for passengers limited in their mobility due to disability, age, or when travelling with young children.\nThe data can be used for as a training or test corpus for aspect-oriented sentiment analysis. Moreover, the corpus can benefit building inclusive public transportation systems. You can find the corpus here: https://github.com/DFKI-NLP/sim3s-corpus, and the description of the corpus here: https://aclanthology.org/2022.csrnlp-1.5.pdf","tags":["Language Understanding","Information Extraction","Sentiment Analysis","Mobility"],"title":"MobASA Corpus","type":"dataset"},{"authors":null,"categories":null,"content":"","date":1593561600,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1593561600,"objectID":"55804a630a9c47f344ed7ef49cca65d6","permalink":"https://dfki-nlp.github.io/dataset/dfki-mobie-corpus/","publishdate":"2020-07-01T00:00:00Z","relpermalink":"/dataset/dfki-mobie-corpus/","section":"dataset","summary":"This repository contains the DFKI MobIE Corpus (formerly \"DAYSTREAM Corpus\"), a dataset of 3,232 German-language documents collected between May 2015 - Apr 2019 that have been annotated with fine-grained geo-entities, such as location-street, location-stop and location-route, as well as standard named entity types (organization, date, number, etc). All location-related entities have been linked to either Open Street Map identifiers or database ids of Deutsche Bahn / Rhein-Main-Verkehrsverbund. The corpus has also been annotated with a set of 7 traffic-related n-ary relations and events, such as Accidents, Traffic jams, and Canceled Routes. It consists of Twitter messages, and traffic reports from e.g. radio stations, police and public transport providers. It allows for training and evaluating both named entity recognition algorithms that aim for fine-grained typing of geo-entities, entity linking of these entities, as well as n-ary relation extraction systems. You can find the description of the corpus here: https://www.dfki.de/web/forschung/projekte-publikationen/publikationen-uebersicht/publikation/11741/","tags":["Language Understanding","Information Extraction","Mobility"],"title":"MobIE Corpus","type":"dataset"},{"authors":["Karolina Zaczynska","Nils Feldhus","Robert Schwarzenberg","Aleksandra Gabryszak","Sebastian Möller"],"categories":[],"content":"","date":1592870400,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1592870400,"objectID":"69575402fe2a9bf17cf5ea06cff1b67c","permalink":"https://dfki-nlp.github.io/publication/swisstext2020-zaczynska-evaluating/","publishdate":"2020-06-23T00:00:00Z","relpermalink":"/publication/swisstext2020-zaczynska-evaluating/","section":"publication","summary":"Pre-trained transformer language models (TLMs) have recently refashioned natural language processing (NLP): Most stateof-the-art NLP models now operate on top of TLMs to benefit from contextualization and knowledge induction. To explain their success, the scientific community conducted numerous analyses. Besides other methods, syntactic agreement tests were utilized to analyse TLMs. Most of the studies were conducted for the English language, however. In this work, we analyse German TLMs. To this end, we design numerous agreement tasks, some of which consider peculiarities of the German language. Our experimental results show that state-of-the-art German TLMs generally perform well on agreement tasks, but we also identify and discuss syntactic structures that push them to their limits.","tags":[],"title":"Evaluating German Transformer Language Models with Syntactic Agreement Tests","type":"publication"},{"authors":["Dmitrii Aksenov","Julian Moreno Schneider","Peter Bourgonje","Robert Schwarzenberg","Leonhard Hennig","Georg Rehm"],"categories":[],"content":"","date":1589587200,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1589587200,"objectID":"691b01834034e508889d3d916be8218c","permalink":"https://dfki-nlp.github.io/publication/lrec2020-aksenov-abstractive/","publishdate":"2020-05-16T00:00:00Z","relpermalink":"/publication/lrec2020-aksenov-abstractive/","section":"publication","summary":"We explore to what extent knowledge about the pre-trained language model that is used is beneficial for the task of abstractive summarization. To this end, we experiment with conditioning the encoder and decoder of a Transformer-based neural model on the BERT language model. In addition, we propose a new method of BERT-windowing, which allows chunk-wise processing of texts longer than the BERT window size. We also explore how locality modeling, i.e., the explicit restriction of calculations to the local context, can affect the summarization ability of the Transformer. This is done by introducing 2-dimensional convolutional self-attention into the first layers of the encoder. The results of our models are compared to a baseline and the state-of-the-art models on the CNN/Daily Mail dataset. We additionally train our model on the SwissText dataset to demonstrate usability on German. Both models outperform the baseline in ROUGE scores on two datasets and show its superiority in a manual qualitative analysis.","tags":[],"title":"Abstractive Text Summarization based on Language Model Conditioning and Locality Modeling","type":"publication"},{"authors":null,"categories":null,"content":"","date":1576508108,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1576508108,"objectID":"2fd1a9eee61a268e57a10ee4b5e09ea7","permalink":"https://dfki-nlp.github.io/dataset/dfki-product-corpus/","publishdate":"2019-12-16T15:55:08+01:00","relpermalink":"/dataset/dfki-product-corpus/","section":"dataset","summary":"The Product Corpus is a dataset of 174 English web pages and social media posts annotated for product and company named entities, and the relation CompanyProvidesProduct. The goal is to make extraction of non-standard, B2B products and relations from unstructured text easier and more reliable. The corpus is also annotated for coreference chains of companies and products.","tags":["Language Understanding","Information Extraction"],"title":"Product Corpus","type":"dataset"},{"authors":null,"categories":null,"content":"","date":1576508108,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1576508108,"objectID":"88423b8faaa8c52b9ff3f68c29045e86","permalink":"https://dfki-nlp.github.io/dataset/dfki-smartdata-corpus/","publishdate":"2019-12-16T15:55:08+01:00","relpermalink":"/dataset/dfki-smartdata-corpus/","section":"dataset","summary":"The SmartData Corpus is a dataset of 2598 German-language documents which has been annotated with fine-grained geo-entities, such as streets, stops and routes, as well as standard named entity types. It has also been annotated with a set of 15 traffic- and industry-related n-ary relations and events, such as Accidents, Traffic jams, Acquisitions, and Strikes. The corpus consists of newswire texts, Twitter messages, and traffic reports from radio stations, police and railway companies. It allows for training and evaluating both named entity recognition algorithms that aim for fine-grained typing of geo-entities, as well as n-ary relation extraction systems.","tags":["Language Understanding","Information Extraction"],"title":"SmartData Corpus","type":"dataset"},{"authors":["Lisa Raithel","Robert Schwarzenberg"],"categories":[],"content":"","date":1569863398,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1569863398,"objectID":"e8e32af7af385deca2d3740c85dea084","permalink":"https://dfki-nlp.github.io/publication/cl-nvc/","publishdate":"2019-09-30T18:09:58+01:00","relpermalink":"/publication/cl-nvc/","section":"publication","summary":"Recently, Neural Vector Conceptualization (NVC) was proposed as a means to interpret samples from a word vector space. For NVC, a neural model activates higher order concepts it recognizes in a word vector instance. To this end, the model first needs to be trained with a sufficiently large instance-to-concept ground truth, which only exists for a few languages. In this work, we tackle this lack of resources with word vector space alignment techniques: We train the NVC model on a high resource language and test it with vectors from an aligned word vector space of another language, without retraining or fine-tuning. A quantitative and qualitative analysis shows that the NVC model indeed activates meaningful concepts for unseen vectors from the aligned vector space. NVC thus becomes available for low resource languages for which no appropriate concept ground truth exists.","tags":[],"title":"Cross-lingual Neural Vector Conceptualization","type":"publication"},{"authors":["Robert Schwarzenberg","Marc Hübner","David Harbecke","Christoph Alt","Leonhard Hennig"],"categories":[],"content":"","date":1569476323,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1569476323,"objectID":"a473d0f5238b0e0755c80265a795c54a","permalink":"https://dfki-nlp.github.io/publication/layerwise-relevance-visualization-in-convolutional-text-graph-classifiers/","publishdate":"2019-09-26T07:38:43+02:00","relpermalink":"/publication/layerwise-relevance-visualization-in-convolutional-text-graph-classifiers/","section":"publication","summary":"Representations in the hidden layers of Deep Neural Networks (DNN) are often hard to interpret since it is difficult to project them into an interpretable domain. Graph Convolutional Networks (GCN) allow this projection, but existing explainability methods do not exploit this fact, i.e. do not focus their explanations on intermediate states. In this work, we present a novel method that traces and visualizes features that contribute to a classification decision in the visible and hidden layers of a GCN. Our method exposes hidden cross-layer dynamics in the input graph structure. We experimentally demonstrate that it yields meaningful layerwise explanations for a GCN sentence classifier.","tags":[],"title":"Layerwise Relevance Visualization in Convolutional Text Graph Classifiers","type":"publication"},{"authors":["Christoph Alt","Marc Hübner","Leonhard Hennig"],"categories":[],"content":"","date":1564358400,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1564358400,"objectID":"584cb9534c1efd44745dff10b6121dc1","permalink":"https://dfki-nlp.github.io/publication/acl2019-alt-finetuning/","publishdate":"2019-08-26T14:09:16+02:00","relpermalink":"/publication/acl2019-alt-finetuning/","section":"publication","summary":"We show that generative language model pre-training combined with selective attention improves recall for long-tail relations in distantly supervised neural relation extraction.","tags":[],"title":"Fine-Tuning Pre-Trained Transformer Language Models to Distantly Supervised Relation Extraction","type":"publication"},{"authors":["Neslihan Iskender","Aleksandra Gabryszak","Tim Polzehl","Leonhard Hennig","Sebastian Möller"],"categories":[],"content":"","date":1561334400,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1561334400,"objectID":"c481a30f36434bd26ba43c2e41347fd9","permalink":"https://dfki-nlp.github.io/publication/qomex2019-iskender-crowd/","publishdate":"2019-06-24T14:09:16+02:00","relpermalink":"/publication/qomex2019-iskender-crowd/","section":"publication","summary":"We analyze the feasibility and appropriateness of micro-task crowdsourcing for evaluation of different summary quality characteristics and report an ongoing work on the crowdsourced evaluation of query-based extractive text summaries","tags":[],"title":"A Crowdsourcing Approach to Evaluate the Quality of Query-based Extractive Text Summaries","type":"publication"},{"authors":["Malte Ostendorff","Peter Bourgonje","Maria Berger","Julian Moreno Schneider","Georg Rehm","Bela Gipp"],"categories":[],"content":"","date":1558953138,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1558953138,"objectID":"a37577f877550a6974c6b3374f48e5ea","permalink":"https://dfki-nlp.github.io/publication/ostendorff2019/","publishdate":"2019-05-27T12:32:18+02:00","relpermalink":"/publication/ostendorff2019/","section":"publication","summary":" In this paper, we focus on the classification of books using short descriptive texts (cover blurbs) and additional metadata. Building upon BERT, a deep neural language model, we demonstrate how to combine text representations with metadata and knowledge graph embeddings, which encode author information. Compared to the standard BERT approach we achieve considerably better results for the classification task. For a more coarse-grained classification using eight labels we achieve an F1- score of 87.20, while a detailed classification using 343 labels yields an F1-score of 64.70. We make the source code and trained models of our experiments publicly available ","tags":[],"title":"Enriching BERT with Knowledge Graph Embedding for Document Classification","type":"publication"},{"authors":["Christoph Alt","Marc Hübner","Leonhard Hennig"],"categories":[],"content":"","date":1558310400,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1558310400,"objectID":"d6c8a5f82560901c80529e3d2c9e706a","permalink":"https://dfki-nlp.github.io/publication/akbc2019-alt-improving/","publishdate":"2019-08-26T14:36:10+02:00","relpermalink":"/publication/akbc2019-alt-improving/","section":"publication","summary":"We show that transfer learning through generative language model pre-training improves supervised neural relation extraction, achieving new state-of-the-art performance on TACRED and SemEval 2010 Task 8.","tags":[],"title":"Improving Relation Extraction by Pre-Trained Language Representations","type":"publication"},{"authors":["Robert Schwarzenberg","Lisa Raithel","David Harbecke"],"categories":[],"content":"","date":1554224121,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1554224121,"objectID":"2e97d4e5206e642fb4fe1c042c0171cc","permalink":"https://dfki-nlp.github.io/publication/nvc/","publishdate":"2019-04-02T17:55:21+01:00","relpermalink":"/publication/nvc/","section":"publication","summary":"Distributed word vector spaces are considered hard to interpret which hinders the understanding of natural language processing (NLP) models. In this work, we introduce a new method to interpret arbitrary samples from a word vector space. To this end, we train a neural model to conceptualize word vectors, which means that it activates higher order concepts it recognizes in a given vector. Contrary to prior approaches, our model operates in the original vector space and is capable of learning non-linear relations between word vectors and concepts. Furthermore, we show that it produces considerably less entropic concept activation profiles than the popular cosine similarity.","tags":[],"title":"Neural Vector Conceptualization for Word Vector Space Interpretation","type":"publication"},{"authors":["Robert Schwarzenberg","David Harbecke","Vivien Macketanz","Eleftherios Avramidis","Sebastian Möller"],"categories":[],"content":"","date":1553791245,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1553791245,"objectID":"704fbe147dc60525fe2463f016efdd9d","permalink":"https://dfki-nlp.github.io/publication/diamat/","publishdate":"2019-03-28T17:40:45+01:00","relpermalink":"/publication/diamat/","section":"publication","summary":"Evaluating translation models is a trade-off between effort and detail. On the one end of the spectrum there are automatic count-based methods such as BLEU, on the other end linguistic evaluations by humans, which arguably are more informative but also require a disproportionately high effort. To narrow the spectrum, we propose a general approach on how to automatically expose systematic differences between human and machine translations to human experts. Inspired by adversarial settings, we train a neural text classifier to distinguish human from machine translations. A classifier that performs and generalizes well after training should recognize systematic differences between the two classes, which we uncover with neural explainability methods. Our proof-of-concept implementation, DiaMaT, is open source. Applied to a dataset translated by a state-of-the-art neural Transformer model, DiaMaT achieves a classification accuracy of 75% and exposes meaningful differences between humans and the Transformer, amidst the current discussion about human parity.","tags":[],"title":"Train, Sort, Explain: Learning to Diagnose Translation Models","type":"publication"},{"authors":["Roland Roller","Christoph Alt","Laura Seiffe","He Wang"],"categories":[],"content":"","date":1543622400,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1543622400,"objectID":"433a17498e14b803ea57862a8b6aadd1","permalink":"https://dfki-nlp.github.io/publication/mex-an-information-extraction-platform-for-german-medical-text/","publishdate":"2019-08-26T14:36:19+02:00","relpermalink":"/publication/mex-an-information-extraction-platform-for-german-medical-text/","section":"publication","summary":"","tags":[],"title":"mEx - an Information Extraction Platform for German Medical Text","type":"publication"},{"authors":["David Harbecke","Robert Schwarzenberg","Christoph Alt"],"categories":[],"content":"","date":1541116800,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1541116800,"objectID":"68f68d704af9c42e6dc03edd211ae360","permalink":"https://dfki-nlp.github.io/publication/learning-explanations-from-language-data/","publishdate":"2019-08-26T14:36:16+02:00","relpermalink":"/publication/learning-explanations-from-language-data/","section":"publication","summary":"PatternAttribution is a recent method, introduced in the vision domain, that explains classifications of deep neural networks. We demonstrate that it also generates meaningful interpretations in the language domain.","tags":[],"title":"Learning Explanations From Language Data","type":"publication"},{"authors":null,"categories":null,"content":"","date":-62135596800,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":-62135596800,"objectID":"f26b5133c34eec1aa0a09390a36c2ade","permalink":"https://dfki-nlp.github.io/admin/config.yml","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/admin/config.yml","section":"","summary":"","tags":null,"title":"","type":"wowchemycms"},{"authors":null,"categories":null,"content":"","date":-62135596800,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":-62135596800,"objectID":"6d99026b9e19e4fa43d5aadf147c7176","permalink":"https://dfki-nlp.github.io/contact/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/contact/","section":"","summary":"","tags":null,"title":"","type":"widget_page"},{"authors":null,"categories":null,"content":"","date":-62135596800,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":-62135596800,"objectID":"c1d17ff2b20dca0ad6653a3161942b64","permalink":"https://dfki-nlp.github.io/people/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/people/","section":"","summary":"","tags":null,"title":"","type":"widget_page"},{"authors":null,"categories":null,"content":"The German Research Center for Artificial Intelligence (Deutsches Forschungszentrum für Künstliche Intelligenz GmbH (DFKI)) and its staff are committed to goal- and risk-oriented information privacy and the fundamental right to the protection of personal data. In this data protection policy we inform you about the processing of your personal data when visiting and using our web site.\nResponsible service provider German Research Center for Artificial Intelligence (DFKI)\nPhone: +49 (0)631 / 205 75-0, Email: info@dfki.de\nData Protection Officer Phone: +49 (0)631 / 205 75-0\nEmail: datenschutz@dfki.de\nIntended use Provision of the information offering in the course of the public communication of the DFKI Establishment of contact and correspondence with visitors and users Anonymous and protected use Visit and usage of our web site are anonymous. At our web site personal data are only collected to the technically necessary extent. The processed data will not be transmitted to any third parties or otherwise disclosed, except on the basis of concrete lawful obligations. Within our information offering we do not embed information or service offerings of third party providers.\nWhile using our web site the data transmission in the internet is being protected by a generally accepted secure encryption procedure and hence cannot easily be eavesdropped or tampered.\nAccess data Every time our website is accessed, usage, traffic and connection data are collected, temporarily stored in a log file and generally deleted after 90 days.\nThe following data is stored for each access/retrieval:\n IP address Transmitted user agent information (in particular type/version of web browser, operating system, etc.) Transmitted referrer information (URL of the referring page) Date and time of access/retrieval Submitted Access Method/Function Transmitted input values (search terms, etc.) Accessed page or retrieved file Amount of data transferred Access/retrieval processing status The processing of the access data is lawful because it is necessary for the purposes of the legitimate interests pursued by DFKI. The legitimate interests pursued by DFKI are the adaptation and optimisation of the information offering and the investigation, detection and prosecution of illegal activities in connection with the usage of our web site.\nThe stored data records can be statistically evaluated in order to adapt and optimize our web site to the needs of our visitors. Any techniques that offer the possibility to retrace the access characteristics of users (tracking) will not be applied. The creation of user profiles and automated decision-making based on it is precluded.\nThe stored data records are not attributable to specific persons. They are generally not being combined with other data sources. However, the stored data can be analysed and combined with other data sources, if we become aware of concrete indications of any illegal usage.\nCookies We use so-called cookies on our web site. Cookies are small files that are being stored by your web browser. The cookies used on our web site do not harm your computer and do not contain any malicious software. They offer a user-friendly and effective usage of our web site. We do not use cookies for marketing purposes.\nWe transmit so-called session cookies to your web browser. They are valid only for the duration of your visit on our web site and they do not have any meaning outside of our web site. The session cookies are needed in order to identify your session with a unique number during your visit and to transmit our contents in your preferred language. At the end of your visit the session cookies will be automatically deleted upon termination of your web browser.\nWe also transmit permanent cookies to your web browser with a validity period of at most 365 days. We are exclusively using these cookies in order to respect your settings for the type of presentation (normal, inverted) and for the font size. Furthermore, it will be recorded whether you’ve taken notice of the information about the usage of cookies in your web browser.\nYou can adjust your web browser such that you will be informed on setting cookies and allow cookies on an individual basis resp. exclude the acceptance of cookies for specific cases or generally. You also can adjust the automatic deletion of cookies upon termination of your web browser. Upon deactivation of cookies the functionality of our web site can be limited. In any case, our information offering is available to its full extent.\nSocial media We do not embed social media plug-ins on our web site. When you are visiting our web site no data are transmitted to social media services. Profiling by any third parties hence is precluded. You though have the option to change over to our information offerings on Facebook, Twitter and YouTube. For the usage of these services we refer to the data protection policies of the respective service providers. We are processing your personal data within the social networks insofar as you post contributions, send messages or otherwise communicate with us.\nCorrespondence You have the option to contact us by e-mail. We will use your e-mail address and other personal contact data for the correspondence with you. Due to lawful obligation every e-mail correspondence will be archived. Subject to our legitimate interests your e-mail address and other personal contact data can be stored in our contact data base. In this case you will receive a corresponding information on the processing of your contact data.\nAccess and Intervention Besides the information in this data protection policy you have the right of access to your personal data. To ensure fair data processing, you have the following rights:\n The right to rectification and completion of your personal data The right to erasure of your personal data The right to restriction of the processing of your personal data The right to object to the processing of your personal data on grounds related to your particular situation To exercise these rights, please contact our data protection officer.\nRight to lodge a complaint You have the right to lodge a complaint with a supervisory authority if you consider that the processing of your personal data infringes statutory data protection regulations.\n","date":-62135596800,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":-62135596800,"objectID":"18d05a63a1c8d7ed973cc51838494e41","permalink":"https://dfki-nlp.github.io/privacy/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/privacy/","section":"","summary":"The German Research Center for Artificial Intelligence (Deutsches Forschungszentrum für Künstliche Intelligenz GmbH (DFKI)) and its staff are committed to goal- and risk-oriented information privacy and the fundamental right to the protection of personal data.","tags":null,"title":"Data Protection Notice","type":"page"},{"authors":null,"categories":null,"content":"Responsible service provider Responsible for the content of the domain dfki-nlp.github.io from the point of view of § 5 TMG:\nDeutsches Forschungszentrum für Künstliche Intelligenz GmbH (DFKI) Management:\nProf. Dr. Antonio Krüger\nHelmut Ditzer\nTrippstadter Str. 122\n67663 Kaiserslautern\nGermany\nPhone: +49 631 20575 0\nFax: +49 631 20575 5030\nEmail: info@dfki.de\nRegister Court: Amtsgericht Kaiserslautern\nRegister Number: HRB 2313\nID-Number: DE 148 646 973\nThe person responsible for the editorial content of the domain cora4nlp.github.io of the German Research Center for Artificial Intelligence GmbH within the meaning of § 18 para. 2 MStV is:\nDr. Leonhard Hennig, Senior Researcher\nDFKI Lab Berlin\nAlt-Moabit 91c\nD-10559 Berlin\nTel: +49 (0)30 / 238 95-0\nEmail: leonhard.hennig@dfki.de\nWebsite URL: www.dfki.de\nLegal notice concerning liability for proprietary content As a content provider in accordance with Section 7 (1) of the German Telemedia Act (Telemediengesetz), the Deutsches Forschungszentrum für Künstliche Intelligenz GmbH (DFKI) is responsible for its own content that is used pursuant to the general laws. The DFKI endeavors to ensure that the information provided on this website is accurate and current. Nevertheless, errors and uncertainties cannot be entirely ruled out. For this reason, the DFKI undertakes no liability for ensuring that the provided information is current, accurate or complete, and is not responsible for its quality. The DFKI is not liable for material or immaterial damages caused directly or indirectly by the use or non-use of the offered information, or by the use of erroneous and incomplete information, unless willful or grossly negligent fault can be demonstrated. This also applies with respect to software or data provided for download. The DFKI reserves the right to modify, expand or delete parts of the website or the entire website without separate announcement, or to cease publication temporarily or definitively.\nLegal notices for third party content and references to external websites As a service provider, we are responsible for our content on these pages in accordance with general law (§ 7 (1) TMG). According to § 8 – 10 TMG, as a service provider, we are not obliged to monior third-party information that is transmitted or stored, or to investigate circumstances that indicate illegal activity. Obligations to remove or block the use of information according to general laws remain unaffected. However, liability in this regard is only possible form the point in time at which we become aware of a specific legal violation. If we become aware of legal violations, we will remove this content immediately. Cross- references (“links”) to the content providers are to be distinguished from own content. Our offer contains links to external third party website. The respective provider is always responsible for the content of the linked external pages. We cannot accept any liability for the content for the content of the linked pages. After this external content was checked by the DFKI when the link was recenty set, it was checked whether there were any preliminary legal violations. At the time oft he review, no legal violations were apparent. However, it cannot be ruled out that the content may be changed afterwards by teh resprective providers. A permanent control oft he content of the linked pages is not reasonable without ecidence of a violation of the law. The DFKI does not constantly check the content to which it refers in ist offer for changes that could justify a new responsibility. If you are oft he option that the linked external pages violate applicable law or have otherwie inappropriate content, please inform us directly: info@dfki.de. Should the DFKI discover or receive a message that an external offer to whicht is has linked triggers civil or criminal liability, the DFKI will remove the link to this offer.\nLegal notice concerning copyright The layout of the homepage, the graphics used and other content on the DFKI website are protected by copyright. The reproduction, processing, distribution and any type of use outside the boundaries of copyright law require the written approval of the DFKI (in writing). Insofar as any content on this page was not prepared by the DFKI, the copyrights of third parties will be observed. If you become aware of a copyright breach nevertheless, please inform us accordingly. Upon becoming aware of relevant legal breaches, the DFKI will remove such content immediately.\n","date":-62135596800,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":-62135596800,"objectID":"9b10c1f64082d3869fd4cb1f85809430","permalink":"https://dfki-nlp.github.io/terms/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/terms/","section":"","summary":"Responsible service provider Responsible for the content of the domain dfki-nlp.github.io from the point of view of § 5 TMG:\nDeutsches Forschungszentrum für Künstliche Intelligenz GmbH (DFKI) Management:\nProf. Dr. Antonio Krüger","tags":null,"title":"Legal Information","type":"page"}]
\ No newline at end of file
diff --git a/index.xml b/index.xml
index 8ce947b..77d12cf 100644
--- a/index.xml
+++ b/index.xml
@@ -1,4 +1,4 @@
-
DFKI-NLP DFKI-NLP Wowchemy (https://wowchemy.com) en-us © DFKI-NLP 2019-2024 Wed, 23 Oct 2024 00:00:00 +0000 https://dfki-nlp.github.io/images/icon_hu15d4cd24fad375030c8e4d8f45deb950_164099_512x512_fill_lanczos_center_2.png DFKI-NLP Unsupervised SapBERT-based bi-encoders for medical concept annotation of clinical narratives with SNOMED CT Wed, 23 Oct 2024 00:00:00 +0000 https://dfki-nlp.github.io/publication/sage2024/ CoXQL: A Dataset for Parsing Explanation Requests in Conversational XAI Systems Thu, 26 Sep 2024 10:33:03 +0200 https://dfki-nlp.github.io/publication/wang-etal-2024-coxql/ Enhancing Editorial Tasks: A Case Study on Rewriting Customer Help Page Contents Using Large Language Models Sat, 21 Sep 2024 10:33:03 +0200 https://dfki-nlp.github.io/publication/gabryszak-etal-2024-enhancing/ LLM-based FAQ Rewrites Wed, 18 Sep 2024 00:00:00 +0000 https://dfki-nlp.github.io/dataset/faq-rewrites/ German Voter Personas can Radicalize LLM Chatbots via the Echo Chamber Effect Sun, 08 Sep 2024 10:33:03 +0200 https://dfki-nlp.github.io/publication/bleick-etal-2024-german/ Two papers accepted to INLG 2024 Thu, 22 Aug 2024 09:24:01 +0200 https://dfki-nlp.github.io/post/inlg2024/ <p>Two papers from DFKI NLP researchers have been accepted at the <a href="https://inlg2024.github.io/" target="_blank" rel="noopener">17th International Natural Language Generation Conference (INLG 2024)</a> that will take place September 23-27 in Tokyo, Japan. One paper presents a case study on using large language models to produce customer-friendly help page contents from more technical text, and includes a text quality evaluation by experienced editors. The other paper analyzes echo chamber effects in LLM-based chatbots in political conversations.</p>
+DFKI-NLP DFKI-NLP Wowchemy (https://wowchemy.com) en-us © DFKI-NLP 2019-2024 Wed, 23 Oct 2024 00:00:00 +0000 https://dfki-nlp.github.io/images/icon_hu15d4cd24fad375030c8e4d8f45deb950_164099_512x512_fill_lanczos_center_2.png DFKI-NLP Unsupervised SapBERT-based bi-encoders for medical concept annotation of clinical narratives with SNOMED CT Wed, 23 Oct 2024 00:00:00 +0000 https://dfki-nlp.github.io/publication/sage2024/ CoXQL: A Dataset for Parsing Explanation Requests in Conversational XAI Systems Thu, 26 Sep 2024 10:33:03 +0200 https://dfki-nlp.github.io/publication/wang-etal-2024-coxql/ Large Language Models for Clinical Text Cleansing Enhance Medical Concept Normalization Wed, 25 Sep 2024 00:00:00 +0000 https://dfki-nlp.github.io/publication/ieee2024/ Enhancing Editorial Tasks: A Case Study on Rewriting Customer Help Page Contents Using Large Language Models Sat, 21 Sep 2024 10:33:03 +0200 https://dfki-nlp.github.io/publication/gabryszak-etal-2024-enhancing/ LLM-based FAQ Rewrites Wed, 18 Sep 2024 00:00:00 +0000 https://dfki-nlp.github.io/dataset/faq-rewrites/ German Voter Personas can Radicalize LLM Chatbots via the Echo Chamber Effect Sun, 08 Sep 2024 10:33:03 +0200 https://dfki-nlp.github.io/publication/bleick-etal-2024-german/ Two papers accepted to INLG 2024 Thu, 22 Aug 2024 09:24:01 +0200 https://dfki-nlp.github.io/post/inlg2024/ <p>Two papers from DFKI NLP researchers have been accepted at the <a href="https://inlg2024.github.io/" target="_blank" rel="noopener">17th International Natural Language Generation Conference (INLG 2024)</a> that will take place September 23-27 in Tokyo, Japan. One paper presents a case study on using large language models to produce customer-friendly help page contents from more technical text, and includes a text quality evaluation by experienced editors. The other paper analyzes echo chamber effects in LLM-based chatbots in political conversations.</p>
<p>
<div class="pub-list-item" style="margin-bottom: 1rem">
<i class="far fa-file-alt pub-icon" aria-hidden="true"></i>
diff --git a/project/bbdc2/index.html b/project/bbdc2/index.html
index 3d8c94a..2d60669 100644
--- a/project/bbdc2/index.html
+++ b/project/bbdc2/index.html
@@ -1,7 +1,7 @@
BBDC2 | DFKI-NLP 
