AMIA 2024 Annual Symposium Tutorial on+ Development and Evaluation of Large Language Models in Healthcare Applications |
+ ||
+ Location: San Francisco, CA, USA + Time: November 9 - 13, 2024 + |
+ ||
Language models are being increasingly used in natural language processing (NLP) applications, which require neither the development of a task-specific architecture nor customized training on large datasets. In particular, large language models (LLMs), such as the GPT1, PaLM2, and Llama-23, have demonstrated significant advances in NLP tasks 4–6. On the other hand, concerns have also been raised about the impact of these tools in health care, education, research, and beyond. One notable concern is the potential for LLMs to reinforce disparities in healthcare, as these models are typically trained on data that is historically biased against certain disadvantaged groups. Another concern is the potential for LLMs to be applied for malicious purposes. Although it is widely accepted that LLMs should be used with integrity, transparency, and honesty, how to appropriately do so and, if needed, regulate the development, and use of this technology needs further discussion.
+ +This course provides students with an understanding of LLMs, using ChatGPT, Llama-2, and other models as examples, and their applications in health. Students will acquire knowledge of natural language processing, large language models, chain-of-though, Retrieval-Augmented Generation (RAG), and the range of prompting methods available for processing clinical text. Hands-on experience and a toolkit will provide useful skills for managing text data to solve a variety of problems in the health domain.
+ +We believe that the proposed tutorial is timely and urgently needed for AMIA stakeholders, including informaticists from a broad array of disciplines, clinicians, software developers, and IT professionals, to learn how to develop and use these models to ensure that their potential benefits are realized while any potential risks and negative consequences are minimized. This tutorial will also likely be one of many conversations at AMIA 2024 about this issue as we learn more about LLMs, their capacity, and their potential impact on healthcare.
+45 min. Topic 1: An introduction to LLMs and their development in the medical domain (Hua Xu)
+45 min. Topic 2: Integration of LLMs into NLP and other clinical decision-making tasks (Yanshan Wang)
+45 min. Topic 3: Multimodal LLMs and their applications (Yifan Peng)
+30 min. Open discussion
+Hua Xu, Ph.D., FACMI, is Robert T. McCluskey Professor and Vice Chair for Research and Development at the Section of Biomedical Informatics and Data Science of Yale School of Medicine. He also serves as Assistant Dean for Biomedical Informatics at Yale School of Medicine. He has worked on different clinical NLP topics and has built multiple clinical NLP systems. Dr. Xu served as the Chair of the AMIA NLP working group between 2014-2015 and currently leads the OHDSI NLP working group. He taught NLP tutorials at various conferences such as AMIA, Medinfo, AIME, etc. Recently, Dr. Xu has worked on building foundation medical LLMs including the recently released Me LLaMA models based on the open LLaMA2 model. He will provide a generation introduction to LLMs and hands-on experience in developing medical LLMs and their applications in clinical NLP tasks such as information extraction. +
+ +Yanshan Wang, Ph.D., FAMIA, is vice chair of Research and assistant professor within the Department of Health Information Management at the University of Pittsburgh. His research interests focus on artificial intelligence (AI), natural language processing (NLP), and machine/deep learning methodologies and applications in health care. Dr. Wang has led several NIH-funded projects, which aimed to develop NLP and AI algorithms to automatically extract information from free-text electronic health records (EHRs). He has over 60 peer-reviewed publications. Dr. Wang has been actively serving the informatics and NLP communities. He has served on a Student Paper Competition Committee for the AMIA Annual Symposium and was an associate editor for MedInfo conference. He is also a regular reviewer for a dozen of prestigious journals, such as Nature Communications, JAMIA, and JBI. Wang also organized several shared tasks, including the first BioCreative/OHNLP challenge in 2018 and the second n2c2/OHNLP challenge in 2019, to encourage the informatics and NLP communities to tackle NLP problems in the clinical domain. He is also a steering committee member for the HealthNLP workshop. In 2020, he was inducted into the Fellows of AMIA (FAMIA). Dr. Wang serves as the Chair of the AMIA NLP working group between 2023-2024. +
+ +Yifan Peng (Moderator), Ph.D., Assistant Professor in the Division of Health Sciences Department of Population Health Sciences at Weill Cornell Medicine. + Dr. Peng's main research interests include BioNLP and medical image analysis. To facilitate research on language representations in the biomedicine domain, one of his studies present the Biomedical Language Understanding Evaluation (BLUE) benchmark, a collection of resources for evaluating and analyzing biomedical natural language representation models. Detailed analysis shows that BLUE can be used to evaluate the capacity of the models to understand the biomedicine text and, moreover, to shed light on the future directions for developing biomedicine language representations. As the panel moderator, Dr. Peng will describe the current state of LLMs and list their unique opportunities and challenges compared to other language models. +
+Please contact Yifan Peng if you have question. The webpage template is by the courtesy of awesome Georgia.
+
