Primer: Promise and Challenges of Language Models in the Clinical Domain
ML/NLP for healthcare, Postdoc at Brigham and Women's Hospital / Harvard Medical School
The field of natural language processing (NLP) has seen rapid advances in recent years with the advent of large language models (LLMs) such as BERT and GPT3. These models are pretrained on large amounts of unlabelled text data and then applied to downstream tasks of interest. This paradigm has demonstrated significant improvement in performance across a range of NLP tasks, including in clinical and biomedical domains. In this primer, I will provide an overview of the architecture and training of the models underlying these advances, motivate the need for domain-specific models such as BioBERT and ClinicalBERT, and describe the promise and challenges of applying LLMs to clinical and biomedical data. We’ll cover how to leverage these models in settings with long texts or with limited annotated data and discuss ethical considerations regarding bias and privacy when training LLMs on clinical text. Finally, I’ll demonstrate how to train a LLM on a task of your choice using the Hugging Face Transformers library.