Data-Driven Lexical Normalization for Medical Social Media

Author(s):
Dirkson, A.R. (0000-0002-4332-0296); Verberne, S. (0000-0002-9609-9505); Sarker, A.; Kraaij, W. (0000-0001-7797-619X)
Published
2019
Publication type
Article / Letter to editor

Abstract

In the medical domain, user-generated social media text is increasingly used as a valuable

complementary knowledge source to scientific medical literature. The extraction of this knowledge is

complicated by colloquial language use and misspellings. However, lexical normalization of such

data has not been addressed effectively. This paper presents a data-driven lexical normalization pipeline with a novel spelling correction module for medical social media. Our method significantly outperforms state-of-the-art spelling correction methods and can detect mistakes with an F1 of 0.63 despite extreme imbalance in the data. We also present the first corpus for spelling mistake detection and correction in a medical patient forum.

Date issued
2019-08-20
Language
English
Publisher
MDPI
Is part of
Multimodal Technologies and Interaction (2019), Vol. 3, no. 3, p. 60
hdl
1887/77332
doi
10.3390/mti3030060
URL
https://doi.org/10.3390/mti3030060