Leiden University Scholarly Publications

Persistent URL of this record https://hdl.handle.net/1887/138695

Documents

In Collections

This item can be found in the following collections:

Dirkson, A.R.; Verberne, S.; Kraaij, W. (2020)

Conversation-Aware Filtering of Online Patient Forum Messages

Article in monograph or in proceedings
Previous approaches to NLP tasks on online patient forums have been limited to single posts as units, thereby neglecting the overarching conversational structure. In this paper we explore the benefit of exploiting conversational context for filtering posts relevant to a specific medical topic. We experiment with two approaches to add conversational context to a BERT model: a sequential CRF layer and manually engineered features. Although neither approach can outperform the F1 score of the BERT baseline, we find that adding a sequential layer improves precision for all target classes whereas adding a non-sequential layer with manually engineered features leads to a higher recall for two out of three target classes. Thus, depending on the end goal, conversation-aware modelling may be beneficial for identifying relevant messages. We hope our findings encourage other researchers in this domain to move beyond studying messages in isolation towards more discourse-based data collection...Show morePrevious approaches to NLP tasks on online patient forums have been limited to single posts as units, thereby neglecting the overarching conversational structure. In this paper we explore the benefit of exploiting conversational context for filtering posts relevant to a specific medical topic. We experiment with two approaches to add conversational context to a BERT model: a sequential CRF layer and manually engineered features. Although neither approach can outperform the F1 score of the BERT baseline, we find that adding a sequential layer improves precision for all target classes whereas adding a non-sequential layer with manually engineered features leads to a higher recall for two out of three target classes. Thus, depending on the end goal, conversation-aware modelling may be beneficial for identifying relevant messages. We hope our findings encourage other researchers in this domain to move beyond studying messages in isolation towards more discourse-based data collection and classification. We release our code for the purpose of follow-up research.
Show less
All authors
Dirkson, A.R.; Verberne, S.; Kraaij, W.
Editor(s)
Gonzalez-Hernandez, G.; Klein, A.Z.; Flores, I.; Weissenbacher, D.; Magge, A.; O'Connor, K.; Sarker, A.; Minard, A.L.; Tutubalina, E.; Miftahutdinov, Z.; Alimova, I.
Date
2020
Title of host publication
Proceedings of the Fifth Social Media Mining for Health Applications Workshop & Shared Task
Pages
11 - 18
Link
https://www.aclweb.org/anthology/2020.smm4h-1.2/

Conference

Conference
Fifth Social Media Mining for Health Applications Workshop and Shared Task at COLING 2020
Date
2020-12-12
Location
Barcelona