Leiden University Scholarly Publications

Persistent URL of this record https://hdl.handle.net/1887/4270917

In Collections

This item can be found in the following collections:

Bakker, R.M.; Kalkman, G.J.; Tolios, I.; Blok, D.; Veldhuis, G.A.; Raaijmakers, S.A.; Boer, M.H.T. de (2023)

Exploring knowledge extraction techniques for system dynamics modelling: comparative analysis and considerations

Article in monograph or in proceedings
Knowledge graphs, semantic networks, and similar formal knowledge modelling structures have become popular solutions to express linguistic entities and relations between them. They are computer representations of human knowledge with all its semantic richness, and therefore powerful tools for structuring all kinds of data. Creating such models is, however, an extensive task which involves meticulous manual work by developers and experts. With the large amount of online textual information, models ideally can be automatically extracted, enriched, and maintained using Natural Language Processing (NLP) techniques. In this paper, we explore these techniques and present a comparative study of traditional knowledge graph extraction techniques (Keyword Extraction, Named Entity Recognition, co-occurrences) and a state-of-the-art large language model (LLM) in the context of systems dynamics modelling. System dynamics modelling is an approach to model the behaviour of complex systems, such...Show moreKnowledge graphs, semantic networks, and similar formal knowledge modelling structures have become popular solutions to express linguistic entities and relations between them. They are computer representations of human knowledge with all its semantic richness, and therefore powerful tools for structuring all kinds of data. Creating such models is, however, an extensive task which involves meticulous manual work by developers and experts. With the large amount of online textual information, models ideally can be automatically extracted, enriched, and maintained using Natural Language Processing (NLP) techniques. In this paper, we explore these techniques and present a comparative study of traditional knowledge graph extraction techniques (Keyword Extraction, Named Entity Recognition, co-occurrences) and a state-of-the-art large language model (LLM) in the context of systems dynamics modelling. System dynamics modelling is an approach to model the behaviour of complex systems, such as policy developments and politics. Often graphs and diagrams are used to support this process. The main contributions of this paper are twofold: first, a comparison and evaluation of traditional NLP methods compared to an LLM; second, an interface for including human feedback, enhancing future collaboration between system and domain expert. Our study informs practitioners about the suitability of these techniques to support problem analysis using system dynamics modelling and paves the way for future research in refining and integrating approaches.
Show less
Information extraction
Knowledge graphs
LLMs
System dynamics
All authors
Bakker, R.M.; Kalkman, G.J.; Tolios, I.; Blok, D.; Veldhuis, G.A.; Raaijmakers, S.A.; Boer, M.H.T. de
Date
2023-11-01

Conference

Conference
35th Benelux Conference, BNAIC/Benelearn 2023
Date
2023-11-08 - 2023-11-10
Location
Delft, Netherlands