To what degree should we ascribe cognitive capacities to Large Language Models (LLMs), such as the ability to reason about intentions and beliefs known as Theory of Mind (ToM)? Here we add to this... Show moreTo what degree should we ascribe cognitive capacities to Large Language Models (LLMs), such as the ability to reason about intentions and beliefs known as Theory of Mind (ToM)? Here we add to this emerging debate by (i) testing 11 base- and instruction-tuned LLMs on capabilities relevant to ToM beyond the dominant false-belief paradigm, including non-literal language usage and recursive intentionality; (ii) using newly rewritten versions of standardized tests to gauge LLMs’ robustness; (iii) prompting and scoring for open besides closed questions; and (iv) benchmarking LLM performance against that of children aged 7-10 on the same tasks. We find that instruction-tuned LLMs from the GPT family outperform other models, and often also children. Base-LLMs are mostly unable to solve ToM tasks, even with specialized prompting. We suggest that the interlinked evolution and development of language and ToM may help explain what instruction-tuning adds: rewarding cooperative communication that takes into account interlocutor and context. We conclude by arguing for a nuanced perspective on ToM in LLMs. Show less
Dijk, B.M.A. van; Kouwenhoven, T.; Spruit, M.; Duijn, M.J. van 2023
Current Large Language Models (LLMs) are unparalleled in their ability to generate grammatically correct, fluent text. LLMs are appearing rapidly, and debates on LLM capacities have taken off, but... Show moreCurrent Large Language Models (LLMs) are unparalleled in their ability to generate grammatically correct, fluent text. LLMs are appearing rapidly, and debates on LLM capacities have taken off, but reflection is lagging behind. Thus, in this position paper, we first zoom in on the debate and critically assess three points recurring in critiques of LLM capacities: i) that LLMs only parrot statistical patterns in the training data; ii) that LLMs master formal but not functional language competence; and iii) that language learning in LLMs cannot inform human language learning. Drawing on empirical and theoretical arguments, we show that these points need more nuance. Second, we outline a pragmatic perspective on the issue of ‘real’ understanding and intentionality in LLMs. Understanding and intentionality pertain to unobservable mental states we attribute to other humans because they have pragmatic value: they allow us to abstract away from complex underlying mechanics and predict behaviour effectively. We reflect on the circumstances under which it would make sense for humans to similarly attribute mental states to LLMs, thereby outlining a pragmatic philosophical context for LLMs as an increasingly prominent technology in society. Show less
The recent surge in deployment and use of generative machine learning models has sparked an interest in the relationships between AI and creativity, or more specifically into the question and... Show moreThe recent surge in deployment and use of generative machine learning models has sparked an interest in the relationships between AI and creativity, or more specifically into the question and debate of whether machines can exhibit human-level creativity. This is by no means a new discussion, going back in time decades if not centuries. The debate was approached from multiple angles, and a general consensus was not yet reached. In this position paper, we present the long-standing debate as it formed across various fields such as cognitive science, philosophy, and computing, approaching it mainly from a historical perspective. Along the way we identify how the various views relate to recent developments in machine learning models and argue our own position regarding the question of whether machines can exhibit human-level creativity. As such we aim to involve computer scientists and AI practitioners into the ongoing debate. Show less
The rapid advancement of Artificial Intelligence (AI) has opened up new frontiers of technological possibilities, yet the traditional education system in Indonesia has largely remained entrenched... Show moreThe rapid advancement of Artificial Intelligence (AI) has opened up new frontiers of technological possibilities, yet the traditional education system in Indonesia has largely remained entrenched in con- ventional practices. Despite Indonesia’s recognition for its vibrant AI innovation scene and its diverse student population, there is a signifi- cant gap in exploring the multifaceted implications of AI in, for, and by education within the country. This paper aims to address this gap by delving into three key areas: AI tutors, governance, and (virtue-based) ethical considerations. Drawing insights from both global and Islamic literature, we first examine the discourse surrounding AI tutors within Indonesia’s education system. Next, we discuss the potential applications of AI in governance, including the role of the government and the emer- gence of AI-related education in Indonesia. Thirdly, we contemplate an ethical framework encompassing issues of inequality, public policy, and Islamic-based principles. Throughout, this paper emphasizes the critical importance of examining these three facets of AI’s impact in education. Ultimately, this research raises the intriguing question of how education and AI will mutually shape each other in the future, urging further ex- ploration of this dynamic relationship. Show less
Onderzoek naar werkwoordspelling laat zien dat fouten in de vervoeging veroorzaakt worden door o.a. tijdsdruk en homofone werkwoordsvormen, zoals 'verhuist'-'verhuisd' (cf. Sandra, Frisson &... Show moreOnderzoek naar werkwoordspelling laat zien dat fouten in de vervoeging veroorzaakt worden door o.a. tijdsdruk en homofone werkwoordsvormen, zoals 'verhuist'-'verhuisd' (cf. Sandra, Frisson & Daems, 1999; Chamalaun, 2023). Wij onderzochten een ander struikelblok voor veel leerlingen: de spelling van leenwerkwoorden. Dergelijke werkwoorden dienen volgens de officiële spellingregels van de Nederlandse taal te worden vervoegd (Taalunie, 2011, p. 36), maar veelgebruikte ezelsbruggetjes, zoals die om de stam te vinden (wij-vorm minus 'en'), leveren hier het incorrecte 'gam' in plaats van 'game' op. Daarnaast wordt de spelling van leenwerkwoorden, zo blijkt uit onze data, vaak 'verengelst' - in plaats van 'genetflixt' wordt 'genetflixed' geschreven. In deze bijdrage beantwoorden wij de vraag in welke mate de leenwoordstatus van een werkwoord een rol speelt bij het maken van spelfouten. Daarnaast laten we zien in hoeverre er een relatie bestaat tot typen leenwerkwoorden uitgesplitst naar stameinde, zoals -e en -x in bovenstaande voorbeelden. Daartoe vergeleken we in een grootschalige analyse van recente data afkomstig van website Gespeld (Reuneker, z.d.) de door leerlingen gemaakte fouten in de vervoegingen van leenwerkwoorden en niet-leenwerkwoorden. We laten zien dat een beperkt aantal stamuitgangen het grootste deel van de fouten veroorzaakt en we doen suggesties voor het spellingonderwijs. Show less
Tseng, R.; Verberne, S.; Putten, P.W.H. van der 2023
In this paper we investigate a fundamental tension in historical games: how they promise to let us experience the past as a playground while at the same time not offering the freedoms to radically... Show moreIn this paper we investigate a fundamental tension in historical games: how they promise to let us experience the past as a playground while at the same time not offering the freedoms to radically explore and experiment with it. Historical games, for all their simulative and immersive power, are still rather stuck in specific forms of past-play. To investigate these borders, and what could lie beyond, we will employ a new political theory of the past, vested in archaeological and anthropological scholarship, as developed by Graeber and Wengrow in their book The Dawn of Everything: A New history of Humanity. In particular, we will use their ideas about fundamental freedoms to analyse how and to what extent processes and moments of radical historical change can be experienced in games. We will do so by focusing on the popular and influential game series Sid Meier’s Civilization. Show less
Digital tracking poses a significant and multifaceted threat to personal privacy and integrity. Tracking techniques, such as the use of cookies and scripts, are widespread on the World Wide Web and... Show moreDigital tracking poses a significant and multifaceted threat to personal privacy and integrity. Tracking techniques, such as the use of cookies and scripts, are widespread on the World Wide Web and have become more pervasive in the past decade. This paper focuses on the historical analysis of tracking practices specifically on educational websites, which require particular attention due to their often mandatory usage by users, including young individuals who may not adequately assess privacy implications. The paper proposes a framework for comparing tracking activities on a specific domain of websites by contrasting a sample of these sites with a control group consisting of sites with comparable traffic levels, but without a specific functional purpose. This comparative analysis allows us to evaluate the distinctive evolution of tracking on educational platforms against a standard benchmark. Our findings reveal that although educational websites initially demonstrated lower levels of tracking, their growth rate from 2012 to 2021 has exceeded that of the control group, resulting in higher levels of tracking at present. Through our investigation into the expansion of various types of trackers, we suggest that the accelerated growth of tracking on educational websites is partly attributable to the increased use of interactive features, facilitated by third-party services that enable the collection of user data. The paper concludes by proposing ways in which web developers can safeguard their design choices to mitigate user exposure to tracking. Show less
Abolghasemi, M.A.; Verberne, S.; Askari, A.; Azzopardi, L. 2023
