To what degree should we ascribe cognitive capacities to Large Language Models (LLMs), such as the ability to reason about intentions and beliefs known as Theory of Mind (ToM)? Here we add to this... Show moreTo what degree should we ascribe cognitive capacities to Large Language Models (LLMs), such as the ability to reason about intentions and beliefs known as Theory of Mind (ToM)? Here we add to this emerging debate by (i) testing 11 base- and instruction-tuned LLMs on capabilities relevant to ToM beyond the dominant false-belief paradigm, including non-literal language usage and recursive intentionality; (ii) using newly rewritten versions of standardized tests to gauge LLMs’ robustness; (iii) prompting and scoring for open besides closed questions; and (iv) benchmarking LLM performance against that of children aged 7-10 on the same tasks. We find that instruction-tuned LLMs from the GPT family outperform other models, and often also children. Base-LLMs are mostly unable to solve ToM tasks, even with specialized prompting. We suggest that the interlinked evolution and development of language and ToM may help explain what instruction-tuning adds: rewarding cooperative communication that takes into account interlocutor and context. We conclude by arguing for a nuanced perspective on ToM in LLMs. Show less
Douben, H.C.W.; Nellist, M.; Unen, L. van; Elfferich, P.; Kasteleijn, E.; Hoogeveen-Westerveld, M.; ... ; Ham, T.J. van 2022
Neurofibromatosis type 1 (NF1) is caused by inactivating mutations in NF1. Due to the size, complexity, and high mutation rate at the NF1 locus, the identification of causative variants can be... Show moreNeurofibromatosis type 1 (NF1) is caused by inactivating mutations in NF1. Due to the size, complexity, and high mutation rate at the NF1 locus, the identification of causative variants can be challenging. To obtain a molecular diagnosis in 15 individuals meeting diagnostic criteria for NF1, we performed transcriptome analysis (RNA-seq) on RNA obtained from cultured skin fibroblasts. In each case, routine molecular DNA diagnostics had failed to identify a disease-causing variant in NF1. A pathogenic variant or abnormal mRNA splicing was identified in 13 cases: 6 deep intronic variants and 2 transposon insertions causing noncanonical splicing, 3 postzygotic changes, 1 branch point mutation and, in 1 case, abnormal splicing for which the responsible DNA change remains to be identified. These findings helped resolve the molecular findings for an additional 17 individuals in multiple families with NF1, demonstrating the utility of skin-fibroblast-based transcriptome analysis for molecular diagnostics. RNA-seq improves mutation detection in NF1 and provides a powerful complementary approach to DNA-based methods. Importantly, our approach is applicable to other genetic disorders, particularly those caused by a wide variety of variants in a limited number of genes and specifically for individuals in whom routine molecular DNA diagnostics did not identify the causative variant. Show less
