Cluster inference based on spatial extent thresholding is a popular analysis method multiple testing in spatial data, and is frequently used for finding activated brain areas in neuroimaging.... Show moreCluster inference based on spatial extent thresholding is a popular analysis method multiple testing in spatial data, and is frequently used for finding activated brain areas in neuroimaging. However, the method has several well-known issues. While powerful for finding regions with some activation, the method as currently defined does not allow any further quantification or localisation of signal. In this paper, we repair this gap. We show that cluster-extent inference can be used (1) to infer the presence of signal in any region of interest and (2) to quantify the percentage of activation in such regions. These additional inferences come for free, i.e. they do not require any further adjustment of the alpha-level of tests, while retaining full family-wise error control. We achieve this extension of the possibilities of cluster inference by embedding the method into a closed testing procedure, and solving the graph-theoretic k-separator problem that results from this embedding. We demonstrate the usefulness of the improved method in a large-scale application to neuroimaging data from the Neurovault database. Show less
We propose a permutation-based method for testing a large collection of hypotheses simultaneously. Our method provides lower bounds for the number of true discoveries in any selected subset of... Show moreWe propose a permutation-based method for testing a large collection of hypotheses simultaneously. Our method provides lower bounds for the number of true discoveries in any selected subset of hypotheses. These bounds are simultaneously valid with high confidence. The methodology is particularly useful in functional Magnetic Resonance Imaging cluster analysis, where it provides a confidence statement on the percentage of truly activated voxels within clusters of voxels, avoiding the well-known spatial specificity paradox. We offer a user-friendly tool to estimate the percentage of true discoveries for each cluster while controlling the family-wise error rate for multiple testing and taking into account that the cluster was chosen in a data-driven way. The method adapts to the spatial correlation structure that characterizes functional Magnetic Resonance Imaging data, gaining power over parametric approaches. Show less
Botvinik-Nezer, R.; Holzmeister, F.; Camerer, C.F.; Dreber, A.; Huber, J.; Johannesson, M.; ... ; Schonberg, T. 2020
Data analysis workflows in many scientific domains have become increasingly complex and flexible. Here we assess the effect of this flexibility on the results of functional magnetic resonance... Show moreData analysis workflows in many scientific domains have become increasingly complex and flexible. Here we assess the effect of this flexibility on the results of functional magnetic resonance imaging by asking 70 independent teams to analyse the same dataset, testing the same 9 ex-ante hypotheses(1). The flexibility of analytical approaches is exemplified by the fact that no two teams chose identical workflows to analyse the data. This flexibility resulted in sizeable variation in the results of hypothesis tests, even for teams whose statistical maps were highly correlated at intermediate stages of the analysis pipeline. Variation in reported results was related to several aspects of analysis methodology. Notably, a meta-analytical approach that aggregated information across teams yielded a significant consensus in activated regions. Furthermore, prediction markets of researchers in the field revealed an overestimation of the likelihood of significant findings, even by researchers with direct knowledge of the dataset(2-5). Our findings show that analytical flexibility can have substantial effects on scientific conclusions, and identify factors that may be related to variability in the analysis of functional magnetic resonance imaging. The results emphasize the importance of validating and sharing complex analysis workflows, and demonstrate the need for performing and reporting multiple analyses of the same data. Potential approaches that could be used to mitigate issues related to analytical variability are discussed.The results obtained by seventy different teams analysing the same functional magnetic resonance imaging dataset show substantial variation, highlighting the influence of analytical choices and the importance of sharing workflows publicly and performing multiple analyses. Show less
Botvinik-Nezer, R.; Holzmeister, F.; Camerer, C.F.; Dreber, A.; Huber, J.; Johannesson, M.; ... ; Schonberg, T. 2020
Data analysis workflows in many scientific domains have become increasingly complex and flexible. Here we assess the effect of this flexibility on the results of functional magnetic resonance... Show moreData analysis workflows in many scientific domains have become increasingly complex and flexible. Here we assess the effect of this flexibility on the results of functional magnetic resonance imaging by asking 70 independent teams to analyse the same dataset, testing the same 9 ex-ante hypotheses(1). The flexibility of analytical approaches is exemplified by the fact that no two teams chose identical workflows to analyse the data. This flexibility resulted in sizeable variation in the results of hypothesis tests, even for teams whose statistical maps were highly correlated at intermediate stages of the analysis pipeline. Variation in reported results was related to several aspects of analysis methodology. Notably, a meta-analytical approach that aggregated information across teams yielded a significant consensus in activated regions. Furthermore, prediction markets of researchers in the field revealed an overestimation of the likelihood of significant findings, even by researchers with direct knowledge of the dataset(2-5). Our findings show that analytical flexibility can have substantial effects on scientific conclusions, and identify factors that may be related to variability in the analysis of functional magnetic resonance imaging. The results emphasize the importance of validating and sharing complex analysis workflows, and demonstrate the need for performing and reporting multiple analyses of the same data. Potential approaches that could be used to mitigate issues related to analytical variability are discussed.The results obtained by seventy different teams analysing the same functional magnetic resonance imaging dataset show substantial variation, highlighting the influence of analytical choices and the importance of sharing workflows publicly and performing multiple analyses. Show less
Botvinik-Nezer, R.; Holzmeister, F.; Camerer, C.F.; Dreber, A.; Huber, J.; Johannesson, M.; ... ; Schonberg, T. 2020
Data analysis workflows in many scientific domains have become increasingly complex and flexible. Here we assess the effect of this flexibility on the results of functional magnetic resonance... Show moreData analysis workflows in many scientific domains have become increasingly complex and flexible. Here we assess the effect of this flexibility on the results of functional magnetic resonance imaging by asking 70 independent teams to analyse the same dataset, testing the same 9 ex-ante hypotheses(1). The flexibility of analytical approaches is exemplified by the fact that no two teams chose identical workflows to analyse the data. This flexibility resulted in sizeable variation in the results of hypothesis tests, even for teams whose statistical maps were highly correlated at intermediate stages of the analysis pipeline. Variation in reported results was related to several aspects of analysis methodology. Notably, a meta-analytical approach that aggregated information across teams yielded a significant consensus in activated regions. Furthermore, prediction markets of researchers in the field revealed an overestimation of the likelihood of significant findings, even by researchers with direct knowledge of the dataset(2-5). Our findings show that analytical flexibility can have substantial effects on scientific conclusions, and identify factors that may be related to variability in the analysis of functional magnetic resonance imaging. The results emphasize the importance of validating and sharing complex analysis workflows, and demonstrate the need for performing and reporting multiple analyses of the same data. Potential approaches that could be used to mitigate issues related to analytical variability are discussed.The results obtained by seventy different teams analysing the same functional magnetic resonance imaging dataset show substantial variation, highlighting the influence of analytical choices and the importance of sharing workflows publicly and performing multiple analyses. Show less
As proposed in a prominent developmental model, social anxiety has different manifestations: social fear, shy temperament. anxious cognitions, and avoidance of social situations. Drawing from this... Show moreAs proposed in a prominent developmental model, social anxiety has different manifestations: social fear, shy temperament. anxious cognitions, and avoidance of social situations. Drawing from this model, we used the network approach to psychopathology to gain a detailed understanding of specific social anxiety components and their associations. The current article investigated (a) how social anxiety components are interconnected within a network, and (b) the consistency of the network over time, in a community sample of children and adolescents. Data from 3 waves of a longitudinal study were used. At Time 1 (T1) the total sample comprised 331 participants (M-age = 13.34 years); at Time 3 (T3) there were 236 participants (M-age = 17.48 years). Social anxiety components were assessed with self-report questionnaires. Networks of 15 nodes (i.e., components) were estimated. Network analysis of T1 components revealed 4 communities: cognitive, social-emotional. avoidance of performance, and avoidance of interaction situations. There were no direct connections between the cognitive and behavioral communities; social-emotional nodes appeared to act as bridge components between the 2 communities. A similar pattern of component associations and communities was found in the T2 and T3 networks. and the longitudinal network incorporating node change trajectories. Networks were estimated on group-level observational data and conclusions about cause-effect relationships are tentative. Although the sample size decreased across the 3 waves, the reliability of parameter estimates were minimally affected. Findings attest to the potential value of applying the network approach to investigate the pattern of associations among social anxiety components in youth. Show less
In cognitive neuroscience there is a growing interest in individual differences. We propose the Multiple Indicators Multiple Causes (MIMIC) model of combined behavioral and fMRI data to determine... Show moreIn cognitive neuroscience there is a growing interest in individual differences. We propose the Multiple Indicators Multiple Causes (MIMIC) model of combined behavioral and fMRI data to determine whether such differences are quantitative or qualitative in nature. A simulation study revealed the MIMIC model to have adequate power for this goal, and parameter recovery to be satisfactory. The MIMIC model was illustrated with a re-analysis of Van Duijvenvoorde et al. (2016) and Blankenstein et al. (2018) decision making data. This showed individual differences in Van Duijvenvoorde et al. (2016) to originate in qualitative differences in decision strategies. Parameters indicated some individuals to use an expected value decision strategy, while others used a loss minimizing strategy, distinguished by individual differences in vmPFC activity. Individual differences in Blankenstein et al. (2018) were explained by quantitative differences in risk aversion. Parameters showed that more risk averse individuals preferred safe over risky choices, as predicted by heightened vmPFC activity. We advocate using the MIMIC model to empirically determine, rather than assume, the nature of individual differences in combined behavioral and fMRI datasets. Show less
Objectives: To adequately monitor the course of cognitive functioning in persons with moderate to severe dementia, relevant cognitive tests for the advanced dementia stages are needed. We examined... Show moreObjectives: To adequately monitor the course of cognitive functioning in persons with moderate to severe dementia, relevant cognitive tests for the advanced dementia stages are needed. We examined the ability of a test developed for the advanced dementia stages, the Severe Impairment Battery Short version (SIB-S), to measure cognitive change over time. Second, we examined type of memory impairment measured with the SIB-S in different dementia stages. Methods: Participants were institutionalized persons with moderate to severe dementia (N = 217). The SIB-S was administered at 6-month intervals during a 2-year period. Dementia severity at baseline was classified according to Global Deterioration Scale criteria. We used mixed models to evaluate the course of SIB-S total and domain scores, and whether dementia stage at baseline affected these courses. Results: SIB-S total scores declined significantly over time, and the course of decline differed significantly between dementia stages at baseline. Persons with moderately severe dementia declined faster in mean SIB-S total scores than persons with moderate or severe dementia. Between persons with moderate and moderately severe dementia, there was only a difference in the rate of decline of semantic items, but not episodic and non-semantic items. Conclusions: Although modest floor and slight ceiling effects were noted in severe and milder cases, respectively, the SIB-S proved to be one of few available adequate measures of cognitive change in institutionalized persons with moderate to severe dementia. Show less
Bergwerff, C.E.; Luman, M.; Weeda, W.D.; Oosterlaan, J. 2019
We examined whether neurocognitive profiles could be distinguished in children with ADHD and typically developing (TD) children, and whether neurocognitive profiles predicted externalizing, social,... Show moreWe examined whether neurocognitive profiles could be distinguished in children with ADHD and typically developing (TD) children, and whether neurocognitive profiles predicted externalizing, social, and academic problems in children with ADHD.Neurocognitive data of 81 children with ADHD and 71 TD children were subjected to confirmatory factor analysis. The resulting factors were used for community detection in the ADHD and TD group.Four subgroups were detected in the ADHD group, characterized by (a) poor emotion recognition, (b) poor interference control, (c) slow processing speed, or (d) increased attentional lapses and fast processing speed. In the TD group, three subgroups were detected, closely resembling Subgroups (a) to (c). Neurocognitive subgroups in the ADHD sample did not differ in externalizing, social, and academic problems.We found a neurocognitive profile unique to ADHD. The clinical validity of neurocognitive profiling is questioned, given the lack of associations with functional outcomes. Show less
Adolescence is a period characterised by increases in risk-taking. This behaviour has been associated with an imbalance in the integration of the networks involved in cognitive control and... Show moreAdolescence is a period characterised by increases in risk-taking. This behaviour has been associated with an imbalance in the integration of the networks involved in cognitive control and motivational processes. We examined whether the influence of emotional cues on cognitive control differs between adolescents who show high or low levels of risk-taking behaviour. Participants who scored especially high or low on a risky decision task were subsequently administered an emotional go/no-go fMRI task comprising angry, happy and calm faces. Both groups showed decreased cognitive control when confronted with appetitive and aversive emotional cues. Activation in the inferior frontal gyrus (IFG) increased in line with the cognitive control demands of the task. Though the risk taking groups did not differ in their behavioural performance, functional connectivity analyses revealed the dorsal striatum plays a more central role in the processing of cognitive control in high than low risk-takers. Overall, these findings suggest that variance in fronto-striatal circuitry may underlie individual differences in risk-taking behaviour. Show less
The most prevalent approach to activation localization in neuroimaging is to identify brain regions as contiguous supra-threshold clusters, check their significance using random field theory, and... Show moreThe most prevalent approach to activation localization in neuroimaging is to identify brain regions as contiguous supra-threshold clusters, check their significance using random field theory, and correct for the multiple clusters being tested. Besides recent criticism on the validity of the random field assumption, a spatial specificity paradox remains: the larger the detected cluster, the less we know about the location of activation within that cluster. This is because cluster inference implies “there exists at least one voxel with an evoked response in the cluster”, and not that “all the voxels in the cluster have an evoked response”. Inference on voxels within selected clusters is considered bad practice, due to the voxel-wise false positive rate inflation associated with this circular inference. Here, we propose a remedy to the spatial specificity paradox. By applying recent results from the multiple testing statistical literature, we are able to quantify the proportion of truly active voxels within selected clusters, an approach we call All-Resolutions Inference (ARI). If this proportion is high, the paradox vanishes. If it is low, we can further “drill down” from the cluster level to sub-regions, and even to individual voxels, in order to pinpoint the origin of the activation. In fact, ARI allows inference on the proportion of activation in all voxel sets, no matter how large or small, however these have been selected, all from the same data. We use two fMRI datasets to demonstrate the non-triviality of the spatial specificity paradox, and its resolution using ARI. We verify that the endless circularity permitted by ARI does not render its estimates overly conservative using both simulation, and a data split. Show less
Kortink, E.D.; Weeda, W.D.; Crowley, M.J.; Gunther Moor, B.; Molen, M.J.W. van der 2018
Monitoring social threat is essential for maintaining healthy social relationships, and recent studies suggest a neural alarm system that governs our response to social rejection. Frontal-midline... Show moreMonitoring social threat is essential for maintaining healthy social relationships, and recent studies suggest a neural alarm system that governs our response to social rejection. Frontal-midline theta (4-8 Hz) oscillatory power might act as a neural correlate of this system by being sensitive to unexpected social rejection. Here, we examined whether frontal-midline theta is modulated by individual differences in personality constructs sensitive to social disconnection. In addition, we examined the sensitivity of feedback-related brain potentials (i.e., the feedback-related negativity and P3) to social feedback. Sixty-five undergraduate female participants (mean age = 19.69 years) participated in the Social Judgment Paradigm, a fictitious peer-evaluation task in which participants provided expectancies about being liked/disliked by peer strangers. Thereafter, they received feedback signaling social acceptance/rejection. A community structure analysis was employed to delineate personality profiles in our data. Results provided evidence of two subgroups: one group scored high on attachment-related anxiety and fear of negative evaluation, whereas the other group scored high on attachment-related avoidance and low on fear of negative evaluation. In both groups, unexpected rejection feedback yielded a significant increase in theta power. The feedback-related negativity was sensitive to unexpected feedback, regardless of valence, and was largest for unexpected rejection feedback. The feedback-related P3 was significantly enhanced in response to expected social acceptance feedback. Together, these findings confirm the sensitivity of frontal midline theta oscillations to the processing of social threat, and suggest that this alleged neural alarm system behaves similarly in individuals that differ in personality constructs relevant to social evaluation. Show less
Kortink, E.D.; Weeda, W.D.; Crowley, M.J.; Gunther, Moor B.; Van der Molen, M.J.W. 2018
Monitoring social threat is essential for maintaining healthy social relationships, and recent studies suggest a neural alarm system that governs our response to social rejection. Frontal-midline... Show moreMonitoring social threat is essential for maintaining healthy social relationships, and recent studies suggest a neural alarm system that governs our response to social rejection. Frontal-midline theta (4-8 Hz) oscillatory power might act as a neural correlate of this system by being sensitive to unexpected social rejection. Here, we examined whether frontal-midline theta is modulated by individual differences in personality constructs sensitive to social disconnection. In addition, we examined the sensitivity of feedback-related brain potentials (i.e., the feedback-related negativity and P3) to social feedback. Sixty-five undergraduate female participants (mean age = 19.69 years) participated in the Social Judgment Paradigm, a fictitious peer-evaluation task in which participants provided expectancies about being liked/disliked by peer strangers. Thereafter, they received feedback signaling social acceptance/rejection. A community structure analysis was employed to delineate personality profiles in our data. Results provided evidence of two subgroups: one group scored high on attachment-related anxiety and fear of negative evaluation, whereas the other group scored high on attachment-related avoidance and low on fear of negative evaluation. In both groups, unexpected rejection feedback yielded a significant increase in theta power. The feedback-related negativity was sensitive to unexpected feedback, regardless of valence, and was largest for unexpected rejection feedback. The feedback-related P3 was significantly enhanced in response to expected social acceptance feedback. Together, these findings confirm the sensitivity of frontal midline theta oscillations to the processing of social threat, and suggest that this alleged neural alarm system behaves similarly in individuals that differ in personality constructs relevant to social evaluation. Show less
Noordermeer, S.D.S.; Luman, M.; Weeda, W.D.; Buitelaar, J.K.; Richards, J.S.; Hartman, C.A.; ... ; Oosterlaan, J. 2017
Whenever parameter estimates are uncertain or observations are contaminated by measurement error, the Pearson correlation coefficient can severely underestimate the true strength of an association.... Show moreWhenever parameter estimates are uncertain or observations are contaminated by measurement error, the Pearson correlation coefficient can severely underestimate the true strength of an association. Various approaches exist for inferring the correlation in the presence of estimation uncertainty and measurement error, but none are routinely applied in psychological research. Here we focus on a Bayesian hierarchical model proposed by Behseta, Berdyyeva, Olson, and Kass (2009) that allows researchers to infer the underlying correlation between error-contaminated observations. We show that this approach may be also applied to obtain the underlying correlation between uncertain parameter estimates as well as the correlation between uncertain parameter estimates and noisy observations. We illustrate the Bayesian modeling of correlations with two empirical data sets; in each data set, we first infer the posterior distribution of the underlying correlation and then compute Bayes factors to quantify the evidence that the data provide for the presence of an association. Show less
This investigation aims to further our understanding of the brain mechanisms underlying the awareness of one's erroneous actions. While all errors are registered as such in the rostral cingulate... Show moreThis investigation aims to further our understanding of the brain mechanisms underlying the awareness of one's erroneous actions. While all errors are registered as such in the rostral cingulate zone, errors enter awareness only when the anterior insula cortex is activated. Aware but not unaware errors elicit autonomic nervous system reactivity. Our aim is to investigate the hypothesis that activation in the insula during error awareness is related to autonomic arousal and to inter-regional interactions with other areas of the brain. To examine the role of the anterior insula in error awareness, we assessed its functional connectivity to other brain regions along with autonomic nervous system reactivity in young healthy participants who underwent simultaneous pupil-diameter and functional magnetic resonance imaging measurements while performing a complex and error-prone task. Error blindness was associated with failures to engage sufficient autonomic reactivity. During aware errors increased pupil-diameter along with increased task-related activation within, and increased connectivity between anterior insula and task-related networks suggested an increased capacity for action-control information transfer. Increased pupil-diameter during aware errors was furthermore associated with decreased activation of the default-mode network along with decreased insular connectivity with regions of the default mode system, possibly reflecting decreased task-irrelevant information processing. This shifting mechanism may be relevant to a better understanding of how the brain and the autonomic nervous system interact to enable efficient adaptive behavior during cognitive challenge. Show less
Janssen, T.W.P.; Bink, M.; Weeda, W.D.; Gelade, K.N.; Van Mourik, R.; Maras, A.; Oosterlaan, J. 2017
Neurofeedback is widely applied as non-pharmacological intervention aimed at reducing symptoms of ADHD, even though efficacy has not been unequivocally established. Neuronal changes during the... Show moreNeurofeedback is widely applied as non-pharmacological intervention aimed at reducing symptoms of ADHD, even though efficacy has not been unequivocally established. Neuronal changes during the neurofeedback intervention that resemble learning can provide crucial evidence for the feasibility and specificity of this intervention. A total of 38 children (aged between 7 and 13 years) with a DSM-IV-TR diagnosis of ADHD, completed on average 29 sessions of theta (4–8 Hz)/beta (13–20 Hz) neurofeedback training. Dependent variables included training-related measures as well as theta and beta power during baseline and training runs for each session. Learning effects were analyzed both within and between sessions. To further specify findings, individual learning curves were explored and correlated with behavioral changes in ADHD symptoms. Over the course of the training, there was a linear increase in participants’ mean training level, highest obtained training level and the number of earned credits (range b = 0.059, −0.750, p < 0.001). Theta remained unchanged over the course of the training, while beta activity increased linearly within training sessions (b = 0.004, 95% CI = [0.0013–0.0067], p = 0.005) and over the course of the intervention (b = 0.0052, 95% CI = [0.0039–0.0065], p < 0.001). In contrast to the group analyses, significant individual learning curves were found for both theta and beta over the course of the intervention in 39 and 53%, respectively. Individual learning curves were not significantly correlated with behavioral changes. This study shows that children with ADHD can gain control over EEG states during neurofeedback, although a lack of behavioral correlates may indicate insufficient transfer to daily functioning, or to confounding reinforcement of electromyographic activity. Clinical Trials Registration: This trial is registered at the US National Institutes of Health (ClinicalTrials.gov, ref. no: NCT01363544); https://clinicaltrials.gov/show/NCT01363544. Show less
Königs, M.; Weeda, W.D.; Van Heurn, L.W.E.; Vermeulen, R.J.; Goslings, J.C.; Luitse, J.S.K.; ... ; Oosterlaan, J. 2017
