In their recent paper, Forbes et al. (2019; FWMK) evaluate the replicability of network models in two studies. They identify considerable replicability issues, concluding that "current 'state-of... Show moreIn their recent paper, Forbes et al. (2019; FWMK) evaluate the replicability of network models in two studies. They identify considerable replicability issues, concluding that "current 'state-of-the-art' methods in the psychopathology network literature [ horizontal ellipsis ] are not well-suited to analyzing the structure of the relationships between individual symptoms". Such strong claims require strong evidence, which the authors do not provide. FWMK identify low replicability by analyzing point estimates of networks; contrast low replicability with results of two statistical tests that indicate higher replicability, and conclude that these tests are problematic. We make four points. First, statistical tests are superior to the visual inspection of point estimates, because tests take into account sampling variability. Second, FWMK misinterpret the statistical tests in several important ways. Third, FWMK did not follow established recommendations when estimating networks in their first study, underestimating replicability. Fourth, FWMK draw conclusions about methodology, which does not follow from investigations of data, and requires investigations of methodology. Overall, we show that the "poor replicability "observed by FWMK occurs due to sampling variability and use of suboptimal methods. We conclude by discussing important recent simulation work that guides researchers to use models appropriate for their data, such as nonregularized estimation routines. Show less
BackgroundChildhood adversity (CA) is strongly associated with mental health problems. Resilience factors (RFs) reduce mental health problems following CA. Yet, knowledge on the nature of RFs is... Show moreBackgroundChildhood adversity (CA) is strongly associated with mental health problems. Resilience factors (RFs) reduce mental health problems following CA. Yet, knowledge on the nature of RFs is scarce. Therefore, we examined RF mean levels, RF interrelations, RF-distress pathways, and their changes between early (age 14) and later adolescence (age 17).MethodsWe studied 10 empirically supported RFs in adolescents with (CA+; n = 631) and without CA (CA−; n = 499), using network psychometrics.ResultsAll inter-personal RFs (e.g. friendships) showed stable mean levels between age 14 and 17, and three of seven intra-personal RFs (e.g. distress tolerance) changed in a similar manner in the two groups. The CA+ group had lower RFs and higher distress at both ages. Thus, CA does not seem to inhibit RF changes, but to increase the risk of persistently lower RFs. At age 14, but not 17, the RF network of the CA+ group was less positively connected, suggesting that RFs are less likely to enhance each other than in the CA− group. Those findings underpin the notion that CA has a predominantly strong proximal effect. RF-distress pathways did not differ in strength between the CA+ and the CA− group, which suggests that RFs have a similarly protective strength in the two groups. Yet, as RFs are lower and distress is higher, RF-distress pathways may overall be less advantageous in the CA+ group. Most RF interrelations and RF-distress pathways were stable between age 14 and 17, which may help explain why exposure to CA is frequently found to have a lasting impact on mental health.ConclusionsOur findings not only shed light on the nature and changes of RFs between early and later adolescence, but also offer some accounts for why exposure to CA has stronger proximal effects and is often found to have a lasting impact on mental health. Show less
Steinley, Hoffman, Brusco, and Sher (2017) proposed a new method for evaluating the performance of psychological network models: fixed-margin sampling. The authors investigated LASSO regularized... Show moreSteinley, Hoffman, Brusco, and Sher (2017) proposed a new method for evaluating the performance of psychological network models: fixed-margin sampling. The authors investigated LASSO regularized Ising models (eLasso) by generating random datasets with the same margins as the original binary dataset, and concluded that many estimated eLasso parameters are not distinguishable from those that would be expected if the data were generated by chance. We argue that fixed-margin sampling cannot be used for this purpose, as it generates data under a particular null-hypothesis: a unidimensional factor model with interchangeable indicators (i.e., the Rasch model). We show this by discussing relevant psychometric literature and by performing simulation studies. Results indicate that while eLasso correctly estimated network models and estimated almost no edges due to chance, fixed-margin sampling performed poorly in classifying true effects as “interesting” (Steinley et al. 2017, p. 1004). Further simulation studies indicate that fixed-margin sampling offers a powerful method for highlighting local misfit from the Rasch model, but performs only moderately in identifying global departures from the Rasch model. We conclude that fixed-margin sampling is not up to the task of assessing if results from estimated Ising models or other multivariate psychometric models are due to chance. Show less
