The main objective of most clinical trials is to estimate the effect of some treatment compared to a control condition. We define the signal-to-noise ratio (SNR) as the ratio of the true treatment... Show moreThe main objective of most clinical trials is to estimate the effect of some treatment compared to a control condition. We define the signal-to-noise ratio (SNR) as the ratio of the true treatment effect to the SE of its estimate. In a previous publication in this journal, we estimated the distribution of the SNR among the clinical trials in the Cochrane Database of Systematic Reviews (CDSR). We found that the SNR is often low, which implies that the power against the true effect is also low in many trials. Here we use the fact that the CDSR is a collection of meta-analyses to quantitatively assess the consequences. Among trials that have reached statistical significance we find considerable overoptimism of the usual unbiased estimator and under-coverage of the associated confidence interval. Previously, we have proposed a novel shrinkage estimator to address this "winner's curse." We compare the performance of our shrinkage estimator to the usual unbiased estimator in terms of the root mean squared error, the coverage and the bias of the magnitude. We find superior performance of the shrinkage estimator both conditionally and unconditionally on statistical significance. Show less
We propose a permutation-based method for testing a large collection of hypotheses simultaneously. Our method provides lower bounds for the number of true discoveries in any selected subset of... Show moreWe propose a permutation-based method for testing a large collection of hypotheses simultaneously. Our method provides lower bounds for the number of true discoveries in any selected subset of hypotheses. These bounds are simultaneously valid with high confidence. The methodology is particularly useful in functional Magnetic Resonance Imaging cluster analysis, where it provides a confidence statement on the percentage of truly activated voxels within clusters of voxels, avoiding the well-known spatial specificity paradox. We offer a user-friendly tool to estimate the percentage of true discoveries for each cluster while controlling the family-wise error rate for multiple testing and taking into account that the cluster was chosen in a data-driven way. The method adapts to the spatial correlation structure that characterizes functional Magnetic Resonance Imaging data, gaining power over parametric approaches. Show less
We propose a permutation-based method for testing a large collection of hypotheses simultaneously. Our method provides lower bounds for the number of true discoveries in any selected subset of... Show moreWe propose a permutation-based method for testing a large collection of hypotheses simultaneously. Our method provides lower bounds for the number of true discoveries in any selected subset of hypotheses. These bounds are simultaneously valid with high confidence. The methodology is particularly useful in functional Magnetic Resonance Imaging cluster analysis, where it provides a confidence statement on the percentage of truly activated voxels within clusters of voxels, avoiding the well-known spatial specificity paradox. We offer a user-friendly tool to estimate the percentage of true discoveries for each cluster while controlling the family-wise error rate for multiple testing and taking into account that the cluster was chosen in a data-driven way. The method adapts to the spatial correlation structure that characterizes functional Magnetic Resonance Imaging data, gaining power over parametric approaches. Show less
Arntzen, V.H.; Fiocco, M.; Leitzinger, N.; Geskus, R.B. 2023
Quarantine length for individuals who have been at risk for infection with SARS-CoV-2 has been based on estimates of the incubation time distribution. The time of infection is often not known... Show moreQuarantine length for individuals who have been at risk for infection with SARS-CoV-2 has been based on estimates of the incubation time distribution. The time of infection is often not known exactly, yielding data with an interval censored time origin. We give a detailed account of the data structure, likelihood formulation and assumptions usually made in the literature: (i) the risk of infection is assumed constant on the exposure window and (ii) the incubation time follows a specific parametric distribution. The impact of these assumptions remains unclear, especially for the right tail of the distribution which informs quarantine policy. We quantified bias in percentiles by means of simulation studies that mimic reality as close as possible. If assumption (i) is not correct, then median and upper percentiles are affected similarly, whereas misspecification of the parametric approach (ii) mainly affects upper percentiles. The latter may yield considerable bias. We suggest a semiparametric method that provides more robust estimates without the need of a parametric choice. Additionally, we used a simulation study to evaluate a method that has been suggested if all infection times are left censored. It assumes that the width of the interval from infection to latest possible exposure follows a uniform distribution. This assumption gave biased results in the exponential phase of an outbreak. Our application to open source data suggests that focus should be on the level of information in the observations, as expressed by the width of exposure windows, rather than the number of observations. Show less
In clinical settings, the absolute risk reduction due to treatment that can be expected in a particular patient is of key interest. However, logistic regression, the default regression model for... Show moreIn clinical settings, the absolute risk reduction due to treatment that can be expected in a particular patient is of key interest. However, logistic regression, the default regression model for trials with a binary outcome, produces estimates of the effect of treatment measured as a difference in log odds. We explored options to estimate treatment effects directly as a difference in risk, specifically in the network meta-analysis setting. We propose a novel Bayesian (meta-)regression model for binary outcomes on the additive risk scale. The model allows treatment effects, covariate effects, interactions and variance parameters to be estimated directly on the linear scale of clinical interest. We compared effect estimates from this model to (1) a previously proposed additive risk model by Warn, Thompson and Spiegelhalter ( "WTS model ") and (2) backtransforming the predictions from a logistic model to the natural scale after regression. The models were compared in a network meta-analysis of 20 hepatitis C trials, as well as in the analysis of simulated single trial settings. The resulting estimates diverged, in particular for small sample sizes or true risks close to 0% or 100%. Researchers should be aware that modelling untransformed risk can yield very different results from default logistic models. The treatment effect in participants with such extreme predicted risks weighed more heavily on the overall treatment effect estimate from our proposed model compared to the WTS model. In our network meta-analysis, this sensitivity of our proposed model was needed to detect all information in the data. Show less
Unobserved individual heterogeneity is a common challenge in population cancer survival studies. This heterogeneity is usually associated with the combination of model misspecification and the... Show moreUnobserved individual heterogeneity is a common challenge in population cancer survival studies. This heterogeneity is usually associated with the combination of model misspecification and the failure to record truly relevant variables. We investigate the effects of unobserved individual heterogeneity in the context of excess hazard models, one of the main tools in cancer epidemiology. We propose an individual excess hazard frailty model to account for individual heterogeneity. This represents an extension of frailty modeling to the relative survival framework. In order to facilitate the inference on the parameters of the proposed model, we select frailty distributions which produce closed-form expressions of the marginal hazard and survival functions. The resulting model allows for an intuitive interpretation, in which the frailties induce a selection of the healthier individuals among survivors. We model the excess hazard using a flexible parametric model with a general hazard structure which facilitates the inclusion of time-dependent effects. We illustrate the performance of the proposed methodology through a simulation study. We present a real-data example using data from lung cancer patients diagnosed in England, and discuss the impact of not accounting for unobserved heterogeneity on the estimation of net survival. The methodology is implemented in the R package IFNS. Show less
Su, X.; Cho, Y.; Liqiang, N.; Liu, L.; Dusseldorp, E. 2022
Moderation analysis is an integral part of precision medicine research. Concerning moderation analysis with categorical outcomes, we start with an interesting observation, which shows that... Show moreModeration analysis is an integral part of precision medicine research. Concerning moderation analysis with categorical outcomes, we start with an interesting observation, which shows that heterogeneous treatment effects could be equivalently estimated via a role exchange between the outcome and the treatment variable in logistic regression models. Hence two estimators of moderating effects can be obtained. We then established the joint asymptotic normality for the two estimators, on which basis refined inference can be made for moderation analysis. The improved precision is helpful in addressing the lack-of-power problem that is common in search of moderators. The above-mentioned results hold for both experimental and observational data. We investigate the proposed method by simulation and provide an illustration with data from a randomized trial on wart treatment. Show less
We use information derived from over 40K trials in the Cochrane Collaboration database of systematic reviews (CDSR) to compute the replication probability, or predictive power of an experiment... Show moreWe use information derived from over 40K trials in the Cochrane Collaboration database of systematic reviews (CDSR) to compute the replication probability, or predictive power of an experiment given its observed (two-sided) P$$ P $$-value. We find that an exact replication of a marginally significant result with P=.05$$ P=.05 $$ has less than 30% chance of again reaching significance. Moreover, the replication of a result with P=.005$$ P=.005 $$ still has only 50% chance of significance. We also compute the probability that the direction (sign) of the estimated effect is correct, which is closely related to the type S error of Gelman and Tuerlinckx. We find that if an estimated effect has P=.05$$ P=.05 $$, there is a 93% probability that its sign is correct. If P=.005$$ P=.005 $$, then that probability is 99%. Finally, we compute the required sample size for a replication study to achieve some specified power conditional on the p$$ p $$-value of the original study. We find that the replication of a result with P=.05$$ P=.05 $$ requires a sample size more than 16 times larger than the original study to achieve 80% power, while P=.005$$ P=.005 $$ requires at least 3.5 times larger sample size. These findings confirm that failure to replicate the statistical significance of a trial does not necessarily indicate that the original result was a fluke. Show less
Benchmark surveillance tests for detecting disease progression (eg, biopsies, endoscopies) in early-stage chronic noncommunicable diseases (eg, cancer, lung diseases) are usually burdensome. For... Show moreBenchmark surveillance tests for detecting disease progression (eg, biopsies, endoscopies) in early-stage chronic noncommunicable diseases (eg, cancer, lung diseases) are usually burdensome. For detecting progression timely, patients undergo invasive tests planned in a fixed one-size-fits-all manner (eg, annually). We aim to present personalized test schedules based on the risk of disease progression, that optimize the burden (the number of tests) and the benefit (shorter time delay in detecting progression is better) better than fixed schedules, and enable shared decision making. Our motivation comes from the problem of scheduling biopsies in prostate cancer surveillance. Using joint models for time-to-event and longitudinal data, we consolidate patients' longitudinal data (eg, biomarkers) and results of previous tests, into individualized future cumulative-risk of progression. We then create personalized schedules by planning tests on future visits where the predicted cumulative-risk is above a threshold (eg, 5% risk). We update personalized schedules with data gathered over follow-up. To find the optimal risk threshold, we minimize a utility function of the expected number of tests (burden) and expected time delay in detecting progression (shorter is beneficial) for different thresholds. We estimate these two in a patient-specific manner for following any schedule, by utilizing a patient's predicted risk profile. Patients/doctors can employ these quantities to compare personalized and fixed schedules objectively and make a shared decision of a test schedule. Show less
The problem of dynamic prediction with time-dependent covariates, given by biomarkers, repeatedly measured over time, has received much attention over the last decades. Two contrasting approaches... Show moreThe problem of dynamic prediction with time-dependent covariates, given by biomarkers, repeatedly measured over time, has received much attention over the last decades. Two contrasting approaches have become in widespread use. The first is joint modeling, which attempts to jointly model the longitudinal markers and the event time. The second is landmarking, a more pragmatic approach that avoids modeling the marker process. Landmarking has been shown to be less efficient than correctly specified joint models in simulation studies, when data are generated from the joint model. When the mean model is misspecified, however, simulation has shown that joint models may be inferior to landmarking. The objective of this article is to develop methods that improve the predictive accuracy of landmarking, while retaining its relative simplicity and robustness. We start by fitting a working longitudinal model for the biomarker, including a temporal correlation structure. Based on that model, we derive a predictable time-dependent process representing the expected value of the biomarker after the landmark time, and we fit a time-dependent Cox model based on the predictable time-dependent covariate. Dynamic predictions based on this approach for new patients can be obtained by first deriving the expected values of the biomarker, given the measured values before the landmark time point, and then calculating the predicted probabilities based on the time-dependent Cox model. We illustrate the approach in predicting overall survival in liver cirrhosis patients based on prothrombin index. Show less
Longitudinal and high-dimensional measurements have become increasingly common in biomedical research. However, methods to predict survival outcomes using covariates that are both longitudinal and... Show moreLongitudinal and high-dimensional measurements have become increasingly common in biomedical research. However, methods to predict survival outcomes using covariates that are both longitudinal and high-dimensional are currently missing. In this article, we propose penalized regression calibration (PRC), a method that can be employed to predict survival in such situations. PRC comprises three modeling steps: First, the trajectories described by the longitudinal predictors are flexibly modeled through the specification of multivariate mixed effects models. Second, subject-specific summaries of the longitudinal trajectories are derived from the fitted mixed models. Third, the time to event outcome is predicted using the subject-specific summaries as covariates in a penalized Cox model. To ensure a proper internal validation of the fitted PRC models, we furthermore develop a cluster bootstrap optimism correction procedure that allows to correct for the optimistic bias of apparent measures of predictiveness. PRC and the CBOCP are implemented in the R package pencal, available from CRAN. After studying the behavior of PRC via simulations, we conclude by illustrating an application of PRC to data from an observational study that involved patients affected by Duchenne muscular dystrophy, where the goal is predict time to loss of ambulation using longitudinal blood biomarkers. Show less
Signorelli, M.; Spitali, P.; Szigvarto, C.A.; Tsonoka, R. 2021
Longitudinal and high-dimensional measurements have become increasingly common in biomedical research. However, methods to predict survival outcomes using covariates that are both longitudinal and... Show moreLongitudinal and high-dimensional measurements have become increasingly common in biomedical research. However, methods to predict survival outcomes using covariates that are both longitudinal and high-dimensional are currently missing. In this article, we propose penalized regression calibration (PRC), a method that can be employed to predict survival in such situations. PRC comprises three modeling steps: First, the trajectories described by the longitudinal predictors are flexibly modeled through the specification of multivariate mixed effects models. Second, subject-specific summaries of the longitudinal trajectories are derived from the fitted mixed models. Third, the time to event outcome is predicted using the subject-specific summaries as covariates in a penalized Cox model. To ensure a proper internal validation of the fitted PRC models, we furthermore develop a cluster bootstrap optimism correction procedure that allows to correct for the optimistic bias of apparent measures of predictiveness. PRC and the CBOCP are implemented in the R package pencal, available from CRAN. After studying the behavior of PRC via simulations, we conclude by illustrating an application of PRC to data from an observational study that involved patients affected by Duchenne muscular dystrophy, where the goal is predict time to loss of ambulation using longitudinal blood biomarkers. Show less
We abstract the concept of a randomized controlled trial as a triple (beta,b,s), where beta is the primary efficacy parameter, b the estimate, and s the standard error (s>0). If the parameter... Show moreWe abstract the concept of a randomized controlled trial as a triple (beta,b,s), where beta is the primary efficacy parameter, b the estimate, and s the standard error (s>0). If the parameter beta is either a difference of means, a log odds ratio or a log hazard ratio, then it is reasonable to assume that b is unbiased and normally distributed. This then allows us to estimate the joint distribution of the z-value z=b/s and the signal-to-noise ratio SNR=beta/s from a sample of pairs (bi,si). We have collected 23 551 such pairs from the Cochrane database. We note that there are many statistical quantities that depend on (beta,b,s) only through the pair (z,SNR). We start by determining the estimated distribution of the achieved power. In particular, we estimate the median achieved power to be only 13%. We also consider the exaggeration ratio which is the factor by which the magnitude of beta is overestimated. We find that if the estimate is just significant at the 5% level, we would expect it to overestimate the true effect by a factor of 1.7. This exaggeration is sometimes referred to as the winner's curse and it is undoubtedly to a considerable extent responsible for disappointing replication results. For this reason, we believe it is important to shrink the unbiased estimator, and we propose a method for doing so. We show that our shrinkage estimator successfully addresses the exaggeration. As an example, we re-analyze the ANDROMEDA-SHOCK trial. Show less
Chalkou, K.; Steyerberg, E.; Egger, M.; Manca, A.; Pellegrini, F.; Salanti, G. 2021
Treatment effects vary across different patients, and estimation of this variability is essential for clinical decision-making. We aimed to develop a model estimating the benefit of alternative... Show moreTreatment effects vary across different patients, and estimation of this variability is essential for clinical decision-making. We aimed to develop a model estimating the benefit of alternative treatment options for individual patients, extending a risk modeling approach in a network meta-analysis framework. We propose a two-stage prediction model for heterogeneous treatment effects by combining prognosis research and network meta-analysis methods where individual patient data are available. In the first stage, a prognostic model to predict the baseline risk of the outcome. In the second stage, we use the baseline risk score from the first stage as a single prognostic factor and effect modifier in a network meta-regression model. We apply the approach to a network meta-analysis of three randomized clinical trials comparing the relapses in Natalizumab, Glatiramer Acetate, and Dimethyl Fumarate, including 3590 patients diagnosed with relapsing-remitting multiple sclerosis. We find that the baseline risk score modifies the relative and absolute treatment effects. Several patient characteristics, such as age and disability status, impact the baseline risk of relapse, which in turn moderates the benefit expected for each of the treatments. For high-risk patients, the treatment that minimizes the risk of relapse in 2 years is Natalizumab, whereas Dimethyl Fumarate might be a better option for low-risk patients. Our approach can be easily extended to all outcomes of interest and has the potential to inform a personalized treatment approach. Show less
The Fine-Gray subdistribution hazard model has become the default method to estimate the incidence of outcomes over time in the presence of competing risks. This model is attractive because it... Show moreThe Fine-Gray subdistribution hazard model has become the default method to estimate the incidence of outcomes over time in the presence of competing risks. This model is attractive because it directly relates covariates to the cumulative incidence function (CIF) of the event of interest. An alternative is to combine the different cause-specific hazard functions to obtain the different CIFs. A limitation of the subdistribution hazard approach is that the sum of the cause-specific CIFs can exceed 1 (100%) for some covariate patterns. Using data on 9479 patients hospitalized with acute myocardial infarction, we estimated the cumulative incidence of both cardiovascular death and non-cardiovascular death for each patient. We found that when using subdistribution hazard models, approximately 5% of subjects had an estimated risk of 5-year all-cause death (obtained by combining the two cause-specific CIFs obtained from subdistribution hazard models) that exceeded 1. This phenomenon was avoided by using the two cause-specific hazard models. We provide a proof that the sum of predictions exceeds 1 is a fundamental problem with the Fine-Gray subdistribution hazard model. We further explored this issue using simulations based on two different types of data-generating process, one based on subdistribution hazard models and other based on cause-specific hazard models. We conclude that care should be taken when using the Fine-Gray subdistribution hazard model in situations with wide risk distributions or a high cumulative incidence, and if one is interested in the risk of failure from each of the different event types. Show less
Ebrahimpoor, M.; Spitali, P.; Goeman, J.J.; Tsonaka, R. 2021
We propose a top-down approach for pathway analysis of longitudinal metabolite data. We apply a score test based on a shared latent process mixed model which can identify pathways with... Show moreWe propose a top-down approach for pathway analysis of longitudinal metabolite data. We apply a score test based on a shared latent process mixed model which can identify pathways with differentially progressing metabolites. The strength of our approach is that it can handle unbalanced designs, deals with potential missing values in the longitudinal markers, and gives valid results even with small sample sizes. Contrary to bottom-up approaches, correlations between metabolites are explicitly modeled leveraging power gains. For large pathway sizes, a computationally efficient solution is proposed based on pseudo-likelihood methodology. We demonstrate the advantages of the proposed method in identification of differentially expressed pathways through simulation studies. Finally, longitudinal metabolite data from a mice experiment is analyzed to demonstrate our methodology. Show less
Goetghebeur, E.; Cessie, S. le; Stavola, B. de; Moodie, E.E.M.; Waernbaum, I.; Topic Grp Causal Inference TG7 2020
Although review papers on causal inference methods are now available, there is a lack of introductory overviews onwhatthey can render and on the guiding criteria for choosing one particular method.... Show moreAlthough review papers on causal inference methods are now available, there is a lack of introductory overviews onwhatthey can render and on the guiding criteria for choosing one particular method. This tutorial gives an overview in situations where an exposure of interest is set at a chosen baseline ("point exposure") and the target outcome arises at a later time point. We first phrase relevant causal questions and make a case for being specific about the possible exposure levels involved and the populations for which the question is relevant. Using the potential outcomes framework, we describe principled definitions of causal effects and of estimation approaches classified according to whether they invoke the no unmeasured confounding assumption (including outcome regression and propensity score-based methods) or an instrumental variable with added assumptions. We mainly focus on continuous outcomes and causal average treatment effects. We discuss interpretation, challenges, and potential pitfalls and illustrate application using a "simulation learner," that mimics the effect of various breastfeeding interventions on a child's later development. This involves a typical simulation component with generated exposure, covariate, and outcome data inspired by a randomized intervention study. The simulation learner further generates various (linked) exposure types with a set of possible values per observation unit, from which observed as well as potential outcome data are generated. It thus provides true values of several causal effects. R code for data generation and analysis is available on , where SAS and Stata code for analysis is also provided. Show less
Steyerberg, E.W.; Nieboer, D.; Debray, T.P.A.; Houwelingen, H.C. van 2019
