Predicting who will benefit from treatment with immune checkpoint inhibition (ICI) in patients with advanced melanoma is challenging. We developed a multivariable prediction model for response to... Show morePredicting who will benefit from treatment with immune checkpoint inhibition (ICI) in patients with advanced melanoma is challenging. We developed a multivariable prediction model for response to ICI, using routinely available clinical data including primary melanoma characteristics. We used a population-based cohort of 3525 patients with advanced cutaneous melanoma treated with anti-PD-1-based therapy. Our prediction model for predicting response within 6 months after ICI initiation was internally validated with bootstrap resampling. Performance evaluation included calibration, discrimination and internal-external cross-validation. Included patients received anti-PD-1 monotherapy (n = 2366) or ipilimumab plus nivolumab (n = 1159) in any treatment line. The model included serum lactate dehydrogenase, World Health Organization performance score, type and line of ICI, disease stage and time to first distant recurrence-all at start of ICI-, and location and type of primary melanoma, the presence of satellites and/or in-transit metastases at primary diagnosis and sex. The over-optimism adjusted area under the receiver operating characteristic was 0.66 (95% CI: 0.64-0.66). The range of predicted response probabilities was 7%-81%. Based on these probabilities, patients were categorized into quartiles. Compared to the lowest response quartile, patients in the highest quartile had a significantly longer median progression-free survival (20.0 vs 2.8 months; P < .001) and median overall survival (62.0 vs 8.0 months; P < .001). Our prediction model, based on routinely available clinical variables and primary melanoma characteristics, predicts response to ICI in patients with advanced melanoma and discriminates well between treated patients with a very good and very poor prognosis. Show less
Objective:To develop a fistula risk score for auditing, to be able to compare postoperative pancreatic fistula (POPF) after pancreatoduodenectomy among hospitals. Background:For proper comparisons... Show moreObjective:To develop a fistula risk score for auditing, to be able to compare postoperative pancreatic fistula (POPF) after pancreatoduodenectomy among hospitals. Background:For proper comparisons of outcomes in surgical audits, case-mix variation should be accounted for. Methods:This study included consecutive patients after pancreatoduodenectomy from the mandatory nationwide Dutch Pancreatic Cancer Audit. Derivation of the score was performed with the data from 2014 to 2018 and validation with 2019 to 2020 data. The primary endpoint of the study was POPF (grade B or C). Multivariable logistic regression analysis was performed for case-mix adjustment of known risk factors. Results:In the derivation cohort, 3271 patients were included, of whom 479 (14.6%) developed POPF. Male sex [odds ratio (OR)=1.34; 95% confidence interval (CI): 1.09-1.66], higher body mass index (OR=1.07; 95% CI: 1.05-1.10), a final diagnosis other than pancreatic ductal adenocarcinoma/pancreatitis (OR=2.41; 95% CI: 1.90-3.06), and a smaller duct diameter (OR=1.43/mm decrease; 95% CI: 1.32-1.55) were independently associated with POPF. Diabetes mellitus (OR=0.73; 95% CI: 0.55-0.98) was independently associated with a decreased risk of POPF. Model discrimination was good with a C-statistic of 0.73 in the derivation cohort and 0.75 in the validation cohort (n=913). Hospitals differed in particular in the proportion of pancreatic ductal adenocarcinoma/pancreatitis patients, ranging from 36.0% to 58.1%. The observed POPF risk per center ranged from 2.9% to 25.4%. The expected POPF rate based on the 5 risk factors ranged from 11.6% to 18.0% among hospitals. Conclusions:The auditing fistula risk score was successful in case-mix adjustment and enables fair comparisons of POPF rates among hospitals. Show less
AimsRisk stratification is used for decisions regarding need for imaging in patients with clinically suspected acute pulmonary embolism (PE). The aim was to develop a clinical prediction model that... Show moreAimsRisk stratification is used for decisions regarding need for imaging in patients with clinically suspected acute pulmonary embolism (PE). The aim was to develop a clinical prediction model that provides an individualized, accurate probability estimate for the presence of acute PE in patients with suspected disease based on readily available clinical items and D-dimer concentrations.Methods and resultsAn individual patient data meta-analysis was performed based on sixteen cross-sectional or prospective studies with data from 28 305 adult patients with clinically suspected PE from various clinical settings, including primary care, emergency care, hospitalized and nursing home patients. A multilevel logistic regression model was built and validated including ten a priori defined objective candidate predictors to predict objectively confirmed PE at baseline or venous thromboembolism (VTE) during follow-up of 30 to 90 days. Multiple imputation was used for missing data. Backward elimination was performed with a P-value <0.10. Discrimination (c-statistic with 95% confidence intervals [CI] and prediction intervals [PI]) and calibration (outcome:expected [O:E] ratio and calibration plot) were evaluated based on internal-external cross-validation. The accuracy of the model was subsequently compared with algorithms based on the Wells score and D-dimer testing. The final model included age (in years), sex, previous VTE, recent surgery or immobilization, haemoptysis, cancer, clinical signs of deep vein thrombosis, inpatient status, D-dimer (in µg/L), and an interaction term between age and D-dimer. The pooled c-statistic was 0.87 (95% CI, 0.85–0.89; 95% PI, 0.77–0.93) and overall calibration was very good (pooled O:E ratio, 0.99; 95% CI, 0.87–1.14; 95% PI, 0.55–1.79). The model slightly overestimated VTE probability in the lower range of estimated probabilities. Discrimination of the current model in the validation data sets was better than that of the Wells score combined with a D-dimer threshold based on age (c-statistic 0.73; 95% CI, 0.70–0.75) or structured clinical pretest probability (c-statistic 0.79; 95% CI, 0.76–0.81).ConclusionThe present model provides an absolute, individualized probability of PE presence in a broad population of patients with suspected PE, with very good discrimination and calibration. Its clinical utility needs to be evaluated in a prospective management or impact study. Show less
Pastena, M. de; Bodegraven, E.A. van; Mungroop, T.H.; Vissers, F.L.; Jones, L.R.; Marchegiani, G.; ... ; Bassi, C. 2023
Objective: To develop 2 distinct preoperative and intraoperative risk scores to predict postoperative pancreatic fistula (POPF) after distal pancreatectomy (DP) to improve preventive and mitigation... Show moreObjective: To develop 2 distinct preoperative and intraoperative risk scores to predict postoperative pancreatic fistula (POPF) after distal pancreatectomy (DP) to improve preventive and mitigation strategies, respectively.Background: POPF remains the most common complication after DP. Despite several known risk factors, an adequate risk model has not been developed yet.Methods: Two prediction risk scores were designed using data of patients undergoing DP in 2 Italian centers (2014-2016) utilizing multivariable logistic regression. The preoperative score (calculated before surgery) aims to facilitate preventive strategies and the intraoperative score (calculated at the end of surgery) aims to facilitate mitigation strategies. Internal validation was achieved using bootstrapping. These data were pooled with data from 5 centers from the United States and the Netherlands (2007-2016) to assess discrimination and calibration in an internal-external validation procedure.Results: Overall, 1336 patients after DP were included, of whom 291 (22%) developed POPF. The preoperative distal fistula risk score (preoperative D-FRS) included 2 variables: pancreatic neck thickness [odds ratio: 1.14; 95% confidence interval (CI): 1.11-1.17 per mm increase] and pancreatic duct diameter (OR: 1.46; 95% CI: 1.32-1.65 per mm increase). The model performed well with an area under the receiver operating characteristic curve of 0.83 (95% CI: 0.78-0.88) and 0.73 (95% CI: 0.70-0.76) upon internal-external validation. Three risk groups were identified: low risk (<10%), intermediate risk (10%-25%), and high risk (>25%) for POPF with 238 (18%), 684 (51%), and 414 (31%) patients, respectively. The intraoperative risk score (intraoperative D-FRS) added body mass index, pancreatic texture, and operative time as variables with an area under the receiver operating characteristic curve of 0.80 (95% CI: 0.74-0.85).Conclusions: The preoperative and the intraoperative D-FRS are the first validated risk scores for POPF after DP and are readily available at: . The 3 distinct risk groups allow for personalized treatment and benchmarking. Show less
Luppino, F.S.; Hollander-Gijsman, M.E. den; Dekker, F.W.; Bartlema, K.A.; Diepen, M. van 2022
Skiing and snowboarding are both popular recreational alpine sports, with substantial injury risk of variable severity. Although skills level has repeatedly been associated with injury risk, a... Show moreSkiing and snowboarding are both popular recreational alpine sports, with substantial injury risk of variable severity. Although skills level has repeatedly been associated with injury risk, a validated measure to accurately estimate the actual skills level without objective assessment is missing. This study aimed to develop a practical validated instrument, to better estimate the actual skills level of recreational skiers, based on the criteria of the Dutch Skiing Federation (DSF), and covering five different skill domains. A sample of Dutch recreational skiers (n = 84) was asked to fill in a questionnaire reflecting seven, a priori chosen predictors by expert opinion, to ski downhill and to be objectively evaluated by expert assessors. The instrument was developed to have a multidimensional character and was validated according to the TRIPOD guideline (Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis). The sample reported an overall incorrect self-reported estimation of their skills, compared with the observed skill score. The instrument showed good calibration and underwent multiple validation methods. The estimated skills score showed to be closer to the observed scores, than self-reportage. Our study provides a practical, multidimensional, and validated instrument to estimate the actual skills level. It proved to better reflect the actual skills levels compared with self-reportage among recreational skiers. Show less
Ratna, M.B.; Bhattacharya, S.; Geloven, N. van; McLernon, D.J. 2022
STUDY QUESTION Can we develop an IVF prediction model to estimate individualized chances of a live birth over multiple complete cycles of IVF in couples embarking on their second complete cycle of... Show moreSTUDY QUESTION Can we develop an IVF prediction model to estimate individualized chances of a live birth over multiple complete cycles of IVF in couples embarking on their second complete cycle of treatment? SUMMARY ANSWER Yes, our prediction model can estimate individualized chances of cumulative live birth over three additional complete cycles of IVF. WHAT IS KNOWN ALREADY After the completion of a first complete cycle of IVF, couples who are unsuccessful may choose to undergo further treatment to have their first child, while those who have had a live birth may decide to have more children. Existing prediction models can estimate the overall chances of success in couples before commencing IVF but are unable to revise these chances on the basis of the couple's response to a first treatment cycle in terms of the number of eggs retrieved and pregnancy outcome. This makes it difficult for couples to plan and prepare emotionally and financially for the next step in their treatment. STUDY DESIGN, SIZE, DURATION For model development, a population-based cohort was used of 49 314 women who started their second cycle of IVF including ICSI in the UK from 1999 to 2008 using their own oocytes and their partners' sperm. External validation was performed on data from 39 442 women who underwent their second cycle from 2010 to 2016. PARTICIPANTS/MATERIALS, SETTING, METHODS Data about all UK IVF treatments were obtained from the Human Fertilisation and Embryology Authority (HFEA) database. Using a discrete time logistic regression model, we predicted the cumulative probability of live birth from the second up to and including the fourth complete cycles of IVF. Inverse probability weighting was used to account for treatment discontinuation. Discrimination was assessed using c-statistic and calibration was assessed using calibration-in-the-large and calibration slope. MAIN RESULTS AND THE ROLE OF CHANCE Following exclusions, 49 314 women with 73 053 complete cycles were included. 12 408 (25.2%) had a live birth resulting from their second complete cycle. Cumulatively, 17 394 (35.3%) had a live birth over complete cycles two to four. The model showed moderate discriminative ability (c-statistic: 0.65, 95% CI: 0.64 to 0.65) and evidence of overprediction (calibration-in-the-large = -0.08) and overfitting (calibration slope 0.85, 95% CI: 0.81 to 0.88) in the validation cohort. However, after recalibration the fit was much improved. The recalibrated model identified the following key predictors of live birth: female age (38 versus 32 years-adjusted odds ratio: 0.59, 95% CI: 0.57 to 0.62), number of eggs retrieved in the first complete cycle (12 versus 4 eggs; 1.34, 1.30 to 1.37) and outcome of the first complete cycle (live birth versus no pregnancy; 1.78, 1.66 to 1.91; live birth versus pregnancy loss; 1.29, 1.23 to 1.36). As an example, a 32-year-old with 2 years of non-tubal infertility who had 12 eggs retrieved from her first stimulation and had a live birth during her first complete cycle has a 46% chance of having a further live birth from the second complete cycle of IVF and an 81% chance over a further three cycles. LIMITATIONS, REASONS FOR CAUTION The developed model was updated using validation data that was 6 to 12 years old. IVF practice continues to evolve over time, which may affect the accuracy of predictions from the model. We were unable to adjust for some potentially important predictors, e.g. BMI, smoking and alcohol intake in women, as well as measures of ovarian reserve such as antral follicle count.These were not available in the linked HFEA dataset. WIDER IMPLICATIONS OF THE FINDINGS By appropriately adjusting for couples who discontinue treatment, our novel prediction model will provide more realistic chances of live birth in couples starting a second complete cycle of IVF. Clinicians can use these predictions to inform discussion with couples who wish to plan ahead. This prediction tool will enable couples to prepare emotionally, financially and logistically for IVF treatment. STUDY FUNDING/COMPETING INTEREST(S) This study was supported by an Elphinstone scholarship scheme at the University of Aberdeen and Aberdeen Fertility Centre, University of Aberdeen. The authors have no conflict of interest. Show less
Patients with acute venous thromboembolism (VTE) require anticoagulant therapy to prevent recurrent VTE and death, which exposes them to an inherent increased risk of bleeding. Identification of... Show morePatients with acute venous thromboembolism (VTE) require anticoagulant therapy to prevent recurrent VTE and death, which exposes them to an inherent increased risk of bleeding. Identification of patients at high risk of bleeding, and mitigating this risk, is an essential component of the immediate and long-term therapeutic management of VTE. The bleeding risk can be estimated by either implicit judgment, weighing individual predictors (clinical variables or biomarkers), or by risk prediction tools developed for this purpose. Management of bleeding risk in clinical practice is, however, far from standardized. International guidelines are contradictory and lack clear and consistent guidance on the optimal management of bleeding risk. This report of the ISTH subcommittee on Predictive and Diagnostic Variables in Thrombotic Disease summarizes the evidence on the prediction of bleeding in VTE patients. We systematically searched the literature and identified 34 original studies evaluating either predictors or risk prediction models for prediction of bleeding risk on anticoagulation in VTE patients. Based on this evidence, we provide recommendations for the standardized management of bleeding risk in VTE patients. Show less
Fosse, N.A. du; Hoorn, M.L.P. van der; Koning, R. de; Mulders, A.G.M.G.J.; Lith, J.M.M. van; Cessie, S. le; Lashley, E.E.L.O. 2022
Objective: To identify, besides maternal age and the number of previous pregnancy losses, additional characteristics of couples with unexplained recurrent pregnancy loss (RPL) that improve the... Show moreObjective: To identify, besides maternal age and the number of previous pregnancy losses, additional characteristics of couples with unexplained recurrent pregnancy loss (RPL) that improve the prediction of an ongoing pregnancy.Design: Hospital-based cohort study in couples who visited specialized RPL units of two academic centers between 2012 and 2020.Setting: Two academic centers in the Netherlands.Patients: Clinical data from 526 couples with unexplained RPL were used in this study.Intervention(s): None.Main Outcome Measures: The final model to estimate the chance of a subsequent ongoing pregnancy was determined using a backward selection process and internally validated using bootstrapping. Model performance was assessed in terms of calibration and discrimination (area under the receiver operating characteristic curve).Results: Subsequent ongoing pregnancy was achieved in 345 of 526 couples (66%). The number of previous pregnancy losses, maternal age, paternal age, maternal body mass index, paternal body mass index, maternal smoking status, and previous in vitro fertilization/intracytoplasmic sperm injection treatment were predictive of the outcome. The optimism-corrected area under the receiver operating characteristic curve was 0.63 compared with 0.57 when using only the number of previous pregnancy losses and maternal age.Conclusions: The identification of additional predictors of a subsequent ongoing pregnancy after RPL, including male characteristics, is significant for both clinicians and couples with RPL. At the same time, we showed that the predictive ability of the current model is still limited and more research is warranted to develop a model that can be used in clinical practice. (C) 2021 by American Society for Reproductive Medicine. Show less
Youssef, A.; Hoorn, M.L.P. van der; Dongen, M.; Visser, J.; Bloemenkamp, K.; Lith, J. van; ... ; Lashley, E.E.L.O. 2021
Study question: What is the predictive performance of a currently recommended prediction model in an external Dutch cohort of couples with unexplained recurrent pregnancy loss (RPL)?Summary answer: .. Show moreStudy question: What is the predictive performance of a currently recommended prediction model in an external Dutch cohort of couples with unexplained recurrent pregnancy loss (RPL)?Summary answer: The model shows poor predictive performance on a new population; it overestimates, predicts too extremely and has a poor discriminative ability.What is known already: In 50-75% of couples with RPL, no risk factor or cause can be determined and RPL remains unexplained. Clinical management in RPL is primarily focused on providing supportive care, in which counselling on prognosis is a main pillar. A frequently used prediction model for unexplained RPL, developed by Brigham et al. in 1999, estimates the chance of a successful pregnancy based on number of previous pregnancy losses and maternal age. This prediction model has never been externally validated.Study design, size, duration: This retrospective cohort study consisted of 739 couples with unexplained RPL who visited the RPL clinic of the Leiden University Medical Centre between 2004 and 2019.Participants/materials, setting, methods: Unexplained RPL was defined as the loss of two or more pregnancies before 24 weeks, without the presence of an identifiable cause for the pregnancy losses, according to the ESHRE guideline. Obstetrical history and maternal age were noted at intake at the RPL clinic. The outcome of the first pregnancy after intake was documented. The performance of Brigham's model was evaluated through calibration and discrimination, in which the predicted pregnancy rates were compared to the observed pregnancy rates.Main results and the role of chance: The cohort included 739 women with a mean age of 33.1 years (±4.7 years) and with a median of three pregnancy losses at intake (range 2-10). The mean predicted pregnancy success rate was 9.8 percentage points higher in the Brigham model than the observed pregnancy success rate in the dataset (73.9% vs 64.0% (95% CI for the 9.8% difference 6.3-13.3%)). Calibration showed overestimation of the model and too extreme predictions, with a negative calibration intercept of -0.46 (95% CI -0.62 to -0.31) and a calibration slope of 0.42 (95% CI 0.11-0.73). The discriminative ability of the model was very low with a concordance statistic of 0.55 (95% CI 0.51-0.59). Recalibration of the Brigham model hardly improved the c-statistic (0.57; 95% CI 0.53-0.62).Limitations, reasons for caution: This is a retrospective study in which only the first pregnancy after intake was registered. There was no time frame as inclusion criterium, which is of importance in the counselling of couples with unexplained RPL. Only cases with a known pregnancy outcome were included.Wider implications of the findings: This is the first study externally validating the Brigham prognostic model that estimates the chance of a successful pregnancy in couples with unexplained RPL. The results show that the frequently used model overestimates the chances of a successful pregnancy, that predictions are too extreme on both the high and low ends and that they are not much more discriminative than random luck. There is a need for revising the prediction model to estimate the chance of a successful pregnancy in couples with unexplained RPL more accurately.Study funding/competing interest(s): No external funding was used and no competing interests were declared.Trial registration number: N/A.Keywords: external validation; miscarriage; prediction model; pregnancy success rate; recurrent pregnancy loss. Show less
Objective To perform a temporal and geographical validation of a prognostic model, considered of highest methodological quality in a recently published systematic review, for predicting survival in... Show moreObjective To perform a temporal and geographical validation of a prognostic model, considered of highest methodological quality in a recently published systematic review, for predicting survival in very preterm infants admitted to the neonatal intensive care unit. The original model was developed in the UK and included gestational age, birthweight and gender. Design External validation study in a population-based cohort. Setting Dutch neonatal wards. Population or sample All admitted white, singleton infants born between 23(+0) and 32(+6) weeks of gestation between 1 January 2015 and 31 December 2019. Additionally, the model's performance was assessed in four populations of admitted infants born between 24(+0) and 31(+6) weeks of gestation: white singletons, non-white singletons, all singletons and all multiples. Methods The original model was applied in all five validation sets. Model performance was assessed in terms of calibration and discrimination and, if indicated, it was updated. Main outcome measures Calibration (calibration-in-the-large and calibration slope) and discrimination (c statistic). Results Out of 6092 infants, 5659 (92.9%) survived. The model showed good external validity as indicated by good discrimination (c statistic 0.82, 95% CI 0.79-0.84) and calibration (calibration-in-the-large 0.003, calibration slope 0.92, 95% CI 0.84-1.00). The model also showed good external validity in the other singleton populations, but required a small intercept update in the multiples population. Conclusions A high-quality prognostic model predicting survival in very preterm infants had good external validity in an independent, nationwide cohort. The accurate performance of the model indicates that after impact assessment, implementation of the model in clinical practice in the neonatal intensive care unit could be considered. Tweetable abstract A high-quality model predicting survival in very preterm infants is externally valid in an independent cohort. Show less
Objectives: In the SPRINT trial, 18% of patients with a tibial shaft fracture (TSF) treated with intramedullary nailing (IMN) had one or more unplanned subsequent surgical procedures. It is... Show moreObjectives: In the SPRINT trial, 18% of patients with a tibial shaft fracture (TSF) treated with intramedullary nailing (IMN) had one or more unplanned subsequent surgical procedures. It is clinically relevant for surgeon and patient to anticipate unplanned secondary procedures, other than operations that can be readily expected such as reconstructive procedures for soft tissue defects. Therefore, the objective of this study was to develop a machine learning (ML) prediction model using the SPRINT data that can give individual patients and their care team an estimate of their particular probability of an unplanned second surgery. Methods: Patients from the SPRINT trial with unilateral TSFs were randomly divided into a training set (80%) and test set (20%). Five ML algorithms were trained in recognizing patterns associated with subsequent surgery in the training set based on a subset of variables identified by random forest algorithms. Performance of each ML algorithm was evaluated and compared based on (1) area under the ROC curve, (2) calibration slope and intercept, and (3) the Brier score. Results: Total data set comprised 1198 patients, of whom 214 patients (18%) underwent subsequent surgery. Seven variables were used to train ML algorithms: (1) Gustilo-Anderson classification, (2) Tscherne classification, (3) fracture location, (4) fracture gap, (5) polytrauma, (6) injury mechanism, and (7) OTA/AO classification. The best-performing ML algorithm had an area under the ROC curve, calibration slope, calibration intercept, and the Brier score of 0.766, 0.954, -0.002, and 0.120 in the training set and 0.773, 0.922, 0, and 0.119 in the test set, respectively. Conclusions: An ML algorithm was developed to predict the probability of subsequent surgery after IMN for TSFs. This ML algorithm may assist surgeons to inform patients about the probability of subsequent surgery and might help to identify patients who need a different perioperative plan or a more intensive approach. Show less
Ramspek, C.L.; Moumni, M. el; Wali, E.; Heemskerk, M.B.A.; Pol, R.A.; Crop, M.J.; ... ; Moers, C. 2021
With a rising demand for kidney transplantation, reliable pre-transplant assessment of organ quality becomes top priority. In clinical practice, physicians are regularly in doubt whether suboptimal... Show moreWith a rising demand for kidney transplantation, reliable pre-transplant assessment of organ quality becomes top priority. In clinical practice, physicians are regularly in doubt whether suboptimal kidney offers from older donors should be accepted. Here, we externally validate existing prediction models in a European population of older deceased donors, and subsequently developed and externally validated an adverse outcome prediction tool. Recipients of kidney grafts from deceased donors 50 years of age and older were included from the Netherlands Organ Transplant Registry (NOTR) and United States organ transplant registry from 2006-2018. The predicted adverse outcome was a composite of graft failure, death or chronic kidney disease stage 4 plus within one year after transplantation, modelled using logistic regression. Discrimination and calibration were assessed in internal, temporal and external validation. Seven existing models were validated with the same cohorts. The NOTR development cohort contained 2510 patients and 823 events. The temporal validation within NOTR had 837 patients and the external validation used 31987 patients in the United States organ transplant registry. Discrimination of our full adverse outcome model was moderate in external validation (C-statistic 0.63), though somewhat better than discrimination of the seven existing prediction models (average C-statistic 0.57). The model's calibration was highly accurate. Thus, since existing adverse outcome kidney graft survival models performed poorly in a population of older deceased donors, novel models were developed and externally validated, with maximum achievable performance in a population of older deceased kidney donors. These models could assist transplant clinicians in deciding whether to accept a kidney from an older donor. Show less
Pavlou, M.; Qu, C.; Omar, R.Z.; Seaman, S.R.; Steyerberg, E.W.; White, I.R.; Ambler, G. 2021
Risk-prediction models for health outcomes are used in practice as part of clinical decision-making, and it is essential that their performance be externally validated. An important aspect in the... Show moreRisk-prediction models for health outcomes are used in practice as part of clinical decision-making, and it is essential that their performance be externally validated. An important aspect in the design of a validation study is choosing an adequate sample size. In this paper, we investigate the sample size requirements for validation studies with binary outcomes to estimate measures of predictive performance (C-statistic for discrimination and calibration slope and calibration in the large). We aim for sufficient precision in the estimated measures. In addition, we investigate the sample size to achieve sufficient power to detect a difference from a target value. Under normality assumptions on the distribution of the linear predictor, we obtain simple estimators for sample size calculations based on the measures above. Simulation studies show that the estimators perform well for common values of the C-statistic and outcome prevalence when the linear predictor is marginally Normal. Their performance deteriorates only slightly when the normality assumptions are violated. We also propose estimators which do not require normality assumptions but require specification of the marginal distribution of the linear predictor and require the use of numerical integration. These estimators were also seen to perform very well under marginal normality. Our sample size equations require a specified standard error (SE) and the anticipated C-statistic and outcome prevalence. The sample size requirement varies according to the prognostic strength of the model, outcome prevalence, choice of the performance measure and study objective. For example, to achieve an SE < 0.025 for the C-statistic, 60-170 events are required if the true C-statistic and outcome prevalence are between 0.64-0.85 and 0.05-0.3, respectively. For the calibration slope and calibration in the large, achieving SE < 0.15 would require 40-280 and 50-100 events, respectively. Our estimators may also be used for survival outcomes when the proportion of censored observations is high. Show less
BackgroundPatients with transposition of the great arteries corrected by an atrial switch operation experience major clinical events during adulthood, mainly heart failure (HF) and arrhythmias, but... Show moreBackgroundPatients with transposition of the great arteries corrected by an atrial switch operation experience major clinical events during adulthood, mainly heart failure (HF) and arrhythmias, but data on the emerging risks remain scarce. We assessed the risk for events during the clinical course in adulthood, and provided a novel risk score for event-free survival.Methods and ResultsThis multicenter study observed 167 patients with transposition of the great arteries corrected by an atrial switch operation (61% Mustard procedure; age, 28 [interquartile range, 24-36] years) for 13 (interquartile range, 9-16) years, during which 16 (10%) patients died, 33 (20%) had HF events, defined as HF hospitalizations, heart transplantation, ventricular assist device implantation, or HF-related death, and 15 (9%) had symptomatic ventricular arrhythmias. Five-year risk of mortality, first HF event, and first ventricular arrhythmia increased from 1% each at age 25 years, to 6% (95% CI, 4%-9%), 23% (95% CI, 17%-28%), and 5% (95% CI, 2%-8%), respectively, at age 50 years. Predictors for event-free survival were examined to construct a prediction model using bootstrapping techniques. A prediction model combining age >30 years, prior ventricular arrhythmia, age >1 year at repair, moderate or greater right ventricular dysfunction, severe tricuspid regurgitation, and mild or greater left ventricular dysfunction discriminated well between patients at low (<5%), intermediate (5%-20%), and high (>20%) 5-year risk (optimism-corrected C-statistic, 0.86 [95% CI, 0.82-0.90]). Observed 5- and 10-year event-free survival rates in low-risk patients were 100% and 97%, respectively, compared with only 31% and 8%, respectively, in high-risk patients.ConclusionsThe clinical course of patients undergoing atrial switch increasingly consists of major clinical events, especially HF. A novel risk score stratifying patients as low, intermediate, and high risk for event-free survival provides information on absolute individual risks, which may support decisions for pharmacological and interventional management. Show less
Simple SummaryA prognostic index for predicting survival of localized prostate cancer (PCa) up to 15 and 20 years was developed. The prognostic index performed well for predicting PCa survival... Show moreSimple SummaryA prognostic index for predicting survival of localized prostate cancer (PCa) up to 15 and 20 years was developed. The prognostic index performed well for predicting PCa survival among screened and non-screened men. The performance of the prediction model was superior to the European Association of Urology (EAU) risk groups as well as a modified cancer of prostate risk assessment (CAPRA) risk score. We further constructed a simplified risk score in an unscreened population, using the three most relevant predictors. The simplified risk score was applied to predict PCa survival at 10 years from diagnosis to provide more accurate risk estimation as the basis for decision making.We developed and validated a prognostic index to predict survival from prostate cancer (PCa) based on the Finnish randomized screening trial (FinRSPC). Men diagnosed with localized PCa (N = 7042) were included. European Association of Urology risk groups were defined. The follow-up was divided into three periods (0-3, 3-9 and 9-20 years) for development and two corresponding validation periods (3-6 and 9-15 years). A multivariable complementary log-log regression model was used to calculate the full prognostic index. Predicted cause-specific survival at 10 years from diagnosis was calculated for the control arm using a simplified risk score at diagnosis. The full prognostic index discriminates well men with PCa with different survival. The area under the curve (AUC) was 0.83 for both the 3-6 year and 9-15 year validation periods. In the simplified risk score, patients with a low risk score at diagnosis had the most favorable survival, while the outcome was poorest for the patients with high risk scores. The prognostic index was able to distinguish well between men with higher and lower survival, and the simplified risk score can be used as a basis for decision making. Show less
Background: Cumulative anticholinergic exposure, also known as anticholinergic burden, is associated with a variety of adverse outcomes. However, studies show that anticholinergic effects tend to... Show moreBackground: Cumulative anticholinergic exposure, also known as anticholinergic burden, is associated with a variety of adverse outcomes. However, studies show that anticholinergic effects tend to be underestimated by prescribers, and anticholinergics are the most frequently prescribed potentially inappropriate medication in older patients. The grading systems and drugs included in existing scales to quantify anticholinergic burden differ considerably and do not adequately account for patients' susceptibility to medications. Furthermore, their ability to link anticholinergic burden with adverse outcomes such as falls is unclear. This study aims to develop a prognostic model that predicts falls in older general practice patients, to assess the performance of several anticholinergic burden scales, and to quantify the added predictive value of anticholinergic symptoms in this context.Methods: Data from two cluster-randomized controlled trials investigating medication optimization in older general practice patients in Germany will be used. One trial (RIME, n = 1,197) will be used for the model development and the other trial (PRIMUM, n = 502) will be used to externally validate the model. A priori, candidate predictors will be selected based on a literature search, predictor availability, and clinical reasoning. Candidate predictors will include socio-demographics (e.g. age, sex), morbidity (e.g. single conditions), medication (e.g. polypharmacy, anticholinergic burden as defined by scales), and well-being (e.g. quality of life, physical function). A prognostic model including sociodemographic and lifestyle-related factors, as well as variables on morbidity, medication, health status, and well-being, will be developed, whereby the prognostic value of extending the model to include additional patient-reported symptoms will be also assessed. Logistic regression will be used for the binary outcome, which will be defined as "no falls" vs. ">= 1 fall" within six months of baseline, as reported in patient interviews.Discussion: As the ability of different anticholinergic burden scales to predict falls in older patients is unclear, this study may provide insights into their relative importance as well as into the overall contribution of anticholinergic symptoms and other patient characteristics. The results may support general practitioners in their clinical decision-making and in prescribing fewer medications with anticholinergic properties. Show less
Ramspek, C.L.; Verberne, W.R.; Buren, M. van; Dekker, F.W.; Bos, W.J.W.; Diepen, M. van 2021
Background. Conservative care (CC) may be a valid alternative to dialysis for certain older patients with advanced chronic kidney disease (CKD). A model that predicts patient prognosis on both... Show moreBackground. Conservative care (CC) may be a valid alternative to dialysis for certain older patients with advanced chronic kidney disease (CKD). A model that predicts patient prognosis on both treatment pathways could be of value in shared decision-making. Therefore, the aim is to develop a prediction tool that predicts the mortality risk for the same patient for both dialysis and CC from the time of treatment decision.Methods. CKD Stage 4/5 patients aged >= 70 years, treated at a single centre in the Netherlands, were included between 2004 and 2016. Predictors were collected at treatment decision and selected based on literature and an expert panel. Outcome was 2-year mortality. Basic and extended logistic regression models were developed for both the dialysis and CC groups. These models were internally validated with bootstrapping. Model performance was assessed with discrimination and calibration.Results. In total, 366 patients were included, of which 126 chose CC. Pre-selected predictors for the basic model were age, estimated glomerular filtration rate, malignancy and cardiovascular disease. Discrimination was moderate, with optimism-corrected C-statistics ranging from 0.675 to 0.750. Calibration plots showed good calibration.Conclusions. A prediction tool that predicts 2-year mortality was developed to provide older advanced CKD patients with individualized prognosis estimates for both dialysis and CC. Future studies are needed to test whether our findings hold in other CKD populations. Following external validation, this prediction tool could be used to compare a patient's prognosis on both dialysis and CC, and help to inform treatment decision-making. Show less
Background: Length of stay (LOS) in the Emergency Department (ED) is correlated with an extended in-hospital LOS and may even increase 30-day mortality. Older patients represent a growing... Show moreBackground: Length of stay (LOS) in the Emergency Department (ED) is correlated with an extended in-hospital LOS and may even increase 30-day mortality. Older patients represent a growing population in the ED and they are especially at risk of adverse outcomes. Screening tools that adequately predict admission could help reduce waiting times in the ED and reduce time to treatment. We aimed to develop and validate a clinical prediction tool for admission, applicable to the aged patient population in the ED.Methods: Data from 7,606 ED visits of patients aged 70 years and older between 2012 and 2014 were used to develop the CLEARED tool. Model performance was assessed with discrimination using logistic regression and calibration. The model was internally validated by bootstrap resampling in Erasmus Medical Center and externally validated at two other hospitals, Medisch Spectrum Twente (MST) and Leiden University Medical Centre (LUMC).Results: CLEARED contains 10 predictors: body temperature, heart rate, diastolic blood pressure, systolic blood pressure, oxygen saturation, respiratory rate, referral status, the Manchester Triage System category, and the need for laboratory or radiology testing. The internally validated area under the curve (AUC) was 0.766 (95% CI [0759; 0.780. External validation in MST showed an AUC of o.797 and in LUMC, an AUC of 0.725.Conclusions: The developed CLEARED tool reliably predicts admission in elderly patients visiting the ED. It is a promising prompt, although further research is needed to implement the tool and to investigate the benefits in terms of reduction of crowding and LOS in the ED. Show less
Ramspek, C.L.; Jong, Y. de; Dekker, F.W.; Diepen, M. van 2020
Background. Prediction tools that identify chronic kidney disease (CKD) patients at a high risk of developing kidney failure have the potential for great clinical value, but limited uptake. The aim... Show moreBackground. Prediction tools that identify chronic kidney disease (CKD) patients at a high risk of developing kidney failure have the potential for great clinical value, but limited uptake. The aim of the current study is to systematically review all available models predicting kidney failure in CKD patients, organize empirical evidence on their validity and ultimately provide guidance in the interpretation and uptake of these tools.Methods. PubMed and EMBASE were searched for relevant articles. Titles, abstracts and full-text articles were sequentially screened for inclusion by two independent researchers. Data on study design, model development and performance were extracted. The risk of bias and clinical usefulness were accessed and combined in order to provide recommendations on which models to use.Results. Of 2183 screened studies, a total of 42 studies were included in the current review. Most studies showed high discriminatory capacity and the included predictors had large overlap. Overall, the risk of bias was high. Slightly less than half the studies (48%) presented enough detail for the use of their prediction tool in practice and few models were externally validated.Conclusions. The current systematic review may be used as a tool to select the most appropriate and robust prognostic model for various settings. Although some models showed great potential, many lacked clinical relevance due to being developed in a prevalent patient population with a wide range of disease severity. Future research efforts should focus on external validation and impact assessment in clinically relevant patient populations. Show less
OBJECTIVE Nonroutine discharge after elective spine surgery increases healthcare costs, negatively impacts patient satisfaction, and exposes patients to additional hospital-acquired complications.... Show moreOBJECTIVE Nonroutine discharge after elective spine surgery increases healthcare costs, negatively impacts patient satisfaction, and exposes patients to additional hospital-acquired complications. Therefore, prediction of nonroutine discharge in this population may improve clinical management. The authors previously developed a machine learning algorithm from national data that predicts risk of nonhome discharge for patients undergoing surgery for lumbar disc disorders. In this paper the authors externally validate their algorithm in an independent institutional population of neurosurgical spine patients.METHODS Medical records from elective inpatient surgery for lumbar disc herniation or degeneration in the Transitional Care Program at Brigham and Women's Hospital (2013-2015) were retrospectively reviewed. Variables included age, sex, BMI, American Society of Anesthesiologists (ASA) class, preoperative functional status, number of fusion levels, comorbidities, preoperative laboratory values, and discharge disposition. Nonroutine discharge was defined as postoperative discharge to any setting other than home. The discrimination (c-statistic), calibration, and positive and negative predictive values (PPVs and NPVs) of the algorithm were assessed in the institutional sample.RESULTS Overall, 144 patients underwent elective inpatient surgery for lumbar disc disorders with a nonroutine discharge rate of 6.9% (n = 10). The median patient age was 50 years and 45.1% of patients were female. Most patients were ASA class II (66.0%), had 1 or 2 levels fused (80.6%), and had no diabetes (91.7%). The median hematocrit level was 41.2%. The neural network algorithm generalized well to the institutional data, with a c-statistic (area under the receiver operating characteristic curve) of 0.89, calibration slope of 1.09, and calibration intercept of -0.08. At a threshold of 0.25, the PPV was 0.50 and the NPV was 0.97.CONCLUSIONS This institutional external validation of a previously developed machine learning algorithm suggests a reliable method for identifying patients with lumbar disc disorder at risk for nonroutine discharge. Performance in the institutional cohort was comparable to performance in the derivation cohort and represents an improved predictive value over clinician intuition. This finding substantiates initial use of this algorithm in clinical practice. This tool may be used by multidisciplinary teams of case managers and spine surgeons to strategically invest additional time and resources into postoperative plans for this population. Show less
