AimsClinical guidelines often recommend treating individuals based on their cardiovascular risk. We revisit this paradigm and quantify the efficacy of three treatment strategies: (i) overall prescr... Show moreAimsClinical guidelines often recommend treating individuals based on their cardiovascular risk. We revisit this paradigm and quantify the efficacy of three treatment strategies: (i) overall prescription, i.e. treatment to all individuals sharing the eligibility criteria of a trial; (ii) risk-stratified prescription, i.e. treatment only to those at an elevated outcome risk; and (iii) prescription based on predicted treatment responsiveness.Methods and resultsWe reanalysed the PROSPER randomized controlled trial, which included individuals aged 70–82 years with a history of, or risk factors for, vascular diseases. We conducted the derivation and internal–external validation of a model predicting treatment responsiveness. We compared with placebo (n = 2913): (i) pravastatin (n = 2891); (ii) pravastatin in the presence of previous vascular diseases and placebo in the absence thereof (n = 2925); and (iii) pravastatin in the presence of a favourable prediction of treatment response and placebo in the absence thereof (n = 2890). We found an absolute difference in primary outcome events composed of coronary death, non-fatal myocardial infarction, and fatal or non-fatal stroke, per 10 000 person-years equal to: −78 events (95% CI, −144 to −12) when prescribing pravastatin to all participants; −66 events (95% CI, −114 to −18) when treating only individuals with an elevated vascular risk; and −103 events (95% CI, −162 to −44) when restricting pravastatin to individuals with a favourable prediction of treatment response.ConclusionPravastatin prescription based on predicted responsiveness may have an encouraging potential for cardiovascular prevention. Further external validation of our results and clinical experiments are needed. Show less
While the opportunities of ML and AI in healthcare are promising, the growth of complex data-driven prediction models requires careful quality and applicability assessment before they are applied... Show moreWhile the opportunities of ML and AI in healthcare are promising, the growth of complex data-driven prediction models requires careful quality and applicability assessment before they are applied and disseminated in daily practice. This scoping review aimed to identify actionable guidance for those closely involved in AI-based prediction model (AIPM) development, evaluation and implementation including software engineers, data scientists, and healthcare professionals and to identify potential gaps in this guidance. We performed a scoping review of the relevant literature providing guidance or quality criteria regarding the development, evaluation, and implementation of AIPMs using a comprehensive multi-stage screening strategy. PubMed, Web of Science, and the ACM Digital Library were searched, and AI experts were consulted. Topics were extracted from the identified literature and summarized across the six phases at the core of this review: (1) data preparation, (2) AIPM development, (3) AIPM validation, (4) software development, (5) AIPM impact assessment, and (6) AIPM implementation into daily healthcare practice. From 2683 unique hits, 72 relevant guidance documents were identified. Substantial guidance was found for data preparation, AIPM development and AIPM validation (phases 1-3), while later phases clearly have received less attention (software development, impact assessment and implementation) in the scientific literature. The six phases of the AIPM development, evaluation and implementation cycle provide a framework for responsible introduction of AI-based prediction models in healthcare. Additional domain and technology specific research may be necessary and more practical experience with implementing AIPMs is needed to support further guidance. Show less
Wynants, L.; Calster, B. van; Bonten, M.M.J.; Collins, G.S.; Debray, T.P.A.; Vos, M. de; ... ; Smeden, M. van 2020
OBJECTIVETo review and critically appraise published and preprint reports of prediction models for diagnosing coronavirus disease 2019 (covid-19) in patients with suspected infection, for prognosis... Show moreOBJECTIVETo review and critically appraise published and preprint reports of prediction models for diagnosing coronavirus disease 2019 (covid-19) in patients with suspected infection, for prognosis of patients with covid-19, and for detecting people in the general population at risk of being admitted to hospital for covid-19 pneumonia.DESIGNRapid systematic review and critical appraisal.DATA SOURCESPubMed and Embase through Ovid, Arxiv, medRxiv, and bioRxiv up to 24 March 2020.STUDY SELECTIONStudies that developed or validated a multivariable covid-19 related prediction model.DATA EXTRACTIONAt least two authors independently extracted data using the CHARMS (critical appraisal and data extraction for systematic reviews of prediction modelling studies) checklist; risk of bias was assessed using PROBAST (prediction model risk of bias assessment tool).RESULTS2696 titles were screened, and 27 studies describing 31 prediction models were included. Three models were identified for predicting hospital admission from pneumonia and other events (as proxy outcomes for covid-19 pneumonia) in the general population; 18 diagnostic models for detecting covid-19 infection (13 were machine learning based on computed tomography scans); and 10 prognostic models for predicting mortality risk, progression to severe disease, or length of hospital stay. Only one study used patient data from outside of China. The most reported predictors of presence of covid-19 in patients with suspected disease included age, body temperature, and signs and symptoms. The most reported predictors of severe prognosis in patients with covid-19 included age, sex, features derived from computed tomography scans, C reactive protein, lactic dehydrogenase, and lymphocyte count. C index estimates ranged from 0.73 to 0.81 in prediction models for the general population (reported for all three models), from 0.81 to more than 0.99 in diagnostic models (reported for 13 of the 18 models), and from 0.85 to 0.98 in prognostic models (reported for six of the 10 models). All studies were rated at high risk of bias, mostly because of non-representative selection of control patients, exclusion of patients who had not experienced the event of interest by the end of the study, and high risk of model overfitting. Reporting quality varied substantially between studies. Most reports did not include a description of the study population or intended use of the models, and calibration of predictions was rarely assessed.CONCLUSIONPrediction models for covid-19 are quickly entering the academic literature to support medical decision making at a time when they are urgently needed. This review indicates that proposed models are poorly reported, at high risk of bias, and their reported performance is probably optimistic. Immediate sharing of well documented individual participant data from covid-19 studies is needed for collaborative efforts to develop more rigorous prediction models and validate existing ones. The predictors identified in included studies could be considered as candidate predictors for new models. Methodological guidance should be followed because unreliable predictions could cause more harm than benefit in guiding clinical decisions. Finally, studies should adhere to the TRIPOD (transparent reporting of a multivariable prediction model for individual prognosis or diagnosis) reporting guideline. Show less
Steyerberg, E.W.; Nieboer, D.; Debray, T.P.A.; Houwelingen, H.C. van 2019
Clinical prediction models aim to provide estimates of absolute risk for a diagnostic or prognostic endpoint. Such models may be derived from data from various studies in the context of a meta... Show moreClinical prediction models aim to provide estimates of absolute risk for a diagnostic or prognostic endpoint. Such models may be derived from data from various studies in the context of a meta-analysis. We describe and propose approaches for assessing heterogeneity in predictor effects and predictions arising from models based on data from different sources. These methods are illustrated in a case study with patients suffering from traumatic brain injury, where we aim to predict 6-month mortality based on individual patient data using meta-analytic techniques (15 studies, n = 11 022 patients). The insights into various aspects of heterogeneity are important to develop better models and understand problems with the transportability of absolute risk predictions. Show less
AimTo provide a comprehensive overview of cardiovascular disease (CVD) risk prediction models for women and models that include female-specific predictors.MethodsWe performed a systematic review of... Show moreAimTo provide a comprehensive overview of cardiovascular disease (CVD) risk prediction models for women and models that include female-specific predictors.MethodsWe performed a systematic review of CVD risk prediction models for women in the general population by updating a previous review. We searched Medline and Embase up to July 2017 and included studies in which; (a) a new model was developed, (b) an existing model was validated, or (c) a predictor was added to an existing model.ResultsA total of 285 prediction models for women have been developed, of these 160 (56%) were female-specific models, in which a separate model was developed solely in women and 125 (44%) were sex-predictor models. Out of the 160 female-specific models, 2 (1.3%) included one or more female-specific predictors (mostly reproductive risk factors). A total of 591 validations of sex-predictor or female-specific models were identified in 206 papers. Of these, 333 (56%) validations concerned nine models (five versions of Framingham, SCORE, Pooled Cohort Equations and QRISK). The median and pooled C statistics were comparable for sex-predictor and female-specific models. In 260 articles the added value of new predictors to an existing model was described, however in only 3 of these female-specific that no competing interests exist. predictors (reproductive risk factors) were added.ConclusionsThere is an abundance of models for women in the general population. Female-specific and sex-predictor models have similar predictors and performance. Female-specific predictors are rarely included. Further research is needed to assess the added value of female-specific predictors to CVD models for women and provide physicians with a well-performing prediction model for women. Show less
