Objectives: To (1) explore trends of risk of bias (ROB) in prediction research over time following key methodological publications, using the Prediction model Risk Of Bias ASsessment Tool (PROBAST)... Show moreObjectives: To (1) explore trends of risk of bias (ROB) in prediction research over time following key methodological publications, using the Prediction model Risk Of Bias ASsessment Tool (PROBAST) and (2) assess the inter-rater agreement of the PROBAST.Study Design and Setting: PubMed and Web of Science were searched for reviews with extractable PROBAST scores on domain and signaling question (SQ) level. ROB trends were visually correlated with yearly citations of key publications. Inter-rater agreement was asResults: One hundred and thirty nine systematic reviews were included, of which 85 reviews (containing 2,477 single studies) on domain level and 54 reviews (containing 2,458 single studies) on SQ level. High ROB was prevalent, especially in the Analysis domain, and overall trends of ROB remained relatively stable over time. The inter-rater agreement was low, both on domain (Kappa 0.04-0.26) and SQ level (Kappa -0.14 to 0.49). Conclusion: Prediction model studies are at high ROB and time trends in ROB as assessed with the PROBAST remain relatively stable. These results might be explained by key publications having no influence on ROB or recency of key publications. Moreover, the trend may suffer from the low inter-rater agreement and ceiling effect of the PROBAST. The inter-rater agreement could potentially be improved by altering the PROBAST or providing training on how to apply the PROBAST.& COPY; 2023 The Authors. Published by Elsevier Inc. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/). Show less
Endt, V.H.W. van der; Milders, J.; Vries, B.B.L.P. de; Trines, S.A.; Groenwold, R.H.H.; Dekkers, O.M.; ... ; Jong, Y. de 2022
Aims Multiple risk scores to predict ischaemic stroke (IS) in patients with atrial fibrillation (AF) have been developed. This study aims to systematically review these scores, their validations... Show moreAims Multiple risk scores to predict ischaemic stroke (IS) in patients with atrial fibrillation (AF) have been developed. This study aims to systematically review these scores, their validations and updates, assess their methodological quality, and calculate pooled estimates of the predictive performance.Methods and results We searched PubMed and Web of Science for studies developing, validating, or updating risk scores for IS in AF patients. Methodological quality was assessed using the Prediction model Risk Of Bias ASsessment Tool (PROBAST). To assess discrimination, pooled c-statistics were calculated using random-effects meta-analysis. We identified 19 scores, which were validated and updated once or more in 70 and 40 studies, respectively, including 329 validations and 76 updates-nearly all on the CHA(2)DS(2)-VASc and CHADS(2). Pooled c-statistics were calculated among 6 267 728 patients and 359 373 events of IS. For the CHA(2)DS(2)-VASc and CHADS(2), pooled c-statistics were 0.644 [95% confidence interval (CI) 0.635-0.653] and 0.658 (0.644-0.672), respectively. Better discriminatory abilities were found in the newer risk scores, with the modified-CHADS(2) demonstrating the best discrimination [c-statistic 0.715 (0.674-0.754)]. Updates were found for the CHA(2)DS(2)-VASc and CHADS(2) only, showing improved discrimination. Calibration was reasonable but available for only 17 studies. The PROBAST indicated a risk of methodological bias in all studies.Conclusion Nineteen risk scores and 76 updates are available to predict IS in patients with AF. The guideline-endorsed CHA(2)DS(2)-VASc shows inferior discriminative abilities compared with newer scores. Additional external validations and data on calibration are required before considering the newer scores in clinical practice. Show less
