'Blinding' involves concealing knowledge of which trial participants received the interventions from participants themselves and other trial personnel throughout the trial. Blinding reduces bias... Show more'Blinding' involves concealing knowledge of which trial participants received the interventions from participants themselves and other trial personnel throughout the trial. Blinding reduces bias arising from the beliefs and expectations of these groups. It is agreed that where possible, blinding should be attempted, for example by ensuring that experimental and control treatments look the same. However, there is a debate about if we should measure whether blinding has been successful, this manuscript will discuss this controversy, including the benefits and risks of measuring blinding within the randomised controlled trial. (c) 2021 Elsevier Inc. All rights reserved. Show less
Verbakel, J.Y.; Steyerberg, E.W.; Uno, H.; Cock, B. de; Wynants, L.; Collins, G.S.; Calster, B. van 2020
Objectives: Receiver operating characteristic (ROC) curves show how well a risk prediction model discriminates between patients with and without a condition. We aim to investigate how ROC curves... Show moreObjectives: Receiver operating characteristic (ROC) curves show how well a risk prediction model discriminates between patients with and without a condition. We aim to investigate how ROC curves are presented in the literature and discuss and illustrate their potential limitations.Study Design and Setting: We conducted a pragmatic literature review of contemporary publications that externally validated clinical prediction models. We illustrated limitations of ROC curves using a testicular cancer case study and simulated data.Results: Of 86 identified prediction modeling studies, 52 (60%) presented ROC curves without thresholds and one (1%) presented an ROC curve with only a few thresholds. We illustrate that ROC curves in their standard form withhold threshold information have an unstable shape even for the same area under the curve (AUC) and are problematic for comparing model performance conditional on threshold. We compare ROC curves with classification plots, which show sensitivity and specificity conditional on risk thresholds.Conclusion: ROC curves do not offer more information than the AUC to indicate discriminative ability. To assess the model's performance for decision-making, results should be provided conditional on risk thresholds. Therefore, if discriminatory ability must be visualized, classification plots are attractive. (C) 2020 Published by Elsevier Inc. Show less
Background Placebo or sham controls are the standard against which the benefits and harms of many active interventions are measured. Whilst the components and the method of their delivery have been... Show moreBackground Placebo or sham controls are the standard against which the benefits and harms of many active interventions are measured. Whilst the components and the method of their delivery have been shown to affect study outcomes, placebo and sham controls are rarely reported and often not matched to those of the active comparator. This can influence how beneficial or harmful the active intervention appears to be. Without adequate descriptions of placebo or sham controls, it is difficult to interpret results about the benefits and harms of active interventions within placebo-controlled trials. To overcome this problem, we developed a checklist and guide for reporting placebo or sham interventions. Methods and findings We developed an initial list of items for the checklist by surveying experts in placebo research (n = 14). Because of the diverse contexts in which placebo or sham treatments are used in clinical research, we consulted experts in trials of drugs, surgery, physiotherapy, acupuncture, and psychological interventions. We then used a multistage online Delphi process with 53 participants to determine which items were deemed to be essential. We next convened a group of experts and stakeholders (n = 16). Our main output was a modification of the existing Template for Intervention Description and Replication (TIDieR) checklist; this allows the key features of both active interventions and placebo or sham controls to be concisely summarised by researchers. The main differences between TIDieR-Placebo and the original TIDieR are the explicit requirement to describe the setting (i.e., features of the physical environment that go beyond geographic location), the need to report whether blinding was successful (when this was measured), and the need to present the description of placebo components alongside those of the active comparator. Conclusions We encourage TIDieR-Placebo to be used alongside TIDieR to assist the reporting of placebo or sham components and the trials in which they are used. Show less
Wynants, L.; Calster, B. van; Bonten, M.M.J.; Collins, G.S.; Debray, T.P.A.; Vos, M. de; ... ; Smeden, M. van 2020
OBJECTIVETo review and critically appraise published and preprint reports of prediction models for diagnosing coronavirus disease 2019 (covid-19) in patients with suspected infection, for prognosis... Show moreOBJECTIVETo review and critically appraise published and preprint reports of prediction models for diagnosing coronavirus disease 2019 (covid-19) in patients with suspected infection, for prognosis of patients with covid-19, and for detecting people in the general population at risk of being admitted to hospital for covid-19 pneumonia.DESIGNRapid systematic review and critical appraisal.DATA SOURCESPubMed and Embase through Ovid, Arxiv, medRxiv, and bioRxiv up to 24 March 2020.STUDY SELECTIONStudies that developed or validated a multivariable covid-19 related prediction model.DATA EXTRACTIONAt least two authors independently extracted data using the CHARMS (critical appraisal and data extraction for systematic reviews of prediction modelling studies) checklist; risk of bias was assessed using PROBAST (prediction model risk of bias assessment tool).RESULTS2696 titles were screened, and 27 studies describing 31 prediction models were included. Three models were identified for predicting hospital admission from pneumonia and other events (as proxy outcomes for covid-19 pneumonia) in the general population; 18 diagnostic models for detecting covid-19 infection (13 were machine learning based on computed tomography scans); and 10 prognostic models for predicting mortality risk, progression to severe disease, or length of hospital stay. Only one study used patient data from outside of China. The most reported predictors of presence of covid-19 in patients with suspected disease included age, body temperature, and signs and symptoms. The most reported predictors of severe prognosis in patients with covid-19 included age, sex, features derived from computed tomography scans, C reactive protein, lactic dehydrogenase, and lymphocyte count. C index estimates ranged from 0.73 to 0.81 in prediction models for the general population (reported for all three models), from 0.81 to more than 0.99 in diagnostic models (reported for 13 of the 18 models), and from 0.85 to 0.98 in prognostic models (reported for six of the 10 models). All studies were rated at high risk of bias, mostly because of non-representative selection of control patients, exclusion of patients who had not experienced the event of interest by the end of the study, and high risk of model overfitting. Reporting quality varied substantially between studies. Most reports did not include a description of the study population or intended use of the models, and calibration of predictions was rarely assessed.CONCLUSIONPrediction models for covid-19 are quickly entering the academic literature to support medical decision making at a time when they are urgently needed. This review indicates that proposed models are poorly reported, at high risk of bias, and their reported performance is probably optimistic. Immediate sharing of well documented individual participant data from covid-19 studies is needed for collaborative efforts to develop more rigorous prediction models and validate existing ones. The predictors identified in included studies could be considered as candidate predictors for new models. Methodological guidance should be followed because unreliable predictions could cause more harm than benefit in guiding clinical decisions. Finally, studies should adhere to the TRIPOD (transparent reporting of a multivariable prediction model for individual prognosis or diagnosis) reporting guideline. Show less
Najafabadi, A.H.Z.; Ramspek, C.L.; Dekker, F.W.; Heus, P.; Hooft, L.; Moons, K.G.M.; ... ; Diepen, M. van 2020
Objectives To assess the difference in completeness of reporting and methodological conduct of published prediction models before and after publication of the Transparent Reporting of a... Show moreObjectives To assess the difference in completeness of reporting and methodological conduct of published prediction models before and after publication of the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement. Methods In the seven general medicine journals with the highest impact factor, we compared the completeness of the reporting and the quality of the methodology of prediction model studies published between 2012 and 2014 (pre-TRIPOD) with studies published between 2016 and 2017 (post-TRIPOD). For articles published in the post-TRIPOD period, we examined whether there was improved reporting for articles (1) citing the TRIPOD statement, and (2) published in journals that published the TRIPOD statement. Results A total of 70 articles was included (pre-TRIPOD: 32, post-TRIPOD: 38). No improvement was seen for the overall percentage of reported items after the publication of the TRIPOD statement (pre-TRIPOD 74%, post-TRIPOD 76%, 95% CI of absolute difference: -4% to 7%). For the individual TRIPOD items, an improvement was seen for 16 (44%) items, while 3 (8%) items showed no improvement and 17 (47%) items showed a deterioration. Post-TRIPOD, there was no improved reporting for articles citing the TRIPOD statement, nor for articles published in journals that published the TRIPOD statement. The methodological quality improved in the post-TRIPOD period. More models were externally validated in the same article (absolute difference 8%, post-TRIPOD: 39%), used measures of calibration (21%, post-TRIPOD: 87%) and discrimination (9%, post-TRIPOD: 100%), and used multiple imputation for handling missing data (12%, post-TRIPOD: 50%). Conclusions Since the publication of the TRIPOD statement, some reporting and methodological aspects have improved. Prediction models are still often poorly developed and validated and many aspects remain poorly reported, hindering optimal clinical application of these models. Long-term effects of the TRIPOD statement publication should be evaluated in future studies. Show less
Background Poorly described placebo/sham controls make it difficult to appraise active intervention benefits and harms. The 12-item Template for Intervention Description and Replication (TIDieR)... Show moreBackground Poorly described placebo/sham controls make it difficult to appraise active intervention benefits and harms. The 12-item Template for Intervention Description and Replication (TIDieR) checklist was developed to improve the reporting of active interventions. The extent to which TIDieR has been used to improve description of placebo or sham control is not known. Materials and methods We systematically identified and examined all placebo/sham-controlled randomised trials published in 2018 in the top six general medical journals. We reported how many of the TIDieR checklist items were used to describe the placebo/sham control(s). We supplemented this with a sample of 100 placebo/sham-controlled trials from any journal and searched Google Scholar to identify placebo/sham-controlled trials citing TIDieR. Results We identified 94 placebo/sham-controlled trials published in the top journals in 2018. None reported using TIDieR, and none reported placebo or sham components completely. On average eight TIDieR items were addressed, with placebo/sham control name (100%) and when and how much was administered (97.9%) most commonly reported. Some items (rationale, 8.5%, whether there were modifications, 25.5%) were less often reported. In our sample of less well-cited journals, reporting was poorer (average of six items) and followed a similar pattern. Since TIDieR's first publication, six placebo-controlled trials have cited it according to Google Scholar. Two of these used the checklist to describe placebo controls; neither one completely desribed the placebo intervention. Conclusions Placebo and sham controls are poorly described within randomised trials, and TIDieR is rarely used to guide these descriptions. We recommend developing guidelines to promote better descriptions of placebo/sham control components within clinical trials. Show less
