Purpose Outcome of endovascular treatment in acute ischemic stroke patients is depending on the collateral circulation maintaining blood flow to the ischemic territory. We evaluated the inter-rater... Show morePurpose Outcome of endovascular treatment in acute ischemic stroke patients is depending on the collateral circulation maintaining blood flow to the ischemic territory. We evaluated the inter-rater reliability and accuracy of raters and an automated algorithm for assessing the collateral score (CS, range: 0-3) in acute ischemic stroke patients. Methods Baseline CTA scans with an intracranial anterior occlusion from the MR CLEAN study (n=500) were used. For each core lab CS, ten CTA scans with sufficient quality were randomly selected. After a training session in collateral scoring, all selected CTA scans were individually evaluated for a visual CS by three groups: 7 radiologists, 13 junior and 9 senior radiology residents. Two additional radiologists scored CS to be used as reference, with a third providing a CS to produce a 2 out of 3 consensus CS in case of disagreement. An automated algorithm was also used to compute CS. Inter-rater agreement was reported with intraclass correlation coefficient (ICC). Accuracy of visual and automated CS were calculated. Results 39 CTA scans were assessed (1 corrupt CTA-scan excluded). All groups showed a moderate ICC (0.689-0.780) in comparison to the reference standard. Overall human accuracy was 65 +/- 7% and increased to 88 +/- 5% for dichotomized CS (0-1, 2-3). Automated CS accuracy was 62%, and 90% for dichotomized CS. No significant difference in accuracy was found between groups with different levels of expertise. Conclusion After training, inter-rater reliability in collateral scoring was not influenced by experience. Automated CS performs similar to residents and radiologists in determining a collateral score. Show less
BACKGROUND AND PURPOSE: Blinded outcome assessment in trials with prospective randomized open blinded end point design is challenging. Unblinding can result in misclassified outcomes and biased... Show moreBACKGROUND AND PURPOSE: Blinded outcome assessment in trials with prospective randomized open blinded end point design is challenging. Unblinding can result in misclassified outcomes and biased treatment effect estimates. An outcome adjudication committee assures blinded outcome assessment, but the added value for trials with prospective randomized open blinded end point design and subjective outcomes is unknown. We aimed to assess the degree of misclassification of modified Rankin Scale (mRS) scores by a central assessor and its impact on treatment effect estimates in a stroke trial with prospective randomized open blinded end point design.METHODS: We used data from the MR CLEAN (Multicenter Randomized Clinical Trial of Endovascular Treatment for Acute Ischemic Stroke in the Netherlands). The primary outcome was the mRS at 90 days. Standardized, algorithm-based telephone interviews to assess the mRS were conducted from a central location by an experienced research nurse, unaware but not formally blinded to treatment allocation (central assessor). Masked reports of these interviews were adjudicated by a blinded outcome committee. Misclassification was defined as an incorrect classification of the mRS by the central assessor. The effect of endovascular treatment on the mRS was assessed with multivariable ordinal logistic regression.RESULTS: In MR CLEAN, 53/500 (10.6%) of the mRS scores were misclassified. The degree and direction of misclassification did not differ between treatment arms (P=0.59). Benefit of endovascular treatment was shown on the mRS when scored by the central assessor (adjusted common odds ratio, 1.60 [95% CI, 1.16-2.21]) and the outcome adjudication committee (adjusted common odds ratio, 1.67 [95% CI, 1.21-2.20]).CONCLUSIONS: Misclassification by the central assessor was small, randomly distributed over treatment arms, and did not affect treatment effect estimates. This study suggests that the added value of a blinded outcome adjudication committee is limited in a stroke trial with prospective randomized open blinded end point design applying standardized, algorithm-based outcome assessment by a central assessor, who is unaware but not formally blinded to treatment allocation. Show less
