Purpose Outcome of endovascular treatment in acute ischemic stroke patients is depending on the collateral circulation maintaining blood flow to the ischemic territory. We evaluated the inter-rater... Show morePurpose Outcome of endovascular treatment in acute ischemic stroke patients is depending on the collateral circulation maintaining blood flow to the ischemic territory. We evaluated the inter-rater reliability and accuracy of raters and an automated algorithm for assessing the collateral score (CS, range: 0-3) in acute ischemic stroke patients. Methods Baseline CTA scans with an intracranial anterior occlusion from the MR CLEAN study (n=500) were used. For each core lab CS, ten CTA scans with sufficient quality were randomly selected. After a training session in collateral scoring, all selected CTA scans were individually evaluated for a visual CS by three groups: 7 radiologists, 13 junior and 9 senior radiology residents. Two additional radiologists scored CS to be used as reference, with a third providing a CS to produce a 2 out of 3 consensus CS in case of disagreement. An automated algorithm was also used to compute CS. Inter-rater agreement was reported with intraclass correlation coefficient (ICC). Accuracy of visual and automated CS were calculated. Results 39 CTA scans were assessed (1 corrupt CTA-scan excluded). All groups showed a moderate ICC (0.689-0.780) in comparison to the reference standard. Overall human accuracy was 65 +/- 7% and increased to 88 +/- 5% for dichotomized CS (0-1, 2-3). Automated CS accuracy was 62%, and 90% for dichotomized CS. No significant difference in accuracy was found between groups with different levels of expertise. Conclusion After training, inter-rater reliability in collateral scoring was not influenced by experience. Automated CS performs similar to residents and radiologists in determining a collateral score. Show less
Weykamp, C.; Franck, P.; Gunnewiek, J.K.; Jonge, R. de; Kuypers, A.; Loon, D. van; ... ; SKML Fdn Quality Assessment Med La 2014
BACKGROUND Equivalence of results among laboratories is a major mission for medical laboratories. Monitoring of test equivalence is structurally integrated in the Dutch External Quality Assessment ... Show moreBACKGROUND Equivalence of results among laboratories is a major mission for medical laboratories. Monitoring of test equivalence is structurally integrated in the Dutch External Quality Assessment (EQA) scheme since 2005. Commutable poolsera, single donation "spy" sera and biological variance tolerance limits have been introduced in the EQA scheme for evaluation of the degree of test equivalence and its determinants. METHODS In the annual cycle scheme 24 samples, covering the (patho)physiological measuring range for 17 analytes, are assayed by 220 participating laboratories at biweekly intervals. Test equivalence was evaluated by calculating overall median interlaboratory coefficients of variation (CVs) and its bias and imprecision components. Data from 2005 and 2010 schemes are evaluated to investigate trends in performance and success of standardization efforts. RESULTS Overall median interlaboratory CVs in 2010 were mostly better than in 2005. Median interlaboratory CVs became <5% for electrolytes and substrates, and <10% for enzymes. Improvement in median interlaboratory CVs over these five years is mainly explained by improved method standardization, especially for enzymes and creatinine. CONCLUSION The Dutch EQA-program proves to be a powerful instrument to evaluate test equivalence. It allows monitoring standardization efforts in a highly effective way and gives insight into remaining standardization potential. Show less
Cobbaert, C.; Weykamp, C.; Franck, P.; Jonge, R. de; Kuypers, A.; Steigstra, H.; ... ; Jansen, R. 2012
BackgroundEquivalence of results among laboratories is a major mission for medical laboratories. Monitoring of test equivalence is structurally integrated in the Dutch External Quality Assessment ... Show moreBackgroundEquivalence of results among laboratories is a major mission for medical laboratories. Monitoring of test equivalence is structurally integrated in the Dutch External Quality Assessment (EQA) scheme since 2005. Commutable poolsera, single donation “spy” sera and biological variance tolerance limits have been introduced in the EQA scheme for evaluation of the degree of test equivalence and its determinants. MethodsIn the annual cycle scheme 24 samples, covering the (patho)physiological measuring range for 17 analytes, are assayed by 220 participating laboratories at biweekly intervals. Test equivalence was evaluated by calculating overall median interlaboratory coefficients of variation (CVs) and its bias and imprecision components. Data from 2005 and 2010 schemes are evaluated to investigate trends in performance and success of standardization efforts. ResultsOverall median interlaboratory CVs in 2010 were mostly better than in 2005. Median interlaboratory CVs became <5% for electrolytes and substrates, and <10% for enzymes. Improvement in median interlaboratory CVs over these five years is mainly explained by improved method standardization, especially for enzymes and creatinine. ConclusionThe Dutch EQA-program proves to be a powerful instrument to evaluate test equivalence. It allows monitoring standardization efforts in a highly effective way and gives insight into remaining standardization potential. Highlights► Equivalence of lab results worldwide is a major mission for medical laboratories. ► Commutable sera have been introduced to evaluate the degree of test equivalence. ► Median interlab CVs became <5% for electrolytes/substrates, and <10% for enzymes. ► Amelioration is mainly explained by improved method standardization. ► Commutable samples in EQAS are a powerful instrument to evaluate test equivalence. Show less
