Quality improvement (QI) projects often employ statistical process control (SPC) charts to monitor process or outcome measures as part of ongoing feedback, to inform successive Plan-Do-Study-Act... Show moreQuality improvement (QI) projects often employ statistical process control (SPC) charts to monitor process or outcome measures as part of ongoing feedback, to inform successive Plan-Do-Study-Act cycles and refine the intervention (formative evaluation). SPC charts can also be used to draw inferences on effectiveness and generalisability of improvement efforts (summative evaluation), but only if appropriately designed and meeting specific methodological requirements for generalisability. Inadequate design decreases the validity of results, which not only reduces the chance of publication but could also result in patient harm and wasted resources if incorrect conclusions are drawn. This paper aims to bring together much of what has been written in various tutorials, to suggest a process for using SPC in QI projects. We highlight four critical decision points that are often missed, how these are inter-related and how they affect the inferences that can be drawn regarding effectiveness of the intervention: (1) the need for a stable baseline to enable drawing inferences on effectiveness; (2) choice of outcome measures to assess effectiveness, safety and intervention fidelity; (3) design features to improve the quality of QI projects; (4) choice of SPC analysis aligned with the type of outcome, and reporting on the potential influence of other interventions or secular trends. These decision points should be explicitly reported for readers to interpret and judge the results, and can be seen as supplementing the Standards for Quality Improvement Reporting Excellence guidelines. Thinking in advance about both formative and summative evaluation will inform more deliberate choices and strengthen the evidence produced by QI projects. Show less
Schie, P. van; Bodegom-Vos, L. van; Zijdeman, T.M.; Nelissen, R.G.H.H.; Mheen, P.J.M. van de 2022
Objective: To assess the effectiveness of a prospective multifaceted quality improvement intervention on patient outcomes after total hip and knee arthroplasty (THA and TKA).Design: Cluster... Show moreObjective: To assess the effectiveness of a prospective multifaceted quality improvement intervention on patient outcomes after total hip and knee arthroplasty (THA and TKA).Design: Cluster randomised controlled trial nested in a national registry. From 1 January 2018 to 31 May 2020 routinely submitted registry data on revision and patient characteristics were used, supplemented with hospital data on readmission, complications and length of stay (LOS) for all patients.Setting: 20 orthopaedic departments across hospitals performing THA and TKA in The Netherlands.Participants: 32 923 patients underwent THA and TKA, in 10 intervention and 10 control hospitals (usual care). Intervention: The intervention period lasted 8 months and consisted of the following components: (1) monthly updated feedback on 1-year revision, 30-day readmission, 30-day complications, long (upper quartile) LOS and these four indicators combined in a composite outcome; (2) interactive education; (3) an action toolbox including evidence-based quality improvement initiatives (QIIs) to facilitate improvement of above indicators; and (4) bimonthly surveys to report on QII undertaken. Main outcome measures: The primary outcome was textbook outcome (TO), an all-or-none composite representing the best outcome on all performance indicators (ie, the absence of revision, readmissions, complications and long LOS).The individual indicators were analysed as secondary outcomes. Changes in outcomes from preintervention to intervention period were compared between intervention versus control hospitals, adjusted for case-mix and clustering of patients within hospitals using random effect binary logistic regression models. The same analyses were conducted for intervention hospitals that did and did not introduce QII. Results: 16,314 patients were analysed in intervention hospitals (12,475 before and 3,839 during intervention) versus 16,609 in control hospitals (12,853 versus 3,756). After the intervention period, the absolute probability to achieve TO increased by 432% (95% confidence interval (CI) 4.30-4.34) more in intervention than control hospitals, corresponding to 21.6 (95%CI 21.5-21.8), i.e., 22 patients treated in intervention hospitals to achieve one additional patient with TO. Intervention hospitals had a larger increase in patients achieving TO (ratio of adjusted odds ratios 1.24, 95%CI 1.05-1.48) than control hospitals, a larger reduction in patients with long LOS (0.74, 95%CI 0.61-0.90) but also a larger increase in patients with reported 30-day complications (1.34, 95% CI 1.00-1.78). Intervention hospitals that introduced QII increased more in TO (1.32, 95% CI 1.10-1.57) than control hospitals, with no effect shown for hospitals not introducing QII (0.93, 95% CI 0.67-1.30). Conclusion: The multifaceted QI intervention including monthly feedback, education, and a toolbox to facilitate QII effectively improved patients achieving TO. The effect size was associated with the introduction of (evidence-based) QII, considered as the causal link to achieve better patient outcomes. Show less
Schie, P. van; Bodegom-Vos, L. van; Zijdeman, T.M.; Nelissen, R.G.H.H.; Mheen, P.J.M. van de 2022
Objective To assess the effectiveness of a prospective multifaceted quality improvement intervention on patient outcomes after total hip and knee arthroplasty (THA and TKA).Design Cluster... Show moreObjective To assess the effectiveness of a prospective multifaceted quality improvement intervention on patient outcomes after total hip and knee arthroplasty (THA and TKA).Design Cluster randomised controlled trial nested in a national registry. From 1 January 2018 to 31 May 2020 routinely submitted registry data on revision and patient characteristics were used, supplemented with hospital data on readmission, complications and length of stay (LOS) for all patients.Setting 20 orthopaedic departments across hospitals performing THA and TKA in The Netherlands.Participants 32 923 patients underwent THA and TKA, in 10 intervention and 10 control hospitals (usual care).Intervention The intervention period lasted 8 months and consisted of the following components: (1) monthly updated feedback on 1-year revision, 30-day readmission, 30-day complications, long (upper quartile) LOS and these four indicators combined in a composite outcome; (2) interactive education; (3) an action toolbox including evidence-based quality improvement initiatives (QIIs) to facilitate improvement of above indicators; and (4) bimonthly surveys to report on QII undertaken.Main outcome measures The primary outcome was textbook outcome (TO), an all-or-none composite representing the best outcome on all performance indicators (ie, the absence of revision, readmissions, complications and long LOS).The individual indicators were analysed as secondary outcomes. Changes in outcomes from preintervention to intervention period were compared between intervention versus control hospitals, adjusted for case-mix and clustering of patients within hospitals using random effect binary logistic regression models. The same analyses were conducted for intervention hospitals that did and did not introduce QII.Results 16,314 patients were analysed in intervention hospitals (12,475 before and 3,839 during intervention) versus 16,609 in control hospitals (12,853 versus 3,756). After the intervention period, the absolute probability to achieve TO increased by 432% (95% confidence interval (CI) 4.30-4.34) more in intervention than control hospitals, corresponding to 21.6 (95%CI 21.5-21.8), i.e., 22 patients treated in intervention hospitals to achieve one additional patient with TO. Intervention hospitals had a larger increase in patients achieving TO (ratio of adjusted odds ratios 1.24, 95%CI 1.05-1.48) than control hospitals, a larger reduction in patients with long LOS (0.74, 95%CI 0.61-0.90) but also a larger increase in patients with reported 30-day complications (1.34, 95% CI 1.00-1.78). Intervention hospitals that introduced QII increased more in TO (1.32, 95% CI 1.10-1.57) than control hospitals, with no effect shown for hospitals not introducing QII (0.93, 95% CI 0.67-1.30).Conclusion The multifaceted QI intervention including monthly feedback, education, and a toolbox to facilitate QII effectively improved patients achieving TO. The effect size was associated with the introduction of (evidence-based) QII, considered as the causal link to achieve better patient outcomes. Show less
Verhagen, M.J.; Vos, M.S. de; Sujan, M.; Hamming, J.F. 2022
When comparing hospitals on their readmission rates as currently done in the Hospital Readmission and Reduction Program (HRRP) in the USA, should we include the competing risk of mortality after... Show moreWhen comparing hospitals on their readmission rates as currently done in the Hospital Readmission and Reduction Program (HRRP) in the USA, should we include the competing risk of mortality after discharge, which precludes the readmission, in the analysis? Not including competing risks in current HRRP metrics was raised recently as a limitation with possible unintended consequences, as financial penalties for higher readmission rates are more severe than for higher mortality rates. Incorrectly including or ignoring competing risks can both induce bias. In this paper, we present a framework to clarify situations when competing risks should be taken into account and when they should not. We argue that the research question and the perspective from which it is asked determine whether the competing risk is also of interest and should be included in the analysis, or if only the event of interest should be considered. This information is often not explicitly reported but is needed to interpret whether the results are valid. Using the examples of readmissions and cancer, we show how different research questions fit different perspectives from which these are asked (patient, system, regulatory/insurance). Slightly changing the research question or perspective may thus change the analysis. Even though some may argue that any introduced bias is likely to be small, in the context of the HRRP, even small changes may mean that a hospital will face (higher) financial penalties. The impact of getting it wrong matters. Show less
Background Hospitals and providers receive feedback information on how their performance compares with others, often using funnel plots to detect outliers. These funnel plots typically use binary... Show moreBackground Hospitals and providers receive feedback information on how their performance compares with others, often using funnel plots to detect outliers. These funnel plots typically use binary outcomes, and continuous variables are dichotomised to fit this format. However, information is lost using a binary measure, which is only sensitive to detect differences in higher values (the tail) rather than the entire distribution. This study therefore aims to investigate whether different outlier hospitals are identified when using a funnel plot for a binary vs a continuous outcome. This is relevant for hospitals with suboptimal performance to decide whether performance can be improved by targeting processes for all patients or a subgroup with higher values.Methods We examined the door-to-needle time (DNT) of all (6080) patients with acute ischaemic stroke treated with intravenous thrombolysis in 65 hospitals in 2017, registered in the Dutch Acute Stroke Audit. We compared outlier hospitals in two funnel plots: the median DNT versus the proportion of patients with substantially delayed DNT (above the 90th percentile (P90)), whether these were the same or different hospitals. Two sensitivity analyses were performed using the proportion above the median and a continuous P90 funnel plot.Results The median DNT was 24 min and P90 was 50 min. In the binary funnel plot for the proportion of patients above P90, 58 hospitals had average performance, whereas in the funnel plot around the median 14 of these hospitals had significantly higher median DNT (24%). These hospitals can likely improve their DNT by focusing on care processes for all patients, not shown by the binary outcome funnel plot. Similar results were shown in sensitivity analyses.Conclusion Using funnel plots for continuous versus binary outcomes identify different outlier hospitals, which may enhance hospital feedback to direct more targeted improvement initiatives. Show less
Background and objective Incident, adverse event (AE) and complaint data are typically used separately, but may be related at the patient level with one event triggering a cascade of events,... Show moreBackground and objective Incident, adverse event (AE) and complaint data are typically used separately, but may be related at the patient level with one event triggering a cascade of events, ultimately resulting in a complaint. This study examined relations between incidents, AEs and complaints that co-occurred in admissions.Methods Independently and routinely collected incident, AE and complaint data were retrospectively linked for surgical admissions in an academic centre (2008-2014). Two investigators reviewed whether incidents/AEs in admissions were clinically related and in what sequence (incident preceding vs following AE). Likelihood of occurrence of AEs and AE cascades (ie, >= 3 AEs) was studied using logistic regression analyses.Results Complaints were filed for 33 (0.1%) of 26 383 admissions. Complaints filed by patients with incidents and/or AEs (n=13) mostly addressed quality/safety problems, whereas other complaints mostly addressed relationship problems. Incidents and AEs co-occurred in 730 (2.8%) admissions, which seemed clinically related in 34% of these cases. Incidents with related AEs preceded as well as followed AEs (56.6%/44.4%). Patients with incidents were at greater risk of AEs than patients without incidents, even for seemingly unrelated AEs (OR 1.4; 95% CI 1.3 to 1.6). Risk of AE cascades was greater when patients with AEs also had incidents, regardless of whether these seemed related (unrelated: OR 2.0; 95% CI 1.6 to 2.5; related: OR 5.7; 95% CI 4.3 to 7.4) or whether incidents preceded or followed AEs in these admissions (53% vs 52%, P> 0.05).Conclusions Patient-level linkage of incident, AE and complaint data can reveal relations between events that otherwise remain obscured, such as incidents that trigger as well as follow AEs, introducing event cascades, regardless of whether clinical relations seem present. Show less
Background Despite widespread use of quality indicators, it remains unclear to what extent they can reliably distinguish hospitals on true differences in performance. Rankability measures what part... Show moreBackground Despite widespread use of quality indicators, it remains unclear to what extent they can reliably distinguish hospitals on true differences in performance. Rankability measures what part of variation in performance reflects 'true' hospital differences in outcomes versus random noise. Objective This study sought to assess whether combining data into composites or including data from multiple years improves the reliability of ranking quality indicators for hospital care. Methods Using the Dutch National Medical Registration (2007-2012) for stroke, colorectal carcinoma, heart failure, acute myocardial infarction and total hiparthroplasty (THA)/ total knee arthroplasty (TKA) in osteoarthritis (OA), we calculated the rankability for in-hospital mortality, 30-day acute readmission and prolonged length of stay (LOS) for single years and 3-year periods and for a dichotomous and ordinal composite measure in which mortality, readmission and prolonged LOS were combined. Rankability, defined as (between-hospital variation/between-hospital+within hospital variation)x100% is classified as low (<50%), moderate (50%-75%) and high (>75%). Results Admissions from 555 053 patients treated in 95 hospitals were included. The rankability for mortality was generally low or moderate, varying from less than 1% for patients with OA undergoing THA/TKA in 2011 to 71% for stroke in 2010. Rankability for acute readmission was low, except for acute myocardial infarction in 2009 (51%) and 2012 (62%). Rankability for prolonged LOS was at least moderate. Combining multiple years improved rankability but still remained low in eight cases for both mortality and acute readmission. Combining the individual indicators into the dichotomous composite, all diagnoses had at least moderate rankability (range: 51%-96%). For the ordinal composite, only heart failure had low rankability (46% in 2008) (range: 46%-95%). Conclusion Combining multiple years or into multiple indicators results in more reliable ranking of hospitals, particularly compared with mortality and acute readmission in single years, thereby improving the ability to detect true hospital differences. The composite measures provide more information and more reliable rankings than combining multiple years of individual indicators. Show less
Marang-van de Mheen, P.J.; Abel, G.A.; Shojania, K.G. 2018
