OBJECTIVES: Many machine learning (ML) models have been developed for application in the ICU, but few models have been subjected to external validation. The performance of these models in new... Show moreOBJECTIVES: Many machine learning (ML) models have been developed for application in the ICU, but few models have been subjected to external validation. The performance of these models in new settings therefore remains unknown. The objective of this study was to assess the performance of an existing decision support tool based on a ML model predicting readmission or death within 7 days after ICU discharge before, during, and after retraining and recalibration.DESIGN: A gradient boosted ML model was developed and validated on electronic health record data from 2004 to 2021. We performed an independent validation of this model on electronic health record data from 2011 to 2019 from a different tertiary care center.SETTING: Two ICUs in tertiary care centers in The Netherlands.PATIENTS: Adult patients who were admitted to the ICU and stayed for longer than 12 hours.INTERVENTIONS: None.MEASUREMENTS AND MAIN RESULTS: We assessed discrimination by area under the receiver operating characteristic curve (AUC) and calibration (slope and intercept). We retrained and recalibrated the original model and assessed performance via a temporal validation design. The final retrained model was cross-validated on all data from the new site. Readmission or death within 7 days after ICU discharge occurred in 577 of 10,052 ICU admissions (5.7%) at the new site. External validation revealed moderate discrimination with an AUC of 0.72 (95% CI 0.67–0.76). Retrained models showed improved discrimination with AUC 0.79 (95% CI 0.75–0.82) for the final validation model. Calibration was poor initially and good after recalibration via isotonic regression.CONCLUSIONS: In this era of expanding availability of ML models, external validation and retraining are key steps to consider before applying ML models to new settings. Clinicians and decision-makers should take this into account when considering applying new ML models to their local settings. Show less
Background: Aims of this study were to investigate the prevalence and incidence of catheter-related infection, identify risk factors, and determine the relation of catheter-related infection with... Show moreBackground: Aims of this study were to investigate the prevalence and incidence of catheter-related infection, identify risk factors, and determine the relation of catheter-related infection with mortality in critically ill COVID-19 patients. Methods: This was a retrospective cohort study of central venous catheters (CVCs) in critically ill COVID-19 patients. Eligible CVC insertions required an indwelling time of at least 48 hours and were identified using a full-admission electronic health record database. Risk factors were identified using logistic regression. Differences in survival rates at day 28 of follow-up were assessed using a log-rank test and proportional hazard model. Results: In 538 patients, a total of 914 CVCs were included. Prevalence and incidence of suspected catheter-related infection were 7.9% and 9.4 infections per 1,000 catheter indwelling days, respectively. Prone ventilation for more than 5 days was associated with increased risk of suspected catheter-related infection; odds ratio, 5.05 (95% confidence interval 2.12-11.0). Risk of death was significantly higher in patients with suspected catheter-related infection (hazard ratio, 1.78; 95% confidence interval, 1.25-2.53). Conclusions: This study shows that in critically ill patients with COVID-19, prevalence and incidence of suspected catheter-related infection are high, prone ventilation is a risk factor, and mortality is higher in case of catheter-related infection. Show less
Purpose: To assess, validate and compare the predictive performance of models for in-hospital mortality of COVID-19 patients admitted to the intensive care unit (ICU) over two different waves of... Show morePurpose: To assess, validate and compare the predictive performance of models for in-hospital mortality of COVID-19 patients admitted to the intensive care unit (ICU) over two different waves of infections. Our models were built with high-granular Electronic Health Records (EHR) data versus less-granular registry data. Methods: Observational study of all COVID-19 patients admitted to 19 Dutch ICUs participating in both the national quality registry National Intensive Care Evaluation (NICE) and the EHR-based Dutch Data Warehouse (hereafter EHR). Multiple models were developed on data from the first 24 h of ICU admissions from February to June 2020 (first COVID-19 wave) and validated on prospective patients admitted to the same ICUs between July and December 2020 (second COVID-19 wave). We assessed model discrimination, calibration, and the degree of relatedness between development and validation population. Coefficients were used to identify relevant risk factors. Results: A total of 1533 patients from the EHR and 1563 from the registry were included. With high granular EHR data, the average AUROC was 0.69 (standard deviation of 0.05) for the internal validation, and the AUROC was 0.75 for the temporal validation. The registry model achieved an average AUROC of 0.76 (standard deviation of 0.05) in the internal validation and 0.77 in the temporal validation. In the EHR data, age, and respiratory-system related variables were the most important risk factors identified. In the NICE registry data, age and chronic respiratory insufficiency were the most important risk factors. Conclusion: In our study, prognostic models built on less-granular but readily-available registry data had similar performance to models built on high-granular EHR data and showed similar transportability to a prospective COVID-19 population. Future research is needed to verify whether this finding can be confirmed for upcoming waves. Show less
Background and objective: Previous pharmacokinetic (PK) studies of ciprofloxacin in intensive care (ICU) patients have shown large differences in estimated PK parameters, suggesting that further... Show moreBackground and objective: Previous pharmacokinetic (PK) studies of ciprofloxacin in intensive care (ICU) patients have shown large differences in estimated PK parameters, suggesting that further investigation is needed for this population. Hence, we performed a pooled population PK analysis of ciprofloxacin after intravenous administration using individual patient data from three studies. Additionally, we studied the PK differences between these studies through a post-hoc analysis.Methods: Individual patient data from three studies (study 1, 2, and 3) were pooled. The pooled data set consisted of 1094 ciprofloxacin concentration-time data points from 140 ICU patients. Nonlinear mixed-effects modeling was used to develop a population PK model. Covariates were selected following a stepwise covariate modeling procedure. To analyze PK differences between the three original studies, random samples were drawn from the posterior distribution of individual PK parameters. These samples were used for a simulation study comparing PK exposure and the percentage of target attainment between patients of these studies.Results: A two-compartment model with first-order elimination best described the data. Inter-individual variability was added to the clearance, central volume, and peripheral volume. Inter-occasion variability was added to clearance only. Body weight was added to all parameters allometrically. Estimated glomerular filtration rate on ciprofloxacin clearance was identified as the only covariate relationship resulting in a drop in inter-individual variability of clearance from 58.7 to 47.2%. In the post-hoc analysis, clearance showed the highest deviation between the three studies with a coefficient of variation of 14.3% for posterior mean and 24.1% for posterior inter-individual variability. The simulation study showed that following the same dose regimen of 400 mg three times daily, the area under the concentration-time curve of study 3 was the highest with a mean area under the concentration-time curve at 24 h of 58 mg·h/L compared with that of 47.7 mg·h/L for study 1 and 47.6 mg·h/L for study 2. Similar differences were also observed in the percentage of target attainment, defined as the ratio of area under the concentration-time curve at 24 h and the minimum inhibitory concentration. At the epidemiological cut-off minimum inhibitory concentration of Pseudomonas aeruginosa of 0.5 mg/L, percentage of target attainment was only 21%, 18%, and 38% for study 1, 2, and 3, respectively.Conclusions: We developed a population PK model of ciprofloxacin in ICU patients using pooled data of individual patients from three studies. A simple ciprofloxacin dose recommendation for the entire ICU population remains challenging owing to the PK differences within ICU patients, hence dose individualization may be needed for the optimization of ciprofloxacin treatment. Show less
Purpose: Describe the differences in characteristics and outcomes between COVID-19 and other viral pneumonia patients admitted to Dutch ICUs. Materials and methods: Data from the National-Intensive... Show morePurpose: Describe the differences in characteristics and outcomes between COVID-19 and other viral pneumonia patients admitted to Dutch ICUs. Materials and methods: Data from the National-Intensive-Care-Evaluation-registry of COVID-19 patients admitted between February 15th and January 1th 2021 and other viral pneumonia patients admitted between January 1st 2017 and January 1st 2020 were used. Patients' characteristics, the unadjusted, and adjusted in-hospital mortality were compared. Results: 6343 COVID-19 and 2256 other viral pneumonia patients from 79 ICUs were included. The COVID-19 patients included more male (71.3 vs 49.8%), had a higher Body-Mass-Index (28.1 vs 25.5), less comorbidities (42.2 vs 72.7%), and a prolonged hospital length of stay (19 vs 9 days). The COVID-19 patients had a significantly higher crude in-hospital mortality rate (Odds ratio (OR) = 1.80), after adjustment for patient characteristics and ICU occupancy rate the OR was respectively 3.62 and 3.58. Conclusion: Higher mortality among COVID-19 patients could not be explained by patient characteristics and higher ICU occupancy rates, indicating that COVID-19 is more severe compared to other viral pneumonia. Our findings confirm earlier warnings of a high need of ICU capacity and high mortality rates among relatively healthy COVID-19 patients as this may lead to a higher mental workload for the staff. (c) 2021 The Authors. Published by Elsevier Inc. This is an open access article under the CC BY license (http:// creativecommons.org/licenses/by/4.0/). Show less
Fleuren, L.M.; Dam, T.A.; Tonutti, M.; Bruin, D.P. de; Lalisang, R.C.A.; Gommers, D.; ... ; Dutch ICU Data Sharing Covid-19 Co 2021
Introduction Determining the optimal timing for extubation can be challenging in the intensive care. In this study, we aim to identify predictors for extubation failure in critically ill patients... Show moreIntroduction Determining the optimal timing for extubation can be challenging in the intensive care. In this study, we aim to identify predictors for extubation failure in critically ill patients with COVID-19. Methods We used highly granular data from 3464 adult critically ill COVID patients in the multicenter Dutch Data Warehouse, including demographics, clinical observations, medications, fluid balance, laboratory values, vital signs, and data from life support devices. All intubated patients with at least one extubation attempt were eligible for analysis. Transferred patients, patients admitted for less than 24 h, and patients still admitted at the time of data extraction were excluded. Potential predictors were selected by a team of intensive care physicians. The primary and secondary outcomes were extubation without reintubation or death within the next 7 days and within 48 h, respectively. We trained and validated multiple machine learning algorithms using fivefold nested cross-validation. Predictor importance was estimated using Shapley additive explanations, while cutoff values for the relative probability of failed extubation were estimated through partial dependence plots. Results A total of 883 patients were included in the model derivation. The reintubation rate was 13.4% within 48 h and 18.9% at day 7, with a mortality rate of 0.6% and 1.0% respectively. The grandient-boost model performed best (area under the curve of 0.70) and was used to calculate predictor importance. Ventilatory characteristics and settings were the most important predictors. More specifically, a controlled mode duration longer than 4 days, a last fraction of inspired oxygen higher than 35%, a mean tidal volume per kg ideal body weight above 8 ml/kg in the day before extubation, and a shorter duration in assisted mode (< 2 days) compared to their median values. Additionally, a higher C-reactive protein and leukocyte count, a lower thrombocyte count, a lower Glasgow coma scale and a lower body mass index compared to their medians were associated with extubation failure. Conclusion The most important predictors for extubation failure in critically ill COVID-19 patients include ventilatory settings, inflammatory parameters, neurological status, and body mass index. These predictors should therefore be routinely captured in electronic health records. Show less
Background The Coronavirus disease 2019 (COVID-19) pandemic has underlined the urgent need for reliable, multicenter, and full-admission intensive care data to advance our understanding of the... Show moreBackground The Coronavirus disease 2019 (COVID-19) pandemic has underlined the urgent need for reliable, multicenter, and full-admission intensive care data to advance our understanding of the course of the disease and investigate potential treatment strategies. In this study, we present the Dutch Data Warehouse (DDW), the first multicenter electronic health record (EHR) database with full-admission data from critically ill COVID-19 patients. Methods A nation-wide data sharing collaboration was launched at the beginning of the pandemic in March 2020. All hospitals in the Netherlands were asked to participate and share pseudonymized EHR data from adult critically ill COVID-19 patients. Data included patient demographics, clinical observations, administered medication, laboratory determinations, and data from vital sign monitors and life support devices. Data sharing agreements were signed with participating hospitals before any data transfers took place. Data were extracted from the local EHRs with prespecified queries and combined into a staging dataset through an extract-transform-load (ETL) pipeline. In the consecutive processing pipeline, data were mapped to a common concept vocabulary and enriched with derived concepts. Data validation was a continuous process throughout the project. All participating hospitals have access to the DDW. Within legal and ethical boundaries, data are available to clinicians and researchers. Results Out of the 81 intensive care units in the Netherlands, 66 participated in the collaboration, 47 have signed the data sharing agreement, and 35 have shared their data. Data from 25 hospitals have passed through the ETL and processing pipeline. Currently, 3464 patients are included in the DDW, both from wave 1 and wave 2 in the Netherlands. More than 200 million clinical data points are available. Overall ICU mortality was 24.4%. Respiratory and hemodynamic parameters were most frequently measured throughout a patient's stay. For each patient, all administered medication and their daily fluid balance were available. Missing data are reported for each descriptive. Conclusions In this study, we show that EHR data from critically ill COVID-19 patients may be lawfully collected and can be combined into a data warehouse. These initiatives are indispensable to advance medical data science in the field of intensive care medicine. Show less
Aims To explore the optimal data sampling scheme and the pharmacokinetic (PK) target exposure on which dose computation is based in the model-based therapeutic drug monitoring (TDM) practice of... Show moreAims To explore the optimal data sampling scheme and the pharmacokinetic (PK) target exposure on which dose computation is based in the model-based therapeutic drug monitoring (TDM) practice of vancomycin in intensive care (ICU) patients. Methods We simulated concentration data for 1 day following four sampling schemes,C-min,C-max+C-min,C-max+Cmid-interval+C-min, and rich sampling where a sample was drawn every hour within a dose interval. The datasets were used for Bayesian estimation to obtain PK parameters, which were used to compute the doses for the next day based on five PK target exposures: AUC(24)= 400, 500, and 600 mg center dot h/L andC(min)= 15 and 20 mg/L. We then simulated data for the next day, adopting the computed doses, and repeated the above procedure for 7 days. Thereafter, we calculated the percentage error and the normalized root mean square error (NRMSE) of estimated against "true" PK parameters, and the percentage of optimal treatment (POT), defined as the percentage of patients who met 400 <= AUC(24)<= 600 mg center dot h/L andC(min)<= 20 mg/L. Results PK parameters were unbiasedly estimated in all investigated scenarios and the 6-day average NRMSE were 32.5%/38.5% (CL/V, whereCLis clearance andVis volume of distribution) in the trough sampling scheme and 27.3%/26.5% (CL/V) in the rich sampling scheme. Regarding POT, the sampling scheme had marginal influence, while target exposure showed clear impacts that the maximum POT of 71.5% was reached when doses were computed based on AUC(24)= 500 mg center dot h/L. Conclusions For model-based TDM of vancomycin in ICU patients, sampling more frequently than taking only trough samples adds no value and dosing based on AUC(24)= 500 mg center dot h/L lead to the best POT. Show less
