Large and complex data sets are increasingly available for research in critical care. To analyze these data, researchers use techniques commonly referred to as statistical learning or machine... Show moreLarge and complex data sets are increasingly available for research in critical care. To analyze these data, researchers use techniques commonly referred to as statistical learning or machine learning (ML). The latter is known for large successes in the field of diagnostics, for example, by identification of radiological anomalies. In other research areas, such as clustering and prediction studies, there is more discussion regarding the benefit and efficiency of ML techniques compared with statistical learning. In this viewpoint, we aim to explain commonly used statistical learning and ML techniques and provide guidance for responsible use in the case of clustering and prediction questions in critical care. Clustering studies have been increasingly popular in critical care research, aiming to inform how patients can be characterized, classified, or treated differently. An important challenge for clustering studies is to ensure and assess generalizability. This limits the application of findings in these studies toward individual patients. In the case of predictive questions, there is much discussion as to what algorithm should be used to most accurately predict outcome. Aspects that determine usefulness of ML, compared with statistical techniques, include the volume of the data, the dimensionality of the preferred model, and the extent of missing data. There are areas in which modern ML methods may be preferred. However, efforts should be made to implement statistical frameworks (e.g., for dealing with missing data or measurement error, both omnipresent in clinical data) in ML methods. To conclude, there are important opportunities but also pitfalls to consider when performing clustering or predictive studies with ML techniques. We advocate careful valuation of new data-driven findings. More interaction is needed between the engineer mindset of experts in ML methods, the insight in bias of epidemiologists, and the probabilistic thinking of statisticians to extract as much information and knowledge from data as possible, while avoiding harm. Show less
Background: The aim of this study was to develop a prediction model for 10-year overall survival (OS) after resection of colorectal liver metastasis (CRLM) based on patient, tumour and treatment... Show moreBackground: The aim of this study was to develop a prediction model for 10-year overall survival (OS) after resection of colorectal liver metastasis (CRLM) based on patient, tumour and treatment characteristics.Methods: Consecutive patients after complete resection of CRLM were included from two centres (1992-2019). A prediction model providing 10-year OS probabilities was developed using Cox regression analysis, including KRAS, BRAF and histopathological growth patterns. Discrimination and calibration were assessed using cross-validation. A web-based calculator was built to predict individual 10-year OS probabilities.Results: A total of 4112 patients were included. The estimated 10-year OS was 30% (95% CI 29 -32). Fifteen patient, tumour and treatment characteristics were independent prognostic factors for 10-year OS; age, gender, location and nodal status of the primary tumour, disease-free interval, number and diameter of CRLM, preoperative CEA, resection margin, extrahepatic disease, KRAS and BRAF mutation status, histopathological growth patterns, perioperative systemic chemotherapy and hepatic arterial infusion pump chemotherapy. The discrimination at 10-years was 0.73 for both centres. A simplified risk score identified four risk groups with a 10-year OS of 57%, 38%, 24%, and 12%.Conclusions: Ten-year OS after resection of CRLM is best predicted with a model including 15 patient, tumour, and treatment characteristics. The web-based calculator can be used to inform patients. This model serves as a benchmark to determine the prognostic value of novel biomarkers. (C) 2022 The Author(s). Published by Elsevier Ltd. Show less
Objective: Prediction models for cardiovascular disease (CVD) mortality come from high-income countries, comprising laboratory measurements, not suitable for resource-limited countries. This study... Show moreObjective: Prediction models for cardiovascular disease (CVD) mortality come from high-income countries, comprising laboratory measurements, not suitable for resource-limited countries. This study aims to develop and validate a non-laboratory model to predict CVD mortality in a middle-income setting. Study design and setting: We used data of population aged 40-80 years from three cohort studies: Tehran Lipid and Glucose Study (n = 5160), Isfahan Cohort Study (n = 4350), and Golestan Cohort Study (n = 45,500). Using Cox proportional hazard models, we developed prediction models for men and women, separately. Cross-validation and bootstrapping procedures were applied. The models' discrimination and calibration were assessed by concordance statistic (C-index) and calibration plot, respectively. We calculated the models' sensitivity, specificity and net benefit fraction in a threshold probability of 5%. Results: The 10-year CVD mortality risks were 5.1% (95%CI: 4.8-5.5) in men and 3.1% (95%CI: 2.9%-3.3%) in women. The optimism-corrected performance of the model was c = 0.774 in men and c = 0.798 in women. The models showed good calibration in both sexes, with a predicted-to-observed ratio of 1.07 in men and 1.09 in women. The sensitivity was 0.76 in men and 0.66 in women. The net benefit fraction was higher in men compared to women (0.46 vs. 0.35). Conclusion: A low-cost model can discriminate well between low-and high-risk individuals, and can be used for screening in low-middle income countries. (C)& nbsp;2021 Elsevier Inc. All rights reserved. Show less
Ramspek, C.L.; Steyerberg, E.W.; Riley, R.D.; Rosendaal, F.R.; Dekkers, O.M.; Dekker, F.W.; Diepen, M. van 2021
Etiological research aims to uncover causal effects, whilst prediction research aims to forecast an outcome with the best accuracy. Causal and prediction research usually require different methods,... Show moreEtiological research aims to uncover causal effects, whilst prediction research aims to forecast an outcome with the best accuracy. Causal and prediction research usually require different methods, and yet their findings may get conflated when reported and interpreted. The aim of the current study is to quantify the frequency of conflation between etiological and prediction research, to discuss common underlying mistakes and provide recommendations on how to avoid these. Observational cohort studies published in January 2018 in the top-ranked journals of six distinct medical fields (Cardiology, Clinical Epidemiology, Clinical Neurology, General and Internal Medicine, Nephrology and Surgery) were included for the current scoping review. Data on conflation was extracted through signaling questions. In total, 180 studies were included. Overall, 26% (n = 46) contained conflation between etiology and prediction. The frequency of conflation varied across medical field and journal impact factor. From the causal studies 22% was conflated, mainly due to the selection of covariates based on their ability to predict without taking the causal structure into account. Within prediction studies 38% was conflated, the most frequent reason was a causal interpretation of covariates included in a prediction model. Conflation of etiology and prediction is a common methodological error in observational medical research and more frequent in prediction studies. As this may lead to biased estimations and erroneous conclusions, researchers must be careful when designing, interpreting and disseminating their research to ensure this conflation is avoided. Show less
Souwer, E.T.D.; Bastiaannet, E.; Steyerberg, E.W.; Dekker, J.W.T.; Bos, F. van den; Portielje, J.E.A. 2020
Background: An increasing number of patients with Colorectal Cancer (CRC) is 65 years or older. We aimed to systematically review existing clinical prediction models for postoperative outcomes of... Show moreBackground: An increasing number of patients with Colorectal Cancer (CRC) is 65 years or older. We aimed to systematically review existing clinical prediction models for postoperative outcomes of CRC surgery, study their performance in older patients and assess their potential for preoperative decision making.Methods: A systematic search in Pubmed and Embase for original studies of clinical prediction models for outcomes of CRC surgery. Bias and relevance for preoperative decision making with older patients were assessed using the CHARMS guidelines.Results: 26 prediction models from 25 publications were included. The average age of included patients ranged from 61 to 76. Two models were exclusively developed for 65 and older. Common outcomes were mortality (n = 10), anastomotic leakage (n = 7) and surgical site infections (n = 3). No prediction models for quality of life or physical functioning were identified. Age, gender and ASA score were common predictors; 12 studies included intraoperative predictors. For the majority of the models, bias for model development and performance was considered moderate to high.Conclusions: Prediction models are available that address mortality and surgical complications after CRC surgery. Most models suffer from methodological limitations, and their performance for older patients is uncertain. Models that contain intraoperative predictors are of limited use for preoperative decision making. Future research should address the predictive value of geriatric characteristics to improve the performance of prediction models for older patients. (C) 2020 The Authors. Published by Elsevier Ltd. Show less
Objective: We aimed to explore the added value of common machine learning (ML) algorithms for prediction of outcome for moderate and severe traumatic brain injury.Study Design and Setting: We... Show moreObjective: We aimed to explore the added value of common machine learning (ML) algorithms for prediction of outcome for moderate and severe traumatic brain injury.Study Design and Setting: We performed logistic regression (LR), lasso regression, and ridge regression with key baseline predictors in the IMPACT-II database (15 studies, n = 11,022). ML algorithms included support vector machines, random forests, gradient boosting machines, and artificial neural networks and were trained using the same predictors. To assess generalizability of predictions, we performed internal, internal-external, and external validation on the recent CENTER-TBI study (patients with Glasgow Coma Scale <13, n = 1,554). Both calibration (calibration slope/intercept) and discrimination (area under the curve) was quantified.Results: In the IMPACT-II database, 3,332/11,022 (30%) died and 5,233(48%) had unfavorable outcome (Glasgow Outcome Scale less than 4). In the CENTER-TBI study, 348/1,554(29%) died and 651(54%) had unfavorable outcome. Discrimination and calibration varied widely between the studies and less so between the studied algorithms. The mean area under the curve was 0.82 for mortality and 0.77 for unfavorable outcomes in the CENTER-TBI study.Conclusion: ML algorithms may not outperform traditional regression approaches in a low-dimensional setting for outcome prediction after moderate or severe traumatic brain injury. Similar to regression-based prediction models, ML algorithms should be rigorously validated to ensure applicability to new populations. (C) 2020 The Authors. Published by Elsevier Inc. Show less
