During the past 14 years, a clinical audit has been used in the Netherlands to provide hospitals with data on their performance in colorectal cancer care. Continuous feedback on the quality of care... Show moreDuring the past 14 years, a clinical audit has been used in the Netherlands to provide hospitals with data on their performance in colorectal cancer care. Continuous feedback on the quality of care provided at each hospital is essential to improve patient outcomes. It is unclear which methods should be used to generate most informative output for the identification of potential quality issues. Our aim is to compare the commonly employed funnel plot with existing cumulative sum (CUSUM) methodology for the evaluation of postoperative survival and hospital stay outcomes of patients who underwent colorectal surgery in the Netherlands. Data from the Dutch ColoRectal Audit on 25367 patients in the Netherlands who underwent surgical resection for colorectal cancer in 71 hospitals between 2019 and 2021 is used to compare four methods for the detection of deviations in the quality of care. Two methods based on binary outcomes (funnel plot, binary CUSUM) and two CUSUM charts based on survival outcomes (BK-CUSUM and CGR-CUSUM) are considered. A novel approach for determining hospital specific control limits for CUSUM charts is proposed. The ability to detect deviations as well as the time until detection are compared for the four methods. Charts were constructed for the inspection of both postoperative survival and hospital stay. Methods using survival outcomes always yielded faster detection times compared to approaches employing binary outcomes. Detections between methods mostly coincided for postoperative survival. For hospital stay detections varied strongly, with methods based on survival outcomes signalling over half the hospitals. Further pros and cons as well as pitfalls of all methods under consideration are discussed. Methodology for the continuous inspection of the quality of care should be tailored to the specific outcome. Properly understanding how the mechanism of a control chart functions is crucial for the correct interpretation of results. This is particularly true for CUSUM charts, which require the choice of a parameter that greatly influences the results. When applying CUSUM charts, consideration of these issues is strongly recommended. Show less
Gomon, D.; Sijmons, J.; Putter, H.; Dekker, J.W.; Tollenaar, R.; Wouters, M.; ... ; Signorelli, M. 2023
During the past 14 years, a clinical audit has been used in the Netherlands to provide hospitals with data on their performance in colorectal cancer care. Continuous feedback on the quality of... Show moreDuring the past 14 years, a clinical audit has been used in the Netherlands to provide hospitals with data on their performance in colorectal cancer care. Continuous feedback on the quality of care provided at each hospital is essential to improve patient outcomes. It is unclear which methods should be used to generate most informative output for the identification of potential quality issues. Our aim is to compare the commonly employed funnel plot with existing cumulative sum (CUSUM) methodology for the evaluation of postoperative survival and hospital stay outcomes of patients who underwent colorectal surgery in the Netherlands. Data from the Dutch ColoRectal Audit on 25367 patients in the Netherlands who underwent surgical resection for colorectal cancer in 71 hospitals between 2019 and 2021 is used to compare four methods for the detection of deviations in the quality of care. Two methods based on binary outcomes (funnel plot, binary CUSUM) and two CUSUM charts based on survival outcomes (BK-CUSUM and CGR-CUSUM) are considered. A novel approach for determining hospital specific control limits for CUSUM charts is proposed. The ability to detect deviations as well as the time until detection are compared for the four methods. Charts were constructed for the inspection of both postoperative survival and hospital stay. Methods using survival outcomes always yielded faster detection times compared to approaches employing binary outcomes. Detections between methods mostly coincided for postoperative survival. For hospital stay detections varied strongly, with methods based on survival outcomes signalling over half the hospitals. Further pros and cons as well as pitfalls of all methods under consideration are discussed. Methodology for the continuous inspection of the quality of care should be tailored to the specific outcome. Properly understanding how the mechanism of a control chart functions is crucial for the correct interpretation of results. This is particularly true for CUSUM charts, which require the choice of a parameter that greatly influences the results. When applying CUSUM charts, consideration of these issues is strongly recommended. Show less
In many haematological diseases, the survival probability is the key outcome. However, when the population of patients is rather old and the follow-up long, a significant proportion of deaths... Show moreIn many haematological diseases, the survival probability is the key outcome. However, when the population of patients is rather old and the follow-up long, a significant proportion of deaths cannot be attributed to the studied disease. This lessens the importance of common survival analysis measures like overall survival and shows the need for other outcome measures requiring more complex methodology. When disease-specific information is of interest but the cause of death is not available in the data, relative survival methodology becomes crucial. The idea of relative survival is to merge the observed data set with the mortality data in the general popu-lation and thus allow for an indirect estimation of the burden of the disease.In this work, an overview of different measures that can be of interest in the field of haema-tology is given. We introduce the crude mortality that reports the probability of dying due to the disease of interest; the net survival that focuses on excess hazard alone and presents the key measure in comparing the disease burden of patients from populations with different general population mortality; and the relative survival ratio which gives a simple comparison of the patients' and the general population survival. We explain the properties of each measure, and some brief notes are given on estimation. Furthermore, we describe how association with cova-riates can be studied. All the methods and their estimators are illustrated on a sub-cohort of older patients who received a first allogeneic hematopoietic stem cell transplantation for myelodys-plastic syndromes or secondary acute myeloid leukemia, to show how different methods can provide different insights into the data. Show less
Munsch, G.; Goumidi, L.; Vlieg, A.V.; Ibrahim-Kosta, M.; Bruzelius, M.; Deleuze, J.F.; ... ; Trégouët, D.A. 2023
BackgroundIn studies of time-to-events, it is common to collect information about events that occurred before the inclusion in a prospective cohort. When the studied risk factors are independent of... Show moreBackgroundIn studies of time-to-events, it is common to collect information about events that occurred before the inclusion in a prospective cohort. When the studied risk factors are independent of time, including both pre- and post-inclusion events in the analyses, generally referred to as relying on an ambispective design, increases the statistical power but may lead to a selection bias. In the field of venous thromboembolism (VT), ABO blood groups have been the subject of extensive research due to their substantial effect on VT risk. However, few studies have investigated their effect on the risk of VT recurrence. Motivated by the study of the association of genetically determined ABO blood groups with VT recurrence, we propose a methodology to include pre-inclusion events in the analysis of ambispective studies while avoiding the selection bias due to mortality.MethodsThis work relies on two independent cohorts of VT patients, the French MARTHA study built on an ambispective design and the Dutch MEGA study built on a standard prospective design. For the analysis of the MARTHA study, a weighted Cox model was developed where weights were defined by the inverse of the survival probability at the time of data collection about the events. Thanks to the collection of information on the vital status of patients, we could estimate the survival probabilities using a delayed-entry Cox model on the death risk. Finally, results obtained in both studies were then meta-analysed.ResultsIn the combined sample totalling 2,752 patients including 993 recurrences, the A1 blood group has an increased risk (Hazard Ratio (HR) of 1.18, p = 4.2 x 10(-3)) compared with the O1 group, homogeneously in MARTHA and in MEGA. The same trend (HR = 1.19, p = 0.06) was observed for the less frequent A2 group.ConclusionThe proposed methodology increases the power of studies relying on an ambispective design which is frequent in epidemiologic studies about recurrent events. This approach allowed to clarify the association of ABO blood groups with the risk of VT recurrence. Besides, this methodology has an immediate field of application in the context of genome wide association studies. Show less
Kantidakis, G.; Putter, H.; Litière, S.; Fiocco, M. 2023
BackgroundIn health research, several chronic diseases are susceptible to competing risks (CRs). Initially, statistical models (SM) were developed to estimate the cumulative incidence of an event... Show moreBackgroundIn health research, several chronic diseases are susceptible to competing risks (CRs). Initially, statistical models (SM) were developed to estimate the cumulative incidence of an event in the presence of CRs. As recently there is a growing interest in applying machine learning (ML) for clinical prediction, these techniques have also been extended to model CRs but literature is limited. Here, our aim is to investigate the potential role of ML versus SM for CRs within non-complex data (small/medium sample size, low dimensional setting).MethodsA dataset with 3826 retrospectively collected patients with extremity soft-tissue sarcoma (eSTS) and nine predictors is used to evaluate model-predictive performance in terms of discrimination and calibration. Two SM (cause-specific Cox, Fine-Gray) and three ML techniques are compared for CRs in a simple clinical setting. ML models include an original partial logistic artificial neural network for CRs (PLANNCR original), a PLANNCR with novel specifications in terms of architecture (PLANNCR extended), and a random survival forest for CRs (RSFCR). The clinical endpoint is the time in years between surgery and disease progression (event of interest) or death (competing event). Time points of interest are 2, 5, and 10 years.ResultsBased on the original eSTS data, 100 bootstrapped training datasets are drawn. Performance of the final models is assessed on validation data (left out samples) by employing as measures the Brier score and the Area Under the Curve (AUC) with CRs. Miscalibration (absolute accuracy error) is also estimated. Results show that the ML models are able to reach a comparable performance versus the SM at 2, 5, and 10 years regarding both Brier score and AUC (95% confidence intervals overlapped). However, the SM are frequently better calibrated.ConclusionsOverall, ML techniques are less practical as they require substantial implementation time (data preprocessing, hyperparameter tuning, computational intensity), whereas regression methods can perform well without the additional workload of model training. As such, for non-complex real life survival data, these techniques should only be applied complementary to SM as exploratory tools of model's performance. More attention to model calibration is urgently needed. Show less
This thesis sprang from an interdisciplinary collaboration between the European Organisation for Research and Treatment of Cancer (EORTC), the Mathematical Institute of Leiden University, and the... Show moreThis thesis sprang from an interdisciplinary collaboration between the European Organisation for Research and Treatment of Cancer (EORTC), the Mathematical Institute of Leiden University, and the Leiden University Medical Center (LUMC) Department of Medical Oncology. Research was split into two separate parts. In Part I, the main goal was to provide modern efficacy thresholds for designing new phase II clinical trials for common histotypes of locally advanced or metastatic soft-tissue sarcoma patients. An update was necessary as well-established values were reported back in 2002 by the EORTC – Soft Tissue and Bone Sarcoma Group.Nowadays, there is a growing interest by the medical community in applying machine learning to predict clinical outcomes. In Part II, the main goal was to investigate the potential of existing and novel machine learning techniques compared with traditional statistical benchmarks for real-life clinical data (small/medium or large sample sizes, low- or high-dimensional settings) with time-to-event endpoints. Findings indicate an urgent need to pay closer attention to calibration (absolute predictive accuracy) of machine learning techniques to achieve a complete comparison with statistical models. Show less
This dissertation focuses on developing new mathematical and statistical methods to properly represent time-varying covariates and model them within the context of time-to-event analysis. This... Show moreThis dissertation focuses on developing new mathematical and statistical methods to properly represent time-varying covariates and model them within the context of time-to-event analysis. This research topic is motivated by specific clinical questions aimed at gaining insights into personalised treatments for cardiological and oncological patients. The main purpose is to enrich the knowledge available for modelling patients’ survival with relevant features related to the time-varying processes of interest.The efforts of this work address the complexities of both (i) developing adequate dynamic characterizations of the processes under study (i.e., representation problem) and (ii) identifying and quantifying the association between time-varying processes and patient survival (i.e., time-to-event modelling problem). In both cases, the main issue is dealing with complex data sources while taking into account the nature of the processes and managing the complex trade-off between clinical interpretability and mathematical formulation.By solving the aforementioned statistical complexities, this work is not only impacting the community of researchers in mathematics and statistics. The development of these novel methodologies may represent a significant step forward in the definition of customized and flexible monitoring tools to support doctors and clinicians in their work.*********This doctoral dissertation was part of a cotutelle agreement between the Politecnico di Milano and Leiden University Show less
Gomon, D.; Putter, H.; Nelissen, R.G.H.H.; Pas, S. van der 2022
Rapidly detecting problems in the quality of care is of utmost importance for the well-being of patients. Without proper inspection schemes, such problems can go undetected for years. Cumulative... Show moreRapidly detecting problems in the quality of care is of utmost importance for the well-being of patients. Without proper inspection schemes, such problems can go undetected for years. Cumulative sum (CUSUM) charts have proven to be useful for quality control, yet available methodology for survival outcomes is limited. The few available continuous time inspection charts usually require the researcher to specify an expected increase in the failure rate in advance, thereby requiring prior knowledge about the problem at hand. Misspecifying parameters can lead to false positive alerts and large detection delays. To solve this problem, we take a more general approach to derive the new Continuous time Generalized Rapid response CUSUM (CGR-CUSUM) chart. We find an expression for the approximate average run length (average time to detection) and illustrate the possible gain in detection speed by using the CGR-CUSUM over other commonly used monitoring schemes on a real-life data set from the Dutch Arthroplasty Register as well as in simulation studies. Besides the inspection of medical procedures, the CGR-CUSUM can also be used for other real-time inspection schemes such as industrial production lines and quality control of services. Show less
Time-varying covariates are of great interest in clinical research since they represent dynamic patterns which reflect disease progression. In cancer studies biomarkers values change as functions... Show moreTime-varying covariates are of great interest in clinical research since they represent dynamic patterns which reflect disease progression. In cancer studies biomarkers values change as functions of time and chemotherapy treatment is modified by delaying a course or reducing the dose intensity, according to patient's toxicity levels. In this work, a Functional covariate Cox Model (FunCM) to study the association between time-varying processes and a time-to-event outcome is proposed. FunCM first exploits functional data analysis techniques to represent time-varying processes in terms of functional data. Then, information related to the evolution of the functions over time is incorporated into functional regression models for survival data through functional principal component analysis. FunCM is compared to a standard time-varying covariate Cox model, commonly used despite its limiting assumptions that covariate values are constant in time and measured without errors. Data from MRC BO06/EORTC 80931 randomised controlled trial for treatment of osteosarcoma are analysed. Time-varying covariates related to alkaline phosphatase levels, white blood cell counts and chemotherapy dose during treatment are investigated. The proposed method allows to detect differences between patients with different biomarkers and treatment evolutions, and to include this information in the survival model. These aspects are seldom addressed in the literature and could provide new insights into the clinical research. Show less
This thesis develops and investigates statistical methods for two frailty models in the analysis of time-to-event data.The first part of the thesis deals with the statistical assessment of the... Show moreThis thesis develops and investigates statistical methods for two frailty models in the analysis of time-to-event data.The first part of the thesis deals with the statistical assessment of the slowing down of human death rates at advanced ages. Such mortality deceleration can be described through the gamma-Gompertz model as an effect of selection in heterogeneous populations. As the frailty variance of this proportional hazards frailty model may lie on the boundary of the parameter space, statistical techniques have to be adjusted to this non-standard condition. Chapter 2 presents the asymptotic properties of likelihood inference in this model. In Chapter 3, aspects of study design are discussed, such as the information loss if samples are restricted to cover only survivors beyond some high age. Chapter 4 introduces focused model selection as a new approach to assessing mortality deceleration.The second part of the thesis is concerned with inference in a joint frailty model for recurrent events and a terminal event in two different observational settings. Chapter 5 considers the situation of intermittent observation of the recurrence process, such that only interval counts of recurrent events are available. Chapter 6 examines the situation of delayed entry, in which individuals can only be included in the recurrent event study if they have not yet experienced the terminal event. Show less
Kantidakis, G.; Putter, H.; Lancia, C.; Boer, J. de; Braat, A.E.; Fiocco, M. 2020
Background Predicting survival of recipients after liver transplantation is regarded as one of the most important challenges in contemporary medicine. Hence, improving on current prediction models... Show moreBackground Predicting survival of recipients after liver transplantation is regarded as one of the most important challenges in contemporary medicine. Hence, improving on current prediction models is of great interest.Nowadays, there is a strong discussion in the medical field about machine learning (ML) and whether it has greater potential than traditional regression models when dealing with complex data. Criticism to ML is related to unsuitable performance measures and lack of interpretability which is important for clinicians. Methods In this paper, ML techniques such as random forests and neural networks are applied to large data of 62294 patients from the United States with 97 predictors selected on clinical/statistical grounds, over more than 600, to predict survival from transplantation. Of particular interest is also the identification of potential risk factors. A comparison is performed between 3 different Cox models (with all variables, backward selection and LASSO) and 3 machine learning techniques: a random survival forest and 2 partial logistic artificial neural networks (PLANNs). For PLANNs, novel extensions to their original specification are tested. Emphasis is given on the advantages and pitfalls of each method and on the interpretability of the ML techniques. Results Well-established predictive measures are employed from the survival field (C-index, Brier score and Integrated Brier Score) and the strongest prognostic factors are identified for each model. Clinical endpoint is overall graft-survival defined as the time between transplantation and the date of graft-failure or death. The random survival forest shows slightly better predictive performance than Cox models based on the C-index. Neural networks show better performance than both Cox models and random survival forest based on the Integrated Brier Score at 10 years. Conclusion In this work, it is shown that machine learning techniques can be a useful tool for both prediction and interpretation in the survival context. From the ML techniques examined here, PLANN with 1 hidden layer predicts survival probabilities the most accurately, being as calibrated as the Cox model with all variables. Show less
Purpose The aim of this study was to evaluate whether the addition of brain CT imaging data to a model incorporating clinical risk factors improves prediction of ischemic stroke recurrence over 5... Show morePurpose The aim of this study was to evaluate whether the addition of brain CT imaging data to a model incorporating clinical risk factors improves prediction of ischemic stroke recurrence over 5 years of follow-up. Methods A total of 638 patients with ischemic stroke from three centers were selected from the Dutch acute stroke study (DUST). CT-derived candidate predictors included findings on non-contrast CT, CT perfusion, and CT angiography. Five-year follow-up data were extracted from medical records. We developed a multivariable Cox regression model containing clinical predictors and an extended model including CT-derived predictors by applying backward elimination. We calculated net reclassification improvement and integrated discrimination improvement indices. Discrimination was evaluated with the optimism-corrected c-statistic and calibration with a calibration plot. Results During 5 years of follow-up, 56 patients (9%) had a recurrence. The c-statistic of the clinical model, which contained male sex, history of hyperlipidemia, and history of stroke or transient ischemic attack, was 0.61. Compared with the clinical model, the extended model, which contained previous cerebral infarcts on non-contrast CT and Alberta Stroke Program Early CT score greater than 7 on mean transit time maps derived from CT perfusion, had higher discriminative performance (c-statistic 0.65,P= 0.01). Inclusion of these CT variables led to a significant improvement in reclassification measures, by using the net reclassification improvement and integrated discrimination improvement indices. Conclusion Data from CT imaging significantly improved the discriminatory performance and reclassification in predicting ischemic stroke recurrence beyond a model incorporating clinical risk factors only. Show less
This thesis is based on five papers on several topics: Survival analysis (Chapters 1-2); Optimal Scaling transformations in the Cox proportional hazards model (Chapter 3) and Generalized Linear... Show moreThis thesis is based on five papers on several topics: Survival analysis (Chapters 1-2); Optimal Scaling transformations in the Cox proportional hazards model (Chapter 3) and Generalized Linear Models (Chapter 4); and the interpretation of verbal probability phrases (Chapter 5).See the thesis’ English/Dutch summary for more detailed information. Show less
The main objective of this thesis was to develop clinically relevant survival models for patients with high-grade soft tissue sarcoma of the extremities, in particular the development and... Show moreThe main objective of this thesis was to develop clinically relevant survival models for patients with high-grade soft tissue sarcoma of the extremities, in particular the development and validation of prediction models for use in clinical practice. The interdisciplinary collaboration between the Mathematical Institute of Leiden University and the Leiden University Medical Center resulted in important contributions to the care of soft tissue sarcoma patients. Show less
Martin, E.; Coert, J.H.; Flucke, U.E.; Slooff, W.B.M.; Ho, V.K.Y.; Graaf, W.T. van der; ... ; Verhoef, C. 2020
Background: Despite curative intents of treatment in localized malignant peripheral nerve sheath tumours (MPNSTs), prognosis remains poor. This study investigated survival and prognostic factors... Show moreBackground: Despite curative intents of treatment in localized malignant peripheral nerve sheath tumours (MPNSTs), prognosis remains poor. This study investigated survival and prognostic factors for overall survival in non-retroperitoneal and retroperitoneal MPNSTs in the Netherlands.Methods: Data were obtained from the Netherlands Cancer Registry and the Dutch Pathology Database. All primary MPNSTs were collected. Paediatric cases (age <= 18 years) and synchronous metastases were excluded from analyses. Separate Cox proportional hazard models were made for retroperitoneal and non-retroperitoneal MPNSTs.Results: A total of 629 localized adult MPNSTs (35 retroperitoneal cases, 5.5%) were included for analysis. In surgically resected patients (88.1%), radiotherapy and chemotherapy were administered in 44.2% and 6.7%, respectively. In retroperitoneal cases, significantly less radiotherapy and more chemotherapy were applied. In non-retroperitoneal MPNSTs, older age (60+), presence of NF1, size >5 cm, and deep-seated tumours were independently associated with worse survival. In retroperitoneal MPNSTs, male sex and age of 60+ years were independently associated with worse survival. Survival of R1 and that of R0 resections were similar for any location, whereas R2 resections were associated with worse outcomes. Radiotherapy and chemotherapy administrations were not associated with survival.Conclusion: In localized MPNSTs, risk stratification for survival can be done using several patient-and tumour-specific characteristics. Resectability is the most important predictor for survival in MPNSTs. No difference is present between R1 and R0 resections in both retroperitoneal and non-retroperitoneal MPNSTs. The added value of radiotherapy and chemotherapy is unclear. (C) 2019 The Author(s). Published by Elsevier Ltd. Show less
Valk, M.J.M. van der; Vuijk, F.A.; Putter, H.; Velde, C.J.H. van de; Beets, G.L.; Hilling, D.E. 2019
In many healthcare settings it is of great interest to be able to predict the risk of events occurring in the future. Usually the interest is in predicting the probability that a patient will... Show moreIn many healthcare settings it is of great interest to be able to predict the risk of events occurring in the future. Usually the interest is in predicting the probability that a patient will survive. In this case the event is the death of the patient, but the event could also be the diagnosis of a disease or hospital discharge. Event history data are routinely collected either as a part of a study or in health registries and they can be used to create statistical models. The models can be used to make personalised predictions that accounts for a patient's specific characteristics. Dynamic prediction models are designed to make predictions not only from baseline, but also during the follow-up of the patient. Hence, predictions are updated as time progresses and incorporate the information that becomes available during follow-up. In recent years, a number of new statistical methods for creating models for event history data have emerged, such as inverse probability weights and pseudo-observations. The objective of this thesis has been to contribute to the statistical methodology by extending the available methods to make dynamic predictions. The thesis focuses on two approaches for making dynamic predictions known as landmarking and joint-modelling. Show less
This dissertation presents methodological advances in the field of frailty models. Time to event data is very common in biomedical applications. Multivariate data, such as recurrent event... Show moreThis dissertation presents methodological advances in the field of frailty models. Time to event data is very common in biomedical applications. Multivariate data, such as recurrent event histories or clustered failures, typically require special modeling techniques. Frailty models extend the proportional hazards models by employing random effects. In this book, the first chapter represents an introduction to the field of frailty models. The second chapter studies the small sample properties of such models in more depth. In particular, it addresses the questions of identifiability of frailty models. The third chapter describes a score test that may be used to test whether a recurrent events process is associated with a terminal event. The fourth chapter studies the problem of event-dependent selection in the context of recurrent events data. This problem is relevant when the data are collected from registries. The fifth chapter introduces a novel R package and presents an overview of available software for estimating frailty models. Show less
Rodriguez-Girondo, M.; Deelen, J.; Slagboom, E.P.; Houwing-Duistermaat, J.J. 2018
