Aims: The aim of this study is to develop and validate a deep learning (DL) methodology capable of automated and accurate segmentation of intravascular ultrasound (IVUS) image sequences in real... Show moreAims: The aim of this study is to develop and validate a deep learning (DL) methodology capable of automated and accurate segmentation of intravascular ultrasound (IVUS) image sequences in real-time. Methods and results: IVUS segmentation was performed by two experts who manually annotated the external elastic membrane (EEM) and lumen borders in the end-diastolic frames of 197 IVUS sequences portraying the native coronary arteries of 65 patients. The IVUS sequences of 177 randomly-selected vessels were used to train and optimise a novel DL model for the segmentation of IVUS images. Validation of the developed methodology was performed in 20 vessels using the estimations of two expert analysts as the reference standard. The mean difference for the EEM, lumen and plaque area between the DL-methodology and the analysts was <0.23mm2 (standard deviation <0.85mm2), while the Hausdorff and mean distance differences for the EEM and lumen borders was <0.19 mm (standard deviation<0.17 mm). The agreement between DL and experts was similar to experts' agreement (Williams Index ranges: 0.754-1.061) with similar results in frames portraying calcific plaques or side branches. Conclusions: The developed DL-methodology appears accurate and capable of segmenting high-resolution realworld IVUS datasets. These features are expected to facilitate its broad adoption and enhance the applications of IVUS in clinical practice and research. Show less
Su, R.S.; Cornelissen, S.A.P.; Sluijs, M. van der; Es, A.C.G.M. van; Zwam, W.H. van; Dippel, D.W.J.; ... ; Walsum, T. van 2021
The Thrombolysis in Cerebral Infarction (TICI) score is an important metric for reperfusion therapy assessment in acute ischemic stroke. It is commonly used as a technical outcome measure after... Show moreThe Thrombolysis in Cerebral Infarction (TICI) score is an important metric for reperfusion therapy assessment in acute ischemic stroke. It is commonly used as a technical outcome measure after endovascular treatment (EVT). Existing TICI scores are defined in coarse ordinal grades based on visual inspection, leading to inter- and intra-observer variation. In this work, we present autoTICI, an automatic and quantitative TICI scoring method. First, each digital subtraction angiography (DSA) acquisition is separated into four phases (non-contrast, arterial, parenchymal and venous phase) using a multi-path convolutional neural network (CNN), which exploits spatio-temporal features. The network also incorporates sequence level label dependencies in the form of a state-transition matrix. Next, a minimum intensity map (MINIP) is computed using the motion corrected arterial and parenchymal frames. On the MINIP image, vessel, perfusion and background pixels are segmented. Finally, we quantify the autoTICI score as the ratio of reperfused pixels after EVT. On a routinely acquired multi-center dataset, the proposed autoTICI shows good correlation with the extended TICI (eTICI) reference with an average area under the curve (AUC) score of 0.81. The AUC score is 0.90 with respect to the dichotomized eTICI. In terms of clinical outcome prediction, we demonstrate that autoTICI is overall comparable to eTICI. Show less
Spatially-resolved omics-data enable researchers to precisely distinguish cell types in tissue and explore their spatial interactions, enabling deep understanding of tissue functionality. To... Show moreSpatially-resolved omics-data enable researchers to precisely distinguish cell types in tissue and explore their spatial interactions, enabling deep understanding of tissue functionality. To understand what causes or deteriorates a disease and identify related biomarkers, clinical researchers regularly perform large-scale cohort studies, requiring the comparison of such data at cellular level. In such studies, with little a-priori knowledge of what to expect in the data, explorative data analysis is a necessity. Here, we present an interactive visual analysis workflow for the comparison of cohorts of spatially-resolved omics-data. Our workflow allows the comparative analysis of two cohorts based on multiple levels-of-detail, from simple abundance of contained cell types over complex co-localization patterns to individual comparison of complete tissue images. As a result, the workflow enables the identification of cohort-differentiating features, as well as outlier samples at any stage of the workflow. During the development of the workflow, we continuously consulted with domain experts. To show the effectiveness of the workflow, we conducted multiple case studies with domain experts from different application areas and with different data modalities. Show less
Segmentation of medical images, particularly late gadolinium-enhanced magnetic resonance imaging (LGE-MRI) used for visualizing diseased atrial structures, is a crucial first step for ablation... Show moreSegmentation of medical images, particularly late gadolinium-enhanced magnetic resonance imaging (LGE-MRI) used for visualizing diseased atrial structures, is a crucial first step for ablation treatment of atrial fibrillation. However, direct segmentation of LGE-MRIs is challenging due to the varying intensities caused by contrast agents. Since most clinical studies have relied on manual, labor-intensive approaches, automatic methods are of high interest, particularly optimized machine learning approaches. To address this, we organized the 2018 Left Atrium Segmentation Challenge using 154 3D LGE-MRIs, currently the world's largest atrial LGE-MRI dataset, and associated labels of the left atrium segmented by three medical experts, ultimately attracting the participation of 27 international teams. In this paper, extensive analysis of the submitted algorithms using technical and biological metrics was performed by undergoing subgroup analysis and conducting hyper-parameter analysis, offering an overall picture of the major design choices of convolutional neural networks (CNNs) and practical considerations for achieving state-of-the-art left atrium segmentation. Results show that the top method achieved a Dice score of 93.2% and a mean surface to surface distance of 0.7 mm, significantly outperforming prior state-of-the-art. Particularly, our analysis demonstrated that double sequentially used CNNs, in which a first CNN is used for automatic region-of-interest localization and a subsequent CNN is used for refined regional segmentation, achieved superior results than traditional methods and machine learning approaches containing single CNNs. This large-scale benchmarking study makes a significant step towards much-improved segmentation methods for atrial LGE-MRIs, and will serve as an important benchmark for evaluating and comparing the future works in the field. Furthermore, the findings from this study can potentially be extended to other imaging datasets and modalities, having an impact on the wider medical imaging community. (C) 2020 Elsevier B.V. All rights reserved. Show less
Manual or automatic delineation of the esophageal tumor in CT images is known to be very challenging. This is due to the low contrast between the tumor and adjacent tissues, the anatomical... Show moreManual or automatic delineation of the esophageal tumor in CT images is known to be very challenging. This is due to the low contrast between the tumor and adjacent tissues, the anatomical variation of the esophagus, as well as the occasional presence of foreign bodies (e.g. feeding tubes). Physicians therefore usually exploit additional knowledge such as endoscopic findings, clinical history, additional imaging modalities like PET scans. Achieving his additional information is time-consuming, while the results are error-prone and might lead to non-deterministic results. In this paper we aim to investigate if and to what extent a simplified clinical workflow based on CT alone, allows one to automatically segment the esophageal tumor with sufficient quality. For this purpose, we present a fully automatic end-to-end esophageal tumor segmentation method based on convolutional neural networks (CNNs). The proposed network, called Dilated Dense Attention Unet (DDAUnet), leverages spatial and channel attention gates in each dense block to selectively concentrate on determinant feature maps and regions. Dilated convolutional layers are used to manage GPU memory and increase the network receptive field. We collected a dataset of 792 scans from 288 distinct patients including varying anatomies with air pockets, feeding tubes and proximal tumors. Repeatability and reproducibility studies were conducted for three distinct splits of training and validation sets. The proposed network achieved a DSC value of 0.79 +/- 0.20, a mean surface distance of 5.4 +/- 20.2mm and 95% Hausdorff distance of 14.7 +/- 25.0mm for 287 test scans, demonstrating promising results with a simplified clinical workflow based on CT alone. Our code is publicly available via https://github.com/yousefis/DenseUnet_Esophagus_Segmentation. Show less
Elmahdy, M.S.; Beljaards, L.; Yousefi, S.; Sokooti, H.; Verbeek, F.; Heide, U.A. van der; Staring, M. 2021
Medical image registration and segmentation are two of the most frequent tasks in medical image analysis. As these tasks are complementary and correlated, it would be beneficial to apply them... Show moreMedical image registration and segmentation are two of the most frequent tasks in medical image analysis. As these tasks are complementary and correlated, it would be beneficial to apply them simultaneously in a joint manner. In this paper, we formulate registration and segmentation as a joint problem via a Multi-Task Learning (MTL) setting, allowing these tasks to leverage their strengths and mitigate their weaknesses through the sharing of beneficial information. We propose to merge these tasks not only on the loss level, but on the architectural level as well. We studied this approach in the context of adaptive image-guided radiotherapy for prostate cancer, where planning and follow-up CT images as well as their corresponding contours are available for training. At testing time the contours of the follow-up scans are not available, which is a common scenario in adaptive radiotherapy. The study involves two datasets from different manufacturers and institutes. The first dataset was divided into training (12 patients) and validation (6 patients), and was used to optimize and validate the methodology, while the second dataset (14 patients) was used as an independent test set. We carried out an extensive quantitative comparison between the quality of the automatically generated contours from different network architectures as well as loss weighting methods. Moreover, we evaluated the quality of the generated deformation vector field (DVF). We show that MTL algorithms outperform their Single-Task Learning (STL) counterparts and achieve better generalization on the independent test set. The best algorithm achieved a mean surface distance of 1.06 +/- 0.3 mm, 1.27 +/- 0.4 mm, 0.91 +/- 0.4 mm, and 1.76 +/- 0.8 mm on the validation set for the prostate, seminal vesicles, bladder, and rectum, respectively. The high accuracy of the proposed method combined with the fast inference speed, makes it a promising method for automatic re-contouring of follow-up scans for adaptive radiotherapy, potentially reducing treatment related complications and therefore improving patients quality-of-life after treatment. The source code is available at https://github.com/moelmahdy/JRS-MTL. Show less
Heutink, F.; Koch, V.; Verbist, B.; Woude, W.J. van der; Mylanus, E.; Huinck, W.; ... ; Caballo, M. 2020
Background and objective: Performing patient-specific, pre-operative cochlea CT-based measurements could be helpful to positively affect the outcome of cochlear surgery in terms of intracochlear... Show moreBackground and objective: Performing patient-specific, pre-operative cochlea CT-based measurements could be helpful to positively affect the outcome of cochlear surgery in terms of intracochlear trauma and loss of residual hearing. Therefore, we propose a method to automatically segment and measure the human cochlea in clinical ultra-high-resolution (UHR) CT images, and investigate differences in cochlea size for personalized implant planning.Methods: 123 temporal bone CT scans were acquired with two UHR-CT scanners, and used to develop and validate a deep learning-based system for automated cochlea segmentation and measurement. The segmentation algorithm is composed of two major steps (detection and pixel-wise classification) in cascade, and aims at combining the results of a multi-scale computer-aided detection scheme with a U-Net-like architecture for pixelwise classification. The segmentation results were used as an input to the measurement algorithm, which provides automatic cochlear measurements (volume, basal diameter, and cochlear duct length (CDL)) through the combined use of convolutional neural networks and thinning algorithms. Automatic segmentation was validated against manual annotation, by the means of Dice similarity, Boundary-F1 (BF) score, and maximum and average Hausdorff distances, while measurement errors were calculated between the automatic results and the corresponding manually obtained ground truth on a per-patient basis. Finally, the developed system was used to investigate the differences in cochlea size within our patient cohort, to relate the measurement errors to the actual variation in cochlear size across different patients.Results: Automatic segmentation resulted in a Dice of 0.90 +/- 0.03, BF score of 0.95 +/- 0.03, and maximum and average Hausdorff distance of 3.05 +/- 0.39 and 0.32 +/- 0.07 against manual annotation. Automatic cochlear measurements resulted in errors of 8.4% (volume), 5.5% (CDL), 7.8% (basal diameter). The cochlea size varied broadly, ranging between 0.10 and 0.28 ml (volume), 1.3 and 2.5 mm (basal diameter), and 27.7 and 40.1 mm (CDL).Conclusions: The proposed algorithm could successfully segment and analyze the cochlea on UHR-CT images, resulting in accurate measurements of cochlear anatomy. Given the wide variation in cochlear size found in our patient cohort, it may find application as a pre-operative tool in cochlear implant surgery, potentially helping elaborate personalized treatment strategies based on patient-specific, image-based anatomical measurements. (C) 2020 Elsevier B.V. All rights reserved. Show less
Optical projection tomography (OPT) is a tomographic 3D imaging technique used for specimens in the millimetre scale. 3D images are computed from a tomogram and therefore OPT is considered as... Show moreOptical projection tomography (OPT) is a tomographic 3D imaging technique used for specimens in the millimetre scale. 3D images are computed from a tomogram and therefore OPT is considered as computational imaging. In order to provide imaging and image analysis solutions for large scale biomedical research, optimisation of the OPT reconstruction is required. The aim of the optimisation presented in this thesis includes: (1) accelerate the reconstruction process; (2) reduce the reconstruction artefacts; (3) improve the image quality of 3D image; (4) Find optimal parameters for the iterative reconstruction.Starting from the optimisations that we have elaborated and implemented in the OPT imaging workflow, we have worked on case studies in zebrafish imaging. In this thesis we present one such particular case study (5) as it falls nicely in the order of magnitude for specimens in OPT imaging. The case study is on quantification of tumours in zebrafish and it is explored with image segmentation and object detection using artificial intelligence (AI) techniques. Show less
Kuijf, H.J.; Biesbroek, J.M.; Bresser, J. de; Heinen, R.; Andermatt, S.; Bento, M.; ... ; Biessels, G.J. 2019
We have studied shape with a particular focus on the zebrafish model system. The shape is an essential appearance of the phenotype of a biological specimen and it can be used to read out a... Show moreWe have studied shape with a particular focus on the zebrafish model system. The shape is an essential appearance of the phenotype of a biological specimen and it can be used to read out a current state or response or to study gene expression. So accurate shape analysis requires a precise shape description. Moreover, a sufficiently large sampling size of the specimens is necessary to ensure a justified and unbiased shape analysis. The latter is very important for high-throughput in compound screening. Therefore, top performance in zebrafish analysis requires high-throughput imaging (HTI). To deal with HTI, we aim to design an elaborate and well-performing HTI architecture. For the essential operations we need computational approaches to obtain the 2D/3D shape representations that are precise and yet can be acquired fast. The quality of the obtained shape descriptions will be validated in a straightforward manner with scalar primitives, i.e., the volume and surface area of a 3D shape. These primitives serve as 3D measurements for a robust primary shape assessment in the phenotype characterisation. Using only shape description is not sufficient, e.g., for high-resolution imaging on tissue and cellular level, so texture should be considered to complement and enhance the shape analysis. Show less
Modern radiotherapy requires accurate region of interest (ROI) inputs for plan optimization and delivery. Target delineation, however, remains operator-dependent and potentially serves as a major... Show moreModern radiotherapy requires accurate region of interest (ROI) inputs for plan optimization and delivery. Target delineation, however, remains operator-dependent and potentially serves as a major source of treatment delivery error. In order to optimize this critical, yet observer-driven process, a flexible web-based platform for individual and cooperative target delineation analysis and instruction was developed in order to meet the following unmet needs: (1) an open-source/open-access platform for automated/semiautomated quantitative interobserver and intraobserver ROI analysis and comparison, (2) a real-time interface for radiation oncology trainee online self-education in ROI definition, and (3) a source for pilot data to develop and validate quality metrics for institutional and cooperative group quality assurance efforts. The resultant software, Target Contour Testing/Instructional Computer Software (TaCTICS), developed using Ruby on Rails, has since been implemented and proven flexible, feasible, and useful in several distinct analytical and research applications. Show less
Suinesiaputra, A.; Koning, P.J.H. de; Zudilova-Seinstra, E.; Reiber, J.H.C.; Geest, R.J. van der 2012
The introductory chapter provides an overview of various aspects related to quantitative analysis of cardiovascular MR (CMR) imaging studies. Subsequently, the thesis describes several automated... Show moreThe introductory chapter provides an overview of various aspects related to quantitative analysis of cardiovascular MR (CMR) imaging studies. Subsequently, the thesis describes several automated methods for quantitative assessment of left ventricular function from CMR imaging studies. Several novel computer algorithms are introduced and validated for automated segmentation of short-axis CMR images and validated by comparing functional results derived from automated segmentation with results derived from manually traced contours. In addition an automated method is presented for assessment of flow through the aorta based on Phase-Contrast flow velocity mapping MRI. Finally a method is presented for accurate assessment of the thickness of the left ventricular myocardium taking advantage of the three-dimensional nature of MRI. Show less
In this thesis we aim at automating the analysis of 3D echocardiography, mainly targeting the functional analysis of the left ventricle. Manual analysis of these data is cumbersome, time-consuming... Show moreIn this thesis we aim at automating the analysis of 3D echocardiography, mainly targeting the functional analysis of the left ventricle. Manual analysis of these data is cumbersome, time-consuming and is associated with inter-observer and inter-institutional variability. Methods for reconstruction of 3D echocardiographic images from fast rotating ultrasound transducers is presented and methods for analysis of 3D echocardiography in general, using tracking, detection and model-based segmentation techniques to ultimately fully automatically segment the left ventricle for functional analysis. We show that reliable quantification of left ventricular volume and mitral valve displacement can be achieved using the presented techniques. Show less
