Sewer pipes are an essential infrastructure in modern society and their proper operation is important for public health. To keep sewer pipes operational as much as possible, periodical inspections... Show moreSewer pipes are an essential infrastructure in modern society and their proper operation is important for public health. To keep sewer pipes operational as much as possible, periodical inspections for defects are performed. Instead of repairing sewer pipes when a problem becomes critical, such inspections allow municipalities to plan maintenance.Sewer pipe inspections are an attractive target for automation. While a potential improvement in terms of assessment quality and processing efficiency is generally promised by automation, in this case we would also decrease the variability which is a current problem. Besides the reasons for automating, the methods for automating are also attractive: a lot of (visual) data has been gathered over the past decades which may be used to train algorithms.This thesis compiles the results of five years of research into the possible automation of sewer pipe inspections with the tools of machine learning and computer vision. In this thesis, three distinct, yet complementary approaches to automating sewer pipe inspections are described:- Image-Based Unsupervised Anomaly Detection- Convolutional Neural Network Classification- Stereovision and Geometry Reconstruction Show less
A convolutional neural network (CNN) is a biologically inspired algorithm, highly capable at processing images and videos. Nowadays, CNNs are widely known and used: they watch our safety from the... Show moreA convolutional neural network (CNN) is a biologically inspired algorithm, highly capable at processing images and videos. Nowadays, CNNs are widely known and used: they watch our safety from the CCTV cameras, help doctors diagnose diseases, navigate cars, and do many other important things. One of the recent trends is to execute CNNs on edge devices: cameras, mobile phones, smart watches, etc. This helps to run CNNs faster and ensures privacy of the data used by the CNNs. This, however, is difficult to do. The problem is that the edge devices are small and often do not have enough resources to execute CNNs. In my dissertation, I study this problem and offer solutions for it. I propose specific manners to design and execute CNNs, so that they can run on edge devices efficiently. Show less
Background Right atrial (RA) area predicts mortality in patients with pulmonary hypertension, and is recommended by the European Society of Cardiology/European Respiratory Society pulmonary... Show moreBackground Right atrial (RA) area predicts mortality in patients with pulmonary hypertension, and is recommended by the European Society of Cardiology/European Respiratory Society pulmonary hypertension guidelines. The advent of deep learning may allow more reliable measurement of RA areas to improve clinical assessments. The aim of this study was to automate cardiovascular magnetic resonance (CMR) RA area measurements and evaluate the clinical utility by assessing repeatability, correlation with invasive haemodynamics and prognostic value. Methods A deep learning RA area CMR contouring model was trained in a multicentre cohort of 365 patients with pulmonary hypertension, left ventricular pathology and healthy subjects. Inter-study repeatability (intraclass correlation coefficient (ICC)) and agreement of contours (DICE similarity coefficient (DSC)) were assessed in a prospective cohort (n = 36). Clinical testing and mortality prediction was performed in n = 400 patients that were not used in the training nor prospective cohort, and the correlation of automatic and manual RA measurements with invasive haemodynamics assessed in n = 212/400. Radiologist quality control (QC) was performed in the ASPIRE registry, n = 3795 patients. The primary QC observer evaluated all the segmentations and recorded them as satisfactory, suboptimal or failure. A second QC observer analysed a random subcohort to assess QC agreement (n = 1018). Results All deep learning RA measurements showed higher interstudy repeatability (ICC 0.91 to 0.95) compared to manual RA measurements (1st observer ICC 0.82 to 0.88, 2nd observer ICC 0.88 to 0.91). DSC showed high agreement comparing automatic artificial intelligence and manual CMR readers. Maximal RA area mean and standard deviation (SD) DSC metric for observer 1 vs observer 2, automatic measurements vs observer 1 and automatic measurements vs observer 2 is 92.4 +/- 3.5 cm(2), 91.2 +/- 4.5 cm(2) and 93.2 +/- 3.2 cm(2), respectively. Minimal RA area mean and SD DSC metric for observer 1 vs observer 2, automatic measurements vs observer 1 and automatic measurements vs observer 2 was 89.8 +/- 3.9 cm(2), 87.0 +/- 5.8 cm(2) and 91.8 +/- 4.8 cm(2). Automatic RA area measurements all showed moderate correlation with invasive parameters (r = 0.45 to 0.66), manual (r = 0.36 to 0.57). Maximal RA area could accurately predict elevated mean RA pressure low and high-risk thresholds (area under the receiver operating characteristic curve artificial intelligence = 0.82/0.87 vs manual = 0.78/0.83), and predicted mortality similar to manual measurements, both p < 0.01. In the QC evaluation, artificial intelligence segmentations were suboptimal at 108/3795 and a low failure rate of 16/3795. In a subcohort (n = 1018), agreement by two QC observers was excellent, kappa 0.84. Conclusion Automatic artificial intelligence CMR derived RA size and function are accurate, have excellent repeatability, moderate associations with invasive haemodynamics and predict mortality. Show less
Tomography is a powerful technique to non-destructively determine the interior structure of an object.Usually, a series of projection images (e.g.\ X-ray images) is acquired from a range of... Show moreTomography is a powerful technique to non-destructively determine the interior structure of an object.Usually, a series of projection images (e.g.\ X-ray images) is acquired from a range of different positions.from these projection images, a reconstruction of the object's interior is computed. Many advanced applications require fast acquisition, effectively limiting the number of projection images and imposing a level of noise on these images. These limitations result in artifacts (deficiencies) in the reconstructed images. Recently, deep neural networks have emerged as a powerful technique to remove these limited-data artifacts from reconstructed images, often outperformingconventional state-of-the-art techniques. To perform this task, the networks are typically trained on a dataset of paired low-quality and high-quality images of similar objects. This is a major obstacle to their use in many practical applications. In this thesis, we explore techniques to employ deep learning in advanced experiments where measuring additional objects is not possible. Show less
This thesis mainly focuses on cross-modal retrieval and single-modal image retrieval via deep learning methods, i.e. by using deep convolutional neural networks.For cross-modal retrieval, Shannon... Show moreThis thesis mainly focuses on cross-modal retrieval and single-modal image retrieval via deep learning methods, i.e. by using deep convolutional neural networks.For cross-modal retrieval, Shannon information entropy and adversarial learning are integrated to learn a common latent space for image data and text data. Furthermore, this thesis explores single-modal image retrieval in an incremental learning context to reduce the catastrophic forgetting of deep models, thereby expanding the continuous retrieval ability. The efficacy of the proposed methods in this thesis is verified by thorough experiments on the considered datasets. This thesis also gives an overview of new ideas and trends for multimodal content understanding. Show less
In this thesis we explore machine and deep learning approaches that address keychallenges in high dimensional problem areas and also in improving accuracy in wellknown problems. In high dimensional... Show moreIn this thesis we explore machine and deep learning approaches that address keychallenges in high dimensional problem areas and also in improving accuracy in wellknown problems. In high dimensional contexts, we have focused on computational fluid dynamics (CFD) simulations. CFD simulations are able to produce complex and large outputs that accurately describe the physical properties of fluids and gases in various domains and they are frequently used for studying the effects of flow pat-terns and design choices on many engineering designs, such as wing, car and engineshapes. Due to the high dimensional aspect of the data, it is difficult to model to-ward achieving critical goals such as optimizing lift and drag forces. The key research question addressed in this thesis is whether we develop automated approaches that accurately abstract this information? We tackle these issues by studying a closely re-lated field, 3D computer vision, and adapt approaches to the particular data type.Moreover, inspired by this data type we propose new, deep learning, approaches that are also applied to traditional computer vision. Show less
Segmentation of medical images, particularly late gadolinium-enhanced magnetic resonance imaging (LGE-MRI) used for visualizing diseased atrial structures, is a crucial first step for ablation... Show moreSegmentation of medical images, particularly late gadolinium-enhanced magnetic resonance imaging (LGE-MRI) used for visualizing diseased atrial structures, is a crucial first step for ablation treatment of atrial fibrillation. However, direct segmentation of LGE-MRIs is challenging due to the varying intensities caused by contrast agents. Since most clinical studies have relied on manual, labor-intensive approaches, automatic methods are of high interest, particularly optimized machine learning approaches. To address this, we organized the 2018 Left Atrium Segmentation Challenge using 154 3D LGE-MRIs, currently the world's largest atrial LGE-MRI dataset, and associated labels of the left atrium segmented by three medical experts, ultimately attracting the participation of 27 international teams. In this paper, extensive analysis of the submitted algorithms using technical and biological metrics was performed by undergoing subgroup analysis and conducting hyper-parameter analysis, offering an overall picture of the major design choices of convolutional neural networks (CNNs) and practical considerations for achieving state-of-the-art left atrium segmentation. Results show that the top method achieved a Dice score of 93.2% and a mean surface to surface distance of 0.7 mm, significantly outperforming prior state-of-the-art. Particularly, our analysis demonstrated that double sequentially used CNNs, in which a first CNN is used for automatic region-of-interest localization and a subsequent CNN is used for refined regional segmentation, achieved superior results than traditional methods and machine learning approaches containing single CNNs. This large-scale benchmarking study makes a significant step towards much-improved segmentation methods for atrial LGE-MRIs, and will serve as an important benchmark for evaluating and comparing the future works in the field. Furthermore, the findings from this study can potentially be extended to other imaging datasets and modalities, having an impact on the wider medical imaging community. (C) 2020 Elsevier B.V. All rights reserved. Show less
Heutink, F.; Koch, V.; Verbist, B.; Woude, W.J. van der; Mylanus, E.; Huinck, W.; ... ; Caballo, M. 2020
Background and objective: Performing patient-specific, pre-operative cochlea CT-based measurements could be helpful to positively affect the outcome of cochlear surgery in terms of intracochlear... Show moreBackground and objective: Performing patient-specific, pre-operative cochlea CT-based measurements could be helpful to positively affect the outcome of cochlear surgery in terms of intracochlear trauma and loss of residual hearing. Therefore, we propose a method to automatically segment and measure the human cochlea in clinical ultra-high-resolution (UHR) CT images, and investigate differences in cochlea size for personalized implant planning.Methods: 123 temporal bone CT scans were acquired with two UHR-CT scanners, and used to develop and validate a deep learning-based system for automated cochlea segmentation and measurement. The segmentation algorithm is composed of two major steps (detection and pixel-wise classification) in cascade, and aims at combining the results of a multi-scale computer-aided detection scheme with a U-Net-like architecture for pixelwise classification. The segmentation results were used as an input to the measurement algorithm, which provides automatic cochlear measurements (volume, basal diameter, and cochlear duct length (CDL)) through the combined use of convolutional neural networks and thinning algorithms. Automatic segmentation was validated against manual annotation, by the means of Dice similarity, Boundary-F1 (BF) score, and maximum and average Hausdorff distances, while measurement errors were calculated between the automatic results and the corresponding manually obtained ground truth on a per-patient basis. Finally, the developed system was used to investigate the differences in cochlea size within our patient cohort, to relate the measurement errors to the actual variation in cochlear size across different patients.Results: Automatic segmentation resulted in a Dice of 0.90 +/- 0.03, BF score of 0.95 +/- 0.03, and maximum and average Hausdorff distance of 3.05 +/- 0.39 and 0.32 +/- 0.07 against manual annotation. Automatic cochlear measurements resulted in errors of 8.4% (volume), 5.5% (CDL), 7.8% (basal diameter). The cochlea size varied broadly, ranging between 0.10 and 0.28 ml (volume), 1.3 and 2.5 mm (basal diameter), and 27.7 and 40.1 mm (CDL).Conclusions: The proposed algorithm could successfully segment and analyze the cochlea on UHR-CT images, resulting in accurate measurements of cochlear anatomy. Given the wide variation in cochlear size found in our patient cohort, it may find application as a pre-operative tool in cochlear implant surgery, potentially helping elaborate personalized treatment strategies based on patient-specific, image-based anatomical measurements. (C) 2020 Elsevier B.V. All rights reserved. Show less
