Rheumatoid Arthritis (RA) is an autoimmune disease that mainly affects joints in the wrist and hands. It typically results in inflamed and painful joints. MRI is one of the most common imaging...Show moreRheumatoid Arthritis (RA) is an autoimmune disease that mainly affects joints in the wrist and hands. It typically results in inflamed and painful joints. MRI is one of the most common imaging modalities to detect and monitor possible inflamed RA-related areas, enabling rheumatologists to treat patients more timely and efficiently. Despite the importance of finding and tracking inflamed areas associated with RA in MRI, there is no previously published work on finding pixel-by-pixel changes related to RA between baseline and follow-up MRIs. Therefore, this paper proposes a hypothesis-free deep learning-based model to discover changes in wrist MRIs on a pixel level to detect changes in inflamed areas related to RA without using prior anatomical information. To do this, a combination of a U-Net-based network and image thresholding was utilised to find pixel-level non-trivial changes between baseline and follow-up MRI images. A wrist MRI dataset including 99 individual pairs of MRI images (each pair constructed of baseline and follow-up images) was used to evaluate the proposed model. Data were collected from patients with clinically suspected arthralgia (CSA), defined as patients at risk of developing RA according to their rheumatologist and already had subclinical inflammation on MRI but could not be diagnosed with RA (yet) since they had not developed clinically detectable arthritis. The obtained results were evaluated using an observer study. The evaluation showed that our proposed model is a promising first step toward developing an automatic model to find RA-related inflammatory changes. Show less
The invention of neural networks marks a critical milestone in the pursuit of true artificial intelligence. Despite their impressive performance on various tasks, these networks face limitations in... Show moreThe invention of neural networks marks a critical milestone in the pursuit of true artificial intelligence. Despite their impressive performance on various tasks, these networks face limitations in learning efficiently as they are often trained from scratch. Deep meta-learning is one approach to improve the learning efficiency by leveraging prior knowledge and experience. Whilst many succesful deep meta-learning techniques have been proposed, our understanding of the performance of these methods remains limited. In this dissertation, we delve deeper into the underlying principles of these algorithms, and aim to gain a comprehensive understanding of why certain algorithms succeed while others fall short. This allows us to design enhanced deep meta-learning algorithms and reason about the impact of specific design choices on the performance of different algorithms. Moreover, we investigate the integration of theoretical principles into meta-learning algorithms to improve their performance. Overall, we make a small step toward a better understanding of deep meta-learning algorithms, paving the way for more robust and principled meta-learning techniques with broader applicability and superior performance. Show less
Pipeline-parallel training has emerged as a popular method to train large Deep Neural Networks (DNNs), as it allows the use of the combined compute power and memory capacity of multiple Graphics... Show morePipeline-parallel training has emerged as a popular method to train large Deep Neural Networks (DNNs), as it allows the use of the combined compute power and memory capacity of multiple Graphics Processing Units (GPUs). However, with the sustaining increase in Deep Learning (DL) model sizes, pipeline parallelism provides only a partial solution to the memory bottleneck in large-scale DNN training. Careful partitioning of the DL model over the available GPUs based on memory usage is required to further alleviate the memory bottleneck and train larger DNNs. mCAP is such a memory-oriented partitioning approach for pipeline parallel systems, but it does not scale to models with many layers and very large hardware setups, as it requires extensive profiling and fails to efficiently navigate the partitioning space to find the most memory-friendly partitioning. In this work, we propose CAPSlog, a scalable memory-centric partitioning approach that can recommend model partitionings for larger and more heterogeneous DL models and for larger hardware setups than existing approaches. CAPSlog introduces a new profiling method and a new, much more scalable algorithm for recommending memory-efficient partitionings. CAPSlog reduces the profiling time by 67% compared to existing approaches, searches the partitioning space for the optimal solution orders of magnitude faster and can train significantly larger models. Show less
Background: Recent advances in data-driven computational approaches have been helpful in devising tools to objectively diagnose psychiatric disorders. However, current machine learning studies... Show moreBackground: Recent advances in data-driven computational approaches have been helpful in devising tools to objectively diagnose psychiatric disorders. However, current machine learning studies limited to small homogeneous samples, different methodologies, and different imaging collection protocols, limit the ability to directly compare and generalize their results. Here we aimed to classify individuals with PTSD versus controls and assess the generalizability using a large heterogeneous brain datasets from the ENIGMA-PGC PTSD Working group. Methods: We analyzed brain MRI data from 3,477 structural-MRI; 2,495 resting state-fMRI; and 1,952 diffusion-MRI. First, we identified the brain features that best distinguish individuals with PTSD from controls using traditional machine learning methods. Second, we assessed the utility of the denoising variational autoencoder (DVAE) and evaluated its classification performance. Third, we assessed the generalizability and reproducibility of both models using leave-one-site-out cross-validation procedure for each modality. Results: We found lower performance in classifying PTSD vs. controls with data from over 20 sites (60 % test AUC for s-MRI, 59 % for rs-fMRI and 56 % for D-MRI), as compared to other studies run on single-site data. The performance increased when classifying PTSD from HC without trauma history in each modality (75 % AUC). The classification performance remained intact when applying the DVAE framework, which reduced the number of features. Finally, we found that the DVAE framework achieved better generalization to unseen datasets compared with the traditional machine learning frameworks, albeit performance was slightly above chance. Conclusion: These results have the potential to provide a baseline classification performance for PTSD when using large scale neuroimaging datasets. Our findings show that the control group used can heavily affect classification performance. The DVAE framework provided better generalizability for the multi-site data. This may be more significant in clinical practice since the neuroimaging-based diagnostic DVAE classification models are much less site-specific, rendering them more generalizable. Show less
Sewer pipes are an essential infrastructure in modern society and their proper operation is important for public health. To keep sewer pipes operational as much as possible, periodical inspections... Show moreSewer pipes are an essential infrastructure in modern society and their proper operation is important for public health. To keep sewer pipes operational as much as possible, periodical inspections for defects are performed. Instead of repairing sewer pipes when a problem becomes critical, such inspections allow municipalities to plan maintenance.Sewer pipe inspections are an attractive target for automation. While a potential improvement in terms of assessment quality and processing efficiency is generally promised by automation, in this case we would also decrease the variability which is a current problem. Besides the reasons for automating, the methods for automating are also attractive: a lot of (visual) data has been gathered over the past decades which may be used to train algorithms.This thesis compiles the results of five years of research into the possible automation of sewer pipe inspections with the tools of machine learning and computer vision. In this thesis, three distinct, yet complementary approaches to automating sewer pipe inspections are described:- Image-Based Unsupervised Anomaly Detection- Convolutional Neural Network Classification- Stereovision and Geometry Reconstruction Show less
Cine and 4D flow cardiac MRI are two important non-invasive MR imaging techniques to assess cardiac function and diagnose cardiovascular diseases. Cine MRI offers great soft tissue detail which... Show moreCine and 4D flow cardiac MRI are two important non-invasive MR imaging techniques to assess cardiac function and diagnose cardiovascular diseases. Cine MRI offers great soft tissue detail which allows clinical experts to evaluate structure and function of the heart. 4D flow MRI further has the ability of three-dimensional time-resolved acquisition of blood flow velocity, which can be used to derive intra-cardiac hemodynamic parameters. In this thesis, we developed deep learning-based approaches to analyze cine and 4D flow cardiac MRI. This thesis proposes deep learning based methods for quantifying cardiac MRI. The described methods can be applied for cine MR image quality classification and ventricle segmentation without any human interactions. Investigating combining and fusing magnitude and velocity images can be helpful for left ventricle segmentation in 4D flow MRI, which is not fully explored yet. Moreover, we proposed a network to predict the blood flow pattern from the cine MRI. By combining visualization of the blood flow and myocardial motion in the routinely acquired standard CMR exams, the method can be potentially used in clinical studies. Show less
Radiography is an important technique to inspect objects, with applications in airports and hospitals. X-ray imaging is also essential in industry, for instance in food safety checks for the... Show moreRadiography is an important technique to inspect objects, with applications in airports and hospitals. X-ray imaging is also essential in industry, for instance in food safety checks for the presence of foreign objects. Computed tomography (CT) enables more accurate visualizations of an object in 3D, but requires more computation time. Spectral X-ray imaging is an important recent development to optimize these conflicting goals of speed and accuracy. This technique enables separation of detected X-ray photons in terms of energy. More information can be extracted from spectral images, which allows for better separation of materials. Deep learning is another important recent technique enabling machines to quickly carry out processing tasks, by training these with large volumes of data for these specific tasks.In this dissertation we present new processing methods that use spectral imaging and machine learning, with a special focus on industrial processes. We design a workflow using CT to efficiently generate large volumes of machine learning training data. In addition, we develop a compression method for efficient processing of large volumes of spectral data and two new spectral CT methods to produce more accurate reconstructions. The presented methods are designed for effective use in industry. Show less
Pulmonary function tests (PFTs) play an important role in screening and following-up pulmonary involvement in systemic sclerosis (SSc). However, some patients are not able to perform PFTs due to... Show morePulmonary function tests (PFTs) play an important role in screening and following-up pulmonary involvement in systemic sclerosis (SSc). However, some patients are not able to perform PFTs due to contraindications. In addition, it is unclear how lung function is affected by changes in lung structure in SSc. Therefore, this study aims to explore the potential of automatically estimating PFT results from chest CT scans of SSc patients and how different regions influence the estimation of PFTs. Deep regression networks were developed with transfer learning to estimate PFTs from 316 SSc patients. Segmented lungs and vessels were used to mask the CT images to train the network with different inputs: from entire CT scan, lungs-only to vessels-only. The network trained on entire CT scans with transfer learning achieved an ICC of 0.71, 0.76, 0.80, and 0.81 for the estimation of DLCO, FEV1, FVC and TLC, respectively. The performance of the networks gradually decreased when trained on data from lungs-only and vessels-only. Regression attention maps showed that regions close to large vessels were highlighted more than other regions, and occasionally regions outside the lungs were highlighted. These experiments show that apart from the lungs and large vessels, other regions contribute to PFT estimation. In addition, adding manually designed biomarkers increased the correlation (R) from 0.75, 0.74, 0.82, and 0.83 to 0.81, 0.83, 0.88, and 0.90, respectively. This suggests that that manually designed imaging biomarkers can still contribute to explaining the relation between lung function and structure. Show less
Monitoring the illegal trade of wool fibres of wild vicun~a (Vicugna vicugna) and guanaco (Lama guanicoe) is highly desirable. The high market value of fleece from these camelid species poses a... Show moreMonitoring the illegal trade of wool fibres of wild vicun~a (Vicugna vicugna) and guanaco (Lama guanicoe) is highly desirable. The high market value of fleece from these camelid species poses a threat to their wild populations. A previous study showed that direct analysis in real time time-of-flight mass spectrometry (DART-TOFMS) effectively identifies wool fibres to species. Producing high-resolution data in a short period of time makes DART-TOFMS a reliable identification tool, even though data analysis can still be improved. The present study proposes a novel data analysing pipeline based on Convolutional Neural Networks (CNN), applicable to any kind of DART-TOF MS data. We tested our proposed method on keratin fibres of four camelid species (Vicugna vicugna: n 1⁄4 19; Vicugna pacos: n 1⁄4 20; Lama guanicoe: n 1⁄4 20, and Lama glama: n 1⁄4 20). Analyses showed that selecting 512 ions with the highest relative intensity provides the best resolution and yields 100% accuracy for species identification. Show less
This thesis describes the steps necessary for the addition of the tumor-stroma ratio (TSR) into clinical practice as high-risk factor besides the TNM classification. The route from laboratory... Show moreThis thesis describes the steps necessary for the addition of the tumor-stroma ratio (TSR) into clinical practice as high-risk factor besides the TNM classification. The route from laboratory biomarker development to clinical implementation is followed. During this process, the relationship of the TSR to other available biomarkers for prognostic information for breast and colon cancer patients is investigated. Additionally, the prognostic value of the TSR in lung cancer is studied. Show less
Hesse, L.S.; Aliasi, M.; Moser, F.; Haak, M.C.; Xie, W.D.; Jenkinson, M.; ... ; INTERGROWTH-21st Consortium 2022
The quantification of subcortical volume development from 3D fetal ultrasound can provide important diagnostic information during pregnancy monitoring. However, manual segmentation of subcortical... Show moreThe quantification of subcortical volume development from 3D fetal ultrasound can provide important diagnostic information during pregnancy monitoring. However, manual segmentation of subcortical structures in ultrasound volumes is time-consuming and challenging due to low soft tissue contrast, speckle and shadowing artifacts. For this reason, we developed a convolutional neural network (CNN) for the automated segmentation of the choroid plexus (CP), lateral posterior ventricle horns (LPVH), cavum septum pellucidum et vergae (CSPV), and cerebellum (CB) from 3D ultrasound. As ground-truth labels are scarce and expensive to obtain, we applied few-shot learning, in which only a small number of manual annotations (n = 9) are used to train a CNN. We compared training a CNN with only a few individually annotated volumes versus many weakly labelled volumes obtained from atlas-based segmentations. This showed that segmentation performance close to intra-observer variability can be obtained with only a handful of manual annotations. Finally, the trained models were applied to a large number (n = 278) of ultrasound image volumes of a diverse, healthy population, obtaining novel US-specific growth curves of the respective structures during the second trimester of gestation. Show less
Tomography is a powerful technique to non-destructively determine the interior structure of an object.Usually, a series of projection images (e.g.\ X-ray images) is acquired from a range of... Show moreTomography is a powerful technique to non-destructively determine the interior structure of an object.Usually, a series of projection images (e.g.\ X-ray images) is acquired from a range of different positions.from these projection images, a reconstruction of the object's interior is computed. Many advanced applications require fast acquisition, effectively limiting the number of projection images and imposing a level of noise on these images. These limitations result in artifacts (deficiencies) in the reconstructed images. Recently, deep neural networks have emerged as a powerful technique to remove these limited-data artifacts from reconstructed images, often outperformingconventional state-of-the-art techniques. To perform this task, the networks are typically trained on a dataset of paired low-quality and high-quality images of similar objects. This is a major obstacle to their use in many practical applications. In this thesis, we explore techniques to employ deep learning in advanced experiments where measuring additional objects is not possible. Show less
Over several decades, a variety of computational methods for drug discovery have been proposed and applied in practice. With the accumulation of data and the development of machine learning methods... Show moreOver several decades, a variety of computational methods for drug discovery have been proposed and applied in practice. With the accumulation of data and the development of machine learning methods, computational drug design methods have gradually shifted to a new paradigm, i.e. deep learning methods have attracted particular interest in drug design. In this study, a new deep learning-based method (DrugEx) was proposed to design de novo drug-like molecules. It was proven that candidate molecules designed by DrugEx had a larger chemical diversity, and better covered the chemical space of known ligands. In order to address the issue of polypharmacology, the DrugEx algorithm was updated with multi-objective optimization towards multiple targets. The results of its application demonstrated the generation of compounds with a diverse predicted selectivity profile toward multiple targets, offering the potential of high efficacy and lower toxicity. In order to improve its generality, DrugEx was further updated to have the capability of designing molecules based on given scaffolds. We extended the architecture of Transformer to deal with each molecule as a graph. As a proof, its effectiveness in that 100% valid molecules are generated and most of them had predicted high affinity towards A2AAR with given scaffolds. Moreover, GenUI was developed as a visualizion software platform that makes it possible to integrate molecular generators within a feature-rich graphical user interface to facilitate collaboration in the disparate communities interested in computer-aided drug discovery.These studies highlight the overwhelming power of AI methods in drug discovery. Show less
Stiphout, J.A. van; Driessen, J.; Koetzier, L.R.; Ruules, L.B.; Willemink, M.J.; Heemskerk, J.W.T.; Molen, A.J. van der 2021
Objective To determine the difference in CT values and image quality of abdominal CT images reconstructed by filtered back-projection (FBP), hybrid iterative reconstruction (IR), and deep learning... Show moreObjective To determine the difference in CT values and image quality of abdominal CT images reconstructed by filtered back-projection (FBP), hybrid iterative reconstruction (IR), and deep learning reconstruction (DLR). Methods PubMed and Embase were systematically searched for articles regarding CT densitometry in the abdomen and the image reconstruction techniques FBP, hybrid IR, and DLR. Mean differences in CT values between reconstruction techniques were analyzed. A comparison between signal-to-noise ratio (SNR) and contrast-to-noise ratio (CNR) of FBP, hybrid IR, and DLR was made. A comparison of diagnostic confidence between hybrid IR and DLR was made. Results Sixteen articles were included, six being suitable for meta-analysis. In the liver, the mean difference between hybrid IR and DLR was - 0.633 HU (p = 0.483, SD +/- 0.902 HU). In the spleen, the mean difference between hybrid IR and DLR was - 0.099 HU (p = 0.925, SD +/- 1.061 HU). In the pancreas, the mean difference between hybrid IR and DLR was - 1.372 HU (p = 0.353, SD +/- 1.476 HU). In 14 articles, CNR was described. In all cases, DLR showed a significantly higher CNR. In 9 articles, SNR was described. In all cases but one, DLR showed a significantly higher SNR. In all cases, DLR showed a significantly higher diagnostic confidence. Conclusions There were no significant differences in CT values reconstructed by FBP, hybrid IR, and DLR in abdominal organs. This shows that these reconstruction techniques are consistent in reconstructing CT values. DLR images showed a significantly higher SNR and CNR, compared to FBP and hybrid IR. Show less
Image registration is the process of aligning images by finding the spatial relation between the images. Assuming two images called fixed and moving images are taken at different time, different... Show moreImage registration is the process of aligning images by finding the spatial relation between the images. Assuming two images called fixed and moving images are taken at different time, different spatial location, or via a different imaging technique, the aim of image registration is to find an optimal transformation that aligns the fixed and the moving images. Performing an automatic fast image registration with less manual finetuning can speed up numerous medical image processing procedures. In addition, an automatic quality assessment of registration can speed up this time-consuming task. In this thesis, we developed a fast learning-based image registration technique called RegNet.Predicting registration error can be useful for evaluation of registration procedures, which is important for the adoption of registration techniques in the clinic. In addition, quantitative error prediction can be helpful in improving the registration quality. In this thesis, we proposed two quality assessment mechanisms using random forests (RF) and convolutional long short term memory (ConvLSTM), in which the latter performs faster and more accurate. Show less
This thesis mainly focuses on cross-modal retrieval and single-modal image retrieval via deep learning methods, i.e. by using deep convolutional neural networks.For cross-modal retrieval, Shannon... Show moreThis thesis mainly focuses on cross-modal retrieval and single-modal image retrieval via deep learning methods, i.e. by using deep convolutional neural networks.For cross-modal retrieval, Shannon information entropy and adversarial learning are integrated to learn a common latent space for image data and text data. Furthermore, this thesis explores single-modal image retrieval in an incremental learning context to reduce the catastrophic forgetting of deep models, thereby expanding the continuous retrieval ability. The efficacy of the proposed methods in this thesis is verified by thorough experiments on the considered datasets. This thesis also gives an overview of new ideas and trends for multimodal content understanding. Show less
In this thesis we explore machine and deep learning approaches that address keychallenges in high dimensional problem areas and also in improving accuracy in wellknown problems. In high dimensional... Show moreIn this thesis we explore machine and deep learning approaches that address keychallenges in high dimensional problem areas and also in improving accuracy in wellknown problems. In high dimensional contexts, we have focused on computational fluid dynamics (CFD) simulations. CFD simulations are able to produce complex and large outputs that accurately describe the physical properties of fluids and gases in various domains and they are frequently used for studying the effects of flow pat-terns and design choices on many engineering designs, such as wing, car and engineshapes. Due to the high dimensional aspect of the data, it is difficult to model to-ward achieving critical goals such as optimizing lift and drag forces. The key research question addressed in this thesis is whether we develop automated approaches that accurately abstract this information? We tackle these issues by studying a closely re-lated field, 3D computer vision, and adapt approaches to the particular data type.Moreover, inspired by this data type we propose new, deep learning, approaches that are also applied to traditional computer vision. Show less
Spark NLP is a Natural Language Processing (NLP) library built on top of Apache Spark ML. It provides simple, performant & accurate NLP annotations for machine learning pipelines that can scale... Show moreSpark NLP is a Natural Language Processing (NLP) library built on top of Apache Spark ML. It provides simple, performant & accurate NLP annotations for machine learning pipelines that can scale easily in a distributed environment. Spark NLP comes with 1100+ pretrained pipelines and models in more than 192+ languages. It supports nearly all the NLP tasks and modules that can be used seamlessly in a cluster. Downloaded more than 2.7 million times and experiencing 9x growth since January 2020, Spark NLP is used by 54% of healthcare organizations as the world’s most widely used NLP library in the enterprise. Show less
Deep neural networks can achieve great successes when presented with large data sets and sufficient computational resources. However, their ability to learn new concepts quickly is limited. Meta... Show moreDeep neural networks can achieve great successes when presented with large data sets and sufficient computational resources. However, their ability to learn new concepts quickly is limited. Meta-learning is one approach to address this issue, by enabling the network to learn how to learn. The field of Deep Meta-Learning advances at great speed, but lacks a unified, in-depth overview of current techniques. With this work, we aim to bridge this gap. After providing the reader with a theoretical foundation, we investigate and summarize key methods, which are categorized into (i) metric-, (ii) model-, and (iii) optimization-based techniques. In addition, we identify the main open challenges, such as performance evaluations on heterogeneous benchmarks, and reduction of the computational costs of meta-learning. Show less
As low-field MRI technology is being disseminated into clinical settings around the world, it is important to assess the image quality required to properly diagnose and treat a given disease and... Show moreAs low-field MRI technology is being disseminated into clinical settings around the world, it is important to assess the image quality required to properly diagnose and treat a given disease and evaluate the role of machine learning algorithms, such as deep learning, in the enhancement of lower quality images. In this post hoc analysis of an ongoing randomized clinical trial, we assessed the diagnostic utility of reduced-quality and deep learning enhanced images for hydrocephalus treatment planning. CT images of post-infectious infant hydrocephalus were degraded in terms of spatial resolution, noise, and contrast between brain and CSF and enhanced using deep learning algorithms. Both degraded and enhanced images were presented to three experienced pediatric neurosurgeons accustomed to working in low-to middle-income countries (LMIC) for assessment of clinical utility in treatment planning for hydrocephalus. In addition, enhanced images were presented alongside their ground truth CT counterparts in order to assess whether reconstruction errors caused by the deep learning enhancement routine were acceptable to the evaluators. Results indicate that image resolution and contrast-to-noise ratio between brain and CSF predict the likelihood of an image being characterized as useful for hydrocephalus treatment planning. Deep learning enhancement substantially increases contrast-to-noise ratio improving the apparent likelihood of the image being useful; however, deep learning enhancement introduces structural errors which create a substantial risk of misleading clinical interpretation. We find that images with lower quality than is customarily acceptable can be useful for hydrocephalus treatment planning. Moreover, low quality images may be preferable to images enhanced with deep learning, since they do not introduce the risk of misleading information which could misguide treatment decisions. These findings advocate for new standards in assessing acceptable image quality for clinical use. Show less
