Learning from small data sets in machine learning is a crucial challenge, especially when dealing with data imbalances and anomaly detection. This thesis delves into the challenges and... Show moreLearning from small data sets in machine learning is a crucial challenge, especially when dealing with data imbalances and anomaly detection. This thesis delves into the challenges and methodologies of learning from small datasets in machine learning, with a particular focus on addressing data imbalances and anomaly detec- tion. It thoroughly explores various strategies for effective small dataset learning in ML, examining both existing approaches and introducing novel techniques. The research pivots around two key questions: firstly, it investigates current methods employed for learning from small datasets in machine learning, and secondly, it assesses the efficacy of batch normalization in enhancing model performance and utilizing salient image segmentation as an augmentation policy in self-supervised learning.The thesis comprehensively reviews techniques for managing small datasets, in- cluding data selection and preprocessing, ensemble methods, transfer learning, regularization techniques, and synthetic data generation. A critical examination of batch normalization reveals its significant role in improving training time and testing errors for minority classes in highly imbalanced datasets. The study also demonstrates that utilizing salient image segmentation as an augmentation policy in self-supervised learning substantially improves representation learning. This improvement is particularly evident in the context of downstream tasks such as image segmentation, highlighting the effectiveness of this technique in enhancing model performance.In summary, this study contributes to the field of machine learning by exploring strategies for learning from small datasets. It offers a detailed analysis of batch normalization, highlighting its potential in improving performance for minority classes in imbalanced datasets. Additionally, the study introduces salient image segmentation as an augmentation policy in self-supervised learning, showing its effectiveness in tasks like image segmentation. These findings provide a solid foundation for further research in small sample learning and present practical insights for machine learning practitioners working with limited data. Show less
The research in this dissertation aims to optimise blood donation processes in the framework of the Dutch national blood bank Sanquin. The primary health risk for blood donors is iron deficiency,... Show moreThe research in this dissertation aims to optimise blood donation processes in the framework of the Dutch national blood bank Sanquin. The primary health risk for blood donors is iron deficiency, which is evaluated based on donors' hemoglobin and ferritin levels. If either of these levels are inadequate, donors are deferred from donation. Deferral due to low hemoglobin levels occurs on-site, meaning that donors have already traveled to the blood bank and then have to return home without donating, which is demotivating for the donor and inefficient for the blood bank. A large part of this dissertation therefore has the objective to develop a prediction model for donors' hemoglobin levels, based on historical measurements and donor characteristics.The prediction model that was developed reduces the deferral rate by approximately 60\% (from 3\% to 1\% for women, and from 1\% to 0.4\% for men), showing the potential of using data to enhance blood bank policy efficiency. Additionally, the model predictions were made explainable, providing the blood bank with insights into why specific predictions are made. These insights increase our understanding of the relationships between donor characteristics and hemoglobin levels. If this prediction model would be implemented in practice, the explanations could also be shared with the donor to help them understand why they are (not) invited to donate, which could also contribute to donor satisfaction and retention.In a collaborative effort with blood banks in Australia, Belgium, Finland and South Africa, the same prediction model was applied on data from each blood bank. Despite differences in blood bank policies and donor demographics, the models found similar associations with the predictor variables in all countries. Differences in performance could mostly be attributed to differences in deferral rates, with blood banks with higher deferral rates obtaining higher model accuracy.Beyond hemoglobin prediction models, additional research questions are explored. One study aims to identify determinants of ferritin levels in donors through repeated measurements, and linking these to environmental variables. Another study involves modeling the pharmacokinetics of antibodies in COVID-19 recovered donors, and finding relationships between patient characteristics, symptoms, and antibody levels over time.In summary, the research in this dissertation shows the potential within the wealth of data collected by blood banks. The proposed data-driven donation strategies not only decrease deferral rates but also increase donor retention and understanding. This comprehensive approach allows Sanquin to provide more personalised feedback to donors regarding their iron status, ultimately optimising the blood donation process and contributing to the overall efficacy of blood banking systems. Show less
Modeling and analysis of cyber-physical systems are still challenging. One reason is that cyber-physical systems involve many different parts (cyber or physical), of different nature (discrete or... Show moreModeling and analysis of cyber-physical systems are still challenging. One reason is that cyber-physical systems involve many different parts (cyber or physical), of different nature (discrete or continuous), and in constant interaction via sensing and actuating.For instance, consider a group of robots, each running a program that takes decision based on the sequence of sensor readings. The sensors that equip a robot return the current position of the robot and the position of any adjacent obstacle. The interactionoccurring between each robot in the group cannot be derived solely from the specification of individual robots. If the field on which the robots roam changes its property, the same group of robots might sense different values, and therefore take different actions. Also, the time at which a robot acts and senses will affect the decision of each controller and will change the resulting collective behavior.This thesis proposes a compositional approach to the design and programming of interacting cyber-physical components. We present an algebra that provides a novel perspective on modeling interaction of cyber-physical components. Using our algebraic framework, one can design a complex cyber-physical system by first designing each part, and then specifying how the parts interact. We formalized the relation between our abstract semantic model and an implementation written in Maude, a programming language based on rewriting logic. We present some applications, including safety and liveness properties of a system consisting of a set of robots, each equipped with a battery, running on a shared field. Show less
Radiography is an important technique to inspect objects, with applications in airports and hospitals. X-ray imaging is also essential in industry, for instance in food safety checks for the... Show moreRadiography is an important technique to inspect objects, with applications in airports and hospitals. X-ray imaging is also essential in industry, for instance in food safety checks for the presence of foreign objects. Computed tomography (CT) enables more accurate visualizations of an object in 3D, but requires more computation time. Spectral X-ray imaging is an important recent development to optimize these conflicting goals of speed and accuracy. This technique enables separation of detected X-ray photons in terms of energy. More information can be extracted from spectral images, which allows for better separation of materials. Deep learning is another important recent technique enabling machines to quickly carry out processing tasks, by training these with large volumes of data for these specific tasks.In this dissertation we present new processing methods that use spectral imaging and machine learning, with a special focus on industrial processes. We design a workflow using CT to efficiently generate large volumes of machine learning training data. In addition, we develop a compression method for efficient processing of large volumes of spectral data and two new spectral CT methods to produce more accurate reconstructions. The presented methods are designed for effective use in industry. Show less
With the advent of multicore processors and data centers, computer hardware has become increasingly parallel, allowing one to run multiple pieces of software at the same time on different machines.... Show moreWith the advent of multicore processors and data centers, computer hardware has become increasingly parallel, allowing one to run multiple pieces of software at the same time on different machines. Coordination of these pieces is best expressed in a coordination language as an explicit interaction protocol that clearly defines the interactions among all components in the software. An explicit interaction protocol not only improves code structure but also enables automated analysis of the protocol to improve execution efficiency of the software. In this thesis, we focus in particular on improving execution efficiency by means of scheduling, which concerns with the allocation of (computing) resources to software tasks. Almost always, scheduling is the responsibility of a general-purpose operating system that makes no assumptions on the software and thereby ignores all relevant scheduling information in that software. As a result, the operating system alone cannot ensure optimally scheduled execution of the software. In this thesis, we propose a solution that changes the software such that it will be efficiently scheduled by the general-purpose operating system. The main idea is to take advantage of the duality between scheduling and coordination. To be precise, we analyze the protocol of the software to determine an optimal scheduling strategy for this software. Then, we enforce this optimal schedule by incorporating the strategy in the original protocol. As a result, we force the ignorant operating scheduler to follow our precomputed optimal schedule. Show less
This thesis manuscript explores the use of video games as tools for conceptual exploration and academic research. The research question of how video games facilitate exploration is investigated... Show moreThis thesis manuscript explores the use of video games as tools for conceptual exploration and academic research. The research question of how video games facilitate exploration is investigated through nine chapters, including empirical studies such as user surveys, video game design artifacts, and user studies. Chapter 2 introduces relevant terminology and related work. Chapter 3 describes the creation of CURIO, a video game developed for classroom use that requires players to ask critical and original questions about topics a teacher defines, revealing the need to highlight information gaps to stimulate curiosity for conceptual exploration. Chapter 4 investigates what types of video games elicit curiosity for exploration through a survey, while Chapter 5 formulates a hypothesis on design patterns for exploration based on the survey results. Chapters 6 and 7 focus on implementing and validating design patterns for exploration and how they influence player behavior and experience. Chapter 8 reflects on the use of video games in research efforts. The final chapter summarizes insights and contributions, and provides directions for future research. Overall, the manuscript presents evidence that video games can effectively facilitate exploration and can be used as tools for academic research. Show less
The focus of this thesis is on the technical methods which help promote the movement towards Trustworthy AI, specifically within the Inspectorate of the Netherlands.The goal is develop and assess... Show moreThe focus of this thesis is on the technical methods which help promote the movement towards Trustworthy AI, specifically within the Inspectorate of the Netherlands.The goal is develop and assess the technical methods which are required to shift the actions of the Inspectorate to a data-driven paradigm, concretely under a supervised classification framework of machine learning.The aspect of reliability is addressed as a data quality concern, viz. missingness and noise.The aspect of fairness is addressed as a counter to bias in the selection process of inspections.The conclusion is that, whilst no complete solution has yet been suggested, it is possible to address the concerns related to data quality and data bias, culminating in well-performing classification models which are reliable and fair. Show less
Predictive maintenance (PdM) is a maintenance policy that uses the past, current, and prognosticated health condition of an asset to predict when timely maintenance should occur. PdM overcomes... Show morePredictive maintenance (PdM) is a maintenance policy that uses the past, current, and prognosticated health condition of an asset to predict when timely maintenance should occur. PdM overcomes challenges of more conservative policies, such as corrective or scheduled maintenance. The remaining useful life (RUL) is a critical notion in PdM that determines the time remaining until a system is no longer useful and requires maintenance. Among the approaches employed to estimate the RUL, data-driven PdM methods have shown to be a good candidate due to their (mostly) domain-agnostic nature and broad applicability mos on the asset’s generated data. Nevertheless, there are various challenges to consider in data-driven PdM, such as algorithm selection, hyperparameter optimization, and uncertainty of the RUL estimation. This thesis proposes solutions and frameworks for these challenges using simulated datasets. We furthermore dive into scheduling optimization which is the next step in PdM and point towards the importance of understanding the data generating process in PdM using real-world data. Finally, we show how a method originally developed for PdM in the automotive industry can lend itself to the medical domain, exhibiting the significance of knowledge-transfer and the versatility of data-driven methods. Show less
Quantum annealing belongs to a family of quantum optimization algorithms designed to solve combinatorial optimization problems using programmable quantum hardware. In this thesis, various methods... Show moreQuantum annealing belongs to a family of quantum optimization algorithms designed to solve combinatorial optimization problems using programmable quantum hardware. In this thesis, various methods are developed and tested to understand how to formulate combinatorial optimization problems for quantum annealing. Use of both purely quantum approaches as well as hybrid quantum-classical algorithms are investigated using state-of-the-art quantum hardware. How the developed methods affect the performance of quantum hardware in practice is tested using both real-world and canonical academic problems. Show less
The learning of software design is known to be a difficult and challenging task for students. This dissertation studies different didactic approaches for learning software design to improve the way... Show moreThe learning of software design is known to be a difficult and challenging task for students. This dissertation studies different didactic approaches for learning software design to improve the way we teach students software design. The research in the dissertation questions whether we can assess software design skills, what guidance is needed for the improvement of students’ understanding of software design and how to motivate and engage students for learning software design. The research explores the following: an instrument for measuring software design skills based on design principles, the gamification of learning software design, revealing students’ software design strategies, the use of peer-reflection for uncovering the difficulties students have during software design tasks, the use of teaching assistants as bridge between the lecturer and the students, the automation of grading software designs with machine learning, guiding feedback by a pedagogical agent and a workshop for engaging students into the process of software development. The research contributes to the future education of software design. Show less
In design optimization problems, engineers typically handcraft design representations based on personal expertise, which leaves a fingerprint of the user experience in the optimization data. Thus,... Show moreIn design optimization problems, engineers typically handcraft design representations based on personal expertise, which leaves a fingerprint of the user experience in the optimization data. Thus, learning this notion of experience as transferrable design features has potential to improve the performance of similar, yet more challenging, design optimization problems. However, engineering design data are unstructured, high-dimensional and often have no canonical representation, which poses several challenges for machine learning algorithms. In this thesis, geometric deep learning techniques, in particular 3D point cloud autoencoders, are utilized to learn novel shape-generative models from engineering optimization data. Through different sets of experiments, it is shown that these autoencoders are scalable to high-dimensional engineering models and have comparable optimization performance to state-of-the-art representations. Furthermore, a novel network feature visualization technique is proposed, which provides a geometric interpretation of the knowledge stored in the network and allows one to select sub-sets of degrees of freedom to modify and optimize shapes. Due to the domain agnosticism of the autoencoders’ latent space, the learned representations are exploited in multi-task optimization problems to enable knowledge transfer between tasks and foster commonality between the optimized shapes. Finally, to improve the state of readiness of the 3D models generated by the point cloud autoencoder for engineering simulations, a novel architecture is proposed: Point2FFD. The novel architecture learns to generate 3D polygonal meshes based on input 3D point clouds and a set of existing handcrafted mesh templates parameterized with free-form deformation. Based on shape-generative and optimization experiments, it is shown that Point2FFD generates 3D models with better overall quality than state-of-the-art point cloud (variational) autoencoders and improves the quality of designs in vehicle aerodynamic optimization problems. Show less
We are living in an information era where the amount of image and video data increases exponentially. It is important to develop intelligent visual understanding systems to satisfy our need for... Show moreWe are living in an information era where the amount of image and video data increases exponentially. It is important to develop intelligent visual understanding systems to satisfy our need for searching information of interest. An important example of such a system that, with the current increasing concern for public security, is urgently required, is an automated person Re-Identification (ReID) system. This thesis mainly focuses on exploring ReID systems via deep learning methods. To enable ReID systems to meet the so-called open-world challenges, we explore three themes that are challenging yet practical in real application scenarios: lifelong learning, unsupervised domain adaptation and cross-modality challenge. Furthermore, this thesis provides numerous experiments and in-depth analysis, which can help motivate further research on the three research themes. Show less
Patients share valuable advice and experiences with their peers in online patient discussion groups. These uncensored experiences can provide a complementaryperspective to that of the health... Show morePatients share valuable advice and experiences with their peers in online patient discussion groups. These uncensored experiences can provide a complementaryperspective to that of the health professional and thereby yield novel hypotheses which could be tested in further rigorous medical research. This thesis focuses on the development of automatic extraction methods to harvest these patient experiences from online patient forums using text mining techniques. We also examine the complementary value of these patient-reported outcomes to traditional sources of medical knowledge for scientific hypothesis generation. Specifically, we focus on the extraction of adverse drug events (i.e., side effects) and coping strategies for dealing with adverse drug events. Show less
A convolutional neural network (CNN) is a biologically inspired algorithm, highly capable at processing images and videos. Nowadays, CNNs are widely known and used: they watch our safety from the... Show moreA convolutional neural network (CNN) is a biologically inspired algorithm, highly capable at processing images and videos. Nowadays, CNNs are widely known and used: they watch our safety from the CCTV cameras, help doctors diagnose diseases, navigate cars, and do many other important things. One of the recent trends is to execute CNNs on edge devices: cameras, mobile phones, smart watches, etc. This helps to run CNNs faster and ensures privacy of the data used by the CNNs. This, however, is difficult to do. The problem is that the edge devices are small and often do not have enough resources to execute CNNs. In my dissertation, I study this problem and offer solutions for it. I propose specific manners to design and execute CNNs, so that they can run on edge devices efficiently. Show less
With the emergence of online social networks (OSNs), the way people create and share information has changed, which becomes faster and broader than traditional social media. Understanding how... Show moreWith the emergence of online social networks (OSNs), the way people create and share information has changed, which becomes faster and broader than traditional social media. Understanding how information (both good and harmful) spreads through OSNs, as well as what elements drive the success of information diffusion, has significant implications for a wide range of real-world applications. In this thesis, we conduct research to analysis the diffusion of information in OSNs via using deep representation learning. Specifically, we aim to develop deep learning- based models to solve two specific tasks, i.e., information cascades modeling and rumor detection. Show less
We present an extensive study of methods for exactly solving stochastic constraint (optimisation) problems (SCPs) in network analysis. These problems are prevalent in science, governance and... Show moreWe present an extensive study of methods for exactly solving stochastic constraint (optimisation) problems (SCPs) in network analysis. These problems are prevalent in science, governance and industry. Both our proposed solving methods aim to strike a good balance between convenience, generality, and speed. The first method we study is generic and decomposes stochastic constraints into a multitude of smaller local constraints that are solved using a constraint programming (CP) or mixed-integer programming (MIP) solver. However, many SCPs are formulated on probability distributions with a monotonic property, meaning that adding a positive decision to a partial solution to the problem cannot cause a decrease in solution quality. The second method is specifically designed for solving global stochastic constraints on monotonic probability distributions (SCMDs) in CP. Both methods use knowledge compilation to obtain a decision diagram encoding of the relevant probability distributions, where we focus on ordered binary decision diagrams (OBDDs). We discuss theoretical advantages and disadvantages of these methods and evaluate them experimentally. We conclude that, while the decomposition method is easy to implement and can be used to solve and SCP, the global stochastic constraint solves problems faster, and is still widely applicable due to the prevalence of monotonicity in real-world problems. Show less
Structural variants (SVs) are the hidden architecture of the human genome, and are critical for us to understand diseases, evolution, and so on. The development of both sequencing technologies and... Show moreStructural variants (SVs) are the hidden architecture of the human genome, and are critical for us to understand diseases, evolution, and so on. The development of both sequencing technologies and computational tools greatly facilitates the detection of SVs, while misinterpreting or even missing complex ones. Detecting and characterizing complex events is a typical field requiring multiple disciplines, i.e., domain knowledge and computer science algorithms. In this thesis, we introduce novel algorithms to detect and validate com- plex events, and assess the reproducibility of current SV detection pipelines for clinical and research settings. Show less
Humans perceive the real world through their sensory organs: vision, taste, hearing, smell, and touch. In terms of information, we consider these different modesalso referred to as different... Show moreHumans perceive the real world through their sensory organs: vision, taste, hearing, smell, and touch. In terms of information, we consider these different modesalso referred to as different channels of information or modals. Considering multiple channels of information, at the same time, is referred to as multimodal and the input as multimedia. By their very nature, multimedia data are complex and often involve intertwined instances of different kinds of information. We can leverage this multimodal perspective to extract meaning and understanding of theworld. This is comparable to how our brain processes these multiple channels, we learn how to combine and extract meaningful information from them. In this thesis, the learning is done by computer programs and smart algorithms. This is referred to as artificial intelligence. To that end, in this thesis, we have studied multimedia information, with a focus on vision and language information representation for semantic mapping. The aims of the semantic mapping learning in this thesis are: (1) visually supervised word embedding learning; (2) fine-grained labellearning for vision representation; (3) kernel-based transformation for image and text association; (4) visual representation learning via a cross-modal contrastivelearning framework. Show less
This thesis involves three topics: benchmarking discrete optimization algorithms, empirical analyses of evolutionary computation, and automatic algorithm configuration. The objective is... Show moreThis thesis involves three topics: benchmarking discrete optimization algorithms, empirical analyses of evolutionary computation, and automatic algorithm configuration. The objective is benchmarking EAs on discrete optimization for the selection and design of better optimizers.In practice, we start with building the IOHprofiler benchmark software, which supports us in testing algorithms on a wide range of problems and allows us to perform and visualize the statistical analysis on algorithms' performance.While performing numerous benchmark studies, we study the impact of mutation rate and population size on the EAs and investigate how crossover and mutation interplay with each other and the impact of population size on the GAs. Moreover, we analyze a smooth way of interpolating between local and non-local search by proposing a new normalized bit mutation.We apply Irace, MIP-EGO, and MIES to configure the GA for ERT and AUC, respectively. Our results suggest that even when interested in ERT, it might be preferable to tune for AUC for the configuration task. We also observe that tuning for ERT is much more sensitive with respect to the budget that is allocated to the target algorithms.At last, we leverage our benchmark data of static algorithms to study dynamic algorithm selection. Show less
