The invention of neural networks marks a critical milestone in the pursuit of true artificial intelligence. Despite their impressive performance on various tasks, these networks face limitations in... Show moreThe invention of neural networks marks a critical milestone in the pursuit of true artificial intelligence. Despite their impressive performance on various tasks, these networks face limitations in learning efficiently as they are often trained from scratch. Deep meta-learning is one approach to improve the learning efficiency by leveraging prior knowledge and experience. Whilst many succesful deep meta-learning techniques have been proposed, our understanding of the performance of these methods remains limited. In this dissertation, we delve deeper into the underlying principles of these algorithms, and aim to gain a comprehensive understanding of why certain algorithms succeed while others fall short. This allows us to design enhanced deep meta-learning algorithms and reason about the impact of specific design choices on the performance of different algorithms. Moreover, we investigate the integration of theoretical principles into meta-learning algorithms to improve their performance. Overall, we make a small step toward a better understanding of deep meta-learning algorithms, paving the way for more robust and principled meta-learning techniques with broader applicability and superior performance. Show less
Deep neural networks can achieve great successes when presented with large data sets and sufficient computational resources. However, their ability to learn new concepts quickly is limited. Meta... Show moreDeep neural networks can achieve great successes when presented with large data sets and sufficient computational resources. However, their ability to learn new concepts quickly is limited. Meta-learning is one approach to address this issue, by enabling the network to learn how to learn. The field of Deep Meta-Learning advances at great speed, but lacks a unified, in-depth overview of current techniques. With this work, we aim to bridge this gap. After providing the reader with a theoretical foundation, we investigate and summarize key methods, which are categorized into (i) metric-, (ii) model-, and (iii) optimization-based techniques. In addition, we identify the main open challenges, such as performance evaluations on heterogeneous benchmarks, and reduction of the computational costs of meta-learning. Show less
Many scientists are focussed on building models. We nearly process all information we perceive to a model. There are many techniques that enable computers to build models as well. The field... Show more Many scientists are focussed on building models. We nearly process all information we perceive to a model. There are many techniques that enable computers to build models as well. The field of research that develops such techniques is called Machine Learning. Many research is devoted to develop computer programs capable of building models (algorithms). Many of such algorithms exist, and these often consist of various options that subtly influence performance (parameters). Furthermore, there is mathematical proof that there exists no single algorithm that works well on every dataset. This complicates the task of selecting the right algorithm for a given task. The field of meta-learning aims to resolve these problems. The purpose is to determine what kind of algorithms work well on which datasets. In order to do so, we developed OpenML. This is an online database on which researches can share experimental results amongst each other, potentially scaling up the size of meta-learning studies. Having earlier experimental results freely accessible and reusable for others, it is no longer required to conduct time expensive experiments. Rather, researchers can answer such experimental questions by a simple database look-up. This thesis addresses how OpenML can be used to answer fundamental meta-learning questions. Show less
