Efficient tuning of automated machine learning pipelines

Nguyen, D.A. (2024)

Doctoral Thesis

Automated Machine Learning (AutoML) is widely used to automatically build a suitable practical Machine Learning (ML) model for an arbitrary real-world problem, reducing the effort of practitioners in the ML development cycle for real-world applications. Optimization is a key part of a typical AutoML framework, and several optimization approaches have been developed to enhance AutoML performance and maximize its ability to find high-performing ML models on a wide range of real-world problems. Many AutoML studies treat the problem as a Hyperparameter Optimization (HPO) problem, which may limit the effectiveness of the underlying optimizer for the actual AutoML problem. Instead of using the HPO-based approach, the problem can be approached as a ML pipeline optimization problem with a hierarchically structured search space. Two optimization algorithms have been developed based on this paradigm to improve the performance of Bayesian Optimization (BO) in solving the AutoML optimization problem. A wide range of experiments indicates that our approaches have significantly improved BO's performance. In summary, this thesis focuses on enhancing AutoML performance through solving the ML pipeline optimization problem.

• AutoML
• CASH
• ML pipeline Optimization
• Hyperparameter Optimization
• Machine Learning
• Bayesian Optimization
• Combinational Sampling
• Contesting Procedure

• All authors: Nguyen, D.A.
• Supervisor: Bäck, T.H.W.; Sendhoff, B.
• Co-supervisor: Kononova, A.V.
• Committee: Bonsangue, M.M.; Plaat, A.; Rijn, J.N. van; Zaharie, D.; Ochoa, G.; Sun, J.
• Qualification: Doctor (dr.)
• Awarding Institution: Leiden Institute of Advanced Computer Science (LIACS), Faculty of Science, Leiden University
• Date: 2024-10-09
• Sponsorship: European Research Council