The conflict between computational budget and quality of found solutions is crucial when dealing with expensive black-box optimization problems from the industry. We show that through multi... Show moreThe conflict between computational budget and quality of found solutions is crucial when dealing with expensive black-box optimization problems from the industry. We show that through multi-objective parameter tuning of the Covariance Matrix Adaptation Evolution Strategy on benchmark functions different optimal algorithm configurations can be found for specific computational budgets and solution qualities. With the obtained Pareto front, tuned parameter sets are selected and transferred to a real-world optimization problem from vehicle dynamics, improving the solution quality and budget needed. The benchmark functions for tuning are selected based on their similarity to a real-world problem in terms of Exploratory Landscape Analysis features. Show less
The algorithm selection problem is of paramount importance in achieving high-quality results while minimizing computational effort, especially when dealing with expensive black-box optimization... Show moreThe algorithm selection problem is of paramount importance in achieving high-quality results while minimizing computational effort, especially when dealing with expensive black-box optimization problems. In this paper, we address this challenge by using randomly generated artificial functions that mimic the landscape characteristics of the original problem while being inexpensive to evaluate. The similarity between the artificial function and the original problem is quantified using Exploratory Landscape Analysis. We demonstrate a significant performance improvement on five real-world vehicle dynamics problems by transferring the parameters of the Covariance Matrix Adaptation Evolution Strategy tuned to these artificial functions.We provide a complete set of simulated values of braking distance for fully enumerated 2D design spaces of all five real-world optimization problems. So, replication of our results and benchmarking directly on the real-world problems is possible. Beyond the scope of this paper, this data can be used as a benchmarking set for multi-objective optimization with up to five objectives. Show less
The performance of the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) is significantly affected by the selection of the specific CMA-ES variant and the parameter values used. Furthermore,... Show moreThe performance of the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) is significantly affected by the selection of the specific CMA-ES variant and the parameter values used. Furthermore, optimal CMA-ES parameter configurations vary across different problem landscapes, making the task of tuning CMA-ES to a specific optimization problem a challenging mixed-integer optimization problem. In recent years, several advanced algorithms have been developed to address this problem, including the Sequential Model-based Algorithm Configuration (SMAC) and the Tree-structured Parzen Estimator (TPE).In this study, we propose a novel approach for tuning CMA-ES by leveraging CMA-ES itself. Therefore, we combine the modular CMA-ES implementation with the margin extension to handle mixed-integer optimization problems. We show that CMA-ES can not only compete with SMAC and TPE but also outperform them in terms of wall clock time. Show less
Rooij, L. van; Winter, R. de; Kononova, A.V.; Stein, N. van 2022
Decisions made in the early phases of ship design have a large influence on the capital and operational expenses of a vessel. In order to support decision making in this phase, big data and machine... Show moreDecisions made in the early phases of ship design have a large influence on the capital and operational expenses of a vessel. In order to support decision making in this phase, big data and machine learning techniques can be of great use. This work shows how Explainable Artificial Intelligence (XAI) and Global Sensitivity Analysis (GSA) combined with Autonomous Identification System (AIS) and static ship data can be used to find important design characteristics of ships. A data collection framework is setup that collects AIS data over a five month time period. Static ship design data is used to predict performance related target features that are calculated from AIS data. By applying XAI and GSA methods to the regression models that predict these target features, insight can be gained on how design features influence the performance of ships. Experiments showed that for most ship types, the overall length is the most important design feature for speed related target features. Besides the overall length, the draught also has a significant impact on the rotation capabilities. Show less
Vermetten, D.L.; Caraffini, F.; Stein, B. van; Kononova, A.V. 2022
Decades of progress in simulation-based surrogate-assisted optimization and unprecedented growth in computational power have enabled researchers and practitioners to optimize previously intractable... Show moreDecades of progress in simulation-based surrogate-assisted optimization and unprecedented growth in computational power have enabled researchers and practitioners to optimize previously intractable complex engineering problems. This paper investigates the possible benefit of a concurrent utilization of multiple simulation-based surrogate models to solve complex discrete optimization problems. To fulfill this, the so-called Self-Adaptive Multi-surrogate Assisted Efficient Global Optimization algorithm (SAMA-DiEGO), which features a two-stage online model management strategy, is proposed and further benchmarked on fifteen binary-encoded combinatorial and fifteen ordinal problems against several state-of-the-art non-surrogate or single surrogate assisted optimization algorithms. Our findings indicate that SAMA-DiEGO can rapidly converge to better solutions on a majority of the test problems which shows the feasibility and advantage of using multiple surrogate models in optimizing discrete problems Show less
Structural Bias (SB) is an important type of algorithmic deficiency within iterative optimisation heuristics. However, methods for detecting structural bias have not yet fully matured, and recent... Show moreStructural Bias (SB) is an important type of algorithmic deficiency within iterative optimisation heuristics. However, methods for detecting structural bias have not yet fully matured, and recent studies have uncovered many interesting questions. One of these is the question of how structural bias can be related to anisotropy. Intuitively, an algorithm that is not isotropic would be considered structurally biased. However, there have been cases where algorithms appear to only show SB in some dimensions. As such, we investigate whether these algorithms actually exhibit anisotropy, and how this impacts the detection of SB. We find that anisotropy is very rare, and even in cases where it is present, there are clear tests for SB which do not rely on any assumptions of isotropy, so we can safely expand the suite of SB tests to encompass these kinds of deficiencies not found by the original tests.We propose several additional testing procedures for SB detection and aim to motivate further research into the creation of a robust portfolio of tests. This is crucial since no single test will be able to work effectively with all types of SB we identify. Show less
Automated symmetry detection is still a difficult task in 2021. However, it has applications in computer vision, and it also plays an important part in understanding art. This paper focuses on... Show moreAutomated symmetry detection is still a difficult task in 2021. However, it has applications in computer vision, and it also plays an important part in understanding art. This paper focuses on aiding the latter by comparing different state-of-the-art automated symmetry detection algorithms. For one of such algorithms aimed at reflectional symmetries, we propose postprocessing improvements to find localised symmetries in images, improve the selection of detected symmetries and identify another symmetry type (rotational). In order to detect rotational symmetries, we contribute a machine learning model which detects rotational symmetries based on provided reflection symmetry axis pairs. We demonstrate and analyze the performance of the extended algorithm to detect localised symmetries and the machine learning model to classify rotational symmetries. Show less
Automated machine learning (AutoML) aims to automatically produce the best machine learning pipeline, i.e., a sequence of operators and their optimized hyperparameter settings, to maximize the... Show moreAutomated machine learning (AutoML) aims to automatically produce the best machine learning pipeline, i.e., a sequence of operators and their optimized hyperparameter settings, to maximize the performance of an arbitrary machine learning problem. Typically, AutoML based Bayesian optimization (BO) approaches convert the AutoML optimization problem into a Hyperparameter Optimization (HPO) problem, where the choice of algorithms is modeled as an additional categorical hyperparameter. In this way, algorithms and their local hyper-parameters are referred to as the same level. Consequently, this approach makes the resulting initial sampling less robust. In this study, we describe a first attempt to formulate the AutoML optimization problem as its nature instead of transfer it into a HPO problem. To take advantage of this paradigm, we propose a novel initial sampling approach to maximize the coverage of the AutoML search space to help BO construct a robust surrogate model. We experiment with 2 independent scenarios of AutoML with 2 operators and 6 operators over 117 benchmark datasets. Results of our experiments demonstrate that the performance of BO significantly improved by using our sampling approach. Show less
