Expensive objectives and constraints are key characteristics of real-world multi-objective optimization problems. In practice, they often occur jointly with inexpensive objectives and constraints.... Show moreExpensive objectives and constraints are key characteristics of real-world multi-objective optimization problems. In practice, they often occur jointly with inexpensive objectives and constraints. This paper presents the Inexpensive Objectives and Constraints Self-Adapting Multi-Objective Constraint Optimization algorithm that uses Radial Basis function Approximations (IOC-SAMO-COBRA) for such problems. This is motivated by the recently proposed Inexpensive Constraint Surrogate-Assisted Non-dominated Sorting Genetic Algorithm II (ICSA-NSGA-II). These algorithms and their counterparts that do not explicitly differentiate between expensive and inexpensive objectives and constraints are compared on 22 widely used test functions. The IOC-SAMOCOBRA algorithm finds significantly better (identical/worse) Pareto fronts in at least 78% (6%/16%) of all test problems compared to IC-SA-NSGA-II measured with both the hypervolume and Inverted Generational Distance+ performance metric. The empirical cumulative distribution functions confirm this advantage for both algorithm variants that exploit the inexpensive constraints. In addition, the proposed method is compared against state-of-the-art practices on a real-world cargo vessel design problem. On this 17-dimensional twoobjective practical problem, the proposed IOC-SAMO-COBRA outperforms SA-NSGA-II as well. From an algorithmic perspective, the comparison identifies specific strengths of both approaches and indicates how they should be hybridized to combine their best components. Show less
Tannemaat, M.R.; Kefalas, M.; Geraedts, V.J.; Remijn-Nelissen, L.; Verschuuren, A.J.M.; Koch, M.; ... ; Bäck, T.H.W. 2022
ObjectiveDistinguishing normal, neuropathic and myopathic electromyography (EMG) traces can be challenging. We aimed to create an automated time series classification algorithm.MethodsEMGs of... Show moreObjectiveDistinguishing normal, neuropathic and myopathic electromyography (EMG) traces can be challenging. We aimed to create an automated time series classification algorithm.MethodsEMGs of healthy controls (HC, n = 25), patients with amyotrophic lateral sclerosis (ALS, n = 20) and inclusion body myositis (IBM, n = 20), were retrospectively selected based on longitudinal clinical follow-up data (ALS and HC) or muscle biopsy (IBM). A machine learning pipeline was applied based on 5-second EMG fragments of each muscle. Diagnostic yield expressed as area under the curve (AUC) of a receiver-operator characteristics curve, accuracy, sensitivity, and specificity were determined per muscle (muscle-level) and per patient (patient-level).ResultsDiagnostic yield of the classification ALS vs. HC was: AUC 0.834 ± 0.014 at muscle-level and 0.856 ± 0.009 at patient-level. For the classification HC vs. IBM, AUC was 0.744 ± 0.043 at muscle-level and 0.735 ± 0.029 at patient-level. For the classification ALS vs. IBM, AUC was 0.569 ± 0.024 at muscle-level and 0.689 ± 0.035 at patient-level.ConclusionsAn automated time series classification algorithm can distinguish EMGs from healthy individuals from those of patients with ALS with a high diagnostic yield. Using longer EMG fragments with different levels of muscle activation may improve performance. Show less
Yarkoni, S.; Raponi, E.; Bäck, T.H.W.; Schmitt, S. 2022
Real-world optimization scenarios under uncertainty and noise are typically handled with robust optimization techniques, which re-formulate the original optimization problem into a robust... Show moreReal-world optimization scenarios under uncertainty and noise are typically handled with robust optimization techniques, which re-formulate the original optimization problem into a robust counterpart, e.g., by taking an average of the function values over different perturbations to a specific input. Solving the robust counterpart instead of the original problem can significantly increase the associated computational cost, which is often overlooked in the literature to the best of our knowledge. Such an extra cost brought by robust optimization might depend on the problem landscape, the dimensionality, the severity of the uncertainty, and the formulation of the robust counterpart.This paper targets an empirical approach that evaluates and compares the computational cost brought by different robustness formulations in Kriging-based optimization on a wide combination (300 test cases) of problems, uncertainty levels, and dimensions. We mainly focus on the CPU time taken to find robust solutions, and choose five commonly-applied robustness formulations: `"mini-max robustness'', "mini-max regret robustness'', "expectation-based robustness'', ``dispersion-based robustness'', and "composite robustness'' respectively. We assess the empirical performance of these robustness formulations in terms of a fixed budget and a fixed target analysis, from which we find that "mini-max robustness'' is the most practical formulation w.r.t.~the associated computational cost. Show less
Manuel Proenca, H.; Grünwald, P.D.; Bäck, T.H.W.; Leeuwen, M. van 2022
We introduce the problem of robust subgroup discovery, i.e., finding a set of interpretable descriptions of subsets that 1) stand out with respect to one or more target attributes, 2) are... Show moreWe introduce the problem of robust subgroup discovery, i.e., finding a set of interpretable descriptions of subsets that 1) stand out with respect to one or more target attributes, 2) are statistically robust, and 3) non-redundant. Many attempts have been made to mine either locally robust subgroups or to tackle the pattern explosion, but we are the first to address both challenges at the same time from a global modelling perspective. First, we formulate the broad model class of subgroup lists, i.e., ordered sets of subgroups, for univariate and multivariate targets that can consist of nominal or numeric variables, including traditional top-1 subgroup discovery in its definition. This novel model class allows us to formalise the problem of optimal robust subgroup discovery using the Minimum Description Length (MDL) principle, where we resort to optimal Normalised Maximum Likelihood and Bayesian encodings for nominal and numeric targets, respectively. Second, finding optimal subgroup lists is NP-hard. Therefore, we propose SSD++, a greedy heuristic that finds good subgroup lists and guarantees that the most significant subgroup found according to the MDL criterion is added in each iteration. In fact, the greedy gain is shown to be equivalent to a Bayesian one-sample proportion, multinomial, or t-test between the subgroup and dataset marginal target distributions plus a multiple hypothesis testing penalty. Furthermore, we empirically show on 54 datasets that SSD++ outperforms previous subgroup discovery methods in terms of quality, generalisation on unseen data, and subgroup list size. Show less
Winter, R. de; Bronkhorst, P.; Stein, B. van; Bäck, T.H.W. 2022
This paper proposes the Self-Adaptive algorithm for Multi-Objective Constrained Optimization by using Radial Basis Function Approximations, SAMO-COBRA. This algorithm automatically determines the... Show moreThis paper proposes the Self-Adaptive algorithm for Multi-Objective Constrained Optimization by using Radial Basis Function Approximations, SAMO-COBRA. This algorithm automatically determines the best Radial Basis Function-fit as surrogates for the objectives as well as the constraints, to find new feasible Pareto-optimal solutions. SAMO-COBRA is compared to a wide set of other state-of-the-art algorithms (IC-SA-NSGA-II, SA-NSGA-II, NSGA-II, NSGA-III, CEGO, SMES-RBF) on 18 constrained multi-objective problems. In the first experiment, SAMO-COBRA outperforms the other algorithms in terms of achieved Hypervolume (HV) after being given a fixed small evaluation budget on the majority of test functions. In the second experiment, SAMO-COBRA outperforms the majority of competitors in terms of required function evaluations to achieve 95% of the maximum achievable Hypervolume. In addition to academic test functions, SAMO-COBRA has been applied on a real-world ship design optimization problem with three objectives, two complex constraints, and five decision variables. Show less
Genetic algorithms have unique properties which are useful when applied to black-box optimization. Using selection, crossover, and mutation operators, candidate solutions may be obtained without... Show moreGenetic algorithms have unique properties which are useful when applied to black-box optimization. Using selection, crossover, and mutation operators, candidate solutions may be obtained without the need to calculate a gradient. In this work, we study results obtained from using quantum-enhanced operators within the selection mechanism of a genetic algorithm. Our approach frames the selection process as a minimization of a binary quadratic model with which we encode fitness and distance between members of a population, and we leverage a quantum annealing system to sample low-energy solutions for the selection mechanism. We benchmark these quantum-enhanced algorithms against classical algorithms over various black-box objective functions, including the OneMax function, and functions from the IOHProfiler library for black-box optimization. We observe a performance gain in the average number of generations to convergence for the quantum-enhanced elitist selection operator in comparison to classical on the OneMax function. We also find that the quantum-enhanced selection operator with ∗Corresponding author email: David.VonDollen@vw.com non-elitist selection outperforms benchmarks on functions with fitness perturbation from the IOHProfiler library. Additionally, we find that in the case of elitist selection, the quantum-enhanced operators outperform classical benchmarks on functions with varying degrees of dummy variables and neutrality Show less
Bayesian optimization is often used to optimize expensive black box optimization problems with long simulation times. Typically Bayesian optimization algorithms propose one solution per iteration.... Show moreBayesian optimization is often used to optimize expensive black box optimization problems with long simulation times. Typically Bayesian optimization algorithms propose one solution per iteration. The downside of this strategy is the sub-optimal use of available computing power. To efficiently use the available computing power (or a number of licenses etc.) we introduce a multi-point acquisition function for parallel efficient multi-objective optimization algorithms. The multi-point acquisition function is based on the hypervolume contribution of multiple solutions simultaneously, leading to well spread solutions along the Pareto frontier. By combining this acquisition function with a constraint handling technique, multiple feasible solutions can be proposed and evaluated in parallel every iteration. The hypervolume and feasibility of the solutions can easily be estimated by using multiple cheap radial basis functions as surrogates with different configurations. The acquisition function can be used with different population sizes and even for one shot optimization. The strength and generalizability of the new acquisition function is demonstrated by optimizing a set of black box constraint multi-objective problem instances. The experiments show a huge time saving factor by using our novel multi-point acquisition function, while only marginally worsening the hypervolume after the same number of function evaluations. Show less
This work introduces RADIUS, a framework for anomaly detection in sewer pipes using stereovision. The framework employs three-dimensional geometry reconstruction from stereo vision, followed by... Show moreThis work introduces RADIUS, a framework for anomaly detection in sewer pipes using stereovision. The framework employs three-dimensional geometry reconstruction from stereo vision, followed by statistical modeling of the geometry with a generic pipe model. The framework is designed to be compatible with existing workflows for sewer pipe defect detection, as well as to provide opportunities for machine learning implementations in the future. We test the framework on 48 image sets of 26 sewer pipes in different conditions collected in the lab. Of these 48 image sets, 5 could not be properly reconstructed in three dimensions due to insufficient stereo matching. The surface fitting and anomaly detection performed well: a human-graded defect severity score had a moderate, positive Pearson correlation of 0.65 with our calculated anomaly scores, making this a promising approach to automated defect detection in urban drainage. Show less
Kefalas, M.; Stein, B. van; Baratchi, M.; Apostolidis, A.; Bäck, T.H.W. 2022
In optimization approaches to engineering applications, time-consuming simulations are often utilized that can be configured to deliver solutions for various fidelity (accuracy) levels. It is... Show moreIn optimization approaches to engineering applications, time-consuming simulations are often utilized that can be configured to deliver solutions for various fidelity (accuracy) levels. It is common practice to train hierarchical surrogate models on objective functions in order to speed up the optimization process. These operate under the assumption that there is a correlation between the different fidelities that can be exploited to gain information cheaply. However, limited guidelines are available to help divide the available computational budget between multiple fidelities in practice. This article evaluates a range of different choices for a two-fidelity setup that provide helpful intuitions about this trade-off. An heuristic method is presented based on subsampling from an initial Design of Experiments to find a suitable division of the computational budget between the fidelity levels. This enables the setting up of multi-fidelity optimizations that utilize the available computational budget efficiently, independently of the multi-fidelity model used. Show less
