The conflict between computational budget and quality of found solutions is crucial when dealing with expensive black-box optimization problems from the industry. We show that through multi... Show moreThe conflict between computational budget and quality of found solutions is crucial when dealing with expensive black-box optimization problems from the industry. We show that through multi-objective parameter tuning of the Covariance Matrix Adaptation Evolution Strategy on benchmark functions different optimal algorithm configurations can be found for specific computational budgets and solution qualities. With the obtained Pareto front, tuned parameter sets are selected and transferred to a real-world optimization problem from vehicle dynamics, improving the solution quality and budget needed. The benchmark functions for tuning are selected based on their similarity to a real-world problem in terms of Exploratory Landscape Analysis features. Show less
Saturation is considered the state-of-the-art method for computing fixpoints with decision diagrams. We present a relatively simple decision diagram operation called REACH that also computes... Show moreSaturation is considered the state-of-the-art method for computing fixpoints with decision diagrams. We present a relatively simple decision diagram operation called REACH that also computes fixpoints. In contrast to saturation, it does not require a partitioning of the transition relation. We give sequential algorithms implementing the new operation for both binary and multi-valued decision diagrams, and moreover provide parallel counterparts. We implement these algorithms and experimentally compare their performance against saturation on 692 model checking benchmarks in different languages. The results show that the REACH operation often outperforms saturation, especially on transition relations with low locality. In a comparison between parallelized versions of REACH and saturation we find that REACH obtains comparable speedups up to 16 cores, although falls behind saturation at 64 cores. Finally, in a comparison with the state-of-the-art model checking tool ITS-tools we find that REACH outperforms ITS-tools on 29% of models, suggesting that REACH can be useful as a complementary method in an ensemble tool. Show less
Tannemaat, M.R.; Kefalas, M.; Geraedts, V.J.; Remijn-Nelissen, L.; Verschuuren, A.J.M.; Koch, M.; ... ; Bäck, T.H.W. 2023
OBJECTIVE\nMETHODS\nRESULTS\nCONCLUSIONS\nSIGNIFICANCE\nDistinguishing normal, neuropathic and myopathic electromyography (EMG) traces can be challenging. We aimed to create an automated time... Show moreOBJECTIVE\nMETHODS\nRESULTS\nCONCLUSIONS\nSIGNIFICANCE\nDistinguishing normal, neuropathic and myopathic electromyography (EMG) traces can be challenging. We aimed to create an automated time series classification algorithm.\nEMGs of healthy controls (HC, n = 25), patients with amyotrophic lateral sclerosis (ALS, n = 20) and inclusion body myositis (IBM, n = 20), were retrospectively selected based on longitudinal clinical follow-up data (ALS and HC) or muscle biopsy (IBM). A machine learning pipeline was applied based on 5-second EMG fragments of each muscle. Diagnostic yield expressed as area under the curve (AUC) of a receiver-operator characteristics curve, accuracy, sensitivity, and specificity were determined per muscle (muscle-level) and per patient (patient-level).\nDiagnostic yield of the classification ALS vs. HC was: AUC 0.834 ± 0.014 at muscle-level and 0.856 ± 0.009 at patient-level. For the classification HC vs. IBM, AUC was 0.744 ± 0.043 at muscle-level and 0.735 ± 0.029 at patient-level. For the classification ALS vs. IBM, AUC was 0.569 ± 0.024 at muscle-level and 0.689 ± 0.035 at patient-level.\nAn automated time series classification algorithm can distinguish EMGs from healthy individuals from those of patients with ALS with a high diagnostic yield. Using longer EMG fragments with different levels of muscle activation may improve performance.\nIn the future, machine learning algorithms may help improve the diagnostic accuracy of EMG examinations. Show less
Many state-of-the-art evolutionary algoritahms (EAs) can be categorized as sequential hybrid EAs, in which various EAs are sequentially executed. The timing to switch from one EA to another is... Show moreMany state-of-the-art evolutionary algoritahms (EAs) can be categorized as sequential hybrid EAs, in which various EAs are sequentially executed. The timing to switch from one EA to another is critical to the performance of the hybrid EA because the switching time determines the allocation of computational resources and thereby it helps balance exploration and exploitation. In this article, a framework for adaptive parameter control for hybrid EAs is proposed, in which the switching time is controlled by a learned agent rather than a manually designed scheme. First the framework is applied to an adaptive differential evolution algorithm, LSHADE, to control when to use the scheme to reduce the population. Then the framework is applied to the algorithm that won the CEC 2018 competition, i.e., the hybrid sampling evolution strategy (HSES), to control when to switch from the univariate sampling phase to the Covariance Matrix Adaptation Evolution Straategy phase. The agents for parameter control in LSHADE and HSES are trained by using Q-learning and deep Q-learning to obtain the learned algorithms Q-LSHADE and DQ-HSES. The results of experiments on the CEC 2014 and 2018 test suites show that the learned algorithms significantly outperform their counterparts and some state-of-the-art EAs. Show less
The algorithm selection problem is of paramount importance in achieving high-quality results while minimizing computational effort, especially when dealing with expensive black-box optimization... Show moreThe algorithm selection problem is of paramount importance in achieving high-quality results while minimizing computational effort, especially when dealing with expensive black-box optimization problems. In this paper, we address this challenge by using randomly generated artificial functions that mimic the landscape characteristics of the original problem while being inexpensive to evaluate. The similarity between the artificial function and the original problem is quantified using Exploratory Landscape Analysis. We demonstrate a significant performance improvement on five real-world vehicle dynamics problems by transferring the parameters of the Covariance Matrix Adaptation Evolution Strategy tuned to these artificial functions.We provide a complete set of simulated values of braking distance for fully enumerated 2D design spaces of all five real-world optimization problems. So, replication of our results and benchmarking directly on the real-world problems is possible. Beyond the scope of this paper, this data can be used as a benchmarking set for multi-objective optimization with up to five objectives. Show less
The performance of the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) is significantly affected by the selection of the specific CMA-ES variant and the parameter values used. Furthermore,... Show moreThe performance of the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) is significantly affected by the selection of the specific CMA-ES variant and the parameter values used. Furthermore, optimal CMA-ES parameter configurations vary across different problem landscapes, making the task of tuning CMA-ES to a specific optimization problem a challenging mixed-integer optimization problem. In recent years, several advanced algorithms have been developed to address this problem, including the Sequential Model-based Algorithm Configuration (SMAC) and the Tree-structured Parzen Estimator (TPE).In this study, we propose a novel approach for tuning CMA-ES by leveraging CMA-ES itself. Therefore, we combine the modular CMA-ES implementation with the margin extension to handle mixed-integer optimization problems. We show that CMA-ES can not only compete with SMAC and TPE but also outperform them in terms of wall clock time. Show less
A building spatial design (BSD) determines external and internal walls and ceilings of a building. The design space has a hierarchical structure, in which decisions on the existence or non... Show moreA building spatial design (BSD) determines external and internal walls and ceilings of a building. The design space has a hierarchical structure, in which decisions on the existence or non-existence of spatial components determine the existence of variables related to these spaces, such as sizing and angles. In the optimization of BSDs it is envisioned to optimize various performance indicators from multiple disciplines in concert, such as structural, functional, thermal, and daylight performance. Existing representations of design spaces suffer from severe limitations, such as only representing orthogonal designs or representing the structures in parametric superstructure, allowing only for limited design variations. This paper proposes prism nets - a new way of representing the search space of BSDs based on triangulations defining space filling collections of triangular prisms that can be combined via coloring parameters to spaces. Prism nets can accommodate for non-orthogonal designs and are flexible in terms of topological variations. We follow the guidelines for representation and operator design proposed in the framework of metric-based evolutionary algorithms. The main contribution of the paper is a detailed discussion of the search space representation and corresponding mutation operators. Moreover, a proof of concept example demonstrates the integration into multi-objective evolutionary algorithms and provides first results on a simple, but reproducible, benchmark problem. Show less
This paper investigates how often the popular configurations of Differential Evolution generate solutions outside the feasible domain. Following previous publications in the field, we argue that... Show moreThis paper investigates how often the popular configurations of Differential Evolution generate solutions outside the feasible domain. Following previous publications in the field, we argue that what the algorithm does with such solutions and how often this has to happen is important for the overall performance of the algorithm and interpretation of results. Significantly more solutions than what is usually assumed by practitioners have to undergo some sort of 'correction' to conform with the definition of the problem's search domain. A wide range of popular Differential Evolution configurations is considered in this study. Conclusions are made regarding the effect the Differential Evolution components and parameter settings have on the distribution of percentages of infeasible solutions generated in a series of independent runs. Results shown in this study suggest strong dependencies between percentages of generated infeasible solutions and every aspect mentioned above. Further investigation of the distribution of percentages of generated infeasible solutions is required. Show less
Early lifecycle demand forecast is critical to consumer technology products with a fast innovation speed, as firms which compete on these products focus on timely responding to market changes... Show moreEarly lifecycle demand forecast is critical to consumer technology products with a fast innovation speed, as firms which compete on these products focus on timely responding to market changes through new product development and efficient product diffusion, rather than sustaining product sales. The challenge for obtaining an accurate long-range forecast is that sales volumes at the early lifecycle stages are small, which limits the forecast accuracy. We propose a two-step lifecycle forecast approach for consumer technology products with limited sales data. First, we segment products based on market and clustering. Second, we apply the Bass model to aggregated products in a group using the average periodic sales of all products in the group and then use the forecast for related new products. We validate our approach using a dataset collected from Philips Netherlands, which contains consumer healthcare products sold in US and China over an 8-year timespan. The results suggest that for forecasting the lifecycle of a new product, models based on aggregated products generally perform better than models based on an individual product. It highlights the value of data aggregation in product lifecycle forecasts. Clustering is also useful for improving the forecast accuracy: when aggregation is done using sufficient product sales data, the aggregated model based on products with which the new product has the most sales pattern similarities could provide a more accurate forecast than other aggregated models. Based on our results, we provide a practical guideline to firms for obtaining an accurate early product lifecycle forecast. Show less
Introducing new algorithmic ideas is a key part of the continuous improvement of existing optimization algorithms. However, when introducing a new component into an existing algorithm, assessing... Show moreIntroducing new algorithmic ideas is a key part of the continuous improvement of existing optimization algorithms. However, when introducing a new component into an existing algorithm, assessing its potential benefits is a challenging task. Often, the component is added to a default implementation of the underlying algorithm and compared against a limited set of other variants. This assessment ignores any potential interplay with other algorithmic ideas that share the same base algorithm, which is critical in understanding the exact contributions being made. We explore a more extensive procedure, which uses hyperparameter tuning as a means of assessing the benefits of new algorithmic components. This allows for a more robust analysis by not only focusing on the impact on performance, but also by investigating how this performance is achieved. We implement our suggestion in the context of the Modular CMA-ES framework, which was redesigned and extended to include some new modules and several new options for existing modules, mostly focused on the step-size adaptation method. Our analysis highlights the differences between these new modules, and identifies the situations in which they have the largest contribution. Show less
Structural Bias (SB) is an important type of algorithmic deficiency within iterative optimisation heuristics. However, methods for detecting structural bias have not yet fully matured, and recent... Show moreStructural Bias (SB) is an important type of algorithmic deficiency within iterative optimisation heuristics. However, methods for detecting structural bias have not yet fully matured, and recent studies have uncovered many interesting questions. One of these is the question of how structural bias can be related to anisotropy. Intuitively, an algorithm that is not isotropic would be considered structurally biased. However, there have been cases where algorithms appear to only show SB in some dimensions. As such, we investigate whether these algorithms actually exhibit anisotropy, and how this impacts the detection of SB. We find that anisotropy is very rare, and even in cases where it is present, there are clear tests for SB which do not rely on any assumptions of isotropy, so we can safely expand the suite of SB tests to encompass these kinds of deficiencies not found by the original tests.We propose several additional testing procedures for SB detection and aim to motivate further research into the creation of a robust portfolio of tests. This is crucial since no single test will be able to work effectively with all types of SB we identify. Show less
