Leiden University Scholarly Publications

Persistent URL of this record https://hdl.handle.net/1887/3246961

Documents

In Collections

This item can be found in the following collections:

Nguyen, D.A.; Kong, J.; Wang, H.; Menzel, S.; Sendhoff, B.; Kononova, A.V.; Bäck, T.H.W. (2021)

Improved automated CASH optimization with tree parzen estimators for class imbalance problems

Article in monograph or in proceedings
The imbalanced classification problem is very relevant in both academic and industrial applications. The task of finding the best machine learning model to use for a specific imbalanced dataset is complicated due to a large number of existing algorithms, each with its own hyperparameters. The Combined Algorithm Selection and Hyperparameter optimization (CASH) has been introduced to tackle both aspects at the same time. However, CASH has not been studied in detail in the class imbalance domain, where the best combination of resampling technique and classification algorithm is searched for, together with their optimized hyperparameters. Thus, we target the CASH problem for imbalanced classification. We experiment with a search space of 5 classification algorithms, 21 resampling approaches and 64 relevant hyperparameters in total. Moreover, we investigate performance of 2 well-known optimization approaches: Random search and Tree Parzen Estimators approach which is a kind of...Show moreThe imbalanced classification problem is very relevant in both academic and industrial applications. The task of finding the best machine learning model to use for a specific imbalanced dataset is complicated due to a large number of existing algorithms, each with its own hyperparameters. The Combined Algorithm Selection and Hyperparameter optimization (CASH) has been introduced to tackle both aspects at the same time. However, CASH has not been studied in detail in the class imbalance domain, where the best combination of resampling technique and classification algorithm is searched for, together with their optimized hyperparameters. Thus, we target the CASH problem for imbalanced classification. We experiment with a search space of 5 classification algorithms, 21 resampling approaches and 64 relevant hyperparameters in total. Moreover, we investigate performance of 2 well-known optimization approaches: Random search and Tree Parzen Estimators approach which is a kind of Bayesian optimization. For comparison, we also perform grid search on all combinations of resampling techniques and classification algorithms with their default hyperparameters. Our experimental results show that a Bayesian optimization approach outperforms the other approaches for CASH in this application domain.
Show less
All authors
Nguyen, D.A.; Kong, J.; Wang, H.; Menzel, S.; Sendhoff, B.; Kononova, A.V.; Bäck, T.H.W.
Date
2021
Title of host publication
2021 IEEE 8th international conference on data science and advanced analytics (DSAA)
Pages
1 - 9
ISBN (print)
9781665421003
ISBN (electronic)
9781665420990
DOI
doi:10.1109/DSAA53316.2021.9564147

Conference

Conference
IEEE 8th International Conference on Data Science and Advanced Analytics (DSAA)
Date
2021-10-06 - 2021-12-09
Location
Porto, Portugal

Funding

Sponsorship
Horizon 2020(H2020)
Grant number
766186