Spatially distributed statistical significance approach for real parameter tuning with restricted budgets

Abstract

Parameter tuning aims to find suitable parameter values for heuristic optimisation algorithms that allows for the practical application of such algorithms. Conventional tuning approaches view the tuning problem as two distinct problems, namely, a stochastic problem to quantify the performance of a parameter vector and a deterministic problem for finding improved parameter vectors in the meta-design space. A direct consequence of this viewpoint is that parameter vectors are sampled multiple times to resolve their respective performance uncertainties. In this study we share an alternative viewpoint, which is to consider the tuning problem as a single stochastic problem for which both the spatial location and performance of the optimal parameter vector are uncertain. A direct implication, of this alternative stance, is that every parameter vector is sampled only once. In our proposed approach, the spatial and performance uncertainties of the optimal parameter vector are resolved by the spatial clustering of candidate parameter vectors in the meta-design space. In a series of numerical experiments, considering 16 test problems, we show that our approach, Efficient Sequential Parameter Optimisation (ESPO), outperforms both F/Race and Sequential Parameter Optimisation (SPO), especially for tuning under restricted budgets.