Time series forecasting using dynamic particle swarm optimizer trained neural networks

Abstract

Time series forecasting is a very important research area because of its practical application in many elds. Due to the importance of time series forecasting, much research e ort has gone into the development of forecasting models and in improving prediction accuracies. The interest in using arti cial neural networks (NNs) to model and forecast time series has been growing. The most popular type of NN is arguably the feedforward NN (FNN). FNNs have structures capable of learning static input-output mappings, suitable for prediction of non-linear stationary time series. To model nonstationary time series, recurrent NNs (RNNs) are often used. The recurrent/delayed connections in RNNs give the network dynamic properties to e ectively handle temporal sequences. These recurent/delayed connections, however, increase the number of weights that are required to be optimized during training of the NN. Particle swarm optimization (PSO) is an e cient population based search algorithm based on the social dynamics of group interactions in bird ocks. Several studies have applied PSO to train NNs for time series forecasting, and the results indicated good performance on stationary time series, and poor performance on non-stationary and highly noisy time series. These studies have assumed static environments, making the original PSO, which was designed for static environments, unsuitable for training NNs for forecasting many real-world time series generated by non-stationary processes. In dealing with non-stationary data, modi ed versions of PSOs for optimization in dynamic environments are used. These dynamic PSOs are yet to be applied to train NNs on forecasting problems. The rst part of this thesis formulates training of a FNN forecaster as a dynamic optimization problem, to investigate the application of a dynamic PSO algorithm to train FNNs in forecasting time series in non-stationary environments. For this purpose, a set of experiments were conducted on ten forecasting problems under nine di erent dynamic scenarios. Results obtained are compared to the results of FNNs trained using a standard PSO and resilient backpropagation (RPROP). The results show that the dynamic PSO algorithm outperform the PSO and RPROP algorithms. These ndings highlight the potential of using dynamic PSO in training FNNs for real-world forecasting applications. The second part of the thesis tests the hypothesis that recurrent/delayed connections are not necessary if a dynamic PSO is used as the training algorithm. For this purpose, set of experiments were carried out on the same problems and under the same dynamic scenarios. Each experiment involves training a FNN using a dynamic PSO algorithm, and comparing the result to that obtained from four di erent types of RNNs (i.e. Elman NN, Jordan NN, Multi-Recurrent NN and Time Delay NN), each trained separately using RPROP, standard PSO and the dynamic PSO algorithm. The results show that the FNNs trained with the dynamic PSO signi cantly outperform all the RNNs trained using any of the algorithms considered. These ndings show that recurrent/delayed connections are not necessary in NNs used for time series forecasting (for the time series considered in this study) as long as a dynamic PSO algorithm is used as the training method.