Abstract
We train the wavelet packet multi-layer perceptron neural network (WP-MLP) by backpropagation for time series prediction. Weights in the backpropagation algorithm are usually initialized with small random values. If the random initial weights happen to be far from a good solution or they are near a poor local optimum, training may take a long time or get trap in the local optimum. Proper weights initialization will place the weights close to a good solution with reduced training time and increased the possibility of reaching a good solution. In this paper, we investigate the effect of weight initialization on WP-MLP using two clustering algorithms. We test the initialization methods on WP-MLP with the sunspots and Mackey-Glass benchmark time series. We show that with proper weight initialization, better prediction performance can be attained.
Chapter PDF
Similar content being viewed by others
Keywords
- Mean Square Error
- Wavelet Packet
- Normalize Mean Square Error
- Hierarchical Cluster Algorithm
- Time Series Prediction
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Reference
Lapedes, A., Farber, R.: Nonlinear Signal Processing Using Neural Network: Prediction and System Modeling. Los Alamos Nat. Lab Tech. Rep, LA-UR-872662 (1987)
Mallat, S. G.: Multifrequency channel decompositions of images and wavelet models. IEEE Transactions on Acoustics, Speech and Signal Processing, Vol. 37,No. 12 (1989) 2091–2110
Mallat, S. G.: A Theory for Multiresolution Signal Decomposition: The Wavelet Representation. IEEE Transactions on Pattern Recognition, Vol. 11,No. 7 (1989) 674–693
Vetterli, M., Herley, C.: Wavelets and filter banks: theory and design. IEEE Transactions on Signal Processing, Vol. 40,No.9 (1992) 2207–2232
Mallat, S. G.: A Wavelet Tour of Signal Processing. Academic Press (1998)
Strang, G., Nguyen, T.: Wavelet and Filter Banks. Wellesly-Cambridge Press (1996)
Zhang, Q., Benveniste, A.: Wavelet Networks. IEEE Transactions on Neural Networks, Vol. 3,No. 6. (1992) 889-898
Kolen, J. F., Pollack, J. B.: Back Propagation is Sensitive to Initial Conditions. Tech. Rep. TR 90-JK-BPSIC, Laboratory for Artifical Intelligence Research, Computer and Information Science Department, (1990)
Coifman, R. R., Meyer, Y., Wickerhauser, M. V.: Entropy-Based Algorithm for Best Basis Selection. IEEE Transactions on Information Theory, Vol. 38,No. 2 Part 2 (1992) 713–718
Vetterli, M., Herley, C.: Wavelets and filter banks: theory and design. IEEE Transactions on Signal Processing, Vol. 40,No.9 (1992) 2207–2232
Hagan, M. T., Menhaj, M. B.: Training Feedforward Networks with the Marquardt Algorithm. IEEE Transactions on Neural Networks, Vol. 5,No. 6 (1994) 989–993
Geva, A. B.: ScaleNet-Multiscale Neural-Network Architecture for Time Series Prediction. IEEE Transactions on Neural Networks, Vol. 9,No. 9 (1998) 1471–1482
Jain, A. K., Murty, M. N., Flynn, P. J.: Data Clustering: A Review. ACM Computing Surveys, Vol. 31,No. 3 (1999) 264–323
Aldenderfer, M. S., Blashfield, R. K.: Cluster Analysis. Sage Publications (1984)
Nielsen, R. H.: Neurocomputing. Addisson-Wesley Publishing Company. (1990)
Wang, L.: On competitive learning. IEEE Transaction on Neural Network, Vol. 8,No. 5 (1997) 1214–1217
Wang, L.: Oscillatory and Chaotic dynamic in neural networks under varying operating conditions. IEEE Transactions on Neural Network, Vol. 7,No. 6 (1996) 1382–1388
Denoeux, T., Lengelle, R.: Initializing back propagation network with prototypes. Neural Computation, Vol. 6 (1993) 351–363
Lehtokangas, M., Saarinen, J., Kaski, K., Huuhtanen, P.: Initialization weights of a multiplayer perceptron by using the orthogonal least squares algorithm. Neural Comput., Vol. 7, (1995) 982–999
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2001 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Teo, K.K., Wang, L., Lin, Z. (2001). Wavelet Packet Multi-layer Perceptron for Chaotic Time Series Prediction: Effects of Weight Initialization. In: Alexandrov, V.N., Dongarra, J.J., Juliano, B.A., Renner, R.S., Tan, C.J.K. (eds) Computational Science - ICCS 2001. ICCS 2001. Lecture Notes in Computer Science, vol 2074. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45718-6_35
Download citation
DOI: https://doi.org/10.1007/3-540-45718-6_35
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-42233-4
Online ISBN: 978-3-540-45718-3
eBook Packages: Springer Book Archive