Abstract
It is well-known that single-hidden-layer feedforward networks (SLFNs) with additive models are universal approximators. However the training of these models was slow until the birth of extreme learning machine (ELM) “Huang et al. Neurocomputing 70(1–3):489–501 (2006)” and its later improvements. Before ELM, the faster algorithms for efficiently training SLFNs were gradient based ones which need to be applied iteratively until a proper model is obtained. This slow convergence implies that SLFNs are not used as widely as they could be, even taking into consideration their overall good performances. The ELM allowed SLFNs to become a suitable option to classify a great number of patterns in a short time. Up to now, the hidden nodes were randomly initiated and tuned (though not in all approaches). This paper proposes a deterministic algorithm to initiate any hidden node with an additive activation function to be trained with ELM. Our algorithm uses the information retrieved from principal components analysis to fit the hidden nodes. This approach considerably decreases computational cost compared to later ELM improvements and overcomes their performance.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Asuncion A, Newman D (2007) UCI machine learning repository. http://www.ics.uci.edu/~mlearn/MLRepository.html. Accessed 8 Sept 2007
Cao J, Lin Z, Huang G (2011) Composite function wavelet neural networks with differential evolution and extreme learning machine. Neural Process Lett 33(3): 251–265
Chen L, Zhou L, Pung HK (2008) Universal approximation and qos violation application of extreme learning machine. Neural Process Lett 28(2): 81–95
Dunn OJ (1961) Multiple comparisons among means. J Am Stat Assoc 56: 52–56
Feng G, Huang GB, Lin Q, Gay R (2009) Error minimized extreme learning machine with growth of hidden nodes and incremental learning. IEEE Trans Neural Netw 20(8): 1352–1357
Friedman M (1940) A comparison of alternative tests of significance for the problem of m rankings. Ann Math Stat 11(1): 86–92
Hochberg Y, Tamhane A (1987) Multiple comparison procedures. Wiley, New York
Huang GB, Chen L (2007) Convex incremental extreme learning machine. Neurocomputing 70(16–18): 3056–3062
Huang GB, Chen L (2008) Enhanced random search based incremental extreme learning machine. Neurocomputing 71(16–18): 3460–3468
Huang GB, Chen L, Siew CK (2006) Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans Neural Netw 17:4
Huang GB, Li MB, Chen L, Siew CK (2008) Incremental extreme learning machine with fully complex hidden nodes. Neurocomputing 71(4–6): 576–583
Huang GB, Zhou H, Ding X, Zhang R (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern B 42(2): 513–529
Huang GB, Zhu Q, Siew C (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1–3): 489–501
Huang GB, Zhu QY, Siew CK (2004) Extreme learning machine: a new learning scheme of feedforward neural networks. IEEE Int Conf Neural Netw Conf Proc 2: 985–990
Jaeger H (2001) The “echo state” approach to analysing and training recurrent neural networks. GMD Rep 148: 1435–2702
Kim J, Shin H, Lee Y, Lee M (2007) Algorithm for classifying arrhythmia using extreme learning machine and principal component analysis. In: 29th Annual international conference of the IEEE, engineering in medicine and biology society, 2007. EMBS, New York, pp 3257–3260
Mich Y, Sorjamaa A, Bas P, Simula O, Jutten C, Lendasse A (2010) OP-ELM: optimally pruned extreme learning machine. IEEE Trans Neural Netw 21(1): 158–162
Miche Y, Sorjamaa A, Lendasse A (2008) Op-elm: theory, experiments and a toolbox. In: Artificial neural networks—ICANN 2008, lecture notes in computer science, vol 5163. Springer, Berlin, pp 145–154
Rong HJ, Ong YS, Tan AH, Zhu Z (2008) A fast pruned-extreme learning machine for classification problem. Neurocomputing 72(1–3): 359–366
Sánchez-Monedero J, Gutiérrez PA, Fernández-Navarro F, Hervás-Martínez C (2011) Weighting efficient accuracy and minimum sensitivity for evolving multi-class classifiers. Neural Process Lett 34(2): 101–116
Schlkopf B, Smola AJ, Müller KR (1999) Kernel principal component analysis. In: Advances in kernel methods: support vector learning. MIT Press, Cambridge, pp 327–352
Storn R, Price K. (1997) Differential evolution: a fast and efficient heuristic for global optimization over continuous spaces. J Glob Optim 11: 341–359
Vapnik VN (1999) The nature of statistical learning theory. Springer, Berlin
Zhang R, Huang GB, Sundararajan N, Saratchandran P (2007) Multicategory classification using an extreme learning machine for microarray gene expression cancer diagnosis. Comput Biol Bioinform IEEE/ACM Trans 4(3): 485–495
Zhu QY, Qin A, Suganthan P, Huang GB (2005) Evolutionary extreme learning machine. Pattern Recognit 38(10): 1759–1763
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Castaño, A., Fernández-Navarro, F. & Hervás-Martínez, C. PCA-ELM: A Robust and Pruned Extreme Learning Machine Approach Based on Principal Component Analysis. Neural Process Lett 37, 377–392 (2013). https://doi.org/10.1007/s11063-012-9253-x
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11063-012-9253-x