Abstract
Extreme Learning Machine (ELM) is a popular method in machine learning with extremely few parameters, fast learning speed and model efficiency. While a significant drawback is that ELM is restricted by its single-layer structure and prized analytic solution. If simply stacking more layers, analytic solution of ELM will be intractable. Then gradient-based optimization method is preferred and that results into normal neural networks. Recently a multi-layer ELM (ML-ELM) is proposed to learn compact feature with a series of ELM auto-encoders, which attempts to extend ELM to a deeper network without sacrificing elegant solution. Compared with ML-ELM and following hierarchical ELM, we introduce a sparse Bayesian learning method to imply a stronger sparse regularization and prune network structure. Experiments on classification verify the efficiency of our proposed new multi-layer ELM for unsupervised feature learning.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)
Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 1, pp. 886–893, 25 June 2005. IEEE (2005)
Huang, G.B., Zhou, H., Ding, X., Zhang, R.: Extreme learning machine for regression and multiclass classification. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 42(2), 513–529 (2012)
Kasun, L.L., Zhou, H., Huang, G.B., Vong, C.M.: Representational learning with extreme learning machine for big data. IEEE Intell. Syst. 28(6), 31–34 (2013)
Kasun, L.L., Yang, Y., Huang, G.B., Zhang, Z.: Dimension reduction with extreme learning machine. IEEE Trans. Image Process. 25(8), 3906–3918 (2016)
Tang, J., Deng, C., Huang, G.B.: Extreme learning machine for multilayer perceptron. IEEE Trans. Neural Netw. Learn. Syst. 27(4), 809–821 (2016)
Yoo, Y., Oh, S.Y.: Fast training of convolutional neural network classifiers through extreme learning machines. In: 2016 International Joint Conference on Neural Networks (IJCNN), 24 July, pp. 1702–1708. IEEE (2016)
Wang, Y., Xie, Z., Xu, K., Dou, Y., Lei, Y.: An efficient and effective convolutional auto-encoder extreme learning machine network for 3D feature learning. Neurocomputing 22(174), 988–998 (2016)
Hoerl, A.E., Kennard, R.W.: Ridge regression: biased estimation for nonorthogonal problems. Technometrics 12(1), 55–67 (1970)
Huang, G.B.: An insight into extreme learning machines: random neurons, random features and kernels. Cogn. Comput. 6(3), 376–390 (2014)
Low, C.Y., Teoh, A.B.: Stacking-based deep neural network: deep analytic network on convolutional spectral histogram features. In: 2017 IEEE International Conference on Image Processing (ICIP), 17 September 2017, pp. 1592–1596. IEEE (2017)
Soria-Olivas, E., Gomez-Sanchis, J., Martin, J.D., Vila-Frances, J., Martinez, M., Magdalena, J.R., Serrano, A.J.: BELM: Bayesian extreme learning machine. IEEE Trans. Neural Netw. 22(3), 505–509 (2011)
Luo, J., Vong, C.M., Wong, P.K.: Sparse Bayesian extreme learning machine for multi-classification. IEEE Trans. Neural Netw. Learn. Syst. 25(4), 836–843 (2014)
Congdon, P.: Bayesian Statistical Modelling. Wiley, New York (2007)
Zhang, N., Ding, S., Shi, Z.: Denoising Laplacian multi-layer extreme learning machine. Neurocomputing 1(171), 1066–1074 (2016)
TU Eindhoven, Leiden University. https://www.openml.org/. Accessed 30 Nov 2018
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, G., Cui, D., Mao, S., Huang, GB. (2020). Sparse Bayesian Learning for Extreme Learning Machine Auto-encoder. In: Cao, J., Vong, C., Miche, Y., Lendasse, A. (eds) Proceedings of ELM 2018. ELM 2018. Proceedings in Adaptation, Learning and Optimization, vol 11. Springer, Cham. https://doi.org/10.1007/978-3-030-23307-5_34
Download citation
DOI: https://doi.org/10.1007/978-3-030-23307-5_34
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-23306-8
Online ISBN: 978-3-030-23307-5
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)