Abstract
This paper presents a novel Quasi-Newton method for the minimization of the error function of a feed-forward neural network. The method is a generalization of Battiti’s well known OSS algorithm. The aim of the proposed approach is to achieve a significant improvement both in terms of computational effort and in the capability of evaluating the global minimum of the error function. The technique described in this work is founded on the innovative concept of “convex algorithm” in order to avoid possible entrapments into local minima. Convergence results as well numerical experiences are presented.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Al Baali, M.: Improved Hessian approximations for the limited memory BFGS method. Numer. Algorithms 22, 99–112 (1999)
Battiti, R.: First- and second-order methods for learning: between steepest descent and Newton’s method. Neural Computation 4, 141–166 (1992)
Bianchini, M., Fanelli, S., Gori, M., Protasi, M.: Non-suspiciousness: a generalization of convexity in the frame of foundations of Numerical Analysis and Learning. In: IJCNN 1998, Anchorage, vol. II, pp. 1619–1623 (1998)
Bianchini, M., Fanelli, S., Gori, M.: Optimal algorithms for well-conditioned nonlinear systems of equations. IEEE Transactions on Computers 50, 689–698 (2001)
Bortoletti, A., Di Fiore, C., Fanelli, S., Zellini, P.: A new class of quasi-newtonian methods for optimal learning in MLP-networks. IEEE Transactions on Neural Networks 14, 263–273 (2003)
Di Fiore, C., Fanelli, S., Zellini, P.: Matrix algebras in quasi-newtonian algorithms for optimal learning in multi-layer perceptrons. In: ICONIP Workshop and Expo, Dunedin, pp. 27–32 (1999)
Di Fiore, C., Fanelli, S., Zellini, P.: Optimisation strategies for nonconvex functions and applications to neural networks. In: ICONIP 2001, Shanghai, vol. 1, pp. 453–458 (2001)
Di Fiore, C., Fanelli, S., Zellini, P.: Computational experiences of a novel algorithm for optimal learning in MLP-networks. In: ICONIP 2002, Singapore, vol. 1, pp. 317–321 (2002)
Di Fiore, C., Fanelli, S., Lepore, F., Zellini, P.: Matrix algebras in Quasi-Newton methods for unconstrained optimization. Numerische Mathematik 94, 479–500 (2003)
Di Fiore, C., Fanelli, S., Zellini, P.: Convex algorithms for optimal learning in MLPnetworks. In: Preparation
Duda, R.O., Hart, P.E.: Pattern Classification and Scene Analysis. Wiley, Chichester (1973)
Frasconi, P., Fanelli, S., Gori, M., Protasi, M.: Suspiciousness of loading problems. IEEE Int. Conf. on Neural Networks 2, 1240–1245 (1997)
Liu, D.C., Nocedal, J.: On the limited memory BFGS method for large scale optimization. Math. Programming 45, 503–528 (1989)
Nocedal, J., Wright, S.J.: Numerical Optimization. Springer, New York (1999)
http://www.mathworks.com/access/helpdesk/help/toolbox/nnet/backpr14.html
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Di Fiore, C., Fanelli, S., Zellini, P. (2004). An Efficient Generalization of Battiti-Shanno’s Quasi-Newton Algorithm for Learning in MLP-Networks. In: Pal, N.R., Kasabov, N., Mudi, R.K., Pal, S., Parui, S.K. (eds) Neural Information Processing. ICONIP 2004. Lecture Notes in Computer Science, vol 3316. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30499-9_74
Download citation
DOI: https://doi.org/10.1007/978-3-540-30499-9_74
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-23931-4
Online ISBN: 978-3-540-30499-9
eBook Packages: Springer Book Archive