Abstract
Trust region methods are a class of effective iterative schemes in numerical optimization. In this paper, a new improved nonmonotone adaptive trust region method for solving unconstrained optimization problems is proposed. We construct an approximate model where the approximation to Hessian matrix is updated by the scaled memoryless BFGS update formula, and incorporate a nonmonotone technique with the new proposed adaptive trust region radius. The new ratio to adjusting the next trust region radius is different from the ratio in the traditional trust region methods. Under some suitable and standard assumptions, it is shown that the proposed algorithm possesses global convergence and superlinear convergence. Numerical results demonstrate that the proposed method is very promising.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
M. Ahookhosh, K. Amini: A nonmonotone trust region method with adaptive radius for unconstrained optimization problems. Comput. Math. Appl. 60 (2010), 411–422.
M. Ahookhosh, K. Amini: An efficient nonmonotone trust-region method for unconstrained optimization. Numer. Algorithms 59 (2012), 523–540.
N. Andrei: An unconstrained optimization test functions collection. Adv. Model. Optim. 10 (2008), 147–161.
N. Andrei: Accelerated scaled memoryless BFGS preconditioned conjugate gradient algorithm for unconstrained optimization. Eur. J. Oper. Res. 204 (2010), 410–420.
R.M. Chamberlain, M. J.D. Powell, C. Lemarechal, H.C. Pedersen: The watchdog technique for forcing convergence in algorithms for constrained optimization. Math. Program. Study 16 (1982), 1–17.
A.R. Conn, N. I.M. Gould, P. L. Toint: Trust-Region Methods. MPS/SIAM Series on Optimization, Society for Industrial and Applied Mathematics; Mathematical Programming Society, Philadelphia, 2000.
N.Y. Deng, Y. Xiao, F. J. Zhou: Nonmonotonic trust region algorithm. J. Optimization Theory Appl. 76 (1993), 259–285.
E.D. Dolan, J. J. Moré: Benchmarking optimization software with performance profiles. Math. Program. 91 (2002), 201–213.
J.-Y. Fan, Y.-X. Yuan: A new trust region algorithm with trust region radius converging to zero. Proceedings of the 5th International Conference on Optimization: Techniques and Applications. Hong Kong, 2001, pp. 786–794.
N. I.M. Gould, D. Orban, P. L. Toint: CUTEr and SifDec: a constrained and unconstrained testing environment, revisited. ACM Trans. Math. Softw. 29 (2003), 373–394.
L. Grippo, F. Lampariello, S. Lucidi: A nonmonotone line search technique for Newton’s method. SIAM J. Numer. Anal. 23 (1986), 707–716.
A. Kamandi, K. Amini, M. Ahookhosh: An improved adaptive trust-region algorithm. Optim. Lett. 11 (2017), 555–569.
D. Li, M. Fukushima: A modified BFGS method and its global convergence in nonconvex minimization. J. Comput. Appl. Math. 129 (2001), 15–35.
N. Maratos: Exact penalty function algorithms for finite dimensional and control optimization problems. Ph.D. Thesis, University of London, London, 1978.
D.W. Marquardt: An algorithm for least-squares estimation of nonlinear parameters. J. Soc. Ind. Appl. Math. 11 (1963), 431–441.
M.R. Peyghami, D.A. Tarzanagh: A relaxed nonmonotone adaptive trust region method for solving unconstrained optimization problems. Comput. Optim. Appl. 61 (2015), 321–341.
M. J.D. Powell: A new algorithm for unconstrained optimization. Nonlinear Programming, Proceedings of a Symposium Conducted by the Mathematics Research Center. Academic Press, New York, 1970, pp. 31–65.
S. Rezaee, S. Babaie-Kafaki: An adaptive nonmonotone trust region algorithm. Optim. Methods Softw. 34 (2019), 264–277.
Z.-J. Shi, J. Guo: A new trust region method with adaptive radius. Comput. Optim. Appl. 41 (2008), 225–242.
W. Sun: Nonmonotone trust region method for solving optimization problems. Appl. Math. Comput. 156 (2004), 159–174.
W. Sun, Y. Yuan: Optimization Theory and Methods. Nonlinear Programming. Springer Optimization and Its Applications 1, Springer, New York, 2006.
D. Winfield: Function minimization by interpolation in a data table. J. Inst. Math. Appl. 12 (1973), 339–347.
J.-L. Zhang, X.-S. Zhang: A nonmonotone adaptive trust region method and its convergence. Comput. Math. Appl. 45 (2003), 1469–1477.
X. Zhang, J. Zhang, L. Liao: An adaptive trust region method and its convergence. Sci. China, Ser. A 45 (2002), 620–631.
Author information
Authors and Affiliations
Corresponding authors
Additional information
This research is supported by National Science Foundation of China (No. 11461021), Shaanxi Science Foundation (No. 2017JM1014), and Guangxi Science Foundation (Nos. 2018GXNSFBA281180).
Rights and permissions
About this article
Cite this article
Xue, Y., Liu, H. & Liu, Z. An improved nonmonotone adaptive trust region method. Appl Math 64, 335–350 (2019). https://doi.org/10.21136/AM.2019.0138-18
Received:
Published:
Issue Date:
DOI: https://doi.org/10.21136/AM.2019.0138-18
Keywords
- unconstrained optimization
- trust region method
- scaled memoryless BFGS update
- nonmonotone technique
- global convergence