Abstract
We consider nonsmooth optimization problems whose objective function is defined by the Difference of Convex (DC) functions. With the aim of computing critical points that are also d(irectional)-stationary for such a class of nonconvex programs we propose an algorithmic scheme equipped with an inertial-force procedure. In contrast to the classical DC algorithm of P. D. Tao and L. T. H. An, the proposed inertial DC algorithm defines trial points whose sequence of functional values is not necessary monotonically decreasing, a property that proves useful to prevent the algorithm from converging to a critical point that is not d-stationary. Moreover, our method can handle inexactness in the solution of convex subproblems yielding trial points. This is another property of practical interest that substantially reduces the computational burden to compute d-stationary/critical points of DC programs. Convergence analysis of the proposed algorithm yields global convergence to critical points, and convergence rate is established for the considered class of problems. Numerical experiments on large-scale (nonconvex and nonsmooth) image denoising models show that the proposed algorithm outperforms the classic one in this particular application, specifically in the case of piecewise constant images with neat edges such as QR codes.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Afonso, M.V., Bioucas-Dias, J.M., Figueiredo, M.A.T.: Fast image recovery using variable splitting and constrained optimization. IEEE Trans. Image Process. 19 (9), 2345–2356 (2010)
Alvarez, F.: Weak convergence of a relaxed and inertial hybrid projection-proximal point algorithm for maximal monotone operators in hilbert space. SIAM J. Optim. 14 (3), 773–782 (2004)
Alvarez, F., Attouch, H.: An inertial proximal method for maximal monotone operators via discretization of a nonlinear oscillator with damping. Set-Val. Anal. 9(1), 3–11 (2001)
An, L.T.H., Tao, P.D.: The DC (difference of convex functions) programming and DCA revisited with DC models of real world nonconvex optimization problems. Ann. Oper. Res. 133(1), 23–46 (2005)
Bagirov, A.M., Yearwood, J.: A new nonsmooth optimization algorithm for minimum sum-of-squares clustering problems. Eur. J. Oper. Res. 170(2), 578–596 (2006)
Beck, A., Teboulle, M.: Fast gradient-based algorithms for constrained total variation image denoising and deblurring problems. IEEE Trans. Image Process. 18 (11), 2419–2434 (2009)
Bello Cruz, J.Y.: On proximal subgradient splitting method for minimizing the sum of two nonsmooth convex functions. Set-Val. Var. Anal. 25(2), 245–263 (2017)
Caselles, V., Chambolle, A., Cremers, D., Novaga, M., Pock, T.: An introduction to total variation for image analysis. Theoretical Foundations and Numerical Methods for Sparse Recovery, De Gruyter, Radon Series. Comp. Appl. Math. 9, 263–340 (2010)
Condat, L.: A primal–dual splitting method for convex optimization involving lipschitzian, proximable and linear composite terms. J. Optim. Theory Appl. 158(2), 460–479 (2013)
Correa, R., Lemaréchal, C.: Convergence of some algorithms for convex minimization. Math. Program. 62(2), 261–275 (1993)
de Oliveira, W.: Proximal Bundle Methods for Nonsmooth DC Programming. Technical report. Available at www.oliveira.mat.br (2017)
Dolan, E., Moré, J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002)
Gaudioso, M., Giallombardo, G., Miglionico, G.: Minimizing piecewise-concave functions over polyhedra. Math. Oper. Res. 43(2), 580–597 (2018)
Gaudioso, M., Giallombardo, G., Miglionico, G., Bagirov, A.M.: Minimizing nonsmooth DC functions via successive DC piecewise-affine approximations. J. Glob. Optim. 71, 37–55 (2018)
Gonzalez, R.C., Woods, R.E.: Digital Image Processing, 3rd edn. Prentice-Hall, Inc., Upper Saddle River (2006)
Gruzdeva, T., Strekalovsky, A.: A D.C. Programming Approach to Fractional Problems, pp 331–337. Springer International Publishing, Cham (2017)
Hiriart-Urruty, J.: Optimisation et analyse convexe. Presses Universitaires de France, France (1998)
Hiriart-Urruty, J., Lemaréchal, C.: Convex Analysis and Minimization Algorithms I, 2nd edn. No. 305 in Grundlehren der mathematischen Wissenschaften. Springer, Berlin (1996)
Hiriart-Urruty, J.B.: Generalized Differentiability/Duality and Optimization for Problems Dealing with Differences of Convex Functions, pp 37–70. Springer, Berlin (1985)
Hiriart-Urruty, J.B., Lemaréchal, C.: Convex Analysis and Minimization Algorithms. No. 305-306 in Grund. der math. Wiss. Springer, Berlin (1993). (two volumes)
Holmberg, K., Tuy, H.: A production-transportation problem with stochastic demand and concave production costs. Math. Program. 85(1), 157–179 (1999)
Izmailov, A.F., Solodov, M.V.: Newton-type Methods for Optimization and Variational Problems, 1st edn. Springer Series in Operations Research and Financial Engineering Springer International Publishing (2014)
Joki, K., Bagirov, A.M., Karmitsa, N., Mäkelä, M.M.: A proximal bundle method for nonsmooth dc optimization utilizing nonconvex cutting planes. J. Glob. Optim. 68(3), 501–535 (2017)
Khalaf, W., Astorino, A., d’Alessandro, P., Gaudioso, M.: A DC optimization-based clustering technique for edge detection. Optim. Lett. 11(3), 627–640 (2017)
Lanza, A., Morigi, S., Sgallari, F.: Convex image denoising via non-convex regularization with parameter selection. J. Math. Imaging Vis. 56(2), 195–220 (2016)
Le Thi, H.A., Pham Dinh, T.: Dc programming in communication systems: challenging problems and methods. Vietnam J. Comput. Sci. 1(1), 15–28 (2014)
Le Thi, H.A., Pham Dinh, T., Ngai, H.V.: Exact penalty and error bounds in dc programming. J. Glob. Optim. 52(3), 509–535 (2012)
w. Lu, C.: Image restoration and decomposition using nonconvex non-smooth regularisation and negative hilbert-sobolev norm. IET Image Process. 6(6), 706–716 (2012)
Moudafi, A., Oliny, M.: Convergence of a splitting inertial proximal method for monotone operators. J. Comput. Appl. Math. 155(2), 447–454 (2003)
Nikolova, M., Ng, M.K., Tam, C.P.: Fast nonconvex nonsmooth minimization methods for image restoration and reconstruction. IEEE Trans. Image Process. 19(12), 3073–3088 (2010)
Ochs, P., Chen, Y., Brox, T., Pock, T.: iPiano: Inertial proximal algorithm for nonconvex optimization. SIAM J. Imaging Sci. 7(2), 1388–1419 (2014)
Pang, J.S., Razaviyayn, M., Alvarado, A.: Computing B-stationary points of nonsmooth DC programs. Math. Oper. Res. 42(1), 95–118 (2017)
Polyak, B.: Some methods of speeding up the convergence of iteration methods. USSR Comput. Math. Math. Phys. 4(5), 1–17 (1964)
Rockafellar, R.: Convex Analysis, 1st edn. Princeton University Press, Princeton (1970)
Souza, J.C.O., Oliveira, P.R., Soubeyran, A.: Global convergence of a proximal linearized algorithm for difference of convex functions. Optim. Lett. 10(7), 1529–1539 (2016)
Strekalovskiy, A.: An exact penalty method for d.c. optimization. AIP Conf. Proc. 1776(1), 060,003 (2016)
Strekalovsky, A., Minarchenko, I.: On Local Search in d.c. Optimization. In: 2017 Constructive Nonsmooth Analysis and Related Topics (Dedicated to the Memory of V.F. Demyanov) (CNSA), pp. 1–4 (2017)
Strekalovsky, A.S.: On local search in d.c. optimization problems. Appl. Math. Comput. 255(1), 73–83 (2015)
Tao, P.D., An, L.T.H.: Convex analysis approach to DC programming: theory, algorithms and applications. Acta Math. Vietnamica 22(1), 289–355 (1997)
Tao, P.D., Souad, E.B.: Algorithms for solving a class of nonconvex optimization problems. methods of subgradients. North-Holland Math. Stud. 129, 249–271 (1986). Fermat Days 85: Mathematics for Optimization
Tuy, H.: Convex Analysis and Global Optimization, 2nd edn. Springer Optimization and Its Applications. Springer (2016)
van Ackooij, W., de Oliveira, W.: DC Programming Techniques with Inexact Subproblems’ Solution for General DC Programs. Technical Report, CMA. Available at www.oliveira.mat.br (2017)
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)
Zavriev, S.K., Kostyuk, F.V.: Heavy-ball method in nonconvex optimization problems. Comput. Math. Model. 4(4), 336–341 (1993)
Acknowledgments
The authors are grateful to the Reviewers and the Associate Editor for their remarks and constructive suggestions that considerably improved the original version of this article.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
de Oliveira, W., Tcheou, M.P. An Inertial Algorithm for DC Programming. Set-Valued Var. Anal 27, 895–919 (2019). https://doi.org/10.1007/s11228-018-0497-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11228-018-0497-0