Abstract
The idea of a finite collection of closed sets having “linearly regular intersection” at a point is crucial in variational analysis. This central theoretical condition also has striking algorithmic consequences: in the case of two sets, one of which satisfies a further regularity condition (convexity or smoothness, for example), we prove that von Neumann’s method of “alternating projections” converges locally to a point in the intersection, at a linear rate associated with a modulus of regularity. As a consequence, in the case of several arbitrary closed sets having linearly regular intersection at some point, the method of “averaged projections” converges locally at a linear rate to a point in the intersection. Inexact versions of both algorithms also converge linearly.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
F.J. Aragón Artacho, A.L. Dontchev, M.H. Geoffroy, Convergence of the proximal point method for metrically regular mappings, ESAIM Proc. 17, 1–8 (2007).
H. Attouch, J. Bolte, P. Redont, A. Soubeyran, Alternating minimization and projection methods for nonconvex problems. arXiv:0801.1780v1, 11 Jan. 2008.
A. Auslender, Méthodes Numériques pour la Résolution des Problèmes d’Optimisation avec Contraintes. PhD thesis, Uni. Grenoble, 1969.
D. Aussel, A. Daniilidis, L. Thibault, Subsmooth sets: functional characterizations and related concepts, Trans. Am. Math. Soc. 357, 1275–1301 (2004).
H.H. Bauschke, J.M. Borwein, On the convergence of von Neumann’s alternating projection algorithm for two sets, Set-Valued Anal. 1, 185–212 (1993).
H.H. Bauschke, J.M. Borwein, On projection algorithms for solving convex feasibility problems, SIAM Rev. 38, 367–426 (1996).
H.H. Bauschke, P.L. Combettes, D.R. Luke, Phase retrieval, error reduction algorithm, and Fienup variants: A view from convex optimization, J. Opt. Soc. Am. 19(7), 1334–1345 (2002).
L.M. Bregman, The method of successive projection for finding a common point of convex sets, Sov. Math. Dokl. 6, 688–692 (1965).
E.J. Candès, J. Romberg, Sparsity and incoherence in compressive sampling, Inverse Probl. 23(3), 969–986 (2007).
X. Chen, M.T. Chu, On the least squares solution of inverse eigenvalue problems, SIAM J. Numer. Anal. 33, 2417–2430 (1996).
M.T. Chu, Constructing a Hermitian matrix from its diagonal entries and eigenvalues, SIAM J. Matrix Anal. 16, 207–217 (1995).
F.H. Clarke, Yu.S. Ledyaev, R.J. Stern, P.R. Wolenski, Nonsmooth Analysis and Control Theory (Springer, New York, 1998).
P.L. Combettes, T. Pennanen, Proximal methods for cohypomonotone operators, SIAM J. Control Opt. 43, 731–742 (2004).
P.L. Combettes, H.J. Trussell, Method of successive projections for finding a common point of sets in metric spaces, J. Optim. Theory Appl. 67(3), 487–507 (1990).
F. Deutsch, Best Approximation in Inner Product Spaces (Springer, New York, 2001).
F. Deutsch, H. Hundal, The rate of convergence for the cyclic projections algorithm I: angles between convex sets, J. Approx. Theory 142, 36–55 (2006).
F. Deutsch, H. Hundal, The rate of convergence for the cyclic projections algorithm II: norms of nonlinear operators, J. Approx. Theory 142, 56–82 (2006).
D. Donoho, Compressed sensing, IEEE Trans. Inf. Theory 52, 1289–1306 (2006).
A.L. Dontchev, A.S. Lewis, R.T. Rockafellar, The radius of metric regularity, AMS Trans. 355, 493–517 (2003).
J. Romberg, E.J. Candès, T. Tao, Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information, IEEE Trans. Inf. Theory 52, 489–509 (2005).
M. Elad, Optimized projections for compressed-sensing, IEEE Trans. Signal Process. 55, 5695–5702 (2007).
J. Fadili, G. Peyré, Personal communication, 2007.
K.M. Grigoriadis, E. Beran, Alternating projection algorithm for linear matrix inequalities problems with rank constraints, in Advances in Linear Matrix Inequality Methods in Control (SIAM, Philadelphia, 2000).
K.M. Grigoriadis, R.E. Skelton, Low-order control design for LMI problems using alternating projection methods, Automatica 32, 1117–1125 (1996).
L.G. Gubin, B.T. Polyak, E.V. Raik, The method of projections for finding the common point of convex sets, USSR Comput. Math. Math. Phys. 7, 1–24 (1967).
A.N. Iusem, T. Pennanen, B.F. Svaiter, Inexact versions of the proximal point algorithm without monotonicity, SIAM J. Optim. 13, 1080–1097 (2003).
D. Klatte, B. Kummer, Optimization methods and stability of inclusions in Banach spaces, Math. Program. 17, 305–330 (2009).
A.Y. Kruger, About regularity of collections of sets, Set-Valued Anal. 14, 187–206 (2006).
A.S. Lewis, D.R. Luke, J. Malick, Local convergence for alternating and averaged nonconvex projections. arXiv:0709.0109v1, 2 Sep. 2007.
A.S. Lewis, J. Malick, Alternating projections on manifolds, Math. Oper. Res. 33, 216–234 (2008).
B.S. Mordukhovich, Maximum principle in the problem of time optimal response with nonsmooth constraints, J. Appl. Math. Mech. 40, 960–969 (1976).
B.S. Mordukhovich, Nonsmooth analysis with nonconvex generalized differentials and adjoint mappings, Dokl. Akad. Nauk BSSR 28, 976–979 (1984).
B.S. Mordukhovich, Variational Analysis and Generalized Differentiation, I: Basic Theory; II: Applications (Springer, New York, 2006).
B.S. Mordukhovich, Failure of metric regularity for major classes of variational systems, Nonlinear Anal. 69, 918–924 (2008).
R. Orsi, Numerical methods for solving inverse eigenvalue problems for nonnegative matrices, SIAM J. Matrix Anal. 28, 190–212 (2006).
R. Orsi, U. Helmke, J. Moore, A Newton-like method for solving rank constrained linear matrix inequalities, Automatica 42, 1875–1882 (2006).
T. Pennanen, Local convergence of the proximal point algorithm and multiplier methods without monotonicity, Math. Oper. Res. 27, 170–191 (2002).
G. Pierra, Eclatement de contraintes en parallèle pour la minimisation d’une forme quadratique, Lect. Notes Comput. Sci. 41, 200–218 (1976).
G. Pierra, Decomposition through formalization in a product space, Math. Program. 28, 96–115 (1984).
R.A. Poliquin, R.T. Rockafellar, L. Thibault, Local differentiability of distance functions, AMS Trans. 352, 5231–5249 (2000).
J. Renegar, Incorporating condition measures into the complexity theory of linear programming, SIAM J. Optim. 5, 506–524 (1995).
J. Renegar, Linear programming, complexity theory and elementary functional analysis, Math. Program. 70, 279–351 (1995).
J. Renegar, Condition numbers, the barrier method, and the conjugate gradient method, SIAM J. Optim. 6, 879–912 (1996).
R.T. Rockafellar, R.J.-B. Wets, Variational Analysis (Springer, Berlin, 1998).
A. Shapiro, Existence and differentiability of metric projections in Hilbert space, SIAM J. Optim. 4, 130–141 (1994).
A. Shapiro, On the asymptotics of constrained local M-estimation, Ann. Stat. 28, 948–960 (2000).
A. Shapiro, F. Al-Khayyal, First-order conditions for isolated locally optimal solutions, J. Optim. Theory Appl. 77, 189–196 (1993).
J.A. Tropp, I.S. Dhillon, R.W. Heath, T. Strohmer, Designing structured tight frames via an alternating projection method, IEEE Trans. Inf. Theory 51, 188–209 (2005).
J. von Neumann, Functional Operators, vol. II (Princeton University Press, Princeton, 1950). Reprint of notes distributed in 1933.
C.A. Weber, J.P. Allebach, Reconstruction of frequency-offset Fourier data by alternating projection on constraint sets, in 24th Allerton Conference Proc., pp. 194–201. Urbana-Champaign, IL, 1986.
K. Yang, R. Orsi, Generalized pole placement via static output feedback: a methodology based on projections, Automatica 42, 2143–2150 (2006).
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Michael Todd.
Research of A.S. Lewis supported in part by National Science Foundation Grant DMS-0504032.
Research of D.R. Luke supported in part by National Science Foundation Grant DMS-0712796.
Rights and permissions
About this article
Cite this article
Lewis, A.S., Luke, D.R. & Malick, J. Local Linear Convergence for Alternating and Averaged Nonconvex Projections. Found Comput Math 9, 485–513 (2009). https://doi.org/10.1007/s10208-008-9036-y
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10208-008-9036-y
Keywords
- Alternating projections
- Averaged projections
- Linear convergence
- Metric regularity
- Distance to ill-posedness
- Variational analysis
- Nonconvexity
- Extremal principle
- Prox-regularity