Abstract
Finding the sparsest solution α for an under-determined linear system of equations D α=s is of interest in many applications. This problem is known to be NP-hard. Recent work studied conditions on the support size of α that allow its recovery using ℓ 1-minimization, via the Basis Pursuit algorithm. These conditions are often relying on a scalar property of D called the mutual-coherence. In this work we introduce an alternative set of features of an arbitrarily given D, called the capacity sets. We show how those could be used to analyze the performance of the basis pursuit, leading to improved bounds and predictions of performance. Both theoretical and numerical methods are presented, all using the capacity values, and shown to lead to improved assessments of the basis pursuit success in finding the sparest solution of D α=s.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Candès, E., Romberg, J.: Quantitative robust uncertainty principles and optimally sparse decompositions. Found. Comput. Math. 6(2), 227–254 (2006)
Candès, E.J., Tao, T.: Decoding by linear programming. IEEE Trans. Inform. Theory 51, 4203–4215 (2005)
Candès, E.J., Romberg, J., Tao, T.: Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52(2), 489–509 (2006)
Chen, S.S.: Basis pursuit. Ph.D. dissertation, Stanford Univ., Stanford (1995)
Chen, S.S., Donoho, D.L., Saunders, M.A.: Atomic decomposition by basis pursuit. SIAM J. Sci. Comput. 20(1), 33–61 (1999)
Donoho, D.L.: Neighborly polytopes and sparse solution of underdetermined linear equations. Technical report, Stanford University, Department of Statistics, # 2005-04 (2005)
Donoho, D.L.: For most large underdetermined systems of linear equations the minimal l 1-norm solution is also the sparsest solution. Commun. Pure Appl. Math. 59(6), 797–829 (2006)
Donoho, D.L., Elad, M.: Optimally sparse representation in general (non- orthogonal) dictionaries via l 1 minimization. Proc. Natl. Acad. Sci. 100(5), 2197–2202 (2003)
Donoho, D.L., Huo, X.: Uncertainty principles and ideal atomic decomposition. IEEE Trans. Inf. Theory 47(7), 2845–2862 (1999)
Donoho, D.L., Tanner, J.: Thresholds for the recovery of sparse solutions vial L1 minimization. In: 40th Annual Conference on Information Sciences and Systems (2006)
Elad, M., Bruckstein, A.M.: A generalized uncertainty principle and sparse representations in pairs of bases. IEEE Trans. Inf. Theory 49, 2558–2567 (2002)
Fuchs, J.J.: On sparse representations in arbitrary redundant bases. IEEE Trans. Inf. Theory 50(6), 1341–1344 (2004)
Gribonval, R., Nielsen, M.: Sparse decompositions in unions of bases. IEEE Trans. Inf. Theory 49(12), 3320–3325 (2003)
Hardy, G.H., Littlewood, J.E., Pólya, G.: Inequalities, 2nd ed., pp. 43–45 and 123. Cambridge University Press, Cambridge (1988)
Malioutov, D.M., Cetin, M., Willsky, A.S.: Optimal sparse representations in general overcomplete bases. In: IEEE International Conference on Acoustics, Speech, and Signal Processing—ICASSP, May 2004, Montreal, Canada (2004)
Natarajan, B.K.: Sparse approximate solutions to linear systems. SIAM J. Comput. 24, 227–234 (1995)
Strohmer, T., Heath, R.W. Jr.: Grassmannian frames with applications to coding and communications. Appl. Comput. Harmon. Anal. 14(3), 257–275 (2003)
Tropp, J.: Random subdictionaries of random dictionaries. Preprint (2006)
Tropp, J.: Greed is good: algoritmic results for sparse approximation. IEEE Trans. Inf. Theory 50(10), 2231–2242 (2004)
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Anna Gilbert.
Rights and permissions
About this article
Cite this article
Shtok, J., Elad, M. Analysis of Basis Pursuit via Capacity Sets. J Fourier Anal Appl 14, 688–711 (2008). https://doi.org/10.1007/s00041-008-9036-y
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00041-008-9036-y