Abstract
Recently, the alternating direction method of multipliers (ADMM) and its variations have gained great popularity in large-scale optimization problems. This paper is concerned with the solution of the tensor equation \(\mathscr{A}\textbf {x}^{m-1}=\textbf {b}\) in which \(\mathscr{A}\) is an m th-order and n-dimensional real tensor and b is an n-dimensional real vector. By introducing certain auxiliary variables, we transform equivalently this tensor equation into a consensus constrained optimization problem, and then propose an ADMM type method for it. It turns out that each limit point of the sequences generated by this method satisfies the Karush-Kuhn-Tucker conditions. Moreover, from the perspective of computational complexity, the proposed method may suffer from the curse-of-dimensionality if the size of the tensor equation is large, and thus we further present a modified version (as a variant of the former) turning to the tensor-train decomposition of the tensor \(\mathscr{A}\), which is free from the curse. As applications, we establish the associated inverse iteration methods for solving tensor eigenvalue problems. The performed numerical examples illustrate that our methods are feasible and efficient.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Bader, B.W., Kolda, T.G., et al.: MATLAB Tensor Toolbox Version 2.6. http://www.sandia.gov/~tgkolda/TensorToolbox/index-2.6.html (2015)
Ballani, J., Grasedyck, L.: A projection method to solve linear systems in tensor format. Numer. Linear Algebra Appl. 20(1), 27–43 (2013)
Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2010)
Chang, T.-H., Hong, M.-Y., Liao, W., Wang, X.-F.: Asynchronous distributed ADMM for large-scale optimization-part i: algorithm and convergence analysis. IEEE Trans. Sig. Process. 64(12), 3118–3130 (2016)
Chen, C.-H., He, B.-S., Yuan, X.-M., Ye, Y.-Y.: The direct extension of ADMM for multi-block convex minimization problems is not necessarily convergent. Math. Program. 155(1-2), 57–79 (2016)
Cooper, J., Dutle, A.: Spectra of uniform hypergraphs. Linear Algebra Appl. 436, 3268–3292 (2012)
Ding, W.-Y., Wei, Y.-M.: Solving multi-linear systems with \(\mathcal {M}\)-tensors. J. Sci. Comput. 68, 689–715 (2016)
Dolgov, S.V., Khoromskij, B.N., Oseledets, I.V.: Fast solution of parabolic problems in the tensor train/quantized tensor train format with initial application to the Fokker-Planck equation. SIAM J. Sci. Comput. 34(6), A3016–A3038 (2012)
Fowler, P.W., Pisanski, T.: HOMO-LUMO Maps for chemical graphs. MATCH Commun. Math. Comput. Chem. 64, 373–390 (2010)
Forero, P.A., Cano, A., Giannakis, G.B.: Distributed clustering using wireless sensor networks. IEEE J. Selected Topics Sig. Process. 5, 707–724 (2011)
Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Comput. Math. Appl. 2, 17–40 (1976)
Grasedyck, L.: Hierarchical singular value decomposition of tensors. SIAM J. Matrix Anal. Appl. 31(4), 2029–2054 (2010)
Grasedyck, L., Kressner, D., Tobler, C.: A literature survey of low-rank tensor approximation techniques. GAMM-Mitt. 36(1), 53–78 (2013)
Gutman, I., Rouvray, D.: An approximate topological formula for the HOMO-LUMO separation in alternate hydrocarbons. Chem. Phys. Lett. 62, 384–388 (1979)
Hackbusch, W., Kuhn, S.: A new scheme for the tensor representation. J. Fourier Anal. Appl. 15(5), 706–722 (2009)
Han, L.-X.: A homotopy method for solving multilinear systems with \(\mathcal {M}\)-tensors. Appl. Math. Lett. 69, 49–54 (2017)
Hastie, T., Tibshirani, R., Friedman, J.: The elements of statistical learning: data mining, inference, and prediction. Springer, New York (2001)
Hillar, C.J., Lim, L.-H.: Most tensor problems are NP-hard. J. ACM. 60(6), 1–39 (2013)
He, B.-S., Tao, M., Yuan, X.-M.: A splitting method for separable convex programming. IMA J. Numer. Anal. 20, 1–33 (2014)
He, B.-S., Yuan, X.-M.: Linearized alternating direction method of multipliers with Gaussian back substitution for separable convex programming. Numer. Algebra Control Optim. 3, 247–260 (2013)
Holtz, S., Rohwedder, T., Schneider, R.: The alternating linear scheme for tensor optimization in the tensor train format. SIAM J. Sci. Comput. 34, A683–A713 (2012)
Hong, M.-Y., Luo, Z.-Q., Razaviyayn, M.: Convergence analysis of alternating direction method of multipliers for a family of nonconvex problems. SIAM J. Optim. 26(1), 337–364 (2016)
Kofidis, E., Regalia, P.A.: On the best rank-1 approximation of higher-order supersymmetric tensors. SIAM J. Matrix Anal. Appl. 23, 863–884 (2002)
Kolda, T.G., Bader, B.W.: Tensor decompositions and applications. SIAM Rev. 51, 455–500 (2009)
Kolda, T.G., Mayo, J.R.: Shifted power method for computing tensor eigenpairs. SIAM J. Matrix Anal. Appl. 32(4), 1095–1124 (2011)
Liavas, A.P., Sidiropoulos, N.D.: Parallel algorithms for constrained tensor factorization via the alternating direction method of multipliers. IEEE Trans. Sig. Process. 63(20), 5450–5462 (2015)
Li, D.-H., Xie, S.-L., Xu, H.-R.: Splitting methods for tensor equations. Numer. Linear Algebra Appl. 24(5), 1–16 (2017)
Li, X.-T., Ng, M.K.: Solving sparse non-negative tensor equations: algorithms and applications. Front. Math. China 10(3), 649–680 (2015)
Lim, L.-H.: Singular values and eigenvalues of tensors: a variational approach. In: proceedings of the 1st IEEE international workshop on computational advances of multi-sensor adaptive processing (CAMSAP), December 13-15, pp. 129–132 (2005)
Liu, J., Chen, J., Ye, J.: Large-scale sparse logistic regression. In: Proceedings of the ACM International Conference on Knowledge Discovery and Data Mining, New York, NY, USA, June 28-July 1, pp. 547–556 (2009)
Liu, D.-D., Li, W., Vong, S.W.: The tensor splitting with application to solve multi-linear systems. J. Comput. Appl. Math. 330(1), 75–94 (2018)
Luo, Z.-Y., Qi, L.-Q., Xiu, N.-H.: The sparsest solutions to Z-tensor complementarity problems. Optim Lett. 11, 471–482 (2017)
Matsuno, Y.: Exact solutions for the nonlinear Klein-Gordon and Liouville equations in four-dimensional Euclidean space. J. Math. Phys. 28(10), 2317–2322 (1987)
Qi, L.-Q.: Eigenvalues of a real supersymmetric tensor. J. Symb. Comput. 40, 1302–1324 (2005)
Qi, L.-Q., Luo, Z.-Y.: Tensor analysis: spectral theory and special tensors. SIAM, Philadelphia (2017)
Ortega, J.M., Rheinboldt, W.C.: Iterative solution of nonlinear equations in several variables. Academic Press, New York (1970)
Oseledets, I.V.: Tensor-train decomposition. SIAM J. Sci. Comput. 33(5), 2295–2317 (2011)
Oseledets, I.V., et al.: TT-Toolbox. https://github.com/oseledets/TT-Toolbox (2016)
Oseledets, I.V., Tyrtyshnikov, E.E.: Breaking the curse of dimensionality, or how to use SVD in many dimensions. SIAM J. Sci. Comput. 31(5), 3744–3759 (2009)
Sun, D.-F., Toh, K.-C., Yang, L.-Q.: A convergent 3-block semiproximal alternating direction method of multipliers for conic programming with 4-type constraints. SIAM J. Optim. 25(2), 882–915 (2015)
Tibshirani, R.: Regression shrinkage and selection via the LASSO. J. Roy. Stat. Soc. B 58, 267–288 (1996)
Wang, Y., Yin, W.-T., Zeng, J.-S.: Global convergence of ADMM in nonconvex nonsmooth optimization. J. Sci. Comput. https://doi.org/10.1007/s10915-018-0757-z (2018)
Xie, Z.-J., Jin, X.-Q., Wei, Y.-M.: A fast algorithm for solving circulant tensor systems. Linear Multilinear Algebra 65(9), 1894–1904 (2017)
Xu, Y.-Y., Yin, W.-T., Wen, Z.-W., Zhang, Y.: An alternating direction algorithm for matrix completion with nonnegative factors. Front. Math. China 7(2), 365–384 (2012)
Zhang, J.-Y., Wen, Z.-W., Zhang, Y.: Subspace methods with local refinements for eigenvalue computation using low-rank tensor-train format. J. Sci. Comput. 70, 478–499 (2017)
Zwillinger, D.: Handbook of Differential Equations, 3rd edn. Academic Press Inc, Boston (1997)
Acknowledgements
The authors are very grateful to the editors and two anonymous referees for their constructive comments and valuable suggestions, which greatly improved the original manuscript of this paper. Especially, the first author would like to thank Dr. Yutao Zheng for his selfless help in the process of programming.
Funding
This work was financially supported by the National Natural Science Foundation of China (Grant nos. 11571004 and 11701456). The research of the first author was also financially supported by the Science Foundation of Education Department of Gansu Province (Grant no. 2017A-078) and Tianshui Normal University (Grant no. TAS1603) as well as the Key Discipline Construction Foundation of Tianshui Normal University. The third author was financially supported by the Fundamental Research Funds for the Central Universities (Grant no. lzujbky-2017-it54) as well.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Liang, M., Zheng, B. & Zhao, R. Alternating iterative methods for solving tensor equations with applications. Numer Algor 80, 1437–1465 (2019). https://doi.org/10.1007/s11075-018-0601-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11075-018-0601-4