Abstract
We propose a new message-passing algorithm for the quadratic optimization problem. As opposed to the min-sum algorithm, the new algorithm involves two minimizations and one summation at each iteration. The new min-sum-min algorithm exploits feedback from last iteration in generating new messages, resembling the Jacobi- relaxation algorithm. We show that if the feedback signal is large enough, the min-sum-min algorithm is guaranteed to converge to the optimal solution. Experimental results show that the min-sum-min algorithm outperforms two reference methods w.r.t. the convergence speed.
Chapter PDF
Similar content being viewed by others
References
Bertsekas, D.P., Tsitsikis, J.N.: Parallel and distributed Computation: Numerical Methods. Athena Scientific, Belmont (1997)
Bishop, C.M.: Pattern Recognition and Machine Learning. Springer (2007)
Johnson, J.K., Malioutov, D.M., Willsky, A.S.: Walk-sum Interpretation and Analysis of Gaussian Belief Propagation. In: Advances in Neural Information Processing Systems, vol. 18. MIT Press, Cambridge (2006)
Johnson, J.K., Bickson, D., Dolev, D.: Fixing Convergence of Gaussian Belief Propagation. In: The International Symposium on Information Theory (2009)
Malioutov, D.M., Johnson, J.K., Willsky, A.S.: Walk-Sums and Belief Propagation in Gaussian Graphical Models. J. Mach. Learn. Res. 7, 2031–2064 (2006)
Moallemi, C.C., Van Roy, B.: Convergence of Min-Sum Message Passing for Quadratic Optimization. IEEE Trans. Inf. Theory 55(5), 2413–2423 (2009)
Moallemi, C.C., Van Roy, B.: Convergence of Min-Sum Message Passing for Convex Optimization. IEEE Trans. Inf. Theory 56(4), 2041–2050 (2010)
Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufman Publishers (1988)
Zhang, G., Heusdens, R.: Linear Coordinate-Descent Message-Passing for Quadratic Optimization. Neural Computation 24(12), 3340–3370 (2012)
Zhang, G., Heusdens, R.: Convergence of Generalized Linear Coordinate-Descent Message-Passing for Quadratic Optimization. In: IEEE International Symposium on Information Theory Proceedings, pp. 1997–2001 (2012)
Matrix Market: Harwell Boeing Collection, http://math.nist.gov/MatrixMarket/index.html
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zhang, G., Heusdens, R. (2014). Convergence of Min-Sum-Min Message-Passing for Quadratic Optimization. In: Calders, T., Esposito, F., Hüllermeier, E., Meo, R. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2014. Lecture Notes in Computer Science(), vol 8726. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-44845-8_23
Download citation
DOI: https://doi.org/10.1007/978-3-662-44845-8_23
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-44844-1
Online ISBN: 978-3-662-44845-8
eBook Packages: Computer ScienceComputer Science (R0)