A new algorithm is described for distributed joint diagonalization of real symmetric or complex Hermitian matrices. The approach, which is based on the Jacobi diagonalization, utilizes distribution of the computational power and memory space, minimizes the communication costs, and runs on clusters of personal computers. It further combines two-step load balancing algorithm with a standard Kalman filter to enable quick but low-cost adaptation to resource varying conditions. Theoretical analysis of its performance shows that the communication costs (when normalized by computational costs) decline linearly with the number and size of the diagonalized matrices. This is also confirmed by experimental results: the measured speedup ratio yields 42.2 when jointly diagonalizing 800 matrices of size 400 × 400 on a cluster of 50 personal computers.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Argonne National Laboratory, MPICH – A Portable Implementation of MPI, http://www-unix.mcs.anl.gov/mpi/mpich/html, (2006).
Barlow J.L., Slapniéar I. (1993) Optimal perturbation bounds for the Hermitian Eigenvalue Problem. Linear Algebra Appl., 309:19–43
Bečka M., Oška G., Vajteršic M. (2002) Dynamic Ordering for Parallel Block-Jacobi SVD Algorithm. Parall. Comput. 28:243–262
Belouchrani A., Amin K.M. (1998) Blind Source Separation Based on Time-Frequency Signal Representation. IEEE Trans. Signal Process. 46:2888–2898
Boashash B. (2001) Time-Frequency Signal Analysis and Processing. Prentice Hall PTR, Englewood Cliffs, NJ
Brent R.P., Luk F.T. (1985) The Solution of Singular-Value and Symmetric Eigenvalue Problems on Multiprocessor Arrays. SIAM J. Sci. Stat. Comput. 6:69–84
Bunse-Gerstner A., Byers R., Mehrmann V. (1993) Numerical Methods for Simultaneous Diagonalization. SIAM J. Matrix Anal. Appl. 14:927–949
J. F. Cardoso, Perturbation of Joint Diagonalizers, Technical. report 94D023, Signal Department Telecom Paris, (1994).
Cardoso J.F., Souloumiac A. (1993) Blind Beamforming for Non-Gaussian Signals. IEE Proc. F 6:362–370
Cardoso J.F., Souloumiac A. (1996) Jacobi Angles for Simultaneous Diagonalization. SIAM J. Mater Anal. Appl. 17:161–164
Daoudi E.M., Lakhouaja A. (1997) Exploiting the Symmetry in the Parallelization of the Jacobi Method. Parall. Comput. 23:137–151
J. W. Demmel and K. Veselic, Jacobi Method is More Accurate Than QR, Technical report 468, Department of Computer Science, Courant Institute of Mathematical Science, New York University (1989).
J. W. Demmel, Trading Off Parallelism and Numerical Stability, Technical report CRPC-TR92422, Center for research on Parallel Computation, Rice University, Huston (1992).
R. Diekmann, A. Frommer, and B. Monien. Efficient Schemes for Nearest Neighbor Load Balancing. Parall. Comput. 789–812 (1999).
Golub G.H., Van Loan C.F. (1996) Matrix Computation, 3rd Ed. The Johns Hopkins University Press, Baltimore
W. Gropp and E. Lusk, Reproducible Measurements of MPI Performance Characteristics, in Proc. 6th European PVM/MPI User’s Group Meeting on Recent Advances in Parallel Virtual Machine and Message Passing Interface, Barcelona, Spain pp. 26–29 (1999).
Gygi F., Fattebert J.L., Schwegler E. (2003) Computation of Maximally Localized Functions Using a Simultaneous Diagonalization Algorithm. Comput. Phys. Commun. 155:1–6
A. Holobar, C. Fevotte, C. Doncarli, and D. Zazula, Single Autoterm Selection for Blind Source Separation in Time-Frequency Plane, in Proc. EUSIPCO’02, Toulouse, France (2002).
A. Holobar and D. Zazula, A New Approach for Blind Source Separation of Convolutive Mixtures of Pulse Trains”, in Proc. BSI’02, Como, Italy pp. 163–166 (2002).
Hyvarinen A., Karhunen J., Oja E. (2001) Independent Component Analysis. Wiley, New York
M. Joho and K. Rahbar, Joint Diagonalization of Correlation Matrices by Using Newton Methods With Application to Blind Signal Separation, in Proc. SAM, Rosslyn pp. 403–407 (2002).
Katagiri T., Kanada Y. (2001) An Efficient Implementation of Parallel Eigenvalue Computation for Massively Parallel Processing, Parall. Comput. 27:1831–1845
T. Lücking, B. Monien, and M. Rode, On the Problem of Scheduling Flows on Distributed Networks, in Proc. MFCS 2002, Vol. 2420, Warsaw, Poland, pp. 495–505 (2002).
Pham D.T. (2001) Joint Approximate Diagonalization of Positive Definite Hermitian Matrices, SIAM J. Matrix Anal. Appl. 22:1136–1152
Rotaru T., Nägeli H-H. (2004) Dynamic Load Balancing by Diffusion in Heterogeneous Systems. J. Parallel Distrib. Comput. 64:481–497
Yeredor A. (2002) Non-Orthogonal Joint Diagonalization in the Least Squares Sense With Application in Blind Source Separation, IEEE Trans. Signal Process. 50:1545–1553
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Holobar, A., Ojsteršek, M. & Zazula, D. Distributed Jacobi Joint Diagonalization on Clusters of Personal Computers. Int J Parallel Prog 34, 509–530 (2006). https://doi.org/10.1007/s10766-006-0025-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10766-006-0025-y