Abstract
In this work we describe the usage of bilinear statistical models as a means of factoring the shape variability into two components attributed to inter-subject variation and to the intrinsic dynamics of the human heart. We show that it is feasible to reconstruct the shape of the heart at discrete points in the cardiac cycle. Provided we are given a small number of shape instances representing the same heart at different points in the same cycle, we can use the bilinear model to establish this.
Using a temporal and a spatial alignment step in the preprocessing of the shapes, around half of the reconstruction errors were on the order of the axial image resolution of 2 mm, and over 90% was within 3.5 mm. From this, we conclude that the dynamics were indeed separated from the inter-subject variability in our dataset.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Abboud, B., & Davoine, F. (2004). Bilinear factorization for facial expression analysis and synthesis. IEE Proceedings—Vision, Image and Signal Processing, 152(3), 327–333.
Bistoquet, A., Oshinski, J., & Škrinjar, O. (2007). Left ventricular deformation recovery from cine MRI using an incompressible model. IEEE Transactions on Medical Imaging, 26(9), 1136–1153.
Blackall, J. M., King, A. P., Penney, G. P., Adam, A., & Hawkes, D. J. (2001). A statistical model of respiratory motion and deformation of the liver. In W. J. Niessen & M. A. Viergever (Eds.), Lecture notes in computer science: Vol. 2208. Proc. 4th int. conf. medical image computing and computer assisted intervention (MICCAI), Utrecht, The Netherlands (pp. 1338–1340). Berlin: Springer.
Bosch, J. G., Mitchell, S. C., Lelieveldt, B. P. F., Nijland, F., Kamp, O., Sonka, M., & Reiber, J. H. C. (2002). Automatic segmentation of echocardiographic sequences by active appearance motion models. IEEE Transactions on Medical Imaging, 21(11), 1374–1383.
Chandrashekara, R., Rao, A., Sanchez-Ortiz, G. I., Mohiaddin, R. H., & Rueckert, D. (2003). Construction of a statistical model for cardiac motion analysis using nonrigid image registration. In C. J. Taylor & J. A. Noble (Eds.), Lecture notes in computer science: Vol. 2732. Proc. 18th int. conf. information processing in medical imaging (IPMI), Ambleside, United Kingdom (pp. 599–610). Berlin: Springer.
Chuang, E., & Bregler, C. (2005). Mood swings: Expressive speech animation. ACM Transactions on Graphics, 24(2), 331–347.
Cootes, T. F., Cooper, D. H., Taylor, C. J., & Graham, J. (1992). Trainable method of parametric shape description. Image & Vision Computing, 10(5), 289–294.
Cootes, T. F., Edwards, G. J., & Taylor, C. J. (2001). Active appearance models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(6), 681–685.
Cootes, T. F., Taylor, C. J., Cooper, D. H., & Graham, J. (1995). Active shape models—their training and application. Computer Vision and Image Understanding, 61(1), 38–59.
Cuzzolin, F. (2006). Using bilinear models for view-invariant action and identity recognition. In Proc. IEEE int. conf. on computer vision and pattern recognition (CVPR), New York, NY, USA (pp. 1701–1708) 2006.
De Lathauwer, L., De Moor, B., & Vandewalle, J. (2000). A multilinear singular value decomposition. SIAM Journal on Matrix Analysis and Applications, 21(4), 1253–1278.
Dryden, I. L., & Mardia, K. V. (1998). Statistical shape analysis. New York: Wiley.
Duncan, J. S., & Ayache, N. (2000). Medical image analysis: progress over two decades and the challenges ahead. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(1), 85–106.
Frangi, A. F., Niessen, W. J., Viergever, M. A., & Lelieveldt, B. P. F. (2005). A survey of three-dimensional modeling techniques for quantitative functional analysis of cardiac images. In L. Landini, V. Positano, & M.F. Santarelli (Eds.), Advanced image processing in magnetic resonance imaging (Chap. 9, pp. 267–342). Boca Raton: CRC Press.
González-Mora, J., De la Torre, F., Murthi, R., Guil, N., & Zapata, E. L. (2007). Bilinear active appearance models. In R. Goecke, S. Lucey, & I. Matthews (Eds.), Proc. int. workshop on non-rigid registration and tracking through learning, Rio de Janeiro, Brazil, 2007.
Goodall, C. (1991). Procrustes methods in shape analysis. Journal of the Royal Statistical Society—Series B: Statistical Methodology, 53(2), 285–339.
Grimes, D. B., & Rao, R. P. N. (2005). Bilinear sparse coding for invariant vision. Neural Computation, 17(1), 47–73.
Hamarneh, G., & Gustavsson, T. (2004). Deformable spatio-temporal shape models: extending active shape models to 2D + time. Image & Vision Computing, 22(6), 461–470.
Hoogendoorn, C., Sukno, F. M., Ordás, S., & Frangi, A. F. (2007). Bilinear models for spatio-temporal point distribution analysis: application to extrapolation of whole heart cardiac dynamics. In: M. Nielsen, W. Niessen & C. F. Westin (Eds.), Proc. 8th int. workshop on mathematical methods in biomedical image analysis (MMBIA), Rio de Janeiro, Brazil, 2007.
Hsu, E., Pulli, K., & Popović, J. (2005). Style translation for human motion. ACM Transactions on Graphics, 23(3), 1082–1089.
Kendall, D. G. (1984). Shape manifolds, procrustean metrics and complex projective spaces. Bulletin of the London Mathematical Society, 16(2), 81–121.
Le, H., & Kendall, D. G. (1993). The Riemannian structure of euclidean shape spaces: A novel environment for statistics. Annals of Statistics, 21(3), 1225–1271.
Lee, C. S., & Elgammal, A. (2004). Gait style and gait content: Bilinear models for gait recognition using gait resampling. In Proc. 6th IEEE int. conf. on automatic face and gesture recognition (FGR), Seoul, Korea (pp. 147–152) 2007.
Lekadir, K., Keenan, N., Pennell, D., & Yang, G. Z. (2007). Shape-based myocardial contractility analysis using multivariate outlier detection. In N. Ayache, S. Ourselin, & A. Maeder (Eds.), Lecture notes in computer science: Vol. 4792. Proc. 10th int. conf. medical image computing and computer assisted intervention (MICCAI), Brisbane, QLD, Australia (pp. 834–841). Berlin: Springer.
Leung, K. Y. E., & Bosch, J. G. (2007). Localized shape variations for classifying wall motion in echocardiograms. In N. Ayache, S. Ourselin, & A. Maeder (Eds.), Lecture notes in computer science: Vol. 4791. Proc. 10th int. conf. medical image computing and computer assisted intervention (MICCAI), Brisbane, QLD, Australia (pp. 52–59). Berlin: Springer.
Liu, H., & Shi, P. (2007). State-space analysis of cardiac motion with biomechanical constraints. IEEE Transactions on Image Processing, 16(4), 901–916.
Lynch, M., Ghita, O., & Whelan, P. (2008). Segmentation of the left ventricle of the heart in 3D + t MRI data using an optimised non-rigid temporal model. IEEE Transactions on Medical Imaging, 27(2), 195–203.
Magnus, J. R., & Neudecker, H. (1988). Matrix differential calculus with applications in statistics and econometrics. New York: Wiley.
Mardia, K. V., & Dryden, I. L. (1989). Shape distributions for landmark data. Advances in Applied Probability, 21(4), 742–755.
Marimont, D. H., & Wandell, B. A. (1992). Linear models of surface and illuminant spectra. Journal of the Optical Society of America A: Optics, Image Science, and Vision, 9(11), 1905–1913.
McInerney, T., & Terzopoulos, D. (1996). Deformable models in medical image analysis: a survey. Medical Image Analysis, 1(2), 91–108.
Mitchell, S. C., Bosch, J. G., Lelieveldt, B. P. F., van der Geest, R. J., Reiber, J. H. C., & Sonka, M. (2002). 3-D active appearance models: segmentation of cardiac MR and ultrasound images. IEEE Transactions on Medical Imaging, 21(9), 1167–1178.
Montagnat, J., & Delingette, H. (2005). 4D deformable models with temporal constraints: application to 4D cardiac image segmentation. Medical Image Analysis, 9(1), 87–100.
Ohnesorge, B. M., Becker, C. R., Flohr, T. G., & Reiser, M. F. (2002). Multi-slice CT in cardiac imaging: technical principles, clinical application and future developments. Berlin: Springer.
Ordás, S., Oubel, E., Leta, R., Carrera, F., & Frangi, A. F. (2007). A statistical shape model of the heart and its application to model-based segmentation. In A. Manduca & X. P. Hu (Eds.), Proc. SPIE medical imaging, San Diego, CA, USA, Vol. 6511.
Pantic, M., & Rothkrantz, L. J. M. (2000). Automatic analysis of facial expressions: the state of the art. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(12), 1424–1445.
Perperidis, D. (2005). Spatio-temporal registration and modelling of the heart using cardiovascular MR imaging. Ph.D. thesis, Imperial College London.
Perperidis, D., Mohiaddin, R. H., & Rueckert, D. (2005). Spatio-temporal free-form registration of cardiac MR image sequences. Medical Image Analysis, 9(5), 441–456.
Shin, D., Lee, H. S., & Kim, D. (2008). Illumination-robust face recognition using ridge regressive bilinear models. Pattern Recognition Letters, 29(1), 49–58.
Styner, M. A., Rajamani, K. T., Nolte, L. P., Zsemliye, G., Székely, G., Taylor, C. J., & Davies, R. H. (2003). Evaluation of 3D correspondence methods for model building. In Lecture notes in computer science: Vol. 2732. Proc. 18th int. conf. information processing in medical imaging (IPMI), Ambleside, United Kingdom (pp. 63–75). Berlin: Springer.
Suri, J. S. (2000). Computer vision, pattern recognition and image processing in left ventricle segmentation: the last 50 years. Pattern Analysis & Applications, 3(3), 209–242.
Syeda-Mahmood, T., Wang, F., Beymer, D., London, M., & Reddy, R. (2007). Characterizing spatio-temporal patterns for disease discrimination in cardiac echo videos. In N. Ayache, S. Ourselin, A. Maeder (Eds.), Lecture notes in computer science: Vol. 4791. Proc. 10th int. conf. medical image computing and computer assisted intervention (MICCAI), Brisbane, QLD, Australia (pp. 261–269). Berlin: Springer.
Tenenbaum, J. B., & Freeman, W. T. (1996). Separating style and content. In M. Mozer, M. I. Jordan & T. Petsche (Eds.), Advances in neural information processing systems, Denver, CO, USA (pp. 662–668) 1996.
Tenenbaum, J. B., & Freeman, W. T. (2000). Separating style and content with bilinear models. Neural Computation, 12(6), 1247–1283.
Turk, M., & Pentland, A. (1991). Eigenfaces for recognition. Journal of Cognitive Neuroscience, 3(1), 71–86.
Vasilescu, M. A. O., & Terzopoulos, D. (2003). Multilinear subspace analysis of image ensembles. In Proc. IEEE int. conf. on computer vision and pattern recognition (CVPR), Madison, WI, USA (pp. II: 93–99), 2006.
Author information
Authors and Affiliations
Corresponding author
Additional information
The work of A.F.F. was supported by the Spanish Ministry of Education and Science under a Ramon y Cajal Research Fellowship. This work was partially developed within the framework of the CENIT-CDTEAM Project funded by the Spanish CDTI-MITYC, and also partially supported by grants MEC TEC2006-03617/TCM and ISCIII FIS2004/40676.
Rights and permissions
About this article
Cite this article
Hoogendoorn, C., Sukno, F.M., Ordás, S. et al. Bilinear Models for Spatio-Temporal Point Distribution Analysis. Int J Comput Vis 85, 237–252 (2009). https://doi.org/10.1007/s11263-009-0212-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-009-0212-6