Abstract
In this paper, we propose a Bayesian framework for estimating finite mixtures of the LISREL model. The basic idea in our analysis is to augment the observed data of the manifest variables with the latent variables and the allocation variables. The Gibbs sampler is implemented to obtain the Bayesian solution. Other associated statistical inferences, such as the direct estimation of the latent variables, establishment of a goodness-of-fit assessment for a posited model, Bayesian classification, residual and outlier analyses, are discussed. The methodology is illustrated with a simulation study and a real example.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Albert, J.H., & Chib, S. (1993). Bayesian analysis of binary and polychotomous response data.Journal of the American Statistical Association, 88, 669–679.
Albert, J.H., & Chib, S. (1995). Bayesian residual analysis for binary regression models.Biometrika, 82, 747–759.
Arminger, G., & Muthén, B.O. (1998). A Bayesian approach to nonlinear latent variable models using the Gibbs sampler and the Metropolis-Hastings algorithm.Psychometrika, 63, 271–300.
Barnett, V., & Lewis, T. (1984).Outliers in statistical data (2nd ed). Chichester: Wiley & Sons.
Bentler, P.M. (1983). Some contributions to efficient statistics for structural models: Specification and estimation of moment structures.Psychometrika, 48, 493–517.
Bentler, P.M. (1992).EQS: Structural equation program manual. Los Angeles, CA: BMDP Statistical Software.
Besag, J., Green, P., Higdon, D., & Mengersen, K. (1995). Bayesian computation and stochastic systems.Statistical Science, 10, 3–66.
Bollen, K.A., & Long, J.S. (Eds.). (1993).Testing structural equation models. Newbury Park, CA: Sage.
Broemeling, L.D. (1985).Bayesian analysis of linear models. New York, NY: Marcel Dekker.
Browne, M.W. (1984). Asymptotically distribution-free methods in the analysis of covariance structures,British Journal of Mathematical and Statistical Psychology, 37, 62–83.
Byrne, B.M. (1994).Structural equation modeling with EQS and EQS/Windows, Thousand Oaks, CA: Sage.
Chaloner, K. (1991). Bayesian residual analysis in the presence of censoring.Biometrika, 78, 637–644.
Chaloner, K., & Brant, R. (1988). A Bayesian approach to outlier detection and residual analysis.Biometrika, 75, 651–659.
Cook, R.D., & Weisberg, S. (1982).Residuals and influence in regression. London: Chapman and Hall.
Crawford, S.L., DeGroot, M.H., Kadane, J.B., & Small, M.J. (1992). Modeling lake chemistry distributions: Approximate Bayesian methods for estimating a finite mixture model.Techometrics 34, 441–453.
Day, N.E. (1969). Estimating the components of a mixture of normal distributions,Biometrika, 56, 463–474.
Dempster, A.P., Laird, N.M., & Rubin, D.B. (1977). Maximum likelihood from incomplete data via the EM algorithm (with discussion).Journal of the Royal Statistical Society, Series B, 39, 1–38.
Diebolt, J., & Robert, C.P. (1994). Estimation of finite mixture distributions through Bayesian sampling.Journal of the Royal Statistical Society, Series B, 56, 163–175.
Do, K., & McLachlan, G.J. (1984). Estimation of mixing proportions: A case study.Applied Statistics, 33, 134–140.
Dolan, C.V., & van der Maas, Han L.J. (1998). Fitting multivariate normal finite mixtures subject to structural equation modeling.Psychometrika, 63, 227–254.
Ganesalingam, S., & McLachlan, G.J. (1981). Some efficiency results for the estimation of the mixing proportion in a mixture of two normal distributions.Biometrics, 37, 23–33.
Gelfand, A.E., & Smith, A.F.M. (1990). Sampling-based approaches to calculating marginal densities.Journal of the American Statistical Association, 85, 398–409.
Gelman, A., Carlin, J.B., Stern, H.S., & Rubin, D.B. (1995).Bayesian data analysis. London: Chapman and Hall.
Gelman, A., Meng, X.L., & Stern, H. (1996). Posterior predictive assessment of model fitness via realized discrepancies.Statistics Sinica, 6, 733–807.
Gelman, A., & Rubin, D.B. (1992). Inference from iterative simulation using multiple sequences.Statistical Science, 7, 457–472.
Geman, S., & Geman, D. (1984). Stochastic relaxation, Gibbs distribution, and the Bayesian restoration of images.IEEE Transactions on Pattern Analysis and Machine Intelligence, 6, 721–741.
Geyer, C.J. (1992). Practical Markov chain Monte Carlo.Statistical Science, 7, 473–511.
Gilks, W.R., Richardson, S., & Spiegelhalter, D.J. (1996). Introducing Markov chain Monte Carlo. In W. R. Gilks, S. Richardson and D. J. Spiegelhalter (Eds.),Markov Chain Monte Carlo in practice (pp. 1–19). London: Chapman and Hall.
Hathaway, R.J. (1985). A constrained formulation of maximum-likelihood estimation for normal mixture distributions.The Annals of Statistics, 13, 795–800.
Hasselblad, V. (1966). Estimation of parameters for a mixture of normal distributions,Technometrics, 8, 431–444.
Hawkins, D.M. (1980).Identification of outliers. London: Chapman and Hall.
Hoyle, R. (Ed.). (1995).Structural equation modeling: Concepts, issues and applications. Thousand Oaks, CA: Sage.
Jedidi, K., Jagpal, H.S., & DeSarbo, W.S. (1997a). Finite-mixture structural equation models for response-based segmentation and unobserved heterogeneity.Marketing Science, 16, 39–59.
Jedidi, K., Jagpal, H.S., & DeSarbo, W.S. (1997b). STEMM: A general finite mixture structural equation model.Journal of Classification, 14, 23–50.
Jöreskog, K.G. (1978). Structural analysis of covariance and correlation matrices.Psychometrika, 43, 443–477.
Jöreskog, K.G., & Sörbom, D. (1996).LISREL 8: Structural equation modeling with the SIMPLIS command language. Hove and London: Scientific Software International.
Lee, S.Y. (1981). A Bayesian approach to confirmatory factor analysis.Psychometrika, 46, 153–160.
Lee, S.Y., & Poon, W.Y. (1992). Two-level analysis of covariance structures for unbalanced designs with small level-one samples.British Journal of Mathematical and Statistical Psychology, 45, 109–124.
Lee, S.Y., & Tsui, K.L. (1982). Covariance structure analysis in several populations.Psychometrika, 57, 409–426.
Lindsay, B.G. (1989). Moment matrices: Applications in mixtures.The Annals of Statistics, 17, 722–740.
Lindsay, B.G., & Baska, P. (1993). Multivariate normal mixtures: A fast consistent method of moments.Journal of the American Statistical Association, 88, 468–476.
Lindley, D.V., & Smith, A.F.M. (1972). Bayes estimates for the linear model (with discussion).Journal of the Royal Statistical Society, Series B, 34, 1–42.
Louis, T.A. (1982). Finding the observed information matrix when using EM algorithm.Journal of the Royal Statistical Society, Series B, 44, 226–233.
Meng, X.L. (1994). Posterior predictivep-values.The Annals of Statistics, 22, 1142–1160.
Mood, A.M., Graybill, F.A., & Boes, D.C. (1974).Introduction to the theory of statistics (3rd ed.). London: McGraw-Hill.
Muthén, B. (1989). Latent variable modeling in heterogeneous populations.Psychometrika, 54, 557–585.
Raiffa, H., & Schlaifer, L.R. (1961).Applied statistical secision theory (2nd ed.). Boston, MA: Harvard, Graduate School of Business Administration.
Redner, R.A., & Walker, H.F. (1984). Mixture densities, maximum likelihood and the EM algorithm.SIAM Review, 26, 195–239.
Richardson, S., & Green, P.J. (1997). On Bayesian analysis of mixtures with an unknown number of components (with discussion).Journal of the Royal Statistical Society, Series B, 59, 731–792.
Robert, C.P. (1996).Mixtures of distributions: Inference and estimation. In W.R. Gilks, S. Richardson, & D.J. Spiegelhalter (Eds.),Markov Chain Monte Carlo in practice (pp. 163–187). London: Chapman and Hall.
Roeder, K., & Wasserman, L. (1997). Practical Bayesian density estimation using mixtures of normals.Journal of the American Statistical Association, 92, 894–902.
Rubin, D.B. (1984).Bayesianly justifiable and relevant frequency calculations for the applied statistician.The Annals of Statistics, 12, 1151–1172.
Scheines, R., Hoijtink, H., & Boomsma, A. (1999). Bayesian estimation and testing of structural equation models.Psychometrika, 64, 37–52.
Shi, J.Q., & Lee, S.Y. (1998).Bayesian sampling-based approach for factor analysis models with continuous and polytomous data.British Journal of Mathematical and Statistical Psychology, 51, 233–252.
Smith, A.F.M., & Makov, U.E. (1978). A quasi-Bayes sequential procedure for mixtures.Journal of the Royal Statistical Society, Series B, 40, 106–112.
Titterington, D.M., Smith, A.F.M., & Makov, U.E. (1985).Statistical analysis of finite mixture distributions. Chichester: Wiley & Sons.
van Buuren, S. (1997). Fitting ARMA time series by structural equation models.Psychometrika, 62, 215–236.
Weiss, R.E. (1994).Residuals and outliers in Bayesian random effects models (Tech. Rep.). Los Angeles, CA: UCLA, School of Public Health, Department of Biostatistics.
Weiss, R.E., & Lazaro, C.G. (1992). Residual plots for repeated measures.Statistics in Medicine, 11, 115–124.
World Values Survey, 1981–1984 and 1990–1993 (ICPSR version). (1994). Ann Arbor, MI: Institute for Social Research [producer], Inter-university Consortium for Political and Social Research [distributor].
Yuan, K.E., & Bentler, P.M. (1997). Mean and covariance structures analysis: theoretical and practical improvements.Journal of the American Statistical Association, 92, 767–774.
Yung, Y.F. (1997). Finite mixtures in confirmatory factor-analysis models.Psychometrika, 62, 297–330.
Zeger, S.L., & Karim, M.R. (1991). Generalized linear models with random effects: A gibbs sampling approach.Journal of the American Statistical Association, 86, 79–86.
Zellner, A. (1971).An Introduction to Bayesian Inference in Econometrics, New York, NY: Wiley & Sons.
Zhu, H.T., & Lee, S.Y. (1999). Statistical analysis of nonlinear factor analysis models.British Journal of Mathematical and Statistical Psychology, 52, 225–242.
Author information
Authors and Affiliations
Corresponding author
Additional information
This research was supported by a Hong Kong UGC Earmarked grant CUHK 4026/97H. The authors are indebted to the Editor, the Associate Editor, and three anonymous reviewers for constructive comments in improving the paper, and also to ICPSR and the relevant funding agency for allowing the use of the data. The assistance of Michael K.H. Leung and Esther L.S. Tam is gratefully acknowledged.
Rights and permissions
About this article
Cite this article
Zhu, HT., Lee, SY. A Bayesian analysis of finite mixtures in the LISREL model. Psychometrika 66, 133–152 (2001). https://doi.org/10.1007/BF02295737
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF02295737