Abstract
A critical issue in analyzing multi-item scales is missing data treatment. Previous studies on this topic in the framework of item response theory have shown that imputation procedures are in general associated with more accurate estimates of item location and discrimination parameters under several missing data generating mechanisms. This paper proposes a model-based multiple imputation procedure for multiple categorical items (dichotomous, multinomial or Likert-type) which relies on the results of latent class analysis to impute missing item responses. The effectiveness of the proposed technique is assessed in the estimation of item response theory parameters using a range of ad hoc measures. The accuracy of the method is assessed with respect to other single and multiple imputation procedures, under different missing data generating mechanisms and different rate of missingness (5% to 30%). The simulation results indicate that the proposed technique performs satisfactorily under all conditions and has the greatest potential with severe rates of missingness and under non ignorable missing data mechanisms. The method was implemented in R code with a function that calls scripts from a latent class analysis routine.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
AGRESTI, A. (2002), Categorical Data Analysis, Hoboken: Wiley-Interscience.
AKE, C.F. (2005), “Rounding After Multiple Imputation with Non-Binary Categorical Covariates”, paper presented at the annual meeting of the SAS User Group International, Philadelphia.
BAKER, F.B., and KIM, S.H. (2004), Item Response Theory: Parameter Estimation Techniques, New York: Dekker.
BARALDI, A.N., and ENDERS, C.K. (2010), “An Introduction to Modern Missing Data Analyses”, Journal of School Psychology, 48, 5–37.
BERNAARDS, C.A., and SIJTSMA, K. (1999), “Factor Analysis of Multidimensional Polytomous Item Response Data from Ignorable Item Non Response”, Multivariate Behavioral Research, 34, 277–314.
BIRNBAUM, A. (1968), “Statistical Theories of Mental Test Scores”, in Some Latent Trait Models and Their Use in Inferring an Examinee’S Ability, eds F.M. Lord and M.R. Novick, Reading: Addsion-Wesley, pp. 395–497.
CARPITA, M., and MANISERA, M. (2011), “On the Imputation of Missing Data in Surveys with Likert-Type Scales”, Journal of Classification, 28, 93–112,
EDELEN, M.O., and REEVE, B.B. (2007), “Applying Item Response Theory (IRT) Modeling to Questionnaire Development, Evaluation and Refinement”, Quality of Life Researches, 16, 5–18
ENDERS, G.K. (2004), “The Impact of Missing Data on Sample Reliability Estimates: Implications of Reliability Reporting Practices”, Educational and Psychological Measurement, 64(3), 419–436,
FINCH, H. (2008), “Estimation of Item Response Theory Parameters in the Presence of Missing Data”, Journal of Educational Measurement, 45(3), 225–245.
FINCH, H. (2010), “Imputation Methods for Missing Categorical Questionnaire Data: A Comparison of Approaches”, Journal of Data Science, 8(8), 361–378.
FINCH, H. (2011), “The Impact of Missing Data on the Detection of Nonuniform Differential Item Functioning”, Educational and Psychological Measurement, 71(4), 663–683.
HUISMAN, M. (1999), Item Nonresponse: Occurence, Causes, and Imputation of Missing Answers to Test Items, Leiden, The Netherlands: DSWO Press.
LINZER, D.A., and LEWIS, J. (2011), “poLCA: Polytomous Variable Latent Class Analysis”, Journal of Statistical Software, 42(10), 1–29.
LITTLE, R.J.A., and RUBIN, D.B. (2002), Statistical Analysis with Missing Data (2nd. ed.), New York: John Wiley.
MULLIS, I.V.S., MARTIN, M.O., FOY, P., and DRUCKER, K.T. (eds.) (2012), “PIRLS 2011 International Results in Reading”, 2012 International Association for the Evaluation of Educational, Chestnut Hill MA: TIMSS & PIRLS International Study Center Boston College.
NYLUND, K., ASPAROUHOV, T., and BENGT, O.M. 2007, “Deciding on the Number of Classes in Latent Class Analysis and Growth Mixture Modeling: A Monte Carlo Simulation Study”, Structural Equation Modeling, 14(4), 535–569.
RAAIJMAKERS, A.W. (1999), “Effectiveness of Different Missing Data Treatments in Surveys with Likert-Type Data: Introducing the Relative Mean Substitution Approach”, Educational and Psychological Measurement, 59(5), 725–748.
RAGHUNATHAN, T.E., LEPKOWSKI, J.M., VAN HOEWYK, J., and SOLENBERGER, P. (2001), “A Multivariate Technique for Multiply Imputing Missing Values Using a Sequence of Regression Models”, Survey Methodology, 27, 85–95.
RIZOPOULOS, D. (2006), “ltm: An R package for Latent Variable Modelling and Item Response Theory Analyses”, Journal of Statistical Software, 17(5), 1–25.
RUBIN, D. (1976), “Inference and Missing Data”, Biometrika, 63, 581–592.
SAMEJIMA, F. (1969), “Estimation Of Ability Using a Response Pattern of Graded Scores”, Psychometrika Monograph, 17.
SCHAFER, J. (1997), Analysis of Incomplete Multivariate Data, London: Chapman and Hall.
SCHAFER, J., and GRAHAM, J.W. (2002), “Missing Data: Our View of the State of the Art”, Psychological Methods, 7(2), 147–177.
SIJTSMA, K., and VAN DER ARK, L.A., (2003), “Investigation and Treatment of Missing Item Scores in Test and Questionnaire Data”, Multivariate Behavioral Research, 38(4), 505–528.
SULIS, I. (2013), “A Further Proposal to Perform Multiple Imputation on a Bunch of Polythomous Items based on Latent Class Analysis”, in Statistical Models for Data Analysis: Studies in Classification, Data Analysis, and Knowledge Organization, eds. P. Giudici, S. Ingrassia, and M. Vichi, Heidelberg: Springer-Verlag.
SULIS, I., and PORCU, P. (2008), “Assessing the Effectiveness of a Stochastic Regression Imputation Method for Ordered Categorical Data”, CRENoS Working Papers, 4.
VAN BUUREN, S., and OUDSHOORN, C.G.M. (2011), “MICE: Multivariate Imputation by Chained Equations”, Journal of Statistical Software, 45(3), 1–67.
VERMUNT, J.K, VAN GINKEL, J.R., VAN DER ARK, L.A., and SIJTSMA, K. (2008), “Multiple Imputation of Categorical Data Using Latent Class Analysis”, Sociological Methodology, 33, 269–297.
WU, W., JIA, F., and ENDERS, C. (2015), “A Comparison of Imputation Strategies for Ordinal Missing Data on Likert Scale Variables”, Multivariate Behavioral Research, 50, 484–503.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
ESM 1
(PDF 94 kb)
Rights and permissions
About this article
Cite this article
Sulis, I., Porcu, M. Handling Missing Data in Item Response Theory. Assessing the Accuracy of a Multiple Imputation Procedure Based on Latent Class Analysis. J Classif 34, 327–359 (2017). https://doi.org/10.1007/s00357-017-9220-3
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00357-017-9220-3