Abstract
A general model is presented for homogeneous, dichotomous items when the answer key is not known a priori. The model is structurally related to the two-class latent structure model with the roles of respondents and items interchanged. For very small sets of respondents, iterative maximum likelihood estimates of the parameters can be obtained by existing methods. For other situations, new estimation methods are developed and assessed with Monte Carlo data. The answer key can be accurately reconstructed with relatively small sets of respondents. The model is useful when a researcher wants to study objectively the knowledge possessed by members of a culturally coherent group that the researcher is not a member of.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Batchelder, W. H., Maher, K. M., & Romney, A. K. (1986).Extensions of a model for test theory without an answer key. Manuscript submitted for publication. (Prepublication draft available upon request).
Batchelder, W. H., & Romney, A. K. (1986). The statistical analysis of a general Condorcet model for dichotomous choice situations. In B. Grofman & G. Owen (Eds.),Information pooling and group decision making (pp. 103–112). Greenwich, CT: JAI Press.
Blackwell, H. R. (1963). Neural theories of simple visual discriminations.Journal of the Optical Society of America, 53, 129–160.
Bliss, C. I. (1967).Statistics in biology (Vol. I). New York: McGraw-Hill.
Carré B. (1979).Graphs and networks. Oxford, England: Oxford University Press.
Clogg, Clifford C. (1981). New developments in latent structure analysis. In D. M. Jackson & E. F. Borgatta (Eds.),Factor analysis and measurement in sociological research. Beverly Hills, CA: Sage.
Comrey, A. L. (1962). The minimum residual method of factor analysis.Psychological Reports, 11, 15–18.
Goodman, L. A. (1974). Exploratory latent structure analysis using both identifiable and unidentifiable models.Biometrika, 61, 219–231.
Green, D. M., & Swets, J. A. (1966).Signal detection theory and psychophysics. New York: Wiley.
Grofman, B., Feld, S. L., & Owen, G. (1984). Group size and the performance of a composite group majority: Statistical truths and empirical results.Organizational Behavior and Human Performance, 33, 350–359.
Guilford, J. P. (1954).Psychometric methods (2nd ed.). New York: McGraw-Hill.
Hogg, R. V., & Craig, A. T. (1978).Introduction to mathematical statistics (4th ed.). New York: Macmillan.
Lazarsfeld, P. F., & Henry, N. W. (1968).Latent structure analysis. New York: Houghton Mifflin.
Loéve, M. (1963).Probability theory. Princeton: Van Nostrand.
Lord, F. M. (1974). Individualized testing and item characteristic curve theory. In D. H. Krantz, R. C. Atkinson, R. D. Luce, & P. Suppes (Eds.),Contemporary developments in mathematical psychology (Vol. 2). San Francisco: Freeman.
Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Reading, Mass.: Addison-Wesley.
Maher, K. M. (1987). A multiple choice model for aggregating group knowledge and estimating individual competencies. Unfinished doctoral dissertation, University of California, Irvine. (Available upon request.)
Nie, N. H., Hull, J. G., Jenkens, J. G., Steinbrenner, K., & Bent, D. M. (1975).Statistical package for the social sciences. New York: McGraw Hill.
Nitzan, S., & Paroush, J. (1982). Optimal decision rules in uncertain dichotomous choice situations.International Economics Review, 23, 289–297.
Rasch, G. (1960).Probabilistic models for some intelligence and attainment tests. Copenhagen: Danmarks Paedagogiske Institut.
Romney, A. K., Weller, S. C., & Batchelder, W. H. (1986). Culture as consensus: A theory of culture and informant accuracy.American Anthropologist, 88, 313–338.
Author information
Authors and Affiliations
Additional information
This research was supported by NSF Grant No. SES-8320173 to the authors. We gratefully acknowledge comments and suggestions from John Boyd, Tarow Indow, and Kathy Maher as well as the editor and several anonymous referees.
Rights and permissions
About this article
Cite this article
Batchelder, W.H., Romney, A.K. Test theory without an answer key. Psychometrika 53, 71–92 (1988). https://doi.org/10.1007/BF02294195
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF02294195