Abstract
This paper offers a framework that has been used for both developing the tasks used to measure literacy in the International Adult Literacy Survey and for understanding the meaning of what has been reported with respect to the comparative literacy proficiencies of adults in participating countries. The framework consists of six parts that represent a logical sequence of steps from needing to define and represent a particular domain of interest, to identifying and operationalizing characteristics used to construct items, to providing an empirical basis for interpreting results. The various parts of the framework are seen as important in that they help to provide a deeper understanding of the construct of literacy and the various processes associated with it. A processing model is proposed and variables associated with performance on the literacy tasks are identified and verified through regression analyses. These variables are shown to account for between 79 and 89 percent of the variance in task difficulty. Collectively, these process variables provide a means for moving away from interpreting performance on large-scale surveys in terms of discrete tasks or a single number towards identifying levels of performance that have generalizability across pools of tasks and towards what Messick has called a higher level of measurement.
Résumé
L’article présente un cadre qui permet l’élaboration de tâches pour mesurer la compréhension de l’écrit ainsi qu’un dépistage plus précis du sens des données analogues dont nous disposons relatives à la compréhension de l’écrit parmi les adultes des pays participants. Le cadre consiste de six parties dont la séquence logique va de la nécessité de définir et de représenter un domaine d’intérêt spécifique à l’établissement d’une base empirique pour l’interprétation des résultats, en passant par l’identification et l’opérationnalisation des caractéristiques qui entrent dans la construction des items. L’importance des éléments de ce cadre se manifeste par leur potentiel de contribuer à la compréhension approfondie de la notion de compréhension de l’écrit et des processus divers qui lui sont associés. On propose un modèle de processus mental et procède à l’identification et à la vérification, moyennant des analyses de régression, des variables déterminant la performance dans les tâches de compréhension de l’écrit. On montre que ces variables expliquent de 79% à 89% de la variance relative à la difficulté des tâches. Dans leur totalité, ces variables de processus mental permettent de sortir du mesurage, courant dans les enquêtes à grande échelle, de la performance par tâches discrètes ou par simple échelle numérique pour arriver à une identification des niveaux de performance propres à être généralisés pour des groupes entiers de tâches et, par là, à ce que Messick a appelé un niveaux supérieur de measure.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Almond, R.G., & Mislevy, R.J. (1998).Graphical models and computerized adaptive testing (TOEFL Tech. Rep. No. 14). Princeton, NJ: Educational Testing Service.
Beach, R., & Appleman, D. (1984). Reading strategies for expository and literacy text types. In A. Purves & O. Niles (Eds.),Becoming readers in a complex society. Eighty-third yearbook of the National Society for the Study of Education. Chicago, IL: University of Chicago Press.
Beaton, A.E., & Allen, N.L. (1992). Interpreting scales through scale anchoring.Journal of Educational Statistics, 17, 191–204.
Cattell, R.B. (1966). The scree test for the number of factors.Multivariate Behavioral Research, 1, 140–161.
Clark, H., & Haviland, S.E. (1977). Comprehension and the given-new contract. In R.O. Freedle (Ed.),Discourse production and comprehension (pp. 1–39). Norwood, NJ: Ablex.
Cook-Gumperz, J., & Gumperz, J. (1981). From oral to written culture: The transition to literacy. In M. Whitman (Ed.),Writing: The nature, development and teaching of written communication (vol. 1). Hillsdale, NJ: Erlbaum.
Crandall, J. (1981).Functional literacy of clerical workers: Strategies for minimizing literacy demands and maximizing available information. Paper presented at the annual meeting of the American Association for Applied Linguistics (December). New York.
Diehl, W. (1980).Functional literacy as a variable construct: An examination of the attitudes, behaviors, and strategies related to occupational literacy. Unpublished doctoral dissertation, Indiana University.
Fisher, D.L. (1981). Functional literacy tests: A model of question-answering and an analysis of errors.Reading Research Quarterly, 16, 418–448.
Graff, H.J. (1979).The literacy myth. New York: Academic Press.
Guthrie, J.T. (1988). Locating information in documents: A computer simulation and cognitive model.Reading Research Quarterly, 23, 178–199.
Heath, S.B. (1980). The functions and uses of literacy.Journal of Communication, 30, 123–133.
Jacob, E. (1982).Literacy on the job: Final report of the ethnographic component of the industrial literacy project. Washington, DC: Center for Applied Linguistics.
Kirsch, I.S., & Guthrie, J.T. (1984a). Adult reading practices for work and leisure.Adult Education Quarterly, 34, 213–232.
Kirsch, I.S., & Guthrie, J.T. (1984b). Prose comprehension and text search as a function of reading volume.Reading Research Quarterly, 19, 331–342.
Kirsch, I.S., & Jungeblut, A. (1986).Literacy: Profiles of America’s young adults — Final report (NAEP Report No. 16-PL-OI). Princeton, NJ: National Assessment of Educational Progress.
Kirsch, I.S., & Jungeblut, A. (1992).Profiling the literacy proficiencies of JTPA and ES/UI populations: Final report to the Department of Labor. Princeton, NJ: Educational Testing Service.
Kirsch, I.S., & Mosenthal, P.B. (1990). Exploring document literacy: Variables underlying the performance of young adults.Reading Research Quarterly, 25, 5–30.
Kirsch, I.S., & Mosenthal, P.B. (1994). Interpreting the IEA Reading Literacy Scales. In M. Binkley, K. Rust, & M. Winglee (Eds.),Methodological Issues in Comparative Educational Studies: The case of the IEA Reading Literacy Study (pp. 135–192). Washington, DC: U.S. Department of Education, National Center for Education Statistics.
Kirsch, I., Jungeblut, A., & Mosenthal, P.B. (1998). The measurement of adult literacy. In T.S. Murray, I.S. Kirsch, & L. Jenkins (Eds.),Adult literacy in OECD countries: Technical report on the first international adult literacy survey (pp. 105–134). Washington, DC: U.S. Department of Education, National Center for Education Statistics.
Kirsch, I.S., Jungeblut, A., Jenkins, L., & Kolstad, A. (1993).Adult literacy in America: A first look at the results of the National Adult Literacy Survey. Washington, DC: U.S. Department of Education.
Lerner, D., & Lasswell, H.D. (1951).The policy sciences: Recent developments in scope and method. Stanford, CA: Stanford University Press.
Messick, S. (1987). Large-scale educational assessment as policy research: Aspirations and limitations.European Journal of Psychology and Education, 2, 157–165.
Messick, S. (1989). Validity. In R. Linn (Ed.),Educational Measurement (3rd ed.). New York: Macmillan.
Messick, S., Beaton, A., & Lord, F. (1983).National Assessment of Educational Progress Reconsidered: A new design for a new era (NAEP Report No. 83-1). Princeton, NJ: National Assessment of Educational Progress.
Mikulecky, L. (1982). Job literacy: The relationship between school preparation and workplace actuality.Reading Research Quarterly, 17, 400–419.
Miller, P. (1982). Reading demands in a high-technology industry.Journal of Reading, 26, 109–115.
Montigny, G., Kelly, K., & Jones, S. (1991).Adult literacy in Canadaresults of a national study. Ottawa: Minister of Industry, Science and Technology (Statistics Canada, Catalogue No. 89-525-XPE).
Mosenthal, P.B., & Kirsch, I.S. (1989-1991). Understanding documents. A monthly column appearing in theJournal of Reading. Newark, DE: International Reading Association.
Mosenthal, P.B., & Kirsch, I.S. (1991). Toward an explanatory model of document process.Discourse Processes, 14, 147–180.
Mosenthal, P.B., & Kirsch, I.S. (1998). A new measure for assessing document complexity: The PMOSE/IKIRSCH document readability formula.Journal of Adolescent and Adult Literacy, 41, 638–657.
National Assessment of Educational Progress (NAEP). (1972).Reading: Summary (Report 02-R-00). Denver, CO: Education Commission of the States.
Organisation for Economic Co-operation and Development (OECD). (1999).Measuring Student Knowledge and Skills: A New Framework for Assessment. Paris: OECD.
Organisation for Economic Co-operation and Development (OECD) and Human Resources Development Canada (HRDC). (1997).Literacy Skills for the Knowledge Society: Further Results of the International Adult Literacy Survey. Paris and Ottawa: OECD and HRDC.
Organisation for Economic Co-operation and Development (OECD) and Statistics Canada (STATCAN). (1992).Adult illiteracy and economic performance. Paris: OECD.
Organisation for Economic Co-operation and Development (OECD) and Statistics Canada (STATCAN). (1995).Literacy, Economy and Society: Results of the First International Adult Literacy Survey. Paris and Ottawa: OECD and STATCAN.
Organisation for Economic Co-operation and Development (OECD) and Statistics Canada (STATCAN). (2000).Literacy in the Information Age: Final report of the International Adult Literacy Survey. Paris and Ottawa: OECD and STATCAN.
Resnick, D., & Resnick, L. (1977). The nature of literacy — An historical exploration.Harvard Educational Review, 43, 370–385.
Scribner, S., & Cole, M. (1981).The psychology of literacy. Cambridge, MA: Harvard University Press.
Sticht, T.G. (Ed.). (1975).Reading for working: A functional literacy anthology. Alexandria, VA: Human Resources Research Organization.
Sticht, T. (1978).Literacy and vocational competency. Columbus, OH: Ohio State University (Occasional Paper 39, National Center for Research in Vocational Education).
Sticht, T. (1982).Evaluation of the reading potential concept for marginally literate adults (Final Report FR-ET50-82-2). Alexandria, VA: Human Resources Research Organization.
Szwed, J. (1981). The ethnography of literacy. In M. Whitman (Ed.),Writing: The nature development, and teaching of written communication (vol. 1). Hillsdale, NJ: Lawrence Erlbaum Associates.
Venezky, R.L. (1983). The origins of the present-day chasm between adult literacy needs and school literacy instruction.Visible Language, 16, 113–136.
Werlich, E. (1976).A text grammar of English. Heidelberg: Quelle & Meyer.
Wickert, R. (1989).No single measure. Canberra, Australia: The Commonwealth Department of Employment, Education and Training.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Kirsch, I.S. The framework used in developing and Interpreting the International Adult Literacy Survey (IALS). Eur J Psychol Educ 16, 335–361 (2001). https://doi.org/10.1007/BF03173187
Received:
Issue Date:
DOI: https://doi.org/10.1007/BF03173187