Abstract
This chapter will describe how the cross-curricular learning area of digital competence has been conceptualized and assessed as part of national and international large-scale assessments. Digital competence can be conceived as an overarching concept that includes ICT literacy (or computer and information literacy) and computational thinking. This chapter will outline the conceptual background as well as the history of developing studies assessing students’ ICT-related knowledge, skills, and understanding, both in selected national assessments and in cross-national studies. It will discuss the challenges faced by assessments in this area with regard to new developments in digital technologies that impact on the aim of obtaining comparable measures of digital competence such as computer and information literacy and computational thinking across national contexts and over time.
Taking the example of the IEA International Computer and Information Literacy Study (ICILS) as the only study dedicated to comprehensively assessing this learning area from a cross-national perspective, this chapter outlines the development of its conceptual framework, the assessment strategy, and the broad range of contextual information gathered as part of this survey as well as the main outcomes from the second assessment cycle conducted in 2018. This chapter also includes a description of the assessment of computational thinking as part of ICILS 2018 and describes the relationship of this new construct with young people’s computer and information literacy. While not indicating a unidimensional construct, results from ICILS 2018 show that both computer information literacy and computational thinking are positively correlated and have highly similar associations with contextual factors. However, there are some interesting differences, in particular, in their respective relationships with students’ gender.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Aesaert, K., van Braak, J., van Nijlen, D., & Vanderlinde, R. (2015). Primary school pupils’ ICT competences: Extensive model and scale development. Computers & Education, 81, 326–344. https://doi.org/10.1016/j.compedu.2014.10.021
Aesaert, K., van Nijlen, D., Vanderlinde, R., & van Braak, J. (2014). Direct measures of digital information processing and communication skills in primary education: Using item response theory for the development and validation of an ICT competence scale. Computers & Education, 76, 168–181. https://doi.org/10.1016/j.compedu.2014.03.013
Ainley, J. (2018). Students and their computer literacy: Evidence and curriculum implications. In J. Voogt, G. Knezek, R. Christensen, & K. W. Lai (Eds.), Second handbook of information technology in primary and secondary education (pp. 69–88). Springer International Handbooks of Education. Springer. https://doi.org/10.1007/978-3-319-53803-7_4-1
Australian Curriculum and Reporting Authority [ACARA]. (2018). National assessment program – ICT literacy years 6 & 10 2017 report. Australian Curriculum and Reporting Authority (ACARA).
Australian Curriculum and Reporting Authority [ACARA]. (2020). General capabilities.. Australian Curriculum. https://acara.edu.au/curriculum/foundation-year-10/general-capabilities
Bienkowski, M., Snow, E., Rutstein, D. W., & Grover, S. (2015). Assessment design patterns for computational thinking practices in secondary computer science: A first look (SRI technical report). SRI International. Retrieved from http://pact.sri.com/resources.html
Bocconi, S., Chioccariello, A., Detori, G., Ferrari, A., & Engelhardt, K. (2016). Developing computational thinking in compulsory education. Implications for policy and practice. Joint research centre report EUR 28295 EN. Publications Office of the European Union. Retrieved from https://doi.org/10.2791/792158.
Brese, F., & Mirazchiyski, P. (2013). Measuring students’ family background in large-scale education studies (IERI monograph series). Issues and methodologies in large-scale assessments (special issue 2).
Chen, G., Shen, J., Barth-Cohen, L., Jiang, S., Huang, X., & Eltoukhy, M. (2017). Assessing elementary students’ computational thinking in everyday reasoning and robotics programming. Computers & Education, 109, 162–175. Retrieved from https://doi.org/10.1016/j.compedu.2017.03.001
Claro, M., Preiss, D., San Martin, E., Jara, I., Hinostraoza, J. E., Valenzuela, S., Cortes, F., & Nussbaum, M. (2012). Assessment of 21st century ICT skills in Chile: Test design and results from high school level students. Computers & Education, 59, 1042–1053. https://doi.org/10.1016/j.compedu.2012.04.004
Ebbs, D., & Friedman, T. (2015). Translation and verification of ICILS 2013 instruments. In J. Fraillon, W. Schulz, T. Friedman, J. Ainley, & E. Gebhardt (Eds.), ICILS 2013 technical report (pp. 55–66). International Association for the Evaluation of Educational Achievement (IEA).
Educational Testing Service [ETS]. (2002). Digital transformation: A framework for ICT literacy. Educational Testing Service. http://www.ets.org/Media/Tests/Information_and_Communication_Technology_Literacy/ictreport.pdf
Fraillon, J. (2018). International studies on information technology literacy in education. In J. Voogt, G. Knezek, R. Christensen, & K. W. Lai (Eds.), Second handbook of information technology in primary and secondary education. Springer International Handbooks of Education.
Fraillon, J. (2020). ICILS 2018 test development. In J. Fraillon, J. Ainley, W. Schulz, T. Friedman, & D. Duckworth (Eds.), ICILS 2018 technical report (pp. 11–30). International Association for the Evaluation of Educational Achievement (IEA).
Fraillon, J., Ainley, J., Schulz, W., Duckworth, D., & Friedman, T. (2019). IEA international computer and information literacy study 2018 assessment framework. Springer. Retrieved from https://www.springer.com/gp/book/9783030193881
Fraillon, J., Ainley, J., Schulz, W., Friedman, T., & Duckworth, D. (2020). Preparing for life in a digital world: IEA international computer and information literacy study, 2018 international report. International Association for the Evaluation of Educational Achievement.
Fraillon, J., Carstens, R., & Meyer, S. (2020). Computer-based assessment systems. In J. Fraillon, J. Ainley, W. Schulz, T. Friedman, & D. Duckworth (Eds.), ICILS 2018 technical report (pp. 31–38). International Association for the Evaluation of educational achievement (IEA).
Fraillon, J., Schulz, W., & Ainley, J. (2013). International computer and information literacy study: Assessment framework. International Association for the Evaluation of Educational Achievement.
Fraillon, J., Schulz, W., & Ainley, J. (2014). Preparing for life in a digital age: The international computer and information literacy study international report. International Association for the Evaluation of Educational Achievement.
Griffin, P., & Care, E. (2015). Assessment and teaching of 21st century skills: Methods and approach. Springer.
Grover, S., & Pea, R. (2013). Computational thinking in K–12: A review of the state of the field. Educational Researcher, 42(1), 38–43.
Hatlevik, O. (2009). How to identify and understand digital literacy among 9th grade Norwegian students: Examining the influences from school and home on students’ digital literacy. Nordic Journal of Digital Literacy, 4, 159–175.
Kaarakainen, M., Kivinen, O., & Vainio, T. (2018). Performance-based testing for ICT skills assessing: A case study of students and teachers’ ICT skills in Finnish schools. Universal Access in the Information Society, 17(2), 349–360.
Kim, H. S., Kil, H. J., & Shin, A. (2014). An analysis of variables affecting the ICT literacy level of Korean elementary school students. Computers & Education, 77, 29–38. Retrieved from https://doi.org/10.1016/j.compedu.2014.04.009.
Kim, J., & Lee, W. (2013). Meanings of criteria and norms: Analyses and comparisons of ICT literacy competencies of middle school students. Computers & Education, 64, 81–94. Retrieved from https://doi.org/10.1016/j.compedu.2012.12.018.
Lähdemäki, J. (2019). Case study: The Finnish national curriculum 2016 – A co-created national education policy. In J. Cook (Ed.), Sustainability, human Well-being, and the future of education. Palgrave Macmillan. https://doi.org/10.1007/978-3-319-78580-6_13
Lockheed, M., & Mandinach, E. B. (1986). Trends in educational computing: Decreasing interest and the changing focus of instruction. Educational Researcher, 15(5), 21–26.
Lye, S. Y., & Koh, J. H. L. (2014). Review on teaching and learning of computational thinking through programming: What is next for K-12? Computers in Human Behavior, 41, 51–61. Retrieved from https://doi.org/10.1016/j.chb.2014.09.012.
Mullis, I. V. S., Martin, M. O., Foy, P., & Hooper, M. (2017). ePIRLS 2016 international results in online informational reading. Retrieved from Boston College, TIMSS & PIRLS International Study Center website: http://timssandpirls.bc.edu/pirls2016/international-results/
National Assessment Governing Board [NAGB]. (2013). 2014 abridged technology and engineering literacy framework for the 2014 national assessment of educational progress. National Assessment Governing Board.
National Center for Educational Statistics [NCES]. (2016). 2014 nations report card: Technology & Engineering Literacy (TEL). https://www.nationsreportcard.gov/tel_2014/
National Center for Educational Statistics [NCES]. (2020). NAEP Technology & Engineering Literacy (TEL) report card. https://www.nationsreportcard.gov/tel/
Ockwell, L., Draganov, A., & Schulz, W. (2020). Scaling procedures for ICILS 2018 test items. In J. Fraillon, J. Ainley, W. Schulz, T. Friedman, & D. Duckworth (Eds.), ICILS 2018 technical report (pp. 133–158).
Organisation for Economic Cooperation and Development [OECD]. (2011). PISA 2009 results: Students on-line digital technologies and performance. OECD.
Organisation for Economic Cooperation and Development [OECD]. (2013). OECD skills outlook 2013: First results from the survey of adult skills. OECD. https://doi.org/10.1787/9789264204256-en.
Organisation for Economic Cooperation and Development [OECD]. (2016). PISA 2015 results volume V. Collaborative problem solving. OECD. https://doi.org/10.1787/9789264285521-en.
Orti-Colon, A. M., & Marato Romo, J. L. (2016). Teaching with scratch in compulsory secondary education. International Journal of Emerging Technologies in Learning, 11(2), 67–70. Retrieved from https://doi.org/10.3991/ijet.v11i02.5094.
Pelgrum, W. J., & Plomp, T. J. (1993). What do students know, learn and think about computers? In W. J. Pelgrum, I. A. J. Rienen, & T. J. Plomp (Eds.), Schools, teachers, students and computers: A cross-national perspective (IEA-Comped study stage 2) (pp. 49–70). IEA ERIC number: ED372734.
Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen, Denmark: Nielsen & Lydiche.
Rohatgi, A., Scherer, R., & Hatlevik, O. (2016). The role of ICT self-efficacy for students’ ICT use and their achievement in a computer and information literacy test. Computers & Education, 102, 103–116.
Rutstein, D. W., Snow, E. B., & Bienkowski, M. (2014). Computational thinking practices: Analyzing and modelling a critical domain in computer science education. Paper presented at the annual meeting of the American Educational Research Association. Retrieved from https://www.aera.net/PublicationsOnline-Paper-Repository/AERA-Online-Paper-Repository
Scheerens, J. (Ed.) (2017). Opportunity to learn, curriculum alignment and test preparation: A research review. Springer, Cham. https://doi.org/10.1007/978-3-319-43110-9
Schulz, W. (2020). The reporting of ICILS 2018 results. In J. Fraillon, W. Schulz, T. Friedman, & S. Meyer (Eds.), ICILS 2018 technical report (pp. 221–233). International Association for the Evaluation of Educational Achievement (IEA).
Schulz, W., & Friedman, T. (2020). Scaling procedures for ICILS 2018 questionnaire items. In J. Fraillon, J. Ainley, W. Schulz, T. Friedman, & D. Duckworth (Eds.), ICILS 2018 technical report (pp. 159–219). International Association for the Evaluation of Educational Achievement (IEA).
Shute, V. J., Sun, C., & Asbell-Clarke, J. (2017). Demystifying computational thinking. Educational Research Review, 22, 142–158. Retrieved from 10.1016/j.edurev.2017.09.003.
Siddiq, F., Hatlevik, O. E., Olsen, R. V., Throndsen, I., & Scherer, R. (2016). Taking a future perspective by learning from the past: A systematic review of assessment instruments that aim to measure primary and secondary school students’ ICT literacy. Educational Research Review, 19, 58–84.
Tatnall, A., & Davey, B. (Eds.). (2014). Reflections on the history of computers in education: Early use of computers and teaching about computing in schools (IFIP Advances in Information and Communication Technology). Springer.
Uerz, D., Volman, M., & Kral, M. (2018). Teacher educators’ competences in fostering student teachers’ proficiency in teaching and learning with technology: An overview of relevant research literature. Teaching and Teacher Education, 70, 12–23. https://doi.org/10.1016/j.tate.2017.11.005
United Nations. (2016). Global indicator framework for the sustainable development goals and targets for the 2030 agenda for sustainable development. Annex: A/RES/21/313. Retrieved from: https://unstats.un.org/sdgs/indicators/Global%20Indicator%20Framework_A.RES.71.313%20Annex.pdf
von Davier, M., Gonzalez, E., & Mislevy, R. (2009). What are plausible values and why are they useful? IERI Monograph Series: Issues and Methodologies in Large-Scale Assessments, 2, 9–36.
Voogt, J., Fisser, P., Good, J., Mishra, P., & Yadav, A. (2015). Computational thinking in compulsory education: Towards an agenda for research and practice. Education and Information Technologies, 20(4), 715–728.
Wang, J. (2005). Relationship between mathematics and science achievement at the 8th grade. International Online Journal of Science and Mathematics Education, 5, 1–17.
Wing, J. M. (2006). Computational thinking. Communications of the ACM, 49, 33–35. Retrieved from https://doi.org/10.1145/1118178.111821
Zhong, B., Wang, Q., Chen, J., & Li, Y. (2016). An exploration of three-dimensional integrated assessment for computational thinking. Journal of Educational Computing Research, 53(4), 562–590. Retrieved from https://journals.sagepub.com/doi/pdf/10.1177/0735633115608444
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Section Editor information
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this entry
Cite this entry
Schulz, W., Fraillon, J., Ainley, J., Duckworth, D. (2022). Digital Competences: Computer and Information Literacy and Computational Thinking. In: Nilsen, T., Stancel-Piątak, A., Gustafsson, JE. (eds) International Handbook of Comparative Large-Scale Studies in Education. Springer International Handbooks of Education. Springer, Cham. https://doi.org/10.1007/978-3-030-88178-8_43
Download citation
DOI: https://doi.org/10.1007/978-3-030-88178-8_43
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-88177-1
Online ISBN: 978-3-030-88178-8
eBook Packages: EducationReference Module Humanities and Social SciencesReference Module Education