1 Introduction

With the rapid advancements in technology, a new literacy concept that defines the relationships with technological tools has come into our lives nowadays in addition to the qualifications such as mathematics and science literacy sought in students. This concept, known as computer and information literacy, was defined by Fraillon et al. (2013, p. 17) as “an individual’s ability to use computers to investigate, create and communicate in order to participate effectively at home, at school, in the workplace and in society”. Gilster (1997) argues that this skill, which is based on interaction with a wide variety of sources, is an essential skill for success in the digital age. Bawden (2008) states that this skill includes collecting information, reading and understanding hypermedia texts, locating the desired information and evaluating it critically and then working in cooperation to communicate the information. While explaining the differences between the concepts of computer and information literacy (CIL) and information and communication technology (ICT), Fraillon et al. (2013) express that CIL has to do more with computer context while ICT, beyond this, has a nature considering these technologies as basic learning instruments.

The concept of CIL includes information about software and hardware applications and understanding of technological concepts, rather than expressing only a form of literacy in the sense of using technology (Kuhlemeier & Hemker, 2007). With the widespread digitalization in every walk of life, computer and information literacy is considered as a necessary skill in almost every field as well as a basic criterion for employment (Martin, 2006). On the other hand, the European Commission states that information and data literacy, digital content creation, communication and collaboration skills are among the basic digital competencies that are thought to be acquired by twenty-first century students (Carretero et al., 2017). Developing these competencies as a policy will help students to cope with their educational or professional problems in the future by themselves and to ensure social equality by closing the development gaps between individuals in this respect. Therefore, students’ digital competencies have started to be measured based on theoretical frameworks (ACARA, 2015; Claro et al., 2018; Fraillon et al., 2013; Lorenceau et al., 2019; Senkbeil et al., 2013) determined through various large-scale initiatives [ICILS (International), PISA (International), ACARA (Australia) TIDE (Chile), TILT (Germany)].

We know that students interact with computers at home or other environments at greater degree than they do at school (Fraillon et al., 2014). This case results in the fact that students gain computer skills on their own (Wittwer & Senkbeil, 2008). On the other hand, although it is known that teachers direct students towards these skills for the activities they request from their students such as presentation, activity etc. both through in-class applications and through out-of-class assignments, it is also known that the teachers do not have sufficient guidance for this process (Hsu, 2011). In the national and international evaluations, the contributions of students' purposes of using computers in in-school / out-of-school environments (homework completion, entertainment, communication, getting news, etc.), frequency of use, and teachers' practices on the CIL developed in students are examined.

In these studies, it is highlighted that students’ characteristics such as self-efficacy levels (Scherer et al., 2017), motivational characteristics (Senkbeil & Ihme, 2017) or gender (Gebhardt et al., 2019) are featured student characteristics in their CIL achievements. On the other hand, there are studies in the literature reflecting adverse views regarding the variable of gender (Claro et al., 2012; Hatlevik & Christophersen, 2013; Kim et al., 2014), there are also studies featuring students’ attitudes (Senkbeil et al., 2013) and their purposes of use (Thompson, 2013; van Deursen & van Dijk, 2014). Besides, the variables such as teachers’ cooperation, participation in professional development (Drossel & Eickelmann, 2017a), ICT acceptance levels (Kreijns et al., 2013; Scherer et al., 2015), beliefs regarding in-class usage (Chien et al., 2014) and teacher self-efficacy (Mumtaz, 2000) are listed as significant variables in the literature.

The literature suggests that a school-wide common vision plan contributes a lot to ICT integration (Tondeur et al., 2008). Research has demonstrated that this integration is also significant with respect to students’ CIL skills. Besides, teacher characteristics also contributes to students’ CIL skills (Gerick et al., 2017; Lorenz et al., 2019). The majority of the studies addressed only student characteristics or teacher characteristics. However, the embedded school characteristics as well as student characteristics affect students’ CIL achievement. Therefore, these hierarchical characteristics should be addressed accordingly. With the current study, it is aimed to examine the relationships in the effect of teacher and student characteristics on CIL achievement, which are not clear in the literature, based on International Computer and Information Literacy Study (ICILS) data, which deals with CIL skills at an international level. This study set out to reveal (a) the differentiation in CIL levels among schools, (b) student characteristics related to students’ CIL achievement, (c) teacher characteristics related to students’ achievements in the context of most successful countries.

2 Theoretical framework

2.1 Computer and information literacy and student characteristics

Among the demographic variables that have effect on students’ CIL levels, home background variables (Chinn & Fairlie, 2010; Fraillon et al., 2014; Hatlevik et al., 2015; Rosén & Gustafsson, 2016; Sutherland-Smith et al., 2003) such as gender (Gebhardt et al., 2019), socio-economic level of the family and parents’ education levels attract notice in the literature. Examining home background variables, Becker (2000) points out that parents of disadvantaged students are less likely to be digital literate. Considering that disadvantaged students’ ICT experiences (computer, smartphone, table pc, etc.) will be weaker, students' home-educational statuses are defined as their cultural capital (Claro et al., 2012).

Schunk and Pajares (2009) emphasize that students who have been provided a positive environment at home and who have received education-related support from their parents are likely to have good self-efficacy beliefs for school learning. The close relationship of students’ self-efficacy levels and self-perceptions with their learning is also observed in students' CIL achievements (Hatlevik et al., 2018; Porat et al., 2018; Rohatgi et al., 2016; Vekiri & Chronaki, 2008). Students' perceptions regarding the effects of ICT on society also have an effect on their CIL levels. Studies have reported that students' positive and negative views, particularly regarding non-academic ICT use, make a difference in their achievements (Salomon & Kolikant, 2016). Christensen and Knezek (2008) emphasized that positive opinions are higher in students who have access to ICT, which contributes positively to performance by affecting students’ efficacy perceptions.

Rohatgi et al. (2016) stress that students’ purposes of using ICT also have an important place in CIL levels. In a process that students mostly control their ICT usage on their own in the home setting (Zhong, 2011), it is seen that the use of related technologies for recreational purposes is ahead of their use for educational purposes (searching for the information needed, doing homework, etc.) (Pedro, 2010). Rohatgi et al. (2016) point out that use of ICT for recreational purposes affects achievement significantly through self-efficacy levels and that this purpose of use has an effect on students' CIL levels ahead of other uses (use of ICT for task learning, use of ICT for study purposes). Gebhardt et al. (2019) argue that this relationship between achievement and the use of ICT for recreational purposes, which is considered to distract students, may have emerged from the familiarity with ICT.

2.2 Computer and information literacy and teacher characteristics

In the literature, it is stated that teachers' attitudes towards ICT have an important place in teachers' employment of ICT in learning and teaching processes (Drossel et al., 2017a; Hatlevik & Hatlevik, 2018). Teachers' perceptions and acceptance levels of ICT play an important role as determinants of these attitudes. The old view that perceived utility levels were determinant for the integration of computers into education (Gressard & Loyd, 1985), is now accepted as valid for all ICT tools (Macedo, 2017). Therefore, how teachers integrate ICT into education is of interest to many researchers (Scherer et al., 2015). However, it is still not clear enough what kind of activities students carry out in activating their twenty-first century skills (Siddiq et al., 2016). Results based on international comparisons revealed that teachers' attitudes towards the use of ICT in learning and teaching processes create a difference in achievements across countries (Fraillon et al., 2019a). Particularly in these comparisons, the variable of teachers’ ICT-related self-efficacy comes to the fore (Hatlevik, 2017; Hatlevik & Hatlevik, 2018; Rohatgi et al., 2016). Additionally, teachers’ demographic characteristics (such as ICT experience or age) are also considered as teacher characteristics featured in research studies (Gil-Flores et al., 2017; Wong & Li, 2008).

Teachers’ types of ICT usage (Bai et al., 2016), the significance attached to deceloping students’ ICT skills (Berger, 2019) and collaborative use of ICT for teaching and learning (Drossel et al., 2017a, 2017b; Tondeur et al., 2012) can be listed as other featured variables. It is seen that these teacher variables, which stand out in the studies, are widely discussed in the context of the teacher, and studies are limited regarding their level of affecting students' CIL skills.

2.3 International Computer and Information Literacy Study (ICILS)

ICILS is a large-scale assessment conducted with randomly selected 8th grade students based on a nationally representative sample. This evaluation, conducted by the International Association for the Evaluation of Educational Achievement (IEA), was first conducted with 18 countries in 2013 (Fraillon et al., 2014). The second cycle of the implementation was conducted with 46,561 students and 26,530 teachers from 2226 schools in 12 countries (Fraillon et al., 2019a). ICILS is the first initiative that evaluates students' computer and communication literacy internationally (Fraillon et al., 2014). With this study, which measures the ICT competencies of the students, it is aimed to evaluate to what extent students know the basic skills of ICT, and how they understand and put into practice these skills. In the first cycle of the study, it was seen in the results that 17% of the students participating at the international level could not even reach the lowest level determined for the study (Fraillon et al., 2014). In the last cycle, it was concluded that less than 25% of students reached two out of five achievement levels. This result shows that students cannot reach basic computer skills substantially in order to participate in society (Fraillon et al., 2019a). These results and similar results, which are deemed important at the international level, have resulted in the inclusion of the ICILS study in education monitoring studies by UNESCO (2017) in line with the 2030 goals (sustainable development goal 4.4).

In the ICILS evaluation, students' learning outcomes are examined by considering background variables and process variables together. The features related to these different contexts include characteristics related to the national level, school, teacher and student levels. This model (See Fig. 1) related to CIL has a structure that covers the system as a whole, such as the dynamic model of educational effectiveness (Creemers & Kyriakides, 2010). The ICILS implementation provides documentation on data obtained from these different levels open to everyone. In addition to these data, it also provides information for data collection processes, data operationalization and advanced analysis to researchers (Jung & Carstens, 2015).

Fig. 1
figure 1

Contexts for ICILS 2018 (Fraillon et al., 2019b)

Full size image

3 The present study

In the reviews of the literature, it is seen that the studies conducted in the field of ICT literacy are mostly at the level of students or pre / in-service teachers. However, it is known that in-school practices and activities are significant for the ICT use skills of students, on the other hand, teachers' experiences and attitudes towards these activities also vary. It is also known that the association between the characteristics of teachers and students and students' ICT literacy has been discussed in a limited number of studies in the literature (i.e. Berger, 2019; Claro et al., 2018). With this research, it is expected that examining teacher and student characteristics together will contribute to the understanding of students’ CIL. The prominent countries (Korea, Finland) in the ICILS study were examined within the scope of this research. Characteristics related to teachers and students are examined in the categories of background variables, use of ICT and attitudes towards ICT. For this purpose, the following questions were sought in the study.

  1. 1.

    What are the student background characteristics that are related to CIL of Korean and Finnish students?

  2. 2.

    What is the level of the relationship between students' attitudes towards ICT and student achievement after controlling for their background characteristics that are related to CIL of Korean and Finnish students?

  3. 3.

    What is the level of the relationship between the students' purposes of using ICT and student achievement after controlling for attitude towards ICT and the student background characteristics that are related to CIL of Korean and Finnish students?

  4. 4.

    What are the teacher background characteristics that are related to CIL of Korean and Finnish students?

  5. 5.

    What is the level of relationship between teachers' attitudes towards ICT and student achievement after controlling for teacher background characteristics that are related to CIL of Korean and Finnish students?

  6. 6.

    What is the level of relationship between teachers' purposes of using ICT and student achievement after controlling for teacher background characteristics and attitudes towards ICT that are related to the CIL of Korean and Finnish students?

  7. 7.

    How is the relationship between the Korean and Finnish teachers and students’ characteristic and student achievement?

4 Method

4.1 Sample and procedure

This study was conducted with ICILS 2018 Korean and Finnish sample. These selected countries are among the top three in the ICILS. Denmark, which ranks first, was not included in this study because it could not provide the required sample size for analysis. These countries were selected to reveal how the student and teacher characteristics featured in the literature for relationships in the most successful countries.

Within the scope of the research, there were 147 teachers and 2616 students in the Korean sample and 142 teachers and 2109 students in the Finnish sample. The average age of the students was 14.21 and 14.80, and the average age of the teachers was 46.32 and 45.52 respectively. Data collection, coding and reporting processes were carried out according to the quality standards predefined by IEA (Fraillon et al., 2014).

4.2 Instruments

ICILS 2018 used questionnaires and scales to assess teacher, student and school characteristics while examining them in their framework in the countries (See Fig. 1). In this study, the documents of the related counties were accessed through IEA webpage to answer the research questions. The scales used in this study were developed within the scope of the ICILS research, and confirmatory factor analysis, item response theory, and Cronbach alpha coefficients were specified in the guide (Fraillon et al., 2020). In addition, five different plausible values (PV1CIL-PV5CIL) calculated for each student expressing the student's CIL achievements were analyzed together.

In the study, students’ CIL achievement assessed within the scope of ICILS was used as the dependent variable. In CIL achievement, students’ computer and information literacies are assessed through a computer-based environment. The questions and tasks in the test environment are arranged in four modules each of which takes a maximum of 30 min to complete. The students completed two of the four modules randomly. In obtaining students’ CIL test scores out of the data of the test environment, Rasch IRT techniques were used.

In the study, the independent variables included students’ and teachers’ background, value beliefs, attitudes, and behaviors relevant to CIL (See Table 1). The Cronbach alpha coefficients of the student scales were 0.67–0.86 for Finland and 0.72–0.89 for Korea. The Cronbach alpha coefficients of the teachers scales were 0.63–0.92 for Finland and 0.78–0.95 for Korea.

Table 1 Level-1 and level-2 variables in the study
Full size table

In the study, which was handled with an explanatory approach, student-level variables were categorized under the titles of home background, use of ICT purposes, and attitude towards ICT variables. A similar classification was made for teacher level variables, as well. Level-2 variables were also classified as teacher background, ICT use activities and attitude towards ICT variables. Table 1 shows the variables that are handled at both levels within the scope of the research.

4.3 Data analysis

It is not possible for this study to perform an independent analysis because the participants included both teachers and students. Two-level design was used in the current study which is appropriate for clustered data considering its hierarchical structure of students (level-1) nested in schools (level-2). In order to answer the research questions, the data were analyzed with multilevel analysis, which is also known as a special regression analysis enabling the analysis of data obtained from different levels (Raudenbush & Bryk, 2002).

The theoretical framework of the study was maintained with the tested six main models. First, the null model was tested which showed whether to adopt a multi-level approach over the single level linear regression, and which did not include research variables. In the null model which was created first, the differences between students and the differences between schools were determined. The next three models addressed students’ demographic characteristics, affective characteristics and their ICT usage variables, respectively. Then teacher characteristics were aggregated to school level model hierarchically. The three models related to teacher characteristics involved teachers’ demographic characteristics, use of ICT in the classroom, and ICT related attitude variables, respectively. Before adding the factors of the next variable sub-cluster, insignificant factors were first removed from the model. In the process, the analysis was continued with only variables (p < .05) showing a significant relationship in the later stages of the model. Using a hierarchical multi-level approach enabled controlling the added-value of the sub-cluster of each variable as explained variable percentage. The data analysis model is presented in Fig. 2.

Fig. 2
figure 2

Data analysis model

Full size image

In the analysis of the data, five plausible values were analyzed. In addition, sampling weights determined for each level were used in the analyses in order to make valid, unbiased estimates over the sample determined (Fraillon et al., 2020). The analyses in HLM software (Raudenbush et al., 2013) were performed with restricted maximum likelihood (REML) estimation method which is used more broadly in multilevel modeling (McNeish & Stapleton, 2014; Snijders & Bosker, 2012). The analyses were performed with the whole data. The default method in HLM were selected for the missing data in level 1 and level 2.

5 Results

5.1 Descriptive statistics

In the study, the mean, standard deviation, minimum and maximum values of the variables were examined before data analysis. In addition, the skewness and kurtosis coefficients of the variables were examined. Descriptive statistics of student level variables are provided in Table 2.

Table 2 Descriptive analysis results regarding student level variables
Full size table

The mean, standard deviation, minimum and maximum values of the teacher level variables in the Korean and Finnish samples were examined at school level. Teachers with missing data were excluded from the analysis. Descriptive statistics of variables regarding the teachers are shown in Table 3.

Table 3 Descriptive analysis results regarding level-2 variables
Full size table

5.2 Analysis of the relationship between student level variables and level of CIL using HLM

The base model (Null Model/Model 0), which does not include any variables within the scope of the research, indicates the variance between schools and between students in Korea and Finland. After this model, which is included in Table 4 as Model 0, variables classified in three categories (demographic, use of ICT purpose, attitude variables) were added as a group. After each analysis, significant variables remained in the analysis and new variables were added to the model. The results are presented in Table 4.

Table 4 Results of analysis of student-level models
Full size table

As a result of the first model showing the difference between schools in CIL levels of students in Korea and Finland, intraclass correlation coefficients were calculated for both countries. Interschool variance (τ00) and variances at the school level (σ2) were considered in this calculation.

$$\uprho \left( {{\text{ICC}}} \right) = \uptau_{00} {/}\uptau_{00} + \upsigma^{2} \left( {\uptau_{00} = {\text{school}}\;{\text{level}}\;{\text{variance}},\;\upsigma^{2} = {\text{individual}}\;{\text{level}}\;{\text{variance}}} \right)$$

The result obtained as a result of the above equation expresses the variability in CIL scores explained by the schools. These values are estimated to be 9% and 13% for Korea and Finland, respectively. It can be deduced from this result that schools in Korea are somewhat more homogeneous in terms of CIL compared to Finland. In other words, it can be said that 91% student-based factors in Korea and 87% student-based factors in Finland are determinant in the CIL achievement of students.

When the variables added to the model were examined, gender (S_SEX), computer experience (S_EXCOMP), socio-economic background (S_NISB) in Model-1 were found to be positively correlated with the CIL achievements of students in both countries, while tablet experience (S_EXTAB) was negatively related. While the education level of the family (S_HOMLIT) and the smartphone experience of the students (S_EXSMART) do not have a significant relationship with the CIL achievements of students in Korea, this relationship is predicted as positive and negative, respectively in Finland. It is observed that students' having internet connection at home (S_INTNET) does not have a significant relationship with the CIL achievements of the students in both countries (p > .05).

When Model-2, which deals with student-level affective variables, is examined, it is seen that while students’ general self-efficacy levels regarding ICT use (S_GENEFF) and students' positive perceptions of ICT use in society (S_ICTPOS) have a positive significant relationship with CIL achievements, self-efficacy levels regarding specialized ICT practices (S_SPECEFF) have a negative correlation with CIL achievements. While negative perceptions of students about ICT use in society (S_ICTNEG) are not related to student achievement in Finland, they are significantly associated with CIL achievements of Korean students (t = -1.98, p = .048). While the variable of working in an ICT focused job or continuing education in the future (S_ICTFUT) does not have a significant relationship in Korea, it has a significant relationship with the CIL achievements of Finnish students.

In Model-3, which includes variables related to ICT usage, it has been observed that, among the variables added to the model, while students’ use of general ICT applications (picture / graphic drawing, presentation applications) in the classroom (S_GENCLASS) and their use of them for activities (S_GENACT) have positive relationship with students' CIL achievements, students' use of specialized ICT applications (Simulation / modeling tools, interactive tools) in the classroom (S_SPECLASS) and their use of ICT for exchanging information (Q&A activities with tools such as forums, blogs) (S_USEINF) are negatively related to students' CIL achievements. It has been observed that students' use of customized ICT applications for activities (S_SPECACT) is not associated with students' CIL achievements in both countries. The relationship between purposes students’ CIL achievements and their use of ICT as a social communication tool (S_USECOM), frequency of access to leisure activities on the internet (places to visit, how to do videos, etc.) (S_ACCONT), and use for studying have been found to be significant only for Finnish students. Among these variables, only their use of ICT as a social communication tool (S_USECOM) has a negative relationship and the other two have a positive relationship in the CIL achievement of Finnish students.

The variance explained by the variables in the final model consisting of Level-1 variables in students' CIL achievement was calculated by comparing with the null model. In this calculation, the following equation is used.

$$\uprho = \left[ {\upsigma^{2} \left( {{\text{null}}\;{\text{model}}} \right) - \upsigma^{2} \left( {{\text{random}}\;{\text{coefficient}}\;{\text{model}}} \right)} \right]{/}\upsigma^{2} \left( {{\text{null}}\;{\text{model}}} \right)*100$$

In line with the above equation, it is seen that the variables in Model-3 explain 31% of the variance in student achievement for the Korean sample and 30% of the variance for the Finnish sample. When all other variables remain constant, it can be said that gender and experience with computers are important variables contributing to CIL achievement.

5.3 Analysis of the relationship between level-2 variables and the level of CIL using HLM

Level-2 models, which include the characteristics of teachers, were created by controlling for variables that have significant relationships at student level. Within the scope of the research, these variables were added to the model as three predetermined groups. The teacher variables that were added to the model were analyzed through addressing them at school level. Coefficients related to the model are presented in Table 5.

Table 5 Results of analysis of teacher-level models
Full size table

In the first model (Model-4) in which teacher variables were dealt with within the scope of the research, demographic characteristics of teachers were included in the model. Teachers' variables of gender (T_SEX), age (T_AGE), use of ICT during the lesson (T_EXLES), and use of ICT for lesson preparation (T_EXPREP), access to computer infrastructure at school (T_RESRC), receiving structured ICT training (T_PROFSTR) or reciprocal ICT training (T_PROFREC) were added to this model. Among these variables, only the variable of access to sound computer infrastructure at school (T_RESRC) is significantly related to the CIL success of schools in Finland (p < .05), but the variable of teacher gender (T_SEX) had a significant relationship with student success in Korean schools. Other variables in the model are not significantly related to the average CIL achievement of schools for both countries (p > .05).

In the next model created with variables related to teachers (Model-5), variables describing teachers' use of ICT in the classroom were added to the model. Teachers' use of ICT in classroom teaching practices (T_ICTPRAC), teachers' employing digital learning tools (edmodo, moodle, etc.) (T_USETOOL) and teachers 'use of general software (word processor, computing applications, etc.) (T_USEUTIL) variables, which were added to the model, and the variable of teachers' ICT usage in in-class activities (T_CLASACT) were present in the model. The effect of all the added variables on the average CIL achievement of schools in both countries is not significant (p > .05).

In the final model (Model-6), which includes variables related to teacher characteristics, teachers' attitudes towards ICT use were examined. In this model, specifically, teachers' collaboration with their colleagues on ICT use (T_COLICT), teachers' negative (T_VWNEG) and positive views (T_VWPOS) about ICT use in the teaching process, emphasis on improving students' ICT-based skills (T_ICTEMP) and teachers' self-efficacy on ICT use (T_ICTEFF) were examined. While the variable of emphasis on improving students' ICT-based skills (T_ICTEMP) in the model has significant relationship with the average CIL achievement of schools in Finland (p < .01), it has been observed that it is not significant in Korea. All other variables added to the model do not have a significant relationship with the average school CIL achievements in both countries (p > .05).

The variance explained by the variables in the final model (Model-6), which consists of Level-2 variables, regarding the average CIL achievement of schools was calculated by comparing it with the full model at the student level (Model-3). In this calculation, the following equation was used.

$$\uprho = \left[ {\upsigma^{2} \left( {{\text{random}}\;{\text{coefficient}}\;{\text{model}}} \right) - \upsigma^{2} \left( {{\text{full}}\;{\text{model}}} \right)} \right]{/}\upsigma^{2} \left( {{\text{random}}\;{\text{coefficient}}\;{\text{model}}} \right)*100$$

The variables in the final model (Model-6) created in line with the above equation corresponds to 12% of the inter-school variance in Korea when all other variables remain constant, and this ratio is 9% for Finland. Considering that the inter-school variance in Korea is calculated as 9% in CIL achievement of students after the null model analysis, 12% of this value is around 1%. Likewise, after the null model, the inter-school variance in students' CIL success is calculated as 13% in Finland. 9% of this value is around 1.25%, as well. This analysis regarding teachers showed that the significant variables in the model correspond to 1% of total variance in students’ CIL achievement.

6 Discussion

Regarding the CIL achievements of the students, the variables of gender, computer experience, tablet experience and socio-economic level of the family are demographic variables that are found to be significantly associated with student achievement in both Korea and Finland. The difference in favor of girls observed in this study regarding gender can also be seen in various large-scale assessments. In the digital reading component of PISA (OECD, 2011), a difference of a quarter standard deviation in favor of girls draws attention. In the results of national ICT literacy evaluation conducted by NAEP in America, this difference in favor of girls was reported as one-sixth of the standard deviation (NCES, 2016). In a similar national assessment in which students in Grades 6 and 10 participated in Australia, this difference was again reported as one-fifth of the standard deviation (ACARA, 2015). In the first cycle of the ICILS research, a difference of one-fifth of the ICILS 2013 standard deviation in favor of girls was identified (Fraillon et al., 2014). A meta-analysis study, which included 121,614 student data as well as ICILS participant country data, reported a low effect (g = 0.12) in favor of girls regarding gender (Siddiq & Scherer, 2019). On the other hand, in studies conducted with Chilean 15-year-old students (Claro et al., 2012) and Norwegian second stage students (Hatlevik & Christophersen, 2013), it was concluded that there was no significant difference in terms of gender. Punter et al. (2017) emphasized that ICT tasks have a differential effect on this difference regarding gender. They stated that there is no significant gender difference in technical matters, but there is a difference in favor of girls in the tasks of accessing information, reflecting and sharing information. Kaarakainen et al. (2018), who conducted research with Finnish students, similarly reported the gender effect on tasks.

Braak and Kavadias (2005) argue that students' computer experiences affect their general beliefs about computers and their perceived efficacy levels. Rohatgi et al. (2016) similarly state that students' computer experience has both a direct effect on students’ CIL achievement and an indirect effect through their basic self-efficacy levels. Hatlevik et al. (2018), in their research with 15 countries, concluded that this variable has a positive relationship in 8 countries. In addition to the computer experience, the tablet experience of the students is also examined in the current research. Drossel and Eickelmann (2017b), in their study with a control group, concluded that the use of tablets by students had a weak but not significant correlation with their CIL achievement. In quasi-experimental studies conducted on tablet experience, this experience was not found to be related even for emergent literacy skills (Brown & Harmon, 2013). In this study, there is a negative relationship between the tablet experience and the CIL achievement of students from both countries. The fact that students' purposes such as watching videos, playing games, and making simple drawings are prominent in their tablet experiences (Marsh et al., 2015) and that these goals are far from CIL skills make these negative relationships understandable. Although some of the studies deal with computers, tablets and mobile devices together in ICT tools, there are limited studies examining the relationship between smart phones and CIL. In the research conducted by Jan (2018), it was concluded that smart phones are not related to students' CIL achievements. In the present study, while there is no significant relationship between student achievement and smartphone experience in Korea, a significant negative relationship is found in the achievement of Finnish students.

Selwyn (2004) defined the difference between those with and without access to ICT resources as digital divide. However, it is now claimed that such a gap (internet infrastructure etc.) is not seen as a problem especially in developed countries (Siddiq, 2016). In this respect, it seems plausible that the variable of the students' internet access at home did not make a difference in the scores of the students in this study. It was observed in another study that even access to the Internet at school does not affect the school achievement of Spanish students (Gil-Flores et al., 2017). Hatlevik and Gudmundsdottir (2013) state that socio-economic variables at home (number of books at home, education level of parents, household income) are associated with students' CIL achievements. In the study, it was concluded that the number of books in students’ homes showed a significant relationship with their success only for Finnish students, while the socioeconomic level of the family was significantly related to student achievement in both Korea and Finland. As a result of the meta-analysis on students’ CIL achievements, it was seen that this significant relationship had a weak correlation value (r2 = 0.214) (Scherer & Siddiq, 2019).

In this study, positive relationships were found between students' general self-efficacy levels for ICT and their CIL achievements, and negative relationships were found between ICT self-efficacy regarding the use of specialist applications and CIL achievements. Although it is seen in studies that the case regarding general self-efficacy is a common expected result (Berger, 2019; Rohatgi et al., 2016; Scherer et al., 2017), the negative relationship with CIL achievements with regard to advanced efficacy is striking. Rohatgi et al. (2016) explain the negative relationship between advanced self-efficacy and CIL achievements with the view that activities requiring advanced skills (programming, database creation, etc.) are less included in the curriculum, so students cannot have a realistic belief about these unfamiliar skills. Besides, the fact that the competencies regarding the basic ICT skills are more prominent in the CIL achievement of students (Fraillon et al., 2014) seems to support this result. In addition, Ng (2012) states in her research on digital literacy that students' positive attitudes towards ICT contribute to their use of ICT tools and improvement of their ICT literacy level. Similar results seen in this study are understandable in this respect.

In the study, students' use of general computer applications for both in-class use and activities emerged as usage purposes variables that are related to CIL achievement. In addition, the use of customized applications in the classroom was negatively related to the CIL achievement of the students, while the use of customized applications for the activities was not found to be associated with the CIL achievement of the students in both countries. The use of ICT in classroom learning activities (use of ICT for task learning) has shown negative (Scherer et al., 2017) and positive relationships (Berger, 2019) with CIL achievements in the literature. In the same studies, the use of ICT for social communication and exchanging information has negative relationships with student achievement as observed in this study. In these studies, the use of ICT for study purposes was also found to have limited relationship (Scherer et al., 2017) or no relationship at all with (Rohatgi et al., 2016) student achievement. The different course of association regarding the use of ICT for study purposes in the literature is reflected in the results of the current study, as well. The use of ICT for recreation purposes has direct and indirect, through self-efficacy of which it is a significant predictor, relationships with CIL achievement (Scherer et al., 2017). Bundsgaard and Gerick (2017), who examined the variables related to the use of ICT for 21 countries by classifying them in low, medium and high categories, concluded that the low and high categories of the relevant variables were negatively related to the students’ CIL achievement. More importantly, in this research and the studies listed, it was seen that the relevant variables (use of ICT variables) had limited contributions (less than 0.03%) in the variance regarding the CIL achievements of the students.

Since the studies report that teachers' background characteristics such as age, gender, ICT experience, etc. cause differences in ICT use in the classroom (Drossel et al., 2017b; Law et al., 2008; Van Braak et al., 2004), this study which examines student achievement included these relevant variables, as well. However, the results of the research revealed that of these variables only the gender variable is related to average CIL success of Korean schools, and other variables do not have significant relationships. In studies where the characteristics of teachers and students were examined together, ambiguity in these issues is evident. In the studies, the variables of teachers’ gender, age (Claro et al., 2018; Gil-Flores et al., 2017), ICT for classroom preparation and ICT use in class (Claro et al., 2018; Kim et al., 2014) did not have any significant relationships. Teachers who received professional development for the use of ICT in lessons tend to use computers more frequently (Drossel et al., 2017a). It was observed that among the Spanish teachers who participated in the TALIS, teachers who stated that they needed professional development used ICT less in-class (Gil-Flores et al., 2017). However, in this study, it was observed that teachers' participation in these activities did not have a significant relationship with students' CIL achievement. On the other hand, considering that there are different motivation tools for professional development in each country, different results are likely to occur in different countries. Studying this relationship longitudinally may give clearer results.

Drossel et al. (2017a) stated that ICT infrastructure in schools is a determinant of teachers' use of ICT. In this study, when the relationship of this variable with students' CIL achievement was examined, a significant relationship with the relevant variable was observed in Finland, but not in Korea. According to the results of the research, the teachers' use of digital learning tools and basic software, and use of ICT for teaching practices and classroom activities were not found to be associated with students' CIL achievement. These variables were mostly evaluated in a form that reported frequency. Drossel et al. (2020) point out that relevant results regarding ICT use should often be viewed with suspicion, because evaluations regarding ICT use require not only quantitative but also qualitative analysis.

In the final model created with regard to teachers’ acceptance of ICT, it was seen that only the variable of ‘emphasis on teachers’ ICT capabilities in class’ was related to the CIL achievements of schools in Finland. These results are in line with Berger’s (2019) study which reported a very low relationship with CIL achievement (less than 0.03%). In this study, the variable of collaboration between teachers in using ICT was not found to be related to average CIL achievements at school in both countries. ICILS 2018 findings also revealed an interesting differentiation in terms of teacher collaboration. Teacher collaboration is highest in countries such as Kazakhstan and lowest in Korea and Germany (Fraillon et al., 2019a). It was concluded that in countries participating in the ICILS implementation, teachers identified as more frequent ICT users collaborate on ICT issues significantly higher than teachers who reported that they were rarely ICT users (Drossel et al., 2020). Teachers' self-efficacy variable addressed at school level in the model did not have a significant relationship with school average CIL achievements. It is also stated that teachers' ICT self-efficacy is a significant predictor of the levels of ICT use (Hatlevik, 2017; Hatlevik & Hatlevik, 2018). This variable which is also the most important predictor of teachers’ ICT acceptance (Siddiq & Scherer, 2016) has been reported not to have a significant correlation with student achievement. Longitudinal studies in the literature (Holzberger et al., 2013) would absolutely contribute to getting an elaborate grasp of this case.

In this study, the variables of teachers' negative and positive views on using ICT in teaching and learning were not found to be significantly correlated with students' CIL achievements in both countries. Lorenz et al., (2019) reported that this variable has an indirect relationship with student achievement through the frequency of ICT use. Drosssel et al. (2017b) reported a similar relationship (its effect on ICT use) as positive for four different countries. These studies are based on the assumption that this situation will have a positive relationship for student achievement, as well. However, considering that studies on student achievement are still limited in the literature, it cannot be said that this assumption is very clear.

7 Conclusion

The current study revealed new findings regarding teacher and student characteristics that are related to students’ CIL levels. The featured variables regarding students’ CIL levels were basically their computer experiences and socio-economic levels. The emerging relationships among these variables that seem as determinants (Aydin, 2021) particularly for first-level digital divide (Van Dijk, 2006) are apparent. The negative relationship, on the other hand, between tablet use and CIL achievements is noteworthy. This study lends its support to research studies (Drossel, & Eickelmann, 2017b) that could not confirm the relationship between computer literacy and CIL achievement though they mostly seem closely related. The competencies regarding these tools commonly used by students and adults may not always be positively associated with CIL competencies.

The self-efficacy levels that seemed related with CIL achievements were observed in both countries’ results. Yet, this study addressed both specialized self-efficacy levels and general self-efficacy levels concurrently, and revealed a negative relationship between specialized self-efficacy levels and students’ achievement. This result may firstly be regarded as evidence to the existence of different self-efficacy areas. This study confirmed the result that competencies in different issues may not be positively related to CIL achievements, and in a sense, these competencies that require expertise are not related to school-context (Rohathi et al., 2016). This shows that general and specialized CIL self-efficacy types diverge in their relationships with achievement. Another variable with which general and specialized applications are addressed was ICT usage in activities. The study demonstrated that ICT usage is positively related with student achievements both for general and specialized activities.

The relationships with student achievements were highly limited in the model with teacher variables. This becomes more understandable considering the sample in the ICILS practice. We know that the ICILS assessment comprises all teachers in a school and therefore some of the teachers may be those who do not teach to the students in the study. Then only a contribution that is limited to the ICT culture in the school may be reflected in students’ CIL achievements. Future studies that include teacher-student matching may help in understanding teacher contribution to students’ CIL achievement thoroughly.

8 Limitations and future studies

This study examined the contribution of student and teacher characteristics to students’ CIL achievement. These characteristics are also related to other countries beyond the ones covered in this study. However, the results are limited to correlational results obtained from the sample of the study. Another limitation of this research study is that it is a limited cross-sectional study. For this reason, attention should be paid to the causality statements among the relevant factors in the interpretation of the research results. It should be known that longitudinal studies will be more appropriate in the defining causality.

Items related to ICT were tested in the scales regarding the opinions of teachers and students. Another limitation of the study is that repeating these statements may increase the likelihood that teachers and students have responded in a similar chain of effects. Besides, the relationship of teachers and students that were included in the sample in ICILS studies does not refer to the relationship between students and teachers from whom those students receive education personally as in TIMSS samples. Therefore, the participating teachers may be in the same schools with the students but may not have encountered in the same instructional activity. Hence it should be noted that school level averages and these characteristics in the analyses may provide limited interpretation for school atmosphere.

This study revealed the teachers’ contribution to students’ CIL achievement in a limited sense. More detailed analyses and research is needed as opposed to denying the contribution of these school-level characteristics. In the future studies, integrating the opinions of school principals or IT coordinators as well as school surveys in increasing the reliability of the data in the evaluation of teacher and student information at school level in the research will make the evaluations effective. It is possible to ensure consistency with the results to be mutually examined through school level analyzes.

The study revealed that some characteristics have different results in the two countries. This gives clues about what to prioritize in the local contexts. In this sense, different studies can be conducted to take a closer look at the results of the countries considered in this study. It would be appropriate to discuss these results together and present valid information to policy makers.