Information and communication technologies use, gender and mathematics achievement: evidence from Italy

Abstract

This study investigates the importance of information and communication technology (ICT) use in the mathematics achievement scores of Italian secondary school students, with particular attention paid to the role of gender in the ICT-maths performance relationship. Data from the 2012 Programme for International Student Assessment study allow to describe (a) how the type and intensity of ICT use are associated with high or low maths achievement and (b) how the association varies according to gender. These issues are examined with respect to different maths domains. The results of multilevel models show a complex scenario. A positive association between ICT use and mathematics achievement occurs only when computers are used for some, not all, activities. In other cases, the association is negative. In general, the ICT-maths performance association is weaker for girls. Some exceptions to this general trend are the benefits of certain ICT applications, only for girls, in Shape and Space and in Uncertainty and Data subscales of mathematics.

1 Introduction

Since infrastructural investments in information and communication technologies (ICTs) have been one of the main priority of education policies in recent years and most countries are investing numerous resources in ICT equipment in schools and in families, it is worthwhile asking whether these investments pay off in terms of students’ performance. The learning achievements of students are, indeed, excellent indicators of future opportunities (Hanushek et al. 2008; OECD 2010), and mathematics performance, in particular, is one of the basic skills that yield productive returns in the labour market (Borghans and ter Weel 2004).

In the past decades, researchers typically considered the effects of computer ownership on learning achievement, but since computer shortage is nowadays less an issue for most families, recent research has instead shifted focus to the effects of ICT use (de Witte and Rogge 2014). One of the main issues that drive such research is whether students using computers less frequently than others perform more poorly and, clearly, distinguishing the different uses of computer can be important. This issue falls within the interest of literature for the equity in educational outcomes, with the aim of developing supplementary support for individuals in disadvantaged positions. An additional aspect usually considered in these studies is gender differences in student performance.

Gender differences in mathematics achievement, with boys outperforming girls, have been explored for many years for the potential implications on the career aspirations and occupational choice of females (Wigfield et al. 2002). A variety of theories that incorporate biological, socio-cultural, pedagogical and attitudinal variables has been used to explain these differences (Gallagher and Kaufman 2005). In this context, differences in ICT use can be an interesting research direction. Studies focused on students in different levels of education has shown a consistent pattern: female students use computers less frequently than males (Notten et al. 2009; Volman et al. 2005), even distinguishing the different use of ICTs (Drabowicz 2014). Thus, if computer use plays a role in maths learning, the fact that girls have less experience and lower use of ICT than boys may be a cause of the gender gap in mathematics achievement. At the same time, the impact of ICT use on learning achievement could be gender differentiated. Despite the potential insights that can be derived from investigations into these issues, however, few studies have been devoted to them (Ferrer et al. 2011; Nævdal 2007). The current research aims to fill this gap in literature.

With reference to the context of Italy, this paper studies (a) how the type and intensity of ICT use are related to student performance in maths and (b) whether the influence of ICT uses differs between boys and girls.

Italy is an interesting case study because the aforementioned issues are particularly relevant in this country. Despite the fact that Italy has relatively high percentages of ICT access, Italian students present one of the lowest percentages of those who use ICTs out of school (Chisalita and Cretu 2014). Moreover, ICT access at school in Italy is one of the lowest among European countries (Chisalita and Cretu 2014): the National Plan for Digital Schools (Piano Nazionale Scuola Digitale) launched in 2007 by the Italian Ministry of Education to mainstream ICT use in Italian schools has probably had a limited effectiveness due to its small budget and ICT equipment is entering Italian schools rather slowly (Avvisati et al. 2013). In addition, in 2012, the mean mathematics score of Italian students was statistically significantly below the OECD average (OECD 2014a), and Italy was one of the countries with the largest difference in maths scores between boys and girls, with the former achieving higher scores. With a difference of around 18 points between boys and girls, only Luxembourg (25), Chile (25) and Austria (22) presented statistically significant gender differences that are higher than those observed in Italy (OECD 2014a, b). These findings highlight the need of ascertaining whether maths performance can improve with a diffusion of ICTs among students and of identifying the potential role of gender in the relationship between ICT use and maths performance.

Data come from the 2012 Programme for International Student Assessment (PISA) study, which represents a particularly relevant data source for the aims of this research. Beginning in 2009, PISA questionnaires have provided information on the type and frequency of ICT-related activities that 15-year-old students engage in, apart from presenting data on test scores. In addition, in PISA 2012 mathematics was the focal point of assessment: thus, the evaluation was directed not only towards overall maths performance but also towards four overarching content areas, namely, Quantity, Uncertainty and Data, Change and Relationships, and Space and Shape. With these recent data, therefore, we can probe into the links between the type and intensity of ICT use and maths learning in different domains.

The rest of the paper is organised as follows. Section 2 presents a review of the literature on the relationship between ICT use and learning outcomes (2.1) and on gender differences in maths achievement and ICT use (2.2). Section 3 describes the data, with particular attention to the appropriate methods for addressing the complexity of PISA data structure. Section 4 discusses the main analytical results, and Sect. 5 concludes.

2 Background

2.1 ICT use and learning outcomes

The past few decades have seen a gradual shift in scholarly interest in the role of ICTs in student performance. Initially, researchers focused on the effects of owning a home computer and on the availability of computers in schools (Attewell and Battle 1999; Fuchs and Wömann 2005; Papanastasiou et al. 2003). The increasing proportions of families with computers has prompted a re-direction of this focus to the use of ICTs (Di Maggio et al. 2004; Gui et al. 2014; Kubiatko and Vlckova 2010; Luu and Freeman 2011).

Empirical research has thus far provided mixed results on the effects of ICT use on student learning and performance. In particular, no univocal and consistent evidence has been presented to support the hypothesis of a positive ICT influence. Some empirical studies suggest that using a computer frequently at home/school is associated with improved performance at school in different domains (Kubiatko and Vlckova 2010; Luu and Freeman 2011; Nævdal 2007; Papanastasiou et al. 2003). ICT use can, indeed, challenge students’ thinking and understanding, as well as improve their problem-solving skills, thus constituting an input in the learning process. Certain scholars are nevertheless critical of technology use, with some indicating that the benefits of computer use are concentrated primarily on the development of ICT skills rather than on other abilities (Wittwer and Senkbeil 2008). Other researchers propose even a negative effect of computer use, arguing that it can distract students, reducing the time they spend on accomplishing homework or learning (van Braak 2001).

Mixed results may be due, at least partly, to the fact that the effects of ICTs on student learning strongly depend on the specific uses of and activities performed with computers and previous studies are not always able to distinguish usage types (Biagi and Loi 2013; Fuchs and Wömann 2005). Some ICT-based activities can improve student competences, but others can distract from study. Lei and Zhao (2007), for example, found that using educational software is usually positively associated with learning performance, whereas Internet use and e-mail exhibit a negative association. At the same time, however, the Internet can help students exploit enormous information possibilities for schooling purposes, increase learning through communication and develop their problem-solving skills. When searching for information over the Internet, for example, one needs problem-solving abilities to identify appropriate information sources, as well as, organise and integrate information from different sources (Tabatabai and Shore 2005). Thus, it is important to distinguish in detail the different activities that students can perform on computers. In general, some researchers suggest that for ICT to exert a positive effect on learning outcomes, the essential requirements are to use computers in a constructive manner and select technology that challenges student competence and reinforces higher-order thinking skills (Papanastasiou et al. 2003).

2.2 Gender gap in maths achievement and ICT use

Researchers have exhaustively examined and explained the substantial advantage of males in standardised maths tests (Gilleece et al. 2010; Cahill 2005; Gallagher and Kaufman 2005; Lavy 2004; Tiedemann 2000). In fact, the conclusions drawn about gender differences in mathematics vary, depending on a number of factors. First, evidence suggests that the gender gap in mathematics is connected to age. No measurable differences in maths performance have been found between boys and girls during elementary years, and nearly no significant differences during the middle secondary level have been observed. Instead, some studies have revealed that differences in maths performance, with boys outperforming girls, start in secondary school (Kaiser and Steisel 2000). Second, some scholars conclude that gender differences in mathematics have been declining over the years (see the review by Liu et al. 2008). Third, evaluating only the total maths score may result in an underestimation of gender differences if males and females are favoured by different mathematical domains (Liu and Wilson 2009).

In particular, the literature has shown that gender differences in mathematics performance are heterogeneous across tasks requiring different mathematical skills since the advantages of boys in maths are not homogeneous across different domains. Mathematical ability is, indeed, as multidimensional (De Lisi and McGillicuddy-De Lisi 2002) as are gender differences in mathematics. Disregarding the variability of gender differences across various maths domains can deter teachers from effectively catering to the differential learning needs of boys and girls and thereby hinder the narrowing of the gender gap. Boys perform better than girls on spatial ability activities, on tasks that assess reasoning and problem-solving abilities and on statistics (Gierl et al. 2003; Liu et al. 2008; Liu and Wilson 2009). Conversely, female students surpass males in algebra and in activities involving the use of formulas, equations and theory (Kaiser and Steisel 2000; Liu et al. 2008). Researchers have suggested that these female advantages in computational tasks originate from the fact that they are more cautious than boys in dealing with arithmetic operations (De Lisi and McGillicuddy-De Lisi 2002). More in general, male performance is superior to female one in tasks that involve logical estimation strategies, multiple solution strategies, the extraction of information and items based on a combination of conceptual understanding skills. By contrast, girls are more effective in tasks that entail routine algorithmic strategies and extensive reading or explanation.

Gender can exert its influence on student maths achievement even through the use of ICTs. Although the gender gap in terms of ICT use has diminished (Cooper 2006), boys and girls continue to exhibit different levels of familiarity with ICTs: studies consistently indicate that boys express more positive views towards the use of technology (Barkatsas et al. 2009; Volman et al. 2005) and use computers more frequently than girls (Livingstone and Helsper 2007; Notten et al. 2009; Smihily 2007). In addition, boys and girls use ICTs for different activities. For example, boys use computers as a mean of entertainment, more often engaging in activities such as gaming and net surfing (Ainley et al. 2008; Tømte and Hatlevik 2011; Volman et al. 2005). They also experiment with a computer’s functions and programming more frequently than do girls. Females, instead, use computers mainly to communicate by e-mail and interact with others in chat rooms, as well to search for information related to homework, being more practically and socially oriented (Mumtaz 2001; Pedró 2007).

If differences in ICT usage figures importantly in the maths learning process, then such differences can, at least partly, explain some gender differences in maths performance. For instance, since a reliable correlation between spatial ability and participation in spatial activities has been found (Nuttal et al. 2005), a higher level of mathematical spatial ability domain among male students than among females could be explained considering that boys tend to have more spatially related experiences than girls through playing computer games (Liu and Wilson 2009). In other words, gender differences in maths performance could be removed controlling for the different ICT activities in which males and females engage.

Another possibility, however, is that boys and girls benefit in different ways from the advantages presented by ICT use (Heemskerk et al. 2009). The different preferences and perceptions of ICTs between boys and girls, with boys having more positive attitudes towards computers than do girls, could, indeed, influence the role of computers in learning. In particular, boys may more strongly benefit from ICT use in terms of learning performance than girls.

Some similar researches in social psychology of education have been focused on how chess education influences maths achievement differently for boys and girls with larger effects for boys than girls (Rosholm et al. 2017). The research has shown, indeed, that chess promotes spatial abilities (Bilalic et al. 2007a, b; Grabner et al. 2007) and higher-order thinking skills (Berkman, 2004). However, chess may be more appealing to boys than girls who may be less interested in this activity (Bilalic et al. 2007a, b). Thus, as for the advantaged from ICT use, girls may less strongly benefit from chess education both due the fact that they play chess less frequently than boys and to the fact that they have less positive attitudes towards chess.

Empirical evidence on these issues is scarce because few studies have been devoted to these matters. Nævdal (2007) found no gender differences in terms of ICT benefit, but this finding is questionable due to the relatively low sample size of some groups. Ferrer et al. (2011) showed that the use of technology exerts a more positive effect on boys than on girls, but this effect may have occurred only because the boys in the sample are those with the worst results.

3 Data

3.1 Data and methods

Data are drawn from the Italian sub-dataset of the PISA 2012 project. PISA is a triennial international survey that since 2000 aims to measure how well prepared 15-year-old students are in meeting the challenges of society, by assessing their performance in three literacies, namely, mathematics, reading and science. Apart from completing cognitive tests, students fill in a questionnaire on their families’ characteristics. In some countries including Italy, the questionnaire investigates students’ familiarity with ICTs, asking students the technologies at their disposal at home and in school, whether, how often and for what purposes they use them.

Every triennial edition of PISA surveys features a different major subject area of inquiry, which is selected from the three literacies tested every time. As previously stated, the main focus of the 2012 edition was mathematics. The results of the tests on this subject are therefore more reliable, since they emerge from a more in-depth investigation (i.e. more tasks on the test than in the previous versions), and performance is evaluated not only in terms of the overall mathematics scale but also with respect to four overarching content areas that encompass the problems arising through interaction with everyday phenomena. The four content areas, or mathematics subscales, are Quantity, Uncertainty and Data, Change and Relationships, and Space and Shape. Each content category is related to specific abilities (described in detail in OECD 2014a, p. 38). Quantity, in particular, may be a domain where the performance of girls would be superior to that of boys because this category revolves around computational tasks and the assessment of reasonableness of results, towards which girls are oriented, as indicated in the literature (Kaiser and Steisel 2000; Liu et al. 2008).

The mathematics scores and the scores on the different content areas are used as our dependent variables and are estimated through plausible values (PVs). The PISA literacy assessments are, indeed, designed to cover a broad range of contents, but time constraints prevent the presentation of all assessment items to each student, and, thus, different test versions containing only a fraction of the entire test items were created. The use of PVs allows to take into account this. In particular, five PVs of the maths score and content area scores were estimated for each student: these values were randomly drawn from the distribution of ability estimates that could reasonably be assigned to a student, and the mean of the PVs should be equal to the expected a posteriori estimator. Using the five PVs in the analyses ensures a correct standard error estimation (OECD 2014a, b).

The test scores and all the variables used in the current study were weighted to represent the schools and the 15-year-old student population of Italy. The PISA sampling procedure was based on a two-stage stratified sampling: in the first stage, schools were sampled with probabilities proportional to size, and in the second stage, students were sampled from the selected schools. In each country, a set of variables were employed for stratification; in Italy’s case, these variables are region and type of school. Given the complexity of this sample design, the use of weights is needed to obtain reliable sampling variances. Specifically, the Fay’s version of Balanced Repeated Replicate (BRR) weights (Judkins 1990), which is an appropriate method for two stage samples, is used in the PISA project. In general, the idea of replicate weight methods is to use a lot of different subsamples (replicate samples) to calculate the parameter of interest in each one of them and the variance is then calculated among the subsample estimates. In the PISA project, it was decided to generate 80 replicate samples and therefore 80 replicate weights (for more details on the construction and use of weights, see OECD 2012).

Apart from the use of PVs and weights, another complexity in the data is the hierarchical structure. The PISA sample comprises students grouped into schools who tend to be more similar to one another than students sampled at random across schools, and thus, ordinary least squares regression can underestimate standard errors (Chiu and Xihua 2008; Snijders and Bosker 1999). To address this problem, we adopt multilevel modelling analysis in which variables are examined at two levels. Independent variables at the student-level include gender and variables on ICT uses as variables of interest, and controls represented by family background characteristics. At the second level, the independent variables regard school type and composition. In particular, models with random intercept and fixed slopes (for details, see Kreft and de Leeuw 1998; Snijders and Bosker 1999) in the formulation as follows are estimated:

$$y_{ij} = \beta_{0j} + \beta_{1j} x_{ij} + \varepsilon_{ij}$$

with

$$\beta_{0j} = \gamma_{00} + \gamma_{01} z_{j} + u_{0j}$$

and

$$\beta_{1j} = \gamma_{10} ,$$

where y_ij denotes the scores (in maths and in the four subscales) of student i attending school j; x_ij is a vector of individual-level characteristics for student i of school j; β_1j is the corresponding vector of regression coefficients; and z_j represents the vector of the characteristics of school j with its corresponding regression coefficients γ₀₁. This type of model has γ₀₀, γ₁₀ and γ₀₁ as fixed effects and two random components:

• the variance of u_0j (between-school residual variance);

• the variance of ε_ij (within-school residual variance).

The models (one for the overall maths score and one for each of the four subscale scores) are based on the SAS macros¹ provided by the OECD for analysing PISA datasets. These macros consider random intercept models with both PVs and weights (OECD, PISA 2012 data analysis manual: SAS users, Paris).

In Italy, 30,559 students (50.7% male) attending secondary schools participated in the PISA 2012 survey. After excluding the cases with missing data, 28,111 students (50.2% male) remained in the final sample. This corresponds to 92% of the original sample indicating a low proportion of missing data.²

3.2 Variables

As previously indicated, ICT and gender serve as the independent variables of interest, whereas the other variables are used as controls.

The questionnaire administered to the students contains questions on the different ways in which ICTs are used. They range from communicative activities and functional ones to learning occupations. Instead of an overall index (frequently used in previous studies), the frequency with which the different activities are performed using ICTs is considered in this work. The approach is similar to that implemented by Biagi and Loi (2013), and it is based on the suggestions of the JRC-IPTS Information Society Unit on Digital Competences (Ferrari 2012). The idea is to divide the different activities (26 items), for which PISA evaluates frequency, into four groups (independently of the sites—school or out of school—where ICTs are used):

• gaming activities³;

• communication and collaboration activities (implying communication through online tools and interaction with others)⁴;

• information management and technical operations (connected to the activities of accessing, retrieving, and organizing information and performing tasks through ICT)⁵;

• creation of content and knowledge and problem-solving.⁶

For each group, an index of usage intensity is obtained considering the average scores measuring the frequency with which each student performs the activities within the group, ranging from 1 to 5 (1 = never or hardly never; 5 = every day). Computer use during maths lessons is also considered through an index that measures how many activities (from a list of seven items⁷) are performed with a computer during maths lessons in the month before the survey.

Family background characteristics are used as controls in the models. In particular, the analyses consider a student’s family structure, socio-economic status and migration background. Family structure is incorporated in the analyses because the literature has shown that students living with two parents score higher in mathematics than those living with one parent or no parents (Chiu and Xihua 2008; Hampden-Thompson and Johnston 2006). In particular, in the current analyses, single-parent families⁸ are distinguished from two-parent families (whether the parents are biological or stepparents is disregarded). Socio-economic status is included to describe the home and family environment of the students, since students with greater resources have more learning opportunities via more education resources, more effective parenting practices and higher parental expectations, and achieve more (Bornstein and Bradley 2003; Chiu and Khoo 2005; Chiu and Xihua 2008; Gilleece et al. 2010). Socio-economic background is described by the combined educational, social and cultural status index proposed by the OECD (for details, see OECD 2014a, b). The index combines six measures: parental occupational status (an index with high scores indicating a high occupational status based on the higher of both parents’ occupations, and derived from the International Standard Classification of Occupation, OECD 2014a, b); parental level of education; home educational resources; cultural resources; indicators of material wealth; and number of books in the home. Low index scores represent a low economic status, and high scores represent a high economic status. The empirical literature reveals that immigrant students underperform compared with native students, even after socio-economic background is controlled for (Azzolini et al. 2012). In the current analyses, the variable on immigrant background distinguishes non-immigrant students from immigrant ones; the former are defined as students born in Italy or those with at least one parent born in Italy, whereas the latter refer to those whose parents were born outside Italy.

Finally, due to the huge territorial differences in students’ mathematical literacy characterizing Italy (Bratti et al. 2007a), the area of residence (described by macro geographical area: North-West, North-East, Centre and South) is also controlled for in the models.

Family characteristics clearly influence students’ performance in education, but school-level factors are equally important. Furthermore, school characteristics may not be independent of family characteristics. For example, families with a high socio-economic status can enrol their children in excellently equipped schools. The school characteristics considered in the analyses are embodied in three variables: type of school, (school-average) socio-economic conditions and percentage of girls in a school.⁹ The type of secondary schools—“high school” (liceo), “technical high-school”, “vocational school”, “regional vocational school”—is a particularly important variable given that the various types of school in Italy strongly differ in terms of student achievement (Barone and Schizzerotto 2006; Gui et al. 2014). The socio-economic composition of a school has been shown to influence educational outcomes over and above student-level socio-economic status (Bratti et al. 2007b; Chiu and McBride-Chang 2006; Cosgrove and Cunningham 2011; Willms 2002). Finally, the proportion of girls in a school is used as an indicator of school composition, following the literature (for example Chiu and Khoo 2005; Dronkers and Robert 2008; Ma 2008).

3.3 Descriptive statistics

Table 1 reports the descriptive statistics of the maths cognitive tests (for the overall mathematics scale and the four subscales) and the independent variables discussed in the previous section for boys and girls in Italy. As expected, the table suggests that boys outperform girls in terms of overall maths score, although the effect size is small (see the values of the Cohen’s d measuring the effect size¹⁰). With regard to the subscales, both genders exhibit the highest scores on Quantity items, and the lowest in Change and Relationships. On all the subscales, boys obtain higher scores than do girls, even in a domain where females normally have some advantages (e.g. Quantity) (see Sect. 2.2). The differences in scores between boys and girls are higher for Space and Shape and lower for Uncertainty and Data.

Table 1 Descriptive statistics for male and female students

All the indexes on ICT use are higher for boys than girls, indicating that boys allocate more time to ICT use in various activities than do girls. The effect size is quite large for gaming activities (d = 0.68). Less strong is the difference between boys and girls with respect to computer use for communication and collaboration activities, towards which literature has shown girls are more oriented. A weak signal of higher computer use among boys than girls occurs also during maths lessons. This result may be attributed to the different types of school attended by boys and girls: the former are more likely to attend technical or vocational schools (where ICT is considered fundamental; see, for example, Calzarossa et al. 2009), whereas a higher percentage of girls attend high schools (where ICT use in school is relatively less common). Students’ background characteristics are instead not gender differentiated.

4 Results

Given that one of the main interests of this study is verifying whether the use of ICT differently affects the mathematics learning outcomes of boys and girls, interactions between gender and the different indexes of ICT use are inserted in the models. A quadratic term (identified in Table 2 as “squared”) for all the indexes of ICT use is also included in the models, following a previously adopted approach (e.g. Gui et al. 2014). This term is intended to account for the potential curvilinearity of the relationship between ICT use and learning outcomes. It is possible, indeed, that a positive effect of ICT use does not hold for students using excessively computer: as shown in other studies, students with the highest test scores may be those who use computers to a moderate extent and not those with a too intensive use (Lei and Zhao 2007; Thiessen and Looker 2007).

Table 2 Factors influencing the maths test scores for the overall mathematics scales and the four subscales

Table 2 lists the results of the multilevel models regarding the scores on the overall mathematics scale and the four overarching content areas. The effects of the control variables occur in the expected direction; that is, students in single-parent families, with low socio-economic status and immigrant backgrounds and from the South of Italy are in disadvantaged positions. Immigrant background and area of residence present a particularly strong effect. The school-level variables are important too. The strongest influence is exerted by the type of school: students attending high schools outperform those from other schools, particularly those from vocational institutions: for example, the latter students have on average an overall maths score which is about 70 (69.87) scores lower than that of students from high schools. Lastly, students in schools with higher socio-economic compositions and a lower girls’ percentages exhibit higher performance. The expected curvilinear trend of computer use is observed, since when a (significant) positive association occurs between ICT use in some activities and maths achievement, the quadratic term is negative (and significant). This means that the positive association is stronger when the ICT use intensity is moderate rather than excessive. The association, however, is quite always positive even with excessive use since the sizes of the interaction coefficients are quite low. Some exceptions are observed (for example, for the association between use of ICT for communication and collaboration activities and the four maths subscales) when the absolute value of the squared term is quite high in comparison with the absolute value of the linear term: in these cases, the association between ICT use and maths achievement can become even negative when ICT use intensity is high.

The stronger performance in all the tests among boys observed in the descriptive analyses persist in the multilevel models, wherein the students’ use of ICTs and the contributions of all the other student- and school-level variables are controlled for (confirming the results of Close and Shiel 2009).

With respect to the role of computer uses, the results indicate that the relationship between the use of ICTs in different activities and maths performance for boys and girls is considerably more complicated than it might initially appear. The intensity of ICT use for gaming is positively associated with the overall maths test scores and the score of each of the four overarching content areas (confirming the findings of Biagi and Loi 2013), but the association is weaker for girls (the interaction terms are significantly negative). For example, the positive effect of ICT use for gaming activities on the overall maths score is 2.24 scores lower for girls than for boys. A particularly strong effect is observed for computer use in information management and technical operations, but again, the positive influence of ICT use is weaker for girls (for example, the effect on the overall maths score is 7.78 scores lower for girls). The positive association between computer use for gaming activities and mathematics achievement may be explained by the fact that gaming stimulates skills such as strategic thinking, memory and fantasy, which can facilitate learning (Kirriemuir 2004).¹¹ Similarly, the positive role of using computers for information management and technical operations may be due to the development of problem-solving abilities that characterised the use of ICTs for locating and organising information. As expected, these benefits of ICT use are stronger for boys than for girls.

For other activities, the situation is more complex and differentiated according to both gender and maths subscale. In particular, for communication and collaboration activities, the intensity of the computer use is negatively associated with the overall mathematics score, although the negative association diminishes for girls, who can probably be able to take some advantages from these activities, until to a positive relationship in Space and Shape and in Uncertainty and Data. These results are probably connected with the socially oriented nature of girls: their use of ICTs for interaction with others may therefore be particularly fruitful, not only in an area such as Uncertainty and Data, which incorporates abilities strictly connected with communication, but also in a domain such as Space and Shape, where their disadvantages in comparison with boys are stronger.

For the creation of content and knowledge and problem-solving activities, the intensity of ICT use tends to be positively correlated only with the students’ test scores in the Change and Relationship and (although weakly) in Space and Shape. This correlation is independent of a student’s gender, but the effect is quite modest. This finding is quite unexpected since the usually important role of problem-solving activities for development of student competences. Nevertheless, this results is common to other studies (for example, Biagi and Loi 2013), which suggested that it can be due to the PISA test’s tendency to measure abilities that are affected mainly by the traditional aspects of the teaching–learning process (Bocconi et al. 2012), rather than by the use of new technologies (Redecker and Johannessen 2013).

Lastly, the intensity of computer use during maths lessons for specific mathematical activities is negatively associated with the maths competencies measured through the overall mathematics scale and the four subscales, except for Space and Shape, where the association is non-significant. The effect is, however, quite modest. The use of ICTs for maths may be at its initial stages: that is, we do not know, for example, if teachers have the skills and knowledge necessary to integrate technology in teaching methods or how ICTs are used as learning tools. In some cases, using computers for maths activities may substitute alternative, more effective forms of teaching and may harm the creativity of students. In general, the simple use of computers does not guarantee to provide students with fundamental problem solving knowledge and skills (Calzarossa et al. 2011). In fact, again the interaction terms with gender are significant in the direction of a lower effect for girls; again for Space and Shape domain, the association is positive.

5 Discussion

This study analysed the importance of ICT use in the mathematics performance of Italian students in secondary schools, with particular attention paid to the role of gender.

Since the literature has suggested that it is not computer use in itself that matters, but rather how computers are used, the current study described how the different uses of ICTs are associated with high or low mathematics achievement. In addition, contrary to previous studies, which did not comprehensively look into the role of gender in the ICT use-maths performance relationship, the current paper examined how such relationship varies according to gender. The use of data from PISA 2012, whose major focus of assessment was mathematics, allowed to measure mathematical literacy with respect not only to the total score but also to performance in different domains of mathematics.

The results indicated that gender differences in mathematics and in the various maths domains, with boys outperforming girls, persist even considering the different activities that boys and girls perform with computers and the contributions of control variables, such as family background and school characteristics. With regard to the importance of computer use in different activities, the scenario is quite complex: ICT use does not necessarily play a positive role in mathematics learning both for boys and girls and in all the maths domains.

In particular, a positive association exists between all the measures of mathematics competence and the intensity of ICT use in gaming as well as in information management and technical operations. A signal of a positive influence of ICT use in the creation of content and knowledge and problem-solving activities is observed also considering the Change and Relationship and Space and Shape domains. Contrastingly, the intensity of computer use for communication and collaboration activities is negatively associated with the overall mathematics score (but not with the four overarching content areas). In the same vein, the intensity of computer use during maths lessons for specific mathematical activities is negatively associated with mathematical competencies (except for Space and Shape). This result suggests a critical point of ICT use in mathematical instruction and the crucial role of teachers in this regard: teachers should reshape their teaching methods, using computers as an integral part of students’ learning process to take full advantage of recent technologies.

These associations are generally weaker for girls: thus, girls’ performance in maths is less influenced by ICT use than boys’ performance. For example, the positive role of using computer for gaming and information management and technical operations is weaker for girls than for boys; simultaneously, however, even the negative association between mathematical achievement and the intensity of computer use during maths lessons is weaker for girls than for boys. In some cases, nevertheless, some signals of the benefits of ICT use occur only for girls: this is the case of the scores on a subscale such as Space and Shape, which is the area where the gender gap is high and for which the intensity of ICT use for communication and collaboration activities and the intensity of computer use during maths lessons is related to high test scores only for girls.

The fact that girls are less influenced by ICT use than boys may be connected to the different attitudes towards technology and confidence in computer usage of boys and girls; as mentioned in Sect. 2.2, indeed, boys express more positive views towards the use of technology and this might imply stronger influence (regardless of direction) than in the case of less positive views or of less confidence in computer, which is typical of girls. Future research could consider the role of ICT usage intensity in different activities also net of these aspects. The benefits of some ICT uses that are observed only for girls can also be more exhaustively explored examining the role of different orientations, preferences and learning strategies of boys and girls: the fact that girls are usually more socially oriented than boys may help them to make the use of computer for communication and collaboration activities fruitful; the different learning strategies, with girls relying more on learning characterised by memorising and following examples and boys being more independent in their learning (Liu and Wilson 2009), may explain the different impact of computer use intensity during maths lessons for specific mathematical activities. An issue that needs further clarification is why the benefits are observed only in some maths subscales, but this suggests that any initiatives aimed at improving equity must take into account differences not only between girls and boys, but also across subscales.

Clearly, these results cannot be read in a cause-and-effect perspective due to the cross-sectional nature of PISA; however, they offer interesting insights into the gender-differentiated relationships that exist between some computer activities and achievement in mathematical literacy. Taking into account the differentiated roles and advantages of ICTs in the performance of boys and girls can be a critical component of the initiatives developed and launched by governments and educational departments. Such consideration is particularly key to teachers, to identify the most effective strategies for incorporating and integrating ICTs in the learning process for boys and girls. Besides the aspects explicitly related to teaching method (and thus oriented towards taking full advantage of recent technologies), in this context, teachers’ expectations and support are essential to student motivation and interest in computers, as well as to stimulating learners to make the most of ICT use (Vekiri 2010).