1 Introduction

In South Africa there is an ‘articulation gap’ between school and higher education curricula which is “manifested in students as a lack of sound foundations for tertiary studies and has a profound effect on students’ ability to respond positively to higher education programmes” (Scott et al. 2007, p. 42). The mismatch between students’ numeracy practices and the expectations of higher education is one of the features of the articulation gap that has a significant impact on student retention and success. The academic numeracy of most prospective university students is not what the higher education sector regards as proficient (Frith and Prince 2016; Prince and Frith 2017). This lack of proficiency can be ascribed largely to shortcomings in the teaching and learning of mathematics and English language at school. According to Spaull and Kotze (2015) “Few would argue that the state of mathematics education in South Africa is something other than dire” (p. 13). Of particular concern for numeracy practices is the fact that “in subjects such as Mathematics … teaching in schools often focuses on algorithms, standard forms and procedural knowledge” (Council on Higher Education 2013, p. 58), which provides little preparation for applying mathematical understandings in disciplinary contexts or in daily life. Similarly disappointing conditions are reported regarding literacy, for example one study showed that 58% of children in South Africa cannot read for meaning by the end of Grade 3 “and remain perpetually behind” (Spaull 2016, p. 1).

In this paper we seek to answer the question: ‘What is the relationship between students’ academic numeracy and their mathematical and language competence.’ Students’ academic numeracy is measured using the South African National Benchmark Tests Project’s quantitative literacy test. We use the performance of a large sample of prospective university students who wrote this test in 2014 as a measure of their vulnerability to failure in higher education, as the result of a mismatch between their school numeracy practices and those of academic disciplines. We use the school-leaving examination (National Senior Certificate) Mathematics and English language subject choices as a proxy measure of mathematical and language competence. The school subject choices that are most significant in terms of their potential effect on academic numeracy are between Mathematics and Mathematical Literacy, and between English Home Language and English First Additional Language (i.e. English at a lower level).

We compare the performance of those who wrote the different school-leaving examinations in the four school subjects mentioned above. The aim is to identify the extent of the vulnerability of students who have taken the different school subjects and to investigate the different relationships between these subject choices and students’ academic numeracy, both separately and in combination. However, we recognise that there are many variables that contribute to which subject a student takes, many of them unrelated to the individual’s academic potential. In using the subject choice as a proxy in this way, we are also simplifying the situation by not taking into account the performance of the individual students in these subjects, but our approach is useful because information about which subject was taken is readily available and can be used easily to identify students who are most likely to deserve additional support in developing academic numeracy. We discuss the implications of the results of our analysis in terms of what the response of higher education should be, and what kinds of interventions are likely to be required.

Ethical clearance for this research was obtained from the Research Ethics Committee of the Faculty of Higher Education Development at the University of Cape Town. This included approving the consent declaration signed by prospective students writing the National Benchmark tests, which allows the use of their results for research purposes and assures anonymity in the use of these data.

2 Student vulnerability

There are high levels of failure and drop-out in South African higher education and these problems are more likely to affect African students, many of whom are poor and working class. “Access, success and completion rates continue to be racially skewed … under 5% of African … youth are succeeding in any form of higher education” (Council on Higher Education 2013, p. 15).

This weakness is exacerbated for the African majority by the fact that the medium of instruction at school and in higher education is English (and in a few cases, Afrikaans), neither of which language is the home language of most students or their teachers. There is much evidence that learning in the medium of a second (or third) language has particularly negative effects on the learning of mathematics (Morgan et al. 2014; Bohlmann and Pretorius 2008; Howie 2003). Furthermore, the quality of the teaching of English is subject to the same limitations as schooling in general, resulting in many students coming to higher education with very limited ability to understand, let alone use, the kind of academic English they encounter in disciplinary studies. This includes the specific language forms that are used to express quantitative concepts, and which form an integral part of academic numeracy (Polito 2014; Kaplan and Rogness 2018; Paxton and Frith 2014). The students most likely to be affected are those who have not studied English at the Home Language level and who have not studied Mathematics in the last 3 years of schooling, since they are most likely to be speakers of African languages who have experienced poor schooling conditions.

The reality of the articulation gap between school outcomes and higher education expectations has resulted in most institutions introducing interventions such as extended degrees and foundation coursesFootnote 1 for the most vulnerable students, as there is little capacity in the school system to improve its outcomes (Council on Higher Education 2013, p. 63). It is therefore imperative that higher education institutions develop systems that can most effectively identify those students most deserving of extra support.

3 Academic numeracy

There is ongoing debate about what constitutes numeracy, especially in England and Australia (where it is usually referred to as numeracy) and in the United States (where it is usually called quantitative literacy, or QL). We adopt the view that academic numeracy is a practice in which people manage situations or solve problems in academic contexts that involve responding to quantitative (mathematical and statistical) information, which may be presented verbally, graphically, in tabular or symbolic form. It requires the activation of a range of enabling knowledge, behaviours and processes and it can be observed when it is expressed in the form of a communication, in written, oral or visual mode. (Frith and Prince 2006, p. 30). This view is strongly influenced by the definition of numeracy used by the Adult Literacy and Lifeskills Survey (Gal et al. 2005). According to Steen (2004), who was writing about numeracy in higher education specifically, “quantitative literacy is … about challenging college-level settings in which quantitative analysis is intertwined with political, scientific, historical, or artistic contexts. Here QL adds a crucial dimension of rigor and thoughtfulness to many of the issues commonly addressed in undergraduate education” (p. 22). Numerous authors (see, e.g., Geiger et al. 2015, and Johnston 1994) have highlighted the importance of a critical orientation to the use of mathematics and statistics in society as an essential element of numerate practice.

One aspect of the debate about numeracy concerns its relationship with mathematics:

Unlike … mathematics, QL is not a discipline but a literacy, not a set of skills but a habit of mind. As physics and finance depend on mathematical skills, so too does quantitative literacy. But as these disciplines differ from mathematics, so too does QL. (Steen 2004, p. 22).

As such, academic numeracy does not limit the mathematics and statistics involved:

there is not a particular level of mathematics associated with it: it is as important for an engineer to be numerate as it is for a primary school child, a parent, a car driver or a gardener. The different contexts will require different mathematics to be activated and engaged in. (Johnston 1994, p. 34).

The second important aspect of academic numeracy is the role played by language. According to Steen (2004), as with literacy, numeracy involves “interpretation, understanding and the power of language” (p. 21). Gee (2008) introduced the notion of Discourses, which “demand certain ways of using language, certain ways of acting and interacting, and the display of certain values and attitudes” (p. 2). Given that academic numeracy is embedded in academic Discourses, and that language is integral to any Discourse, academic numeracy and language development cannot be disentangled.

Gee’s theory underlies the academic literacies approach (Chapman and Lee 1990, Street 2005, Street and Baker 2006, Lillis et al. 2015) which conceptualises literacy and numeracy as social practice. The concept of practice offers a way of linking semiosis with what individuals as socially-situated actors do, both at the level of the context of a specific situation and at the level of the context of culture. The term “practice” is defined as “habitualised ways, tied to particular times and places, in which people apply resources (material or symbolic) to act together in the world” (Chouliaraki and Fairclough 1999, p. 21). This notion of practice was originally applied to numeracy by a number of theorists such as Street (2000, 2005), Baynham and Baker (2002), Barton (2006), and Kelly et al. (2007), and has more recently been adopted by researchers in the USA (Oughton 2018; Craig and Guzman 2018). Viewing numeracy as social practice rather than a collection of skills to be learned, means that the mismatch between students’ practices and the requirements of higher education is not described in terms of student deficit, but rather as an articulation gap that higher education has a responsibility to address.

The conceptualisation of numeracy as practice once again highlights the integral role of language in numeracy. As Barton (2006) found in his study of numeracy in everyday life, “when people talked of having difficulty with numbers … the language and literacy associated with numeracy were part of the issue” (p. 29). According to the the conceptual framework for the assessment of numeracy developed for the OECD’s Programme for the International Assessment of Adult Competencies (PIAAC):

the structure of the tasks and demands in adults’ lives shows that [literacy and numeracy] cannot be considered as mutually exclusive. Mathematical or statistical information is carried by or embedded in text in some … contexts in which adults have to function. … one’s performance on numeracy tasks will depend not only on formal mathematical or statistical knowledge but possibly also on literacy-related factors such as vocabulary, reading comprehension, reading strategies, or prior literacy experiences. (Gal et al. 2009, p. 19).

The accurate expression of quantitative ideas requires very precise use of specific vocabulary and language forms, what MacNeal (1994) termed ‘Mathsemantics’. For example, Schield (2008) pointed out that “The comparison of ratios, rates and percentages in ordinary language requires using English in a very precise manner. Small changes in syntax can produce large changes in semantics” (p. 94).

4 Academic numeracy assessment in the National Benchmark Tests Project

Higher Education South Africa (HESA), a membership organisation representing South Africa’s universities, commissioned the National Benchmark Tests Project in 2005 with its main aim being to create tests to assess the academic proficiency of prospective students wishing to enter higher education. These criterion-referenced tests provide a measure of students’ readiness for higher education and are “constructed to provide information about the level of a test-taker’s performance in relation to clearly defined domains of content and/or behaviours (e.g. reading, writing, mathematics) that requires mastery” (Foxcroft 2006, p. 9). Minimum benchmark scores for three different proficiency levels are established through a rigorous standard-setting process.

The National Benchmark Quantitative Literacy test aims to measure the levels of proficiency in academic numeracy of school-leavers who are aspiring to enter higher education. The construct of this test was presented by Frith and Prince (2006), and the test was described in detail by Frith and Prince (2018). In practical terms, this test assesses students’ ability to interpret and reason with quantitative information presented in a variety of modes. For example, they must read and interpret tables, graphs, charts, diagrams and texts that use common quantitative terms and phrases. They must apply quantitative procedures in various situations to do simple calculations and estimations, which may involve multiple steps. The test questions are designed to assess numeracy practices and do not assume that students have the knowledge of any particular school subject.

4.1 Structure, administration and scoring

There are 50 multiple choice items in the National Benchmark Quantitative Literacy test, which are selected in accordance with the specification table (Frith and Prince 2006). This specifies the proportions of items that should address each of the competencies, mathematical and statistical ideas and levels of cognitive complexity defined in the test construct. In addition to the 50 items the test also contains 20 unscored ‘pre-test’ piloted items.

The National Benchmark tests are administered at test centres across South Africa to upwards of 80,000 prospective higher education students annually, under standardised conditions, by specially trained invigilators. Calculators are not used, but students are required to calculate only with simple numbers, for example, with fractions that can easily be simplified by cancellation. Many questions can be answered by estimation.

Responses are dichotomised (either 1 for right or 0 for wrong). The unidimensional three parameter Item Response Theory (IRT) model, (Yen and Fitzpatrick 2006) is used to determine a student’s ability and generate a score for the candidate on a scale of 0–100%. Results for different versions of the test are linked and equated using anchor items and the Stocking and Lord method (Holland and Dorans 2006), to ensure that a candidate’s score is independent of the version of the test that they wrote.

4.2 Proficiency categories

The scores defining the proficiency categories are established every 3 years at standards-setting workshops by panels of diverse South African academics who teach courses relevant to the domain. It is carried out using the “modified Angoff” method (Hambleton and Pitoniak 2006) in which each item is rigorously assessed by higher education professionals in order to establish benchmarks that reflect the expectations of the higher education curriculum. Table 1 provides the score ranges of the three main academic numeracy proficiency categories for degree study, and the recommendations for appropriate responses by institutions for students whose scores place them in these categories.

Table 1 National Benchmark Quantitative Literacy test benchmarks for degree study, set in 2012
Full size table

It has also been useful to differentiate between different levels of support that would be most appropriate for students with scores in the Intermediate category and so this level is divided in the middle into Upper Intermediate and Lower Intermediate bands as shown in Table 2. This differentiation is not done through the standards-setting workshops but is effective for pragmatic reasons, as the majority of scores are in the Intermediate category.

Table 2 National Benchmark Quantitative Literacy test: Intermediate benchmarks for degree study and how they should be interpreted
Full size table

5 The National Senior Certificate (NSC) school-leaving examinations

5.1 Mathematics and mathematical literacy

All candidates for the school-leaving qualification must write the examinations for either Mathematics or Mathematical Literacy, which are both cognate with but not the same as academic numeracy, as can be seen from their descriptions in the National Curriculum and Assessment Policy Statement (CAPS) documents.

These documents describe Mathematics as,

a language that makes use of symbols and notations for describing numerical, geometric and graphical relationships. It is a human activity that involves observing, representing and investigating patterns and quantitative relationships in physical and social phenomena and between mathematical objects themselves. (Department of Basic Education 2011a, p. 8).

The Curriculum and Assessment Policy Statement document claims that studying Mathematics will develop a student’s ability to think logically, critically and creatively and to be able to solve problems in order to understand real-world phenomena. This suggests strong similarities with the definition of numeracy, but the main focus of the subject is in fact on learning the discipline of mathematics itself to ensure access to further study of the mathematical sciences and in a variety of career paths.

On the other hand, the Curriculum and Assessment Policy Statement document for Mathematical Literacy states that the competencies it develops should,

allow individuals to make sense of, participate in and contribute to the twenty-first century world—a world characterised by numbers, numerically based arguments and data represented and misrepresented in a number of different ways (Department of Basic Education 2011b, p. 8).

It suggests that these competencies, which include the ability to reason, solve problems, interpret information and use technology, should be developed by exposing learners to elementary mathematical content applied to authentic real-life contexts. An emphasis on using mathematical knowledge and skills in context makes this subject similar to academic numeracy, but with life-related rather than academic contexts being stressed. Given that the mathematical content of the subject is elementary, it is clear that the mathematical competence of students who have studied this subject will be at a lower level than that of students who studied Mathematics, although one might expect it to provide some preparation for academic numeracy.

5.2 Languages

Language learning in South African schools includes the eleven official languages, Afrikaans, English, isiNdebele, isiXhosa, isiZulu, Sesotho sa Leboa, Sesotho, Setswana, siSwati, Tshivenda, Xitsonga, as well as non-official languages. These languages are offered at different levels, the first two being the Home Language and First Additional Language levels. Many South African schools do not offer the home languages of some or all of the enrolled students. As a result, the labels Home Language and First Additional Language refer to the proficiency levels at which the language is offered and not necessarily to the native (home) or acquired (as in the additional languages) language of the student.

The Home Language level provides for language proficiency that reflects the mastery of interpersonal communication skills required in social situations and the cognitive academic language proficiency (Cummins 1979) essential for learning across the curriculum. The First Additional Language level assumes that students do not necessarily have any knowledge of the language when they arrive at school and that by the time they finish school, they should be reasonably proficient in their first additional language with regard to both interpersonal communication and cognitive academic language proficiency. However, the reality is that many learners still cannot communicate well in their additional language at this stage (Uys et al. 2007). This implies that many students who took English First Additional Language will not have the cognitive academic language skills required for higher education study in the medium of English. It also means that these students will have suffered the known negative effects of learning mathematics in school in a language that is not their home language and in which they are not competent (Morgan et al. 2014, Bohlmann and Pretorius 2008, Howie 2003).

6 Method

The National Benchmark Quantitative Literacy test results were analysed for school students from across South Africa who wrote one version of this test in 2014 during their last year at school, indicating that they were intending to apply to higher education institutions for study in 2015. The sample used for this study (n = 7464) includes data only for those who then went on to write the school-leaving examinations and who obtained a result that allowed them to progress to some kind of higher education study. Strictly speaking, these are therefore prospective higher education students, but for the sake of brevity we will refer to them as students.

Descriptive statistics were calculated and box-and-whisker plots were constructed for the distributions of scores for the whole sample, for the students who wrote each school subject and for the students in four mathematics–language subject combination groups, as follows: Mathematics and English Home Language; Mathematics and English First Additional Language; Mathematical Literacy and English Home Language; Mathematical Literacy and English First Additional Language. For each of these groups of students the proportions of scores falling in the different proficiency bands were also calculated and presented as multiple bar charts.

To investigate whether differences observed between the four mathematics–language subject combination groups were statistically significant, a one-way analysis of variance (ANOVA) was conducted, to compare the differences between the mean scores. Post hoc comparisons using the Tukey honesty significant difference (HSD) test were then carried out to investigate which of the groups had statistically significantly different means (using a 5% significance level).

In order to examine how students responded to individual items, the proportion of students who chose each alternative answer was recorded. This was done for the whole cohort and separately for the students in four mathematics–language subject combination groups.

The statistical package R (R Core Team 2018) was used for the data analysis and the R package ggplot2 (Wickham 2016) was used to create the graphical representations.

7 Results

We first present some information about the characteristics of the students in the sample. We then present the distribution of scores on the National Benchmark Quantitative Literacy test for the whole sample as well as separately for those who wrote Mathematics, Mathematical Literacy, English Home Language, and English First Additional Language school-leaving examinations. This is followed by distributions of scores to illustrate the relative performance on the test of students in the four mathematics–language subject combination groups. We present the results of the statistical tests used to determine if the difference between the mean test scores for students who wrote the different mathematics–language subject combinations is significant. In order to highlight some particular areas of difficulty that students experienced, we show in more detail the performance of the different mathematics–language subject combination groups on two examples of individual items.

7.1 Characteristics of the sample

Some characteristics of the students in the sample are shown in Table 3. Approximately 60% were African and the majority did not have English as their home language. English home language speakers however formed the largest language group, comprising nearly 40%. There were considerably more females than males in this sample (about 60% and 40% respectively). The preponderance of females is also generally observed in the larger cohorts of all National Benchmark Quantitative Literacy test takers. Just more than half of the students in the sample took the combination of Mathematics and English Home Language subjects at school while about 5% took Mathematical Literacy and English First Additional Language.

Table 3 Characteristics of the students in the sample (n = 7464)
Full size table

7.2 Distributions of scores for the National Benchmark Quantitative Literacy test by school subject

Figure 1 represents the distributions of scores obtained for the National Benchmark Quantitative Literacy test for students who wrote different school subjects and for the sample as a whole. The scores of the students who wrote Mathematical Literacy (MTL), except for the outliers, are consistently lower than those of the Mathematics (MTH) students, showing that they would be less prepared for the numeracy demands of higher education. The students who wrote English First Additional Language (ENF), except for the outliers, performed considerably worse on the test and would be much more vulnerable in terms of not meeting the numeracy demands of higher education than would those who wrote English Home Language (ENH).

Fig. 1
figure 1

Distributions of test scores by school subject and for the whole sample. (MTH Mathematics, MTL Mathematical Literacy, ENF English First Additional Language, ENH English Home Language). The horizontal lines at 38% and 70% define the lower limit of the Intermediate category and the lower limit of the Proficient category respectively

Full size image

In Fig. 2 the relative proportions of scores falling in the different benchmark categories are shown for the students who wrote different school subjects as well as for the whole sample. The results presented in this way show even more clearly that those who have taken Mathematical Literacy (MTL) or English First Additional Language (ENF) have a very high risk of not meeting the numeracy demands of higher education. For both subjects most of the test scores were in the Basic category which means that it is highly likely that students will not cope with degree-level study without extensive and long-term support (see Table 1).

Fig. 2
figure 2

Relative proportions of test scores in different proficiency categories by school subject and for the whole sample. (MTH Mathematics, MTL Mathematical Literacy, ENF English First Additional Language, ENH English Home Language)

Full size image

7.3 Comparison between test scores of different mathematics–language subject combination groups

From the above results it is clear that students who have taken Mathematics and English Home Language at school are somewhat better prepared for the academic numeracy demands of higher education and that those who have taken Mathematical Literacy and English First Additional Language will be the most vulnerable to failure as a result of their academic numeracy challenges. It follows that analysing the data for student subgroups defined in terms of the combination of language and mathematical school subjects will provide insights into the ways in which school practices impact on students’ academic numeracy competencies (as measured in the National Benchmark Quantitative Literacy test).

In Figs. 3 and 4 the test performance is presented in a way similar to Figs. 1 and 2 respectively, but for groups defined by school mathematics and language subject group combinations. As expected, the Mathematical Literacy and English First Additional Language (MTL & ENF) group are the most vulnerable and the Mathematics and English Home Language (MTH & ENH) group are the least vulnerable in terms of the numeracy demands of higher education. By comparing the distribution for the Mathematical Literacy and English Home Language (MTL & ENH) group to the distribution for the Mathematics and English Home Language (MTH & ENH) group, we can see that for English Home Language students, having written Mathematical Literacy instead of Mathematics was associated with a reduction of the median score from 55 to 37%. For those who wrote Mathematics, having studied English First Additional Language rather than Home Language was associated with a reduction in median score of a similar magnitude (comparing the distribution for the Mathematics and English First Additional Language (MTH & ENF) to the distribution for the Mathematics and English Home Language (MTH & ENH) group). This suggests that not having studied English at the Home Language level is as strongly related to a students’ academic numeracy practice as not having studied Mathematics at school. This reinforces the claim that language plays a substantial role in academic numeracy practices, and that mathematical proficiency does not guarantee a high level of academic numeracy.

Fig. 3
figure 3

Distributions of test scores by school subject combination and for the whole sample. (MTH & ENH Mathematics and English Home Language, MTH & ENF Mathematics and English First Additional Language, MTL & ENH Mathematical Literacy and English Home Language, MTL & ENF Mathematical Literacy and English First Additional Language). The horizontal lines at 38% and 70% define the lower limit of the Intermediate category and the lower limit of the Proficient category respectively

Full size image
Fig. 4
figure 4

Relative proportions of test scores in different proficiency categories by combinations of school subject and for the whole sample. (MTH & ENH Mathematics and English Home Language, MTH & ENF Mathematics and English First Additional Language, MTL & ENH Mathematical Literacy and English Home Language, MTL & ENF Mathematical Literacy and English First Additional Language)

Full size image

The one-way ANOVA analysis revealed that there is a significant difference in mean values [F(3, 7460) = 1149, p = 0.000] of the academic numeracy test scores between the four school subject combinations groups. Table 4 and Fig. 5 show the results of the Tukey HSD test for difference between the means. There was a statistically significant difference between five of the six pairs of groups. The only difference that was not significant (p = 0.1 > 0.05) was between the mean for the Mathematical Literacy and English Home Language (MTL & ENH) group and the mean for the Mathematics and English First Additional Language (MTH & ENF) group. This is compatible with the similarity in the distributions of scores for these two mathematics-language subject groups that was noted above (see Fig. 3).

Table 4 Differences between mean test scores for different mathematics-language subject groups with 95% confidence intervals and p values
Full size table
Fig. 5
figure 5

Differences between mean scores for different mathematics-language subject groups with 95% confidence intervals. Only one confidence interval, for the mean difference between (MTL & ENH) and (MTH & ENF), contains the value zero, indicating that this is the only case where the difference between the means is not significant

Full size image

7.4 Performance on selected individual items

The examples described in this section illustrate how a detailed examination of the proportions of students who chose different alternative answers in certain items can be used to illuminate the ways in which language competence and mathematical proficiency combine to determine students’ degree of vulnerability in higher education in terms of their academic numeracy. The examples also show how the pattern of responses in certain items reveals the ways in which the language practices of academic numeracy play a major role in students’ performance.

Example 1: Interpreting percentage values in a table A question that required students to interpret the meaning of a percentage given in a table was based on a table with the structure illustrated in Table 5. The table did not include a row or column of “100%” values to make explicit which category (sex or age) was used as the denominator in the percentage calculation.

Table 5 Structure of a table used in a test question requiring interpreting percentages
Full size table

Figure 6 shows the results for an item which refers to a table with data similar to those in Table 5. The question required students to select the correct verbal description of a given percentage value in the table.

Fig. 6
figure 6

For a question requiring interpretation of a percentage value in a table, the proportions of students who chose each alternative answer, for the total cohort and for each mathematics-language subject combination group. The correct alternative is D and N stands for “Not answered”

Full size image

Only the students who had written Mathematics and English Home Language chose the correct description (D) in the majority of cases. For example, for the value 76.9% in Table 5, this is equivalent to ‘76.9% of the people under 15 years were males’. The incorrect description (A) favoured by all other groups (and by just under half of all students) was the one which would correspond to the statement ‘76.9% of the males were under 15 years’. Schield (2008) reported a similar observation for a survey of college students in the United States, where 19% made the equivalent mistake in reading a table of percentages. Thus, in our sample, most students (and even 40% of those who wrote English Home Language) could not analyse the structure of the table correctly or interpret the English sentence describing a percentage to correctly identify the numerator and denominator. Polito (2014) also reported that college students find it difficult “to pick out a single percentage and decide whether it is a percentage out of the row, the column, or out of some other whole” (p. 8).

Many disciplines in higher education will require students to interpret tables of data and most lecturers will assume that students can understand these representations. The data in Fig. 6 show that more than half of first year students will have trouble interpreting the language used to describe percentages, and with making sense of percentage data in tables.

The pattern of the responses shown in Fig. 6 confirms that the group who wrote Mathematical Literacy and English First Additional Language (MTL & ENF) had the worst performance (the least percentage answering correctly and the greatest percentage choosing the most attractive incorrect alternative) and those who wrote Mathematics and English Home Language (MTH & ENH) had the strongest performance. It is noteworthy that both of the other two groups performed almost exactly equally, showing that poor language practices and poor mathematical practices (as reflected by which NSC subject was taken) in this case are equally strongly related to students’ reduced likelihood of success in this question.

A very similar pattern (in terms of the relative sizes of the proportions of students in each of the different groups who selected the correct answer) is repeated in approximately 80% of the 70 items (50 test items and 20 pre-test items) analysed, lending support to our argument that poorer language practices and poorer mathematical practices are approximately equally related to students’ vulnerability to failure as a result of their academic numeracy preparedness for higher education, and that the most vulnerable are those who wrote English First Additional Language and did not write Mathematics in the NSC examinations.

Example 2: Integrating information from more than one source and calculating a percentage of a percentage Given values in a bar chart for the proportion of the population in each province of a country that is, say, over 15 years of age, as well as the values of the total population for those provinces (as relative percentages of the total in a pie chart), one might ask ‘What percentage of the population of the country is over 15 years and lives in Province X?’. This requires students to determine the percentage of the Province X’s population that is under 15 (from the bar chart) and combine it (by multiplying) with the percentage of the country’s population that lives in this province (from the pie chart).

The results for a question of this kind are shown in Fig. 7. The numbers involved were very simple, so that the answer could easily be identified without doing a calculation if the student understood the question conceptually.

Fig. 7
figure 7

For a question requiring calculating a percentage of a percentage using values derived from more than one chart, the proportions of students who chose each alternative answer, for the total cohort and for each mathematics-language subject combination group. The correct alternative is A and N stands for ‘Not answered’

Full size image

The vast majority of students in all the groups chose alternative C, which is the percentage of the population that is under 15 years in the given province (not in the whole country), which is read off the bar chart only. Another alternative (B) that was more attractive than the correct answer to all but the strongest group (MTH & ENH), is the percentage of the population in the given province, read off the pie chart only.

It appears that most students were unable to interpret a sentence that describes a percentage of a whole (country’s population) defined in terms of two characteristics (a subgroup defined by age of a subgroup in a particular province) in such a way that they recognised the need to combine two pieces of information. The fact that such a small proportion of the students could answer this question correctly illustrates quite dramatically how the language used to express quantitative concepts (particularly proportions) can be very difficult for these students to understand and use correctly. Once again this highlights the need to include appropriate language development activities in any educational interventions intending to improve students’ academic numeracy. Polito (2014) reported a similar lack of fluency with the language of percentages in the United States college population and suggests some approaches for teaching this language.

8 Discussion and conclusion

The fact that only a very small proportion of prospective students who write the National Benchmark tests are deemed proficient by higher education standards, in terms of their academic numeracy practices, indicates that ideally there needs to be widespread and systematic modification of both school and higher education curricula to address this situation for all students. Ideally the articulation gap between school outcomes and higher education requirements should be addressed by the schools as well as by higher education, but, as noted previously, there is little capacity for the school system to improve in the short term. Thus, the responsibility for responding to the articulation gap must be taken by the higher education sector. Part of this responsibility is to introduce interventions to improve students’ chances of success through developing their academic numeracy. Another reason for such an intervention is to ensure that graduates enter society with well-developed numeracy practices in order to contribute optimally to society as professionals and as critical citizens.

However, resource constraints and conservatism within the higher education system can be expected to limit the extent to which the articulation gap can be addressed. The main structure that currently exists in South African higher education to address this gap is the provision of foundation courses and extended degrees for students identified as most in need of additional support. At our university, apart from teaching foundation numeracy courses, we also provide numeracy interventions (such as workshops and computer-based tutorials) that are integrated as part of other disciplinary course structures and are intended to develop students’ numeracy practices within the disciplinary discourse. Various criteria are used to identify students for these interventions, including the National Benchmark tests. Since resources for the provision of this support are limited, it is important to ensure that the most deserving students are selected in the most efficient way.

Socio-economic conditions in South Africa are such that students with Mathematical Literacy and English First Additional Language school subject choices are very likely to come from disadvantaged educational backgrounds. The results of this study confirm that these students constitute a particularly vulnerable group who will be the most in need of extra support in terms of the academic numeracy demands of higher education. This means that considering the subjects written in the school-leaving examinations could provide an inexpensive and freely available mechanism for identifying students most in need of support.

Given that there is a demonstrated need for extra support for students in terms of developing their academic numeracy practices, the question remains what form this support should take. Since academic numeracy is embedded within disciplinary discourses, ideally the development of students’ academic numeracy should be carried out by the disciplinary lecturers as part of their teaching in their disciplines: “numeracy needs to be seen as an integral part of subjects across the curriculum” (Bennison, 2015, p. 561). This means that in schools and in higher education, as indicated by Callingham et al. (2015), “all teachers should have both content knowledge about the numeracy aspects of their subject and pedagogical content knowledge about the ways in which numeracy might be developed in their students” (p. 552). The challenges discussed by Callingham et al. (2015) regarding equipping teachers to be able to develop numeracy across the curriculum also apply in the higher education context, with the result that, in practice, interventions (where they are provided) in higher education in South Africa are usually delivered by ‘numeracy specialists’, which can have the effect that it is difficult to persuade students of the relevance of numeracy as part of their disciplinary discourse.

Whatever the nature of the interventions that can be achieved in practice, it is clear from the results of this study that such interventions must pay at least as much attention to developing students’ use of appropriate quantitative language as to their mathematical competence. This makes provision of such support even more challenging, as designing academic numeracy support that includes appropriate language development for students in the context of many different disciplines clearly requires specialised skills. Although in the literature there is some recognition of the idea that writing and quantitative reasoning would both benefit from being taught in combination (Lutsky 2008; Wolfe 2010), few authors have focussed on the teaching of the precise language specific to expressing quantitative ideas in academic contexts. There are not, to our knowledge, many resources available for academics wishing to do this. Nevertheless, this study has shown that, to be effective in developing students’ academic numeracy practices, interventions will have to address not only students’ mathematical competence in context, but equally their ability to comprehend and express ideas using appropriate quantitative language. Establishing how best to do this could be an important area for future research in numeracy education.