Following the White Paper on education (2011), one of Slovenia’s most important goals in the field of education today is the establishment of a culture of quality and assessment, which is based on the concept of evidence-based policy, where participation in large-scale assessments (ILSAs) plays an important role. Beginning in 1996, Slovenia has participated in different ILSAs (PISA, TALIS, PIAAC, TIMSS, PIRLS, ICCS, and ICILS). In this chapter, we focus on PISA results, which on the one hand demonstrate that throughout the cycles beginning in 2006, Slovenian students have achieved mainly above-average results in science, mathematics and reading. On the other hand, they report rather low motivation to learn. Further, national secondary analysis results also reveal significant disparities in achievement between boys and girls, students enrolled in different educational programmes, from different socio-economic backgrounds, with different immigration backgrounds, and languages spoken at home. In this chapter, we emphasise the importance of the not-self-evident treatment of the above-average results on an international scale as the great efficacy of the national education system. At the end, the main challenges of using ILSA results to develop Slovenian educational policy and future practices are discussed.

The Education System of the Republic of Slovenia

The education system of the Republic of Slovenia is organised as a public service rendered by public and private institutions that provide officially recognised or accredited programmes. By law, public schools are secular and the school space is autonomous. The providers of public service are under supervision of the school inspectorate. The Slovenian education system is organised into several levels of education: Pre-school education (is optional, and encompasses the centre-based early general pre-school education and care. Children are legally entitled to a place in a kindergarten from the age of 11 months to the age of compulsory schooling); compulsory basic education (is organized in a single-structure 9-year basic school attended by pupils aged 6–15 years); upper secondary education (takes 2–5 years, typical age of students is 15–19, educational programmes include vocational, professional and general gymnasium programmes); tertiary education (includes short-cycle higher vocational education and higher education study programmes); and adult education (is marked by its considerable diversity of programmes and institutions). At the end of grades 6 and 9 of compulsory education, pupils undertake compulsory national assessment in three subjects. Tracking of students begins in upper secondary education, typically at the age of 15, after they finish grade 9. Students may choose freely among general and vocational programmes. If the number of candidates exceeds the number of places, schools may limit enrolment in the first year. In this case student’s scores from grades 7 to 9 are considered, and in some cases scores on national assessment in grade 9 are also taken into account. At the end of upper secondary education, students take final exams (school leaving examination in 2- and 3-year programmed and vocational or general Matura in 4-year programmes). Matura is a national external examination which allows the students to enrol in tertiary study programmes (EACEA 2019).

Introduction to the International and National Assessment Contexts and Their History

The educational system in present-day Slovenia has a long history, which can be divided into four phases: imperialistic (until World War II), supervised (from World War II until the 1990s), sovereign (post 1991), and globalised education policy (post 2004) (e.g. Štremfel 2015). An important turning point in its development occurred in the 1990s, following Slovenia’s independence in 1991. The comprehensive education reform, which occurred during that time, was characterised by a desire to break away from socialist ideological influences and get closer to the modern standards of the developed Western Europe (e.g. Gaber 2008). In addition to adopting comprehensive legislation (1991–1996) and curriculum reform (1997–1999), a major focus in Slovenia in establishing its sovereign educational system was to identify and provide the quality of education (see Table 19.1) consistent with international trends. At the same time, it served as a mirror of several issues and dilemmas about the present system and the further development of education in Slovenia (e.g. Kos Kecojević and Gaber 2011). How important it was for Slovenia to follow international trends of global standards of educational quality and achievement is also evident from the White Paper on education (1996, p. 71): “In Slovenia, one of the goals of the renewed school system is to allow achieving internationally comparable standards at the end of the primary school”. The end of comprehensive education reform (and the recognised need for evaluating its effects) in Slovenia coincided with a global paradigmatic shift towards a knowledge-based society/economy. This has in Slovenia, as elsewhere, undoubtedly concentrated the focus of education on measuring achievement and setting new standards of quality assurance, as indicated by the growing number of evaluations of educational programmes and institutions (Kos Kecojević and Gaber 2011). Many authors (e.g. Biesta 2007) believe that the shift towards outcome-centred education is closely associated with the concept of evidence-based policy-making. The empirical study (Štremfel 2013) reveals that, according to the perception of key national actors (8 policy-makers and 22 experts participating in the study), the concept of evidence-based education in Slovenia is still to be developed.

Table 19.1 Involvement of Slovenia in international large-scale assessments
Full size table

Great aspirations for following international trends in education since the beginning of its sovereignty on the one hand, and the paradigmatic shift towards outcome-centred education from 2000 onwards, on the other hand, resulted in increasing Slovenian involvement in large-scale, international assessments (ILSAs; Table 19.1).

In the 1990s, the TIMSS and PIRLS framework, which is more curriculum-based, allowed the exploration of student achievement in reading, mathematics, and science for the periods before, during and after the reform, which is why the focus at the time was on the results of these studies. A greater incentive for Slovenia to participate in PISA in 2006 for the first time was not only Slovenia’s candidature for membership in the OECD (2007–2011), but also its accession to the EU in 2004. The EU benchmark measures the percentage of 15-year-olds who fail to achieve basic levels of reading, mathematics and science literacy in PISA and encourage member states to attain the common EU goal (less than 15% of low achievers by 2020) by comparing their attainment and sharing good practices in attaining it. In the meantime, Slovenia joined other ILSAs measuring different competences and skills, such as language competence (Language Education Study, ESLC), civic competence (CIVICS and ICCS), and computer and information literacy (SITES, ICILS), as well as adult skills (IALS, PIAAC). Slovenia’s involvement in various ILSAs allows international comparison of the achievements of Slovenian students and adults in different educational contexts, and also measures trends when participating in the same study in more cycles over longer periods of time. At the same time, the requirement of ILSA to achieve the technical standards of data collection, particularly at the beginning, contributed to improving the quality of Slovenian research, as a country without a strong previous tradition in this field (Štraus 2005).

How important ILSAs remain in the Slovenian educational system is also evident from the White Paper (2011, p. 24–25), which states:

One of the important goals of Slovenian education is to ensure internationally comparable education for our pupils and students. … To achieve internationally comparable education of our students, in addition to internationally harmonized curricula and standards of knowledge, we must also achieve international harmonization of the criteria for assessing knowledge, of course with those countries that we want to compare. … At the state level, we have to clearly set and pave the way to the goal, that according to the quality of the presented knowledge, Slovenian students rank to the top that is at least the top third of the achievements of the students of the developed countries.

The next section discusses the role of ILSAs in Slovenia’s current assessment framework.

International and National Assessments in Slovenia Today

Following the EU Strategic framework of Education and Training 2020, one of the most important goals for Slovenia in the field of education today is the establishment of a so-called culture of quality and assessment, based on the concept of evidence-based policy. Therefore, Slovenia is currently upgrading an existing framework of assessment and educational quality assurance, which has been built since the end of the 1990s. The framework is based on the following forms of (internal and external) systematic assessment:

  • Internal self-evaluation in schools (From 2008, schools in Slovenia must carry out annual internal evaluations according to the Organization and Financing of Education Act.);

  • External knowledge tests (National assessment of knowledge at the end of Grades 6 and 9, defined in the Elementary School Act (1996), General Matura and Vocational Matura, defined in the Matura Examination Act (2006)) conducted by the National Examinations Centre;

  • National (2-year) evaluation studies, established by the Organisation and Financing of Education Act, defined by ministerial acts and conducted by research institutes and universities;

  • External evaluation of schools defined by the School Inspection Act (1996) and conducted by the Inspectorate of the Republic of Slovenia for Education and Sport;

  • External evaluation of the system by participating in different ILSAs defined as a priority in the White Paper on Education (2011) and conducted by research institutes (Educational Research Institute and Slovenian Institute for Adult Education).

Such an approach aims mainly at establishing professional cores that could support teachers in their process of empowerment in the fields of formative assessment and evaluation of their own work and their students’ work and knowledge, and at assessing and improving the quality of the educational system as a whole (MIZS 2017; OECD 2016b). In the latter, participation in ILSAs plays an important role.

National results in international comparison are an important part of establishing evidence-based policy and enhancing the quality of the Slovenian educational system (White Paper on Education, 2011). Consequently, today a country participates in different ILSAs, mainly those carried out by the International Association for the Evaluation of Educational Achievement (IEA) and the Organisation for Economic Co-operation and Development (OECD), that is, OECD PISA, OECD TALIS, OECD PIAAC, IEA TIMSS, IEA PIRLS, IEA ICCS, and IEA ICILS (Table 19.1).

Currently, the main goal on a national level is to establish a systematic approach to quality assurance in education that would incorporate the advantages and common parts of all of the above-mentioned studies, as stated by Wagemaker (2014, p.13): provision of high-quality data to improve policy-makers’ understanding of key school-based and non-school-based factors influencing teaching and learning, provision of high-quality data as a resource for identifying areas of concern and action, and for preparing and evaluating educational reform and development, and improvement of the capacity of educational systems to engage in national strategies for educational monitoring and improvement.

In this chapter, we discuss an example of ILSA results in Slovenia. In this context, we decided to take a closer look at PISA results and trends, because they represent an important aspect of educational policy at national and international levels.

An Example of International and National Assessment Findings: The PISA Study

Beginning already in 1995, Slovenia has participated in different ILSA studies, in the case of PISA, from cycle 2006 onwards. Throughout the cycles, Slovenian 15-year-olds mainly achieved above the OECD average in all three measured PISA domains (science, mathematics and reading; Table 19.2). The exceptions are reading achievements in 2009 and 2012, which were significantly below the OECD average. Trends in all three domains also tend to be at least stable, if not positive (OECD 2016a).

Table 19.2 Average PISA performance in mathematics, science and reading in Slovenia, 2006–2015
Full size table

In the case of science PISA literacy, the past 9 years indicate stable, statistically significant above-OECD-average results. Although scores for Slovenia on an international science scale dropped an average of 1.5 points in every cycle, the difference did not prove to be statistically significant. The last data from the 2015 cycle also revealed that 85% (the OECD average is 79%) of Slovenian students achieve the baseline level of proficiency in science (Level 2 on the science literacy scale). It is expected that all students should attain Level 2 by the time they leave compulsory education. This way, they are able to successfully continue their secondary education and are able to tackle everyday tasks related to different literacy contexts. According to PISA 2015 data, 11% of students attained the highest level of proficiency in science, which is also above the OECD average (8%). The percentage of high-performing students has decreased by 2 percentage points from the 2006 cycle (from 12.9% in 2006 to 10.6% in 2015), but proved to be stable in the last two PISA cycles (OECD 2016a; Štraus et al. 2017).

Achievement in mathematics in Slovenia also proved to be stable and significantly above the OECD average over time. In PISA 2015, Slovenian students achieved on average 510 points on the mathematics PISA test; students in OECD countries achieved 490 points. Only European students from Estonia (520 points) and Switzerland (521 points) scored higher than Slovenian students. Between the 2012 and 2015 cycles, there was a statistically significant increase (9 points) in mathematics achievement on the PISA test in Slovenia, but the average 3-year positive trend (1.7 points) did not prove to be significant. From PISA 2015 data, it is also evident that there was a decrease in the percentage of students attaining the lowest levels (below Level 2) on the mathematics literacy scale (from 20% to 16%) between cycles 2012 and 2015, while the percentage of students attaining the highest level remained stable. According to PISA 2015 data, 84% of Slovenian students achieved baseline proficiency in mathematics (Level 2; OECD average is 77%), and 13% (the OECD average is 11%) of students achieved the highest levels (Levels 5 or 6; ibid.).

Reading literacy proved to be the weakest domain of Slovenian 15-year-olds, according to PISA 2009 and 2012 results, with scores significantly below the OECD average. However, from the cycle 2009, the average 3-year trend was positive and among the highest (11 points) between participating countries. For example, in the cycle 2015, students scored significantly higher on the reading literacy scale than in 2012, that is, 24 points (481 points in 2012 and 505 in 2015), and also above the OECD average (493 points). According to PISA 2015 data, 85% (vs. 80% for the OECD) of Slovenian students achieved the baseline proficiency level (Level 2) in reading, which is 6 percentage points more than in 2012, and 9% (vs. 8% for the OECD) of students achieve the highest levels (Levels 5 and 6) on the reading literacy scale. As such, Slovenia is the only one of the participating countries where the percentage of high-proficiency students in reading increased between 2012 and 2015, and the percentage of low-proficiency students decreased (ibid.).

On the other hand, national data demonstrate that there are significant differences in the achievement according to achievement predictors, for example, gender, educational programme, socio-economic status, immigrant background, language spoken at home and motivation to learn.

PISA 2015 results indicate that Slovenia is still among the countries with the largest significant gender gap in reading achievement (43 points) in favour of girls (528 points vs. 484 points). Results also reveal significant differences in science achievement between girls and boys, also in favour of girls (516 points vs. 510 points). In mathematics, boys on average performed slightly better than girls (512 points vs. 508 points), but the difference didn’t prove to be statistically significant (Štraus et al. 2017).

Further analysis of national data (e.g. Šterman Ivančič and Puklek Levpušček 2018) also revealed significant differences in achievement in all three domains according to the student’s educational programme. Results of the secondary analysis demonstrate that, for example, students from general gymnasium programmes scored on average 584 points on the science literacy scale, students from technical educational programmes scored 499 points, and students from vocational-educational programmes scored 418 points. The difference in scores between students in general gymnasium and vocational programmes is approximately 160 points, which corresponds to approximately 5 years of schooling.

PISA 2015 national results also revealed that students from different educational programmes vary according to their reported socio-economic background: for example, the index of socio-economic status for students in general gymnasium programmes has a value of 0.53 and the value of the index for students from vocational-educational programmes is −0.61. Furthermore, PISA 2015 results indicate that students from the bottom quarter of index of socio-economic status on average scored 471 points on PISA science scale, and students from the top quarter on average scored 560 points. The gap between the two quarters corresponds to 88 points, which is equal to the gap identified in the OECD average. Such results could therefore indicate that the differences in science scores between the educational programmes in Slovenia could at least partially result from the differences in students’ socio-economic background, with the difference between the achievements of students with different socio-economic background not significantly different from the average difference in other OECD countries.

According to PISA 2015 data, there are also significant differences in science achievement between students with a migrant background (7.8% of participating students) and non-migrant students (92.2%). On average, students with a migrant background achieved 449 points on science PISA 2015 test, which is 71 points lower than their peers with non-migrant background (520 points). The difference in the proportion of low-performing science students with a migrant background and non-migrant students is 7.8% in favour of non-migrant students, which is also relatively high according to the EU average (European Commission 2017).

Significant differences in science achievement were also found between students whose language at home is Slovenian and students for whom it is not. Students who speak Slovenian at home on average scored 88 score-points higher than students whose mother tongue is not Slovenian, which corresponds to almost 3 years of schooling (European Commission 2017).

Already from the PISA 2009 cycle on, the results demonstrate that, despite the fact that Slovenian 15-year-olds achieve high scores in all three measured domains, they report rather low learning motivation: for example, the value of the index of enjoyment in learning science for Slovenia was significantly below the average (−0.36); similar results were reported for the index of interest in broader science topics (−0.32); students in PISA 2015 also reported below-OECD-average instrumental motivation to learn (−0.45). The same was found for the motivation to read (In PISA 2009, students reported on average below-average enjoyment in reading (−0.20).) and motivation to learn mathematics in the PISA 2012 cycle (−0.03; e.g. Kozina and Štraus 2017; Šterman Ivančič 2017; Šterman Ivančič and Puklek Levpušček 2016). TIMSS 2015 also yielded similar results in 2015, especially regarding low motivation to learn science (Japelj Pavešić 2017). The contradiction between high achievement results, on the one hand, and low motivation to learn, on the other hand, has been widely discussed by national educational experts and the research community, and is still one of the important topics when addressing ILSA results in Slovenia.Footnote 1

We can conclude that, despite the above-average results of Slovenian 15-year-old students in science, reading, and mathematics, further analysis of PISA data is crucial to actually understand the nature of the results and, most importantly, that the above-average achievement on an international proficiency scale should not be self-evidently treated as evidence of the great efficacy of the national educational system. In the next chapter, we discuss further the role of ILSA results in the improvement in students’ academic achievement and the challenges of using ILSA results to develop Slovenian education policies and practices.

How to Go Further? A Critical Discussion of Assessment Policies, Practices and Results

ILSA results reveal that disparities in student achievement according to gender, educational programme, socio-economic background, immigrant status, language spoken at home, and learning motivation present an important challenge to the Slovenian educational system and respecting equity as one of the main principles of the White Paper (2011).

Based on the case studies (e.g. Japelj Pavešić 2013; Klemenčič 2010; Štremfel 2013; Šimenc 2012), these and other ILSA results play an important role in the assessment and improvement of academic achievement in Slovenia. The results of case studies have, from different research perspectives and taking the international framework of particular ILSA into account, identified the following influences of ILSAs on the process and content of education policy in Slovenia.Footnote 2 The findings (Japelj Pavešić 2013) suggest that TIMSS results represented an argumentation for some directly and indirectly curricula and syllabus changes over the years.Footnote 3 These were accompanied by intensified teacher trainings and changes in teaching and learning process (e.g. introduction of experimental learning and use of TIMSS type of exercises in the teaching process) (ibid.). Šimenc (2012) also found a relatively high content match among civic and citizenship curricula (1999, 2011) and ICCS 2009 framework. However, according to the authors, we cannot claim that CIVED 1999 and ICCS 2009 directly affected these curricula.

Klemenčič (2010) reported that PIRLS 2001 results presented one of the foundations for forming the National Literacy Strategy. Štremfel (2013) identified the influences, which the below-average results of Slovenian students in PISA 2009 caused in the education policy process. These involve the organisation of national conferences, regional discussions and workshops, targeted projects for improving the level of reading literacy among Slovenian students, and intensive media attention to below-average results. Adam (2014), Štefanc (2008) and Vezjak (2014) exposed that involvement of Slovenia in international integrations in the field of education (including, but not solely, in ILSAs), resulted in some unintended changes in the national education policy (e.g. orientation to the economic dimension of education).

We can assume that although Slovenia as a new (post-socialist) EU member state has been very receptive to comparisons with the developed West (ideational pressure), it has also been confronted with institutional difficulties in translating these ideas into a national context, because of institutional and organisational constraints (e.g. lack of sources and researchers dealing with in-depth secondary analysis of the ILSA results) (Štremfel 2013). The main challenges to using ILSA results to develop a Slovenian education policy and practices in the future are:

  • Systematic planning and involvement in the forthcoming ILSA, which does not depend solely on the (non)availability of public funding for involvement, but on expert and political consensus (e.g. which competences of Slovenian youth are strategically important for Slovenia to compare internationally, which trends in knowledge of Slovenian students is it important to follow, etc.). The lack of public funding has already initiated the debates of (non)involvement of Slovenia in different ILSAs (e.g. ICILS in 2018).

  • Responding to the ILSA results not only when faced with below-average results. The important power the ILSAs have over participating states lies in the fear of being below average, which is becoming more and more common in these times of increasingly fierce competitiveness in the globalised world (e.g. Ozga 2003). Silova (2012) explains this fear is even more evident in post-socialist states, where any deviation from the Western “norm” is immediately reflected in the emerging narratives of “crisis”, “danger”, and “decline”. The research evidence (e.g. Štremfel 2013) confirms the increasing (political) attention to ILSA results for below-average results in Slovenia. This, among others things, resulted in paying more attention to and investing in additional secondary analyses of PISA results to better understand the broader social, psychological, and economic contexts of the results. Kodelja (2005) points out that excessive emphasis on poor results as a form of political response to unsatisfactory results is not necessarily a bad thing, providing they are justified. He believes that they may even significantly contribute to improving the situation, as they shape a social climate that favours change. However, when they are based on simplifications, sweeping generalisations, and hasty conclusions, and there has been no lack of these in Slovenia in the past, they certainly do not work in favour of either identifying the real causes of the current situation or the search for solutions to improve it.

  • Not using the ILSA results for politically motivated changes. The existence of ILSA (neutral) expert data does not ensure that these data are not used for politically motivated changes. Empirical evidence (Klemenčič 2010; Kodelja 2005) actually reports on the use of ILSA data for politically motivated changes. In addition, the empirical study (Štremfel 2013) reveals that 63% of policy-makers, 81% of experts, and 84% of principals participating in the study agreed with the statement “International comparative assessment studies in Slovenia are often used as an argument for politically motivated changes in the field of education”.

  • Developing advanced research infrastructure for in-depth secondary analysis of the ILSA results. Slovenian representatives emphasise that the relevant skills are required not only for the purposes of conducting ILSA, but for the interpretation and contextualisation of student results. For an increasingly large amount of data, made possible by modern technology in conjunction with a not fully developed culture of this type of educational research and scarcity of human resources, the in-depth interpretation of these data remains a challenge for Slovenia and several other participating states.

  • Developing evidence-based education. In accordance with the theoretical assumption of the governance of problems, increasing participation in ILSAs allows a wider identification of the weaknesses and shortcomings of a national educational system. The case study (Štremfel 2013) confirmed that Slovenian actors (policy-makers and experts) believe that the results of ILSAs allow identification of national policy problems when it comes to Slovenia’s below-average results. However, inadequately perfected institutional structures for processing and interpreting data are insufficient pathways for developing country-specific solutions to perceived policy problems.

In any case, an evidence-based policy approach that also acknowledges international assessment results when tackling different issues in the educational arena evolved over time in Slovenia and is still evolving in an encouraging way. Still, much effort will be required to achieve a status where ILSA results will be acknowledged and incorporated in system-level decision-making as a whole, not only about the final achievement in mathematics, reading or science scale. Areas of improvement in this manner are being further discussed.

It seems that an important step towards addressing the above-identified challenges presents the new framework for identifying and ensuring quality in the field of education. The new framework aims to unite different existing (internal and external) approaches of monitoring and evaluating educational institutions and system in single comprehensive model. Among other things, the new framework established the so-called Coordination and Analytical Centre of quality assurance at the Ministry of Education, Science and Sport (the Office for the Development of Education), which is responsible for preparing a joint evaluation of the educational system (partly at the annual level and summary quality report, presumably for 3 years). It is foreseen that it will significantly contribute to a more systematic, higher quality, and more comprehensive planning of measures and development policies in the field of education on the basis of expert data and evidence (including from ILSAs; MIZS 2017).