Since 1968, when it joined the International Association for the Evaluation of Educational Achievement (IEA), Hungary has participated in approximately 25 international large-scale student assessments. Participation in these assessments and the development of a national assessment system are intended to inform educational policy makers, professionals and the public. This chapter presents the history and the current state of the Hungarian assessment system with special focus on international studies and the National Assessment of Basic Competencies as its main pillars. The chapter’s main focus is on equity, a key issue in Hungarian public education, which our assessment data sheds light on. Results from PISA 2015, TIMSS 2015, PIRLS 2016 and NABC 2017 are used to explore differences between schools in terms of socio-economic status and academic achievement, as well as the strength of relationship between the former and the latter. Policy recommendations regarding assessment and the public education system as a whole are made.

Introduction to the International Assessment Context and Its History in Hungary

In Hungary, compulsory schooling (including 3 years in kindergarten) lasts from age 3 to age 16 in accordance with the Act on National Public Education (2011), and the public education system is based on 8 years in primary school and 4–5-years in secondary school. Students, however, may change to grammar schools as early as after 4th grade – the system offers grammar schools of 8, 6 and 4 grades, of which the last-named is the most popular. All academic-track secondary schools end with the Matura exit exam at the end of 12th grade that works as a university admissions exam as well. Vocational training can be started after 8th grade either in vocational secondary schools that offer also Matura or in vocational schools that focus on occupation-related qualifications.

According to the Hungarian Central Statistical Office in 2011, at the time of the last population census in Hungary, approximately 68% of the young adults aged 20–24 finished their secondary education with a Matura examination, 16% had vocational qualification without a Matura and 16% had not finished upper secondary education (Hungarian Central Statistical Office n.d., Table 2.1.1). The proportion of early school leavers, the share of the population aged 18–24 with at most lower secondary education who were not involved in any education or training during the 4 weeks preceding the survey, was 12.5% in 2018 according to Eurostat (Eurostat n.d., Table SDG_04_10).

International and National Studies Before the Millennium

Hungary joined the International Association for the Evaluation of Educational Achievement (IEA) in 1968, shortly after IEA had successfully conducted its first international large-scale student assessments, the Pilot Study and the First International Mathematics Survey. Hungary was the first Eastern European communist country to join the IEA and in fact the only one from the Soviet bloc up to its collapse in 1990 (Brassói and Kádár-Fülöp 2011); although Poland and Yugoslavia did participate in the first IEA Pilot Study in 1960, and Poland and Romania sought to join some parts of the Six Subject Survey in 1970–1971. Unfortunately, Poland did not manage to finish the survey with a published dataset and Romania only participated in the French as a foreign language part of the study. Only Hungary had permanent and almost full participation in the tests.

Hungary took part in a couple of IEA studies before the millennium, including the Second International Science Study, the Second IEA Study on Reading Literacy and TIMSS. Hungarian researchers and policy makers considered mathematics, science and reading comprehension as essential domains of interest from early on (Kádár-Fülöp 2015). Additionally, Hungary joined many other innovative areas of research conducted by IEA, for example, studies about civic education, ICT skills and composition skills (for a detailed list of IEA studies in which Hungary participated, see Brassói and Kádár-Fülöp 2011, Table 1, p. 436).

The membership in IEA had a stimulating effect on the Hungarian educational research community (Brassói and Kádár-Fülöp 2011). Báthory (1992) identified three areas of benefits for the educational system: (1) methodological advances, (2) the ‘window effect’ and (3) introduction to system-level analysis.

Peer-learning on fields such as conceptualization of educational assessments, framework development, field operations, data processing and analysis resulted in methodological advances in developing a national assessment system (Halász and Lukács 1987; Kádár-Fülöp 2015). Besides the expertise coming from working day-to-day on the national implementation of the studies, IEA organized workshops and trainings from its early days, and Hungarian educational researchers benefited a lot professionally from these occasions too. As an example, articles about the subsequent Hungarian reforms on the national curriculum usually mention the International Curriculum Seminar held in Gränna in 1971 as having a long-lasting effect on researchers involved in those reforms (Ballér 2001; Brassói and Kádár-Fülöp 2011). Hungarian experts were also able to join in the work of IEA and contribute to the development of the international assessments. Tamás Varga, the leading figure behind the renewal of the Hungarian mathematics curriculum and the ‘new math’ movement was a member of the International Mathematics Committee of the Second International Mathematics Study (Travers 2011). Zoltán Báthory, the representative of Hungary in IEA’s Standing Committee until 1994 and an honorary member of IEA, contributed to the mathematics and science assessments of IEA in various ways.

The ‘window effect’, meaning the possibility of international research cooperation in an era of strict ideological isolation, had its merits alongside the direct professional development of researchers. Through IEA studies and regular meetings with researchers all over the world, Hungary had the possibility of viewing its own educational system in a wider, global context (Báthory 1992; Kádár-Fülöp 2015).

The third and the most evident effect of IEA studies was the introduction of system-level analyses in Hungary. Although the IEA studies were unprecedented sources of comparable data on many aspects of educational systems worldwide and IEA always emphasized that the studies were meant to research the processes and methodological differences leading to different student outcomes, the main interest of the Hungarian politicians and educational professionals alike was the overall achievement of the system (Kádár-Fülöp 2015). The international and national reports of IEA studies up to 1990 showed good results in mathematics and science, and poor results in reading comprehension. The latter shocked the Ministry of Education and experts as well. Methodological changes and the liberalization of methods of teaching reading from the late 1970s were linked directly to these IEA findings (Báthory 1992). The liberalization processes operated in parallel with other endeavours towards freedom of education (e.g. free school choice for parents and students, more autonomy for schools in relation to the selection of methods and materials, the right for non-governmental organizations to maintain primary and secondary schools) and resulted in new regulations such as the 1985 Public Education Act (“Az oktatásról. 1985. évi I. törvény” 1985).

Researchers and institutes working on IEA studies were also involved in developing a national assessment system (Halász and Lukács 1987; Kádár-Fülöp 2015). The first nationally representative sample-based study, TOF-80 assessed 4th and 8th grade students’ abilities in various subject areas (Báthory 1983). Though it was a stand-alone study with no follow-up, the experiences benefited the planning and implementation of the Monitor Studies. The Monitor Studies, first implemented in 1986, were created with the goal to regularly monitor student achievement in reading, mathematics, science, information technology and cognitive skills. The study was first repeated in 1991 and from that time it became a bi-annual study until its termination in 2005. The Monitor Studies complemented IEA studies with trend data on student achievement. Whenever it was possible, the Monitor Studies used the same samples as IEA studies (Vári 1997; Vári et al. 2000).

International and National Assessments in the New Millennium

With the long history of international student assessments, it was no surprise when Hungary joined the OECD PISA project when it was launched in 1997. PISA 2000 results were received with a sense of disappointment in Hungary, because not only reading literacy, but also mathematics and science literacy performance were mediocre compared to previous good results in IEA studies (Kádár-Fülöp 2015). The seemingly contradictory results of IEA’s PIRLS and TIMSS and OECD’s PISA were and are in the focus of much attention in Hungary since then. The first article about Hungarian PISA 2000 results gave various possible explanations for the differences, for example, the differences in standardization and populations (the computation of average scores in IEA studies vs. the OECD average in PISA, grade-based vs. age-based sample, 4th and 8th graders vs. 15-year-olds); different emphases of the studies (research on educational processes vs. indicators of the quality of education); differences in the frameworks, especially the contexts in which the scientific or mathematical problems are embedded (textbook-like stems in TIMSS items, based on curricula vs. situations from everyday life in which students have to use their mathematical and scientific abilities to solve the problems in PISA) (Vári et al. 2002). The OECD’s more intense communication and ‘advertising’ of PISA and the poorer results of Hungary in PISA both contributed to the greater attention among the media and the general public following PISA and other international student assessments in the new millennium (Kádár-Fülöp 2015).

Based on experiences from the international assessments and the Monitor Studies, a new, annual student assessment system was initiated in Hungary in 2001 (Berényi et al. 2013). The National Assessment of Basic Competencies (NABC) is a constantly developing system, and during the more than 15 years of its existence the characteristics of the study have changed considerably. However, the basic aims of the study did not change a lot from the beginning; the main aim was and is to give schools objective, nationally comparable data on their students’ abilities in important literacy domains (Ostorics 2015). Based on that, every year all students in grades 6, 8 and 10 participate in these assessments of reading and mathematics competencies. School Reports have been available to schools – and from 2007 on to the general public as well – in order to help them evaluate their results. Results are reported not only in absolute terms, but also in comparison to students’ socio-economic status (from 2003) and to earlier results (from 2010) as well (Balázsi 2016).

Despite the evident effects of IEA studies mentioned before, researchers felt that the full potential of the studies has never been reached in Hungary. For example, Brassói and Kádár-Fülöp (2011) wrote that “In spite of our zealous participation in several IEA surveys, their results had little direct impact on education policy or instruction in Hungary” (pp. 435–436). The several merits of participation in international assessments have been overshadowed by the fact that during our long history, policy changes and system development initiatives were rarely able to improve the results or equity of the system.

International Assessments Today

Today, international assessments are a constant element of the Hungarian assessment system and provide a wider perspective alongside periodic national assessments and exams of various goals, stakes and scope that cover the span of primary and secondary schooling (Fig. 13.1.).

Fig. 13.1
A screenshot of international large-scale student assessments, national student assessments with low stakes for students, and national high-stakes entrance and leaving exams for grades first to thirteenth.

The national assessment system in Hungary

Full size image

National assessments and exams range from no-stake university-developed diagnostic programmes to state-run centrally developed high-stakes examinations. Diagnostic assessments are available to schools from grade 1 to grade 6. These are developed by Szeged University’s Centre for Research on Learning and Instruction and cover domains such as prerequisite knowledge fields of school readiness for first graders delivered via paper-based tests (writing-movement coordination, relational vocabulary, basic calculations, experimental deduction, basic social skills) and digital reading, mathematics and science test tools for higher grades. Schools are legally required to utilize the former in cases when kindergarten reports or the experiences of the first month of schooling compel them to do so (Centre for Research on Learning and Instruction n.d.).

Centrally developed and legally full-cohort albeit low-stake assessments are the National Assessment of Basic Competences (NABC) and the Assessment of First Foreign Language (AFFL), both the responsibility of the Department of Assessment and Evaluation in the Educational Authority (Szabó et al. 2018). These are administered as pen-and-paper tests. The NABC assesses non-curriculum-based domains including reading and mathematics literacy and has a well-developed and extensive on-line administration and reporting system. Reports are available from student to national level and provide mean scores and distributions of students across proficiency levels per grade, school type and cognitive domain compared to the national average and to the results of different subpopulations (Ostorics 2015). Besides, the reports communicate value-added analyses outcomes as students’ results are compared to their expected results based on their socio-economic status and on their earlier results where available. Software is also provided to schools to allow for further analysis of their own results. Compared to the NABC, the AFFL is a limited instrument. Students in 6th and 8th grades are legally required to take the curriculum-based test; however, central administration and reports are still lacking.

Unlike national tests, participation in international assessments is not obligatory for schools, students or teachers. Still, participation rates are constantly high in TIMSS, PISA and PIRLS, the three major international assessments that Hungary participates in since their first administration. Although not an achievement study, Hungary took part in OECD TALIS 2018 involving teachers in ISCED 2 (lower secondary) schools as well.

As seen, assessments examine the educational system at various points. One way of looking at these points is taking note of the grades and school types involved. At the stage when PIRLS is administered, at grade 4, all students attend primary schools. In fact, this is the last point in Hungarian schooling when all students go to a single type of school. The 6th grade NABC assesses students in lower secondary and in 8 grade grammar schools. TIMSS and NABC in the 8th grade look at students in lower secondary and 6 and 8 grade grammar schools, while PISA assesses 15-year-old students from 7th to 10th grade in lower secondary schools (grade 7 and 8), all three tracks of academic grammar schools, and the two tracks of vocational training. The 10th grade NABC assesses students is all secondary tracks.

A set of international, state and local assessments administered in the same national educational context does not necessarily constitute a system. In the case of Hungary, however, the implementation of the student measurement identifier (SMID) provides a possibility to link the datasets yielded by such diverse sources. The SMID was implemented in the 2007/2008 academic year and has the dual goal of protecting student privacy and allowing for the examination and analysis of relations between results of national tests (such as the NABC) and international tests (such as TIMSS, PISA and PIRLS) (Ostorics 2015).

An Example: The Issue of Equity in the Hungarian School System

International assessment results are usually the focus of attention because of the rankings and national mean scores. As noted above, IEA and OECD programmes provide seemingly contradictory results in Hungary: PIRLS and TIMSS outcomes are always above international average and are sometimes excellent, while the results of Hungarian 15-year-olds rarely attain the OECD average in PISA. The latter showed further decline with the shift from printed to digital assessment mode in 2015. While rankings are attracting more attention, the main lessons learnt from international assessments are related to equity.

Issues of equity were in the focus from the first IEA assessments in Hungary. Báthory (1992) cites a table presented at the 1986 IEA General Assembly meeting by Sixten Marklund showing how the variation in the science performances of students is distributed between school and student levels in different countries in 1970. Among the nine countries presented there, Hungary had the highest between-school variance portion (40%, the country average was 26%) in grade 4 and above-average between-school variance (34%, the country average was 29%) in grade 8. Also, the effect of the socio-economic status on student performance was around the IEA average in reading and somewhat below the IEA average in science, with several countries having a lower impact of SES on performance (Báthory 1992). These findings were particularly disturbing for Hungary, where the declared socialist state policy was egalitarian, and equality of opportunity – the educational and social mobility of working class children – was one of the most important aims of the educational system.

Almost 50 years later, after fundamental changes in the political and educational systems, equity is still a serious problem in Hungary. PISA and NABC have shown that differences between schools in terms of the socio-economic status and the performance of students are still considerable. About 25–36% of the differences in student performance in primary and lower secondary level come from differences between schools (Table 13.1). For the upper secondary level, the differences between schools are even more pronounced; above 50% of the variance originates from school-level differences in all domains of PISA and NABC. These values are high compared to other countries, with the PISA estimates for Hungary well above the corresponding OECD averages of between-school variance proportions (Organisation for Economic Cooperation and Development 2016b, Table I.6.9, p. 409). The increase in between-school variances between grades 8 and 10 originates from the structure of the educational system as described earlier, including the selection of students for academic secondary schools.

Table 13.1 The differences between schools and the effect of socio-economic status on the performance in various studies
Full size table

There are considerable differences in students’ socio-economic status between schools as well. PISA’s index of social inclusion (the proportion of variance coming from differences within schools) is one of the lowest for Hungary at 62.6% (OECD average is 76.5%), meaning that the intra-class correlation (rho), a measure of between-school variance in SES, is one of the highest (Organisation for Economic Cooperation and Development 2016b, Table I.6.10, p. 410). Hence, social segregation is high in the Hungarian education system with schools differing considerably on their student intake and having more homogenous student populations within schools. Moreover, the index of social inclusion is already low on the primary level. Based on the Family Background index of NABC 2017, between 45% and 50% of the variance of students’ socio-economic status comes from differences between schools in all grades. According to this, while tracking in Hungary on the secondary level increases the academic segregation, social segregation is already large in lower grades and does not increase much more when further tracking takes place.

It should be noted that in Hungary parents are free to choose the primary school for their children since the 1985 Act I on Public Education. While this may further school competition, which helps with matching school offers to student demands, it might be equally detrimental for equity (Musset 2012). Free school choice favours families who have sufficient resources to take into account criteria other than low expenses and short distance from home when choosing a suitable school for their children (Organisation for Economic Cooperation and Development 2014).

The relationship between students’ socioeconomic status (SES) and performance is also high in Hungary and does not change considerably between grades (Table 13.1). In PISA, the effect of the ESCS (index of the economic, social and cultural status) is above the OECD average, and the strength of the relationship between student ESCS and performance is one of the strongest among PISA 2015 countries (Organisation for Economic Cooperation and Development 2016b, Table I.6.12a, p. 412). Given the high levels of segregation in the Hungarian educational system, it is not surprising that the relationship between the economic, social and cultural status and performance of students is most evident at the school level. While the effect of students’ ESCS within schools is small, 80.1% of variation between schools in science performance is explained by students’ and schools’ ESCS in Hungary. Overall, taken together, student and school ESCS explain 43.4% of the variation in student performance, which is the highest value in PISA 2015 and almost twice the OECD average 22.4%.

Because of the high social disparities in the performance of students and schools in Hungary and the high proportions of low achievers, research on resiliency (the ability of students to succeed in school against the adversities that may arise from coming from a disadvantaged family background) is very important for us. International and national student assessments are excellent sources of data for that purpose as well. PISA has an interest in student resiliency since the 2006 cycle. In research on factors associated with resilience, or the ability of disadvantaged students to perform well, the OECD found that in Hungary and in many other countries, students’ self-efficacy and the number of hours students reported spending on regular lessons at school learning science were related to student resilience in science (Organisation for Economic Cooperation and Development 2011). Other factors, such as general interest in science, participation in science-related activities, competitiveness and selectivity of the schools based on academic record or the activities of schools to promote the learning of science and quality of school educational resources were not significantly related to resiliency.

Agasisti et al. (2018) reported that, after controlling for student gender, the ESCS of students and schools, and differences between the language spoken at home and the language of instruction, schools with a more positive school climate and lower absenteeism were significantly better in promoting resiliency in PISA 2015 in Hungary. In contrast, school resources indicators were not connected to resiliency. The effects of the ratio of computers to students and the average class size were not statistically significant, while the number of extracurricular activities at school had a negative relationship with resiliency. The last two had a positive effect in many other countries and on average among OECD countries too, which highlights the need to examine school-level factors in a national context.

The reports on the NABC strive to take differences in student intake between schools into account and to show how schools are able to deal with student groups with various social and cultural backgrounds. As a first step to examine school level factors fostering resiliency, the Educational Authority identifies schools that score significantly above what is estimated based on their students’ socio-economic status or previous achievement. A non-ranking list of these schools has been published annually since the 2014–2015 school year (Educational Authority 2018).

Policy Recommendations

The Hungarian national assessment system currently focuses on the domains of reading literacy, mathematics and science. Among international assessments, only PISA has an option to assess other domains. Participation in other international large-scale student assessments (e.g. in IEA studies of civic and citizenship education or computer and information literacy) might help to plan curriculum development and reforms of teaching and learning materials, practices, etc. in those specific fields.

However, current evidence based on ongoing international and national assessments is plentiful concerning the structural, quality and equity issues of the Hungarian educational system, so limited resources are probably better used in disseminating outcomes, engaging in secondary analysis of existing databases and making policy recommendations based on evidence coming from them. The student measurement identifier (SMID) provides the possibility of linking the data of international assessments to the NABC and other national data sources at the student level. This, in turn, expands the research and policy topics that could be addressed, for example, by following up students’ educational career from grade 4 to the end of their tertiary education.

PISA, TIMSS and PIRLS are pillars of the Hungarian assessment system and continuing participation in them provides invaluable trend data on the quality and characteristics of the educational system with the possibility of international comparison and opportunity to learn from good practices from all over the world. The long history of participating in international large-scale studies and the constant development of Hungary’s national monitoring and assessment system yield data that can inform policy makers on the structural changes that may be beneficial for Hungarian public education, allow for the identification of regional, local and school-level issues, and offer educational researchers the possibility to conduct in-depth analysis. The main lessons are summarized below.

Achievement indicators provided by international assessments show that Hungarian students at the beginning of lower secondary education are more proficient than their peers in many other education systems, while 15-year-olds’ results lag behind the OECD average in all assessment domains, even more so when the data are collected using digital tools. This raises the question of the quality of lower secondary schooling, how we can better support literacy development of children, and which policy initiatives can improve teaching and learning in lower secondary schools.

Free school choice seems to affect the student intake of the public education system in a way that results in schools which are homogenous in not just academic achievement but in socio-economic status as well. This hinders equity, favours well-off families and does not facilitate the prevention of early school leaving. Policies to mitigate the adverse effects of the current completely free school choice regulations should be explored.

The system of national assessments needs to adapt to serve demands that include supporting early drop-out prevention programmes. Currently, several projects financed by EU structural funds or the state of Hungary aim to reach the above-stated goals. These projects are overseen by the Educational Authority, as the state agency responsible for collecting and reporting assessment data.

As for preventing early dropout and fostering resilience, it should be noted again that schools that can constitute a basis for these processes can be identified by the means of the NABC reports and databases. A considerable number of schools can achieve significantly better results than that estimated based on their students’ ESCS-index or their students’ previous proficiency scores. The existence of such schools indicates that the effects of socio-cultural and socio-economic disadvantage can be mitigated, even in the current structure of the education system. A relevant further step is examining school-level factors and exploring the good practices that may result in student resiliency. On the other hand, schools that do not attain expected levels based on their students’ socio-cultural status or previous results also can be identified with the use of NABC data. One of the main goals of the projects mentioned above is to build tools to facilitate communication between schools to promote networked learning.

Other relevant goals include promoting the use of digital technologies and moving the national assessment system online, adding new domains to the existing ones, including science and digital literacy, and fostering the use of existing data by making it more accessible to both the general public and professionals.