1 Introduction

The consequences of accountability policies through standardized testing have been analyzed in diverse educational systems. The best-known examples of high stakes testing systems are England, Australia, and the USA (Collins et al., 2010; Hursh, 2013; Lingard et al., 2016), but there is also evidence from low-stake accountability contexts, such as Germany (Thiel et al., 2017). While there are variations among these countries, they all share the following common consequences as described by Stobart (2008): (i) to motivate teachers to work more and be more effective, although not necessarily more productive; (ii) to cause teachers to prioritize measurable content and test preparation, thus narrowing the curricula taught; and (iii) to maximize results by aligning content with the required standards, training for the test, and even by cheating. These effects have also been registered in the German low-stake accountability context (Thiel et al., 2017). Regarding students’ experiences, Howell (2017) found Australian children often experience the National Assessment Program: Literacy and Numeracy in negative and distressful ways. Regarding teachers’ reactions, in Sweden, Lunneblad and Asplund Carlsson (2012) found classroom practices to be framed by several national tests and that they were not necessarily related to what is intended by the test, but to what teachers perceived to be its intentions.

In Chile, accountability policies were introduced in a context of increasing privatization. The structure and model of Chilean education, established in the 1980s during the military dictatorship period (1973–1989), are described as a quasi-educational market operating under neoliberal principles (Verger et al., 2016). The role of the State is largely limited to funding public and privately managed schools as well as monitoring and quality control by means of tests embedded in what is known as the Measurement of Educational Quality System (Sistema de Medición de la Calidad de la Educación; SIMCE).

In line with neoliberal principles, the SIMCE tests were originally designed to stimulate quality improvement in competitive contexts, and they have continued to be key components of market and accountability educational policies in Chile despite a number of corrections over three decades of use (Gysling, 2016). However, in the last decade or so, the role of SIMCE has become the subject of strong criticism on the part of researchers, policy analysts (Falabella & De la Vega, 2016), and activist groups led by “Alto al SIMCE” (Stop SIMCE) (Montero et al., 2018). Also, high school students protested against SIMCE by boycotting the test in different regions since 2015.

Studying the way in which SIMCE is recontextualized (Ball, 1993) in schools becomes relevant to understand the effects of educational policy in specific contexts, especially when it is aimed towards school improvement (Thrupp et al., 2007). Moreover, the context of disadvantaged schools is key to understanding the effects of these policies where improvement is more urgent because they concentrate minority students (Hickok, 1998). Therefore, this article examines the way SIMCE, as an evaluation device within the Chilean educational system, is lived in the daily practices and interpreted by Chilean school leadership teams and teachers. The article presents results from an ethnographic study conducted in three Chilean public municipal schools located in disadvantaged neighborhoods and with different achievement levels as determined by the Quality Agency of Education. It is part of a more extensive study on the enactment of educational policies in the micro-political school space.

1.1 Educational policies in Chile and the role of the SIMCE

The Chilean school system consists of a large publicly funded set of schools, both fully public and private subsidized, comprising together over 90% of all schools. The rest are fully private. During the 1980s, a significant education reform took place in Chile that transferred public schools from national to local municipal administration. The system for disbursing public funds to both public and private subsidized schools was also transformed into a portable subsidy, where schools receive a variable amount of money based on students’ enrollment and attendance. It was assumed that as schools improved in quality, they would attract more students and thereby compete favorably in the school market (Bellei & Vanni, 2015; Donoso, 2013; Assaél et al., 2011). Municipal schools had to compete in unequal conditions, and their enrollment declined from 78% in 1981 to 37% in 2015 (Ministerio de Educación, 2015).

Starting in the 2000s, the national government began to use what have been described as “new public management” or at-a-distance government policies (Ball, 1997; Rizvi & Lingard, 2010) with the purpose of securing school quality improvement and better learning opportunities. Assuming school’s management is autonomous, these policies focus on checking results through vigilance and comparison devices, thus freeing the State from political and economic costs (Bellei, 2015; Olmedo, 2013). They are based on auditing practices, which rely on quantification and provide a sense of objectivity and authority (Hardy, 2015). Key among such policies have been those addressing teacher performance evaluation and an increase in standardized assessment (SIMCE tests), together with participation in international studies such as PISA, TIMSS, and LLECE (Cox, 2012).

SIMCE was first applied in Chile in 1988 with the purposes of (i) informing educational authorities about the schools’ curriculum coverage and learning results and (ii) stimulating school competition for students within the framework of the quasi-educational market referred to above (Himmel & Comisión para el desarrollo y uso del sistema de medición de la calidad de la educación, 2003). After the return to democratic governments in 1990, the original objectives of the SIMCE tests have persisted, and its mechanisms have been perfected. The results from each school were widely published and compared. The number of tests has increased, and SIMCE has been used to provide monetary incentives to teachers and leadership teams, as well as to evaluate schools’ improvement plans.

SIMCE tests have also been subjected to criticism, regarding their effects on narrowing the curriculum to what they measure, as well as on resulting public comparison among schools. This led to a revision of its mode of operation in the 2000s and more recently in 2015. However, while important shortcomings were noted, this did not lead to major changes in its application and uses.

Currently, SIMCE comprises tests taken by all students in 4th, 6th, 8th, and 10th grades covering reading and mathematics. Other tests in writing, natural sciences, and social sciences are administered annually or every other year. The SIMCE tests have an important impact on schools in that results are linked to the degree of autonomy they may have over management of their resources (Ministerio de Educación, 2008), salary supplements for teachers, and involve the possibility of school closure if results are not improved over 4 years (Ministerio de Educación, 2011).

Particularly, since the enactment of a law known as the “Preferential School Voucher” (Ministerio de Educación, 2008) similar to the U.S. policy known as “No Child Left Behind,” the Ministry of Education has been providing additional resources to schools with disadvantaged students requiring they improve in areas that are mainly measured by the SIMCE tests, according to an improvement plan they must design. Both “in-recovery” and “emerging” schools must submit their improvement plan for approval by the Quality of Education Agency, while “autonomous” ones do not have this obligation.

According to the law, in terms of their performance, all schools are classified as (a) autonomous with systematic high SIMCE results and not in need of support, (b) emerging or moving towards autonomy, and (c) in-recovery or schools which repeatedly underperform (Ministerio de Educación, 2008). “Emerging schools” have 4 years to improve, and they are re-classified as “in-recovery” if they do not. In turn, “in-recovery” schools that do not reach the standards of “autonomous” or “emerging” after support has been provided, risk being shut down by the Ministry of Education.

While in Chile there is little evidence regarding the effects of SIMCE on teachers’ daily work, some studies provide clues as to their use. Following Stobart’s (2008) classification of consequences, there is evidence that in terms of motivation, teachers and principals believe that SIMCE encourages improvement and stimulates greater professional responsibility for results (CIDE, 2012), but that in terms of teaching priority, they feel pressured to concentrate their work on the tasks, contents, and strategies that are measured by the tests (Carrasco & Fromm, 2016; CIDE, 2012; Falabella, 2016; Falabella & De la Vega, 2016; Flórez, 2015; Manzi et al., 2014; Ruminot, 2017; Falabella, 2020).

There is also evidence that teachers use different strategies to maximize results, such as focusing on short-term processes with impact such as hiring mock tests services (Asesorías para el Desarrollo, 2014) or increased coaching of students that perform poorly (Manzi et al., 2014). In the same way, the information provided by the SIMCE test is not considered by families in the process of choosing schools (Corvalán & Román, 2012). According to research by Falabella and Opazo (2014) and Falabella (2015), the harmful effects of the test tend to appear more in marginal urban schools with low performance than in up-market schools with high performance levels. Research has also shown that schools make their low-performing students miss the test to improve the school’s performance to the extent that it modifies the category they are placed in the SEP classification, especially schools serving disadvantaged students (Hofflinger & von Hippel, 2020). In addition, tensions among teachers between concentrating on the test or on content and skills considered to be important but go unmeasured have been described (Falabella, 2016; Flórez, 2015), as well as evidence of teacher stress and anxiety (Manzi et al., 2014; Rojas & Leyton, 2014). More recently, national authorities have recognized that excessive pressure on schools because of the evaluations does not improve learning (Mineduc, 2014; Ministerio de Educación, 2016). This pressure and the emotional effects it has on teachers in a competitive context entails a competition ethics as it positions teachers, students, and principals in a better or worse position relative to others (Falabella, 2020).

1.2 Conceptual basis: re-contextualization of policies at school level

Over time, educational policy research has been paying special attention to the space between policies as conceived by policymakers and actual school practices. In contrast to the rational project, critical policy analysis conceptualized this policy as a social practice of cultural production influenced by different actors in different locations (Stone, 2012). This has enabled a greater understanding of the policy cycle and recognition that the design, development, and application of policy are part of a dynamic process that is iterative and nonlinear (Ball et al., 2012). From this perspective, schools are contexts of practice, where policies are actively produced according to their specific characteristics (Ball, 1987, 1993).

The implication here is that educational policy analysis should encompass an understanding of how the various school actors experience, resist, or redefine policies. In this regard, the concept of re-contextualization (Ball et al., 2012; Bernstein, 1994) provides a useful tool for analyzing the productive nature of educational policy within school contexts. According to Bernstein (1994), the practice of recontextualization is carried out within specific contexts where various agencies, agents, and practices regulate and delimit the circulation of discourses and practices between the primary context and the secondary context, which, in this case, is the daily space of school institutions. This grants a degree of relative autonomy to schools, where the different contexts that frame it are influenced in a variable way. Ball et al. (2012) have delved into some of these contexts, particularly, how educational policies and external advisory and control systems contribute to understanding the heterogeneity, as well as the similarities, in the trajectories of schools.

To explore the process of re-contextualization, we focused on the concepts of interpretation and translation (Ball et al., 2012). Interpretation refers to the political process of explaining, clarifying, and setting forth an institutional agenda. Through this process, different agents establish priorities and value of different policies. Policy translation, on the other hand, refers to the re-coding of materials, practices, concepts, procedures, and policy directions within contexts, cultures, and specific practices (Spillane et al., 2002). This creative and productive process implies that there could be a significant difference between official policy texts and their enactments (Ball et al., 2012).

As far as policy translation is concerned, different actors take on distinct roles or positions, both as recipients and agents. These processes occur daily in the formal and informal institutional spaces of schools, in their work dynamics and social interactions, and in the framing of decisions about management and resource allocation, to name but a few (Braun et al., 2010). They underscore how schools translate and therefore produce their local education policy as a form of responding to what comes to them as official policy. Thus, as Rizvi and Lingard (2010) argue, public policy analysis requires an understanding of the different logics of practice and meaning-making processes that occur in the many locations where policies are continuously re-contextualized.

Considering the above concepts and based on our research, we aim to respond in this article to the following two questions:

  1. 1.

    How do key school actors (teachers and school authorities) translate the accountability policies that touch them in their schools’ daily life, particularly the effects of the SIMCE test?

  2. 2.

    How do teachers and school authorities interpret the impact of accountability policies on their everyday school life considering the broader issues derived from market policies and school competitiveness?

2 Methods

In this study, the concept of ethnography is understood as an approach where theoretical concern is combined with the in-depth description and interpretation of particular social and cultural realities, delimited in time and space (Rockwell, 2009). On our research, an ethnographic case study approach was used to reach a deep understanding of the day-to-day phenomena occurring in the selected schools’ social contexts. This approach enables the understanding of how narratives are constructed in those environments (Rockwell, 2009) along with a comprehensive description of each school, without losing sight of the broader context (LeCompte et al., 1993). Focusing on the discourse and meanings embedded in the day-to-day school’s experience allowed us to understand how educational policy is interpreted and translated in concrete school contexts (Anderson-Levitt, 2011; Gerrard & Farrel, 2013; Nichols & Griffith, 2009). This shed light on the way policies associated with SIMCE are re-contextualized in these schools.

2.1 The schools studied

Three municipal schools located in disadvantaged neighborhoods in Santiago, Chile, were studied. Two schools offered elementary education (grades 1–8), and one offered secondary education (grades 7–12). Schools were selected according to three criteria: school categorization, geographic location, and willingness to participate in the study. (1) School categorization: Each school was categorized according to its performance (mainly SIMCE scores): “autonomous” (high performance), “emerging” (medium performance), and “emerging” with a recent previous experience of being “in-recovery” (low performance). We assumed each school would experience accountability in diverse ways, hypothesizing that both “emerging” and recently “in-recovery” schools would be under greater pressure to improve learning results along with the established indicators than would “autonomous” schools. (2) Geographic location: The three schools were in a metropolitan area (Santiago), which exhibited high poverty rates, deficits in access to public services, as well as micro trafficking of drugs. Table 1 summarizes the schools’ intakes.

Table 1 Schools’ intakes, classification, and socioeconomic group
Full size table

Escuela Sur and Rosa were part of a previous study about policy enactment, which allowed us to analyze interviews and observations conducted over a period of 4 years, enabling us to enrich understanding of the schools’ cultures and its relationship with how educational policy is interpreted and translated these schools. We were also able to gauge the institutional and material fragility of the schools (Acuña et al., 2014). To address the focus on SIMCE, we included a third school, aiming to provide insight into a high-performing case. The complete set of analyzed materials is summarized in Table 2.

Table 2 Interviews and observations per school
Full size table

Our analysis, however, focuses on one of the 4 years, when we observed classes on the day students took the tests: four in the two elementary schools and two in the high school. Also, each teacher of the grade evaluated by SIMCE was interviewed in each school. This allowed us to observe and analyze the reactions of the participants to a concrete event where the meanings of the accountability polices were mobilized. The interviews were conducted with headteachers and the school leadership team (principal, General Inspector and Technical-Pedagogical Unit Chief). Furthermore, observations from teachers and head team meetings were gathered to understand the way daily life at school was managed. We observed the application of four tests in Escuelas Nova and Sur and two in Liceo Rosa. Common school situations such as recess and official acts were observed for context. Each observation and interview were informed by the previous; formulating questions based on preliminary analysis of earlier observations and interviews allowed us a deeper final analysis.

During 2015, we visited the schools 1 or 2 days each week for approximately 4 h each visit. During this time, we did contextual observations of the schools’ activities and teacher meetings with the purpose of getting acquainted with their practices. We documented our observations in field notes at the end of each visit following Hammersley and Atkinson’s (1995) recommendations. In addition to conducting semi-structured interviews with school authorities and teachers from the evaluated grades, we held spontaneous conversations before and after the test application with different school actors (Guber, 2011). Audio interviews were recorded and transcribed verbatim.

Below, we describe each school providing information regarding Ball et al. (2012) contextual dimensions: situated contexts (localization and school intakes), material contexts (human and financial resources), professional culture, and external contexts (external support and pressures). Table 1 summarizes the main schools’ characteristics.

Escuela Sur routinely performs below the mean of its district on the SIMCE tests and is under constant pressure to improve. At the time of the study, it had managed to move from “in-recovery” to “emerging” status. The school was committed to conducting its “educational improvement plan,” using some of the funds provided by the government to finance it, to engage external support that pushed teaching staff to work on strategies considered to improve results. Additionally, during 2015, it was undergoing progressive staff changes. The period of the previous principal had ended. The General Inspector, who had been working at the school for two decades, became acting principal. The other members of the leadership team had also worked in the school for decades. Almost half of the teachers had retired or were going to retire soon, and the school administrator (municipality) had not filled many of the vacancies due to the lack of funding from low enrollment. However, a sense of camaraderie and community, related to the many years they had worked at the school, persisted among the teachers. According to teachers, students came mainly from poor backgrounds and had social, cultural, and cognitive challenges. They reported that their school received the most troubled students and that those students would leave for another school when they “got better.”

Liceo Rosa’s SIMCE results had oscillated from being the worst to being the best in its district. When the first data were collected, during the previous ethnography study, the school was categorized as in-recovery. At the time of this study, it had been upgraded from “in-recovery” to “emerging,” but it still was performing below the mean in SIMCE. The school was also working along with the required “educational improvement plan.” However, municipal authorities had been talking for years about closing the school as it had lost many students and was judged to be financially “unviable.” They progressively closed most of the primary levels. At the time of the study, the school had six classes, one per level from grades 7th to 12th, and very few students. It closed completely at the end of 2015.

The school leadership team members were the principal, who had held the position for 10 years, the Technical-Pedagogical Unit Chief, and the General Inspector. The principal was in constant conflict with the Technical-Pedagogical Unit Chief and the General Inspector regarding the day-to-day management of the school. However, they agreed that poor teacher quality and lack of support from local authorities were challenges for their school. Teachers were from two generations: an older one that had been working in the school system for many years and a younger one that came to fill in for retired teachers and did not stay long. Up to 90% of students received extra funding from the government due to their vulnerability. Half of the students’ family incomes were below the minimum wage.

Escuela Nova had 248 students from the 1st to 8th grades of primary school in 2015. It had been performing around the mean or slightly below in SIMCE for years. It was classified as autonomous in 2014, which gave the school greater freedom of management being exempted from the need of approval of its improvement plan by the Quality Agency of Education.

The head team was the acting principal (who had formerly been the General Inspector), the General Inspector, and the Technical-Pedagogical Unit Chief (who was indefinitely absent from the school). Teachers were, as in the other schools, from two generations: an older one that was about to retire and a newer one that was replacing the former. Sixty-four to 80% of students were considered vulnerable. The principal described the school’s students as unsupported by their families and living in environments that included drug abuse and trafficking.

2.2 Data analysis procedures

The data analysis process was driven by inductive logic embedded in a semantic analysis of the data. This involved inductive analysis, constant comparison, and the building of categories emerging from different discourses, with the purpose of examining how accountability policies were being re-contextualized and expressed in these discourses (Goetz & LeCompte, 1988; Hammersley & Atkinson, 1995; LeCompte et al., 1993). Transcribed interviews, observation records, and field notes were analyzed together, with careful consideration to not lose sight of the context of each case. The analyst team searched for connections between these texts that would enable us to achieve the objectives, thus using inductive logic. We interpreted the texts according to their context of production, this is, the particular position of the participants and their schools. The contrast between categories generated new categories, reducing their quantity, and increasing the level of abstraction, thereby enhancing the explanatory power of similar phenomena. To increase the inter-rater reliability, cross-reviews were conducted among researchers for the codes and categories of the analysis, and different analysts shared the rationale of the categories and codes they used. In case of discrepancies, new categories were recoded and created to reduce the differences between coders (Cheung & Tai, 2023). We focused on those emerging categories that illustrated the complexity of the practices and discourses that each school builds around the SIMCE test. Based on this integrated analysis of interviews and observations, we produced an ethnographic text that delves into the daily work and the specific forms of translation and interpretation of SIMCE-related policies found in these schools.

3 Results

At the beginning, we offer a description of events on the day of application of the SIMCE tests in the schools and examine teachers’ and principals’ discourse related to the institution’s strategies in preparing for it (policy translation). Then, we analyze diverse actors’ interpretation of responsibility regarding possible results, as well as teacher and principals’ interpretation of the SIMCE system and its purposes given their school contexts.

3.1 The SIMCE event

Every year in October, all Chilean schools host the SIMCE examiners charged with the application process in each one of the relevant classes. The scheduling of the test is planned by the Quality Agency of Education and made public throughout the various levels of the school system. In our research, we observed that each school provided relevant information about SIMCE to their teachers’ council, management team, families, and students in the days before the application.

In September there is expectation. They, [the SIMCE examiners] are due to come, the children are nervous. You will find, in the files, our communications to families to make sure they are included in the process. They are notified one month before (Interview, Luis, General Inspector, Escuela Nova, 2015).

The day the three schools took the SIMCE examination was no ordinary day. New actors appeared at the schools, lesson times were altered, and some teachers’ tasks were rearranged. This moment was important in each school. These days were lived with anxiety and expectation and with the school itself becoming something of a stage. The schools prepared their students to ensure they take the test in the best way possible, using a variety of strategies to explain to the students why this test is important for them and the school.

At Escuela Sur, the SIMCE day began with a special breakfast for students where teachers and the principal hoped to transmit a sense of tranquility and reinforce the importance of the test for the school.

You must do very well in this test, children. You have taken many tests, and this is one more of them. You should write what you know, what you have learned and not be afraid, be brave, let the ‘little southern’ [reference to the name of the school] emerge from inside of you [applause] (Field notes, José, principal, Escuela Sur, 2015).

We also witnessed a collective breakfast during which teachers and principal harangued the students at Escuela Nova. They also offered practical suggestions and recommendations: “Read the test content carefully, be confident, and take the assessment seriously.”

Unlike the other schools, Liceo Rosa did not prepare a special breakfast, although they did communicate to students the importance of the assessment. As in the other schools, students were asked to go for “better scores,” “to work conscientiously,” and were reassured that “everything will be fine” (Field notes, Liceo Rosa, 2015). They were also told that the test was important for showing what the teachers had been teaching them throughout the year:

Outside the library there was a group of four girls who were being ordered to go in to take the SIMCE (…) one of them intervened and said: ‘it is terribly boring, so,’ while the other agreed. The teacher replied that they must do it ‘conscientiously.’ That ultimately, they should do it for their teacher. The girls showed discontent on their faces (Field notes, Liceo Rosa, 2015).

What was striking in all schools was the role given to students and how this was communicated before the test. They were deemed responsible for what the school does, were made to value the preparation provided by the school before entering the “scene,” and were praised for exhibiting courage in facing the enemy. In all three schools, the examiner considered a sort of “enemy” who embodied the school staff’s discomfort with the test:

We open the classrooms for him; we check what is or should not be on the walls. The atmosphere [produced by the examiner] is a very punitive one. They hand out the questions; they invade our space; no one can come near it. It is as if everything was somewhat sacred (Interview, Luis, General Inspector, Escuela Nova, 2015)

The constant worry about the SIMCE test began to fade towards the end of the year when the last examination was taken, marking a new stage in the school’s annual cycle. The teachers were relieved. Some activities, such as test rehearsals, disappeared. But anxiety about SIMCE reappears in the following school year. First, when the school received its results (around the second month of the academic year), then when they were told of the school’s new classification based on test results and whether teachers in schools with high results would get a salary bonus, and finally, when the new test application approached.

3.2 Preparing for SIMCE

The SIMCE tests, its results, and consequences were present in the three schools long before it was due to be taken. At the time of learning about the school’s results and classification, and whether standards were satisfied or not, management and teachers look apprehensively to the next SIMCE and the possibility that their school may lower its results. Learning about their schools’ classification was also a source of anxiety as they were aware it meant that in the future, they would have to either match or improve their current results. So, even though the test may be over, uncertainty looms about what the next year’s results would be.

Regarding the next SIMCE application, teachers and principals reacted with expectant concern. One of the teachers pointed out: The pressure [of the test] is terrible. The whole school talks about SIMCE … “When will we go on a trip?” “After SIMCE.” Everything is based on SIMCE (Interview, Juan, Teacher, Escuela Sur, 2015).

One of the school principals refers to SIMCE as “a continuous torment” (Field notes, Escuela Sur, 2015). The teachers’ feelings of pressure did not only derive from the external judgment constructed around the school and its results. In fact, they internalized these judgments, feeling somewhat culprit even in absence of external monitoring of their work: “I would say that this is chaotic at times. We feel a lot of stress, even when we are not supervised. It is a personal matter” (Interview, Juan, Teacher, Liceo Sur, 2013).

Reacting to the external pressure, principals and particularly teachers used a confrontational discourse between what they saw as SIMCE’s demands and what they considered relevant from an educational point of view. Their daily dilemma was whether to spend most of their time focused on student learning and personal development or to prepare students to improve SIMCE scores. Although teachers did use different strategies in response to SIMCE requirements, they still had misgivings about whether their role was to prepare for the test, or whether they should get on with regular teaching and learning. While we did observe teachers working in what we might call “getting students used to” the test’s demands, we also witnessed the teachers and principals’ misgivings about having to engage in such preparation.

During our observations, we saw a focusing of resources on SIMCE preparation illustrated by the provision of extra time for teaching in the target subjects, of the carrying out of tasks associated with SIMCE, and in curricular narrowing. Along these lines, the following excerpt from an interview with the principal of one the schools illustrate her mode of dealing with a retirement request from one of her teachers:

The first thing Silvia [the teacher] told me was: ‘I want to be here, at this school, until I retire.’ Well, I said, you may stay here until you retire, as long as you take the first grade this year and prepare them during the next four years for a powerful SIMCE test. Okay? And she made the commitment. And we supported it. We injected many resources into that class (Interview, Pedro, principal, Liceo Rosa, 2013).

We observed what could be described as preparing students to “get used” to the SIMCE items. While we saw teachers engaging in test preparation, such as mock tests, they nevertheless told us that they considered it pedagogically unsound to rehearse the test’s contents and that instead they prepared their students to manage its format. Not only were teachers negative about their schools preparing in the contents of the test, but they also had negative views about test preparation in other schools:

Many schools do it. For example, to prepare for SIMCE they eliminate art education, they eliminate technology education and let us go at language, mathematics, history, since that is what SIMCE will measure at the end. (Interview, Julieta, Teacher, Liceo Rosa, 2013).

Finally, in the three schools, there were teachers who manifested strong opposition to test preparation, arguing that teachers should be confident about their work:

There are some schools that prepare for SIMCE, which is wrong because you should be confident about what you taught. Come, test me! Children already studied; they studied it the entire year. I do not need to prepare them for a SIMCE. I do not need to threaten them with SIMCE (Interview, Laura, Teacher, Liceo Rosa, 2013).

3.3 Responsibility for SIMCE results

Teachers received contradictory discourses about their responsibility for test results, which were more and less direct across different actors and school categorization. While official public pronouncements assert that the SIMCE tests do not evaluate teachers, a series of practices and statements at different levels of the education system pointed to the opposite. Schools categorized as in-recovery or emerging reported that the municipal or district level authorities labeled and signaled out institutional and individual causes for results. Teachers said that schools were individualized and stigmatized when results were poor and teacher work was directly questioned by local authorities, thus becoming a source of teacher malaise:

Well, they somehow tell you off. The Municipal Education Department calls us, they hold a meeting and show the schools’ graphs. It is a lousy situation. If we have slipped -because we do vary- (...) we feel awful, because we had only been trying to do our best (Interview, Rosario, Teacher, Liceo Rosa, 2015).

The schools’ leadership teams were emphatic in stating that, on occasions, there was specific pressure by local authorities to target the teacher of a given class as being individually responsible for the school’s performance: “If results are poor … the DEM (Municipal Education Department) asks ‘who is the teacher responsible for SIMCE?’” (Interview, principal, Liceo Rosa, 2012). In contrast, in the schools categorized as autonomous, the form in which this responsibility was conveyed was not always direct:

Well, some of the teachers feel frustrated. They’ve worked all their life, have been congratulated, have achievements and suddenly it is assumed that they failed, although we are never told that in so many words (Interview, Luis, General Inspector, Escuela Nova, 2015).

The schools’ leadership teams in the three schools contributed to an implicit and contradictory blaming of teachers pointing to their collective responsibility for SIMCE results. For example, we heard a school authority frustrated by not being able to achieve better results: “one has to plow with the oxen we’ve got” (Interview, Pedro, principal, Liceo Rosa, 2012), suggesting that the school’s poor performance was owed to teachers not being capable of doing more. Conversely, teacher responsibility was cited when the results were satisfactory. Individual teachers were commended for improvements in their classes, meaning to be competent or get good results. Despite the criticism and negative feelings that surrounded the SIMCE test, when school results were good, schools’ authorities congratulated their teachers and schools for such success:

Clearly, for us, it [to be classified as autonomous] is an achievement. … It is the result of an effort carried out over the years. It is only fair that we receive such recognition, the recognition of having performed well’ (Interview, Luis, General Inspector, Escuela Nova, 2015).

These anecdotes show that in schools with low outcomes, school leadership teams conveyed ambiguity about possible teacher responsibility or guilt. In schools with high outcomes, results were attributed to the school’s “competent teachers.”

Similarly, in schools where results on the test were improved, teachers felt proud of themselves and their students:

Last year my students increased their SIMCE score (...) Not much, perhaps, on a broader level, but I am very proud because this is an achievement for all the years we have been working (…) These are small achievements. Small achievements generally go unseen. We are capable of getting these results because you accomplish things, this school has accomplished things, increasing SIMCE, but not on a national level (Interview, Rosario, Teacher, Escuela Sur, 2015).

In this way, teachers’ sense of responsibility for student results was explicit: “Let’s not beat around the bush, SIMCE also measures us” (Interview, Andrés, Teacher, Escuela Nova, 2015). Thus, in the three schools we studied, teachers believed that SIMCE evaluates their own work, and this increases their feelings of being under pressure.

Despite the above, teachers did not always agree that they were personally responsible for student test scores. They claimed that they are not able to control the set of situations that explain such results. The following example refers to a teacher who blames the arrival of new students, well on in the year, for the SIMCE results achieved:

One month before the test there were six new children arriving in my class. We had fourteen and now there are twenty. So, imagine what it is when six new unrepresentative pupils suddenly arrive. They don’t bring with them the fund of learning achieved by the class since the beginning of the year. In this school there is so much pupil mobility that in the end you just can’t assess what you’ve been teaching (Interview, Lorena, Teacher, Liceo Rosa, 2015)

In sum, teacher responsibility appears paradoxical. Although teachers were not directly blamed by the assessment system, local-level authorities did blame them. They requested the names of the teachers in charge of the SIMCE classes, while teachers and leadership teams attempted to defend themselves. Teachers contended that there were many external factors that explain low results, but when SIMCE test scores were good, teachers and leadership teams took this to be their own success.

3.4 “The real school is invisible to the SIMCE test”

Even though in the three schools some teachers and school leadership team members agreed evaluations are necessary, participants criticized the SIMCE test as an assessment instrument. According to teachers and leadership teams of all schools, the SIMCE test does not consider the complexity of their schools. In their view, the evaluation system does not take in the real conditions under which teachers work. Therefore, they considered it inappropriate to compare the results of one school with those of another. Even though the Quality Agency of Education compares a school’s results against other schools with a Similar Socioeconomic Group (SSG) classification, participants considered school comparison unfair because it did not consider the complexity of their students and contexts, which were beyond the broad classification of SSG. They stated: “We have students with real difficulties and yet we’re measured by a system that uses the same parameters for all situations” (Interview, Juan, Teacher, Escuela Sur, 2013).

School staff constantly complained about what they considered to be an unfair and “hardly objective” feature of the standardized evaluation system. They argued that the system does not seriously consider the context and that real schools were invisible to the test.Footnote 1 Their critical discourses constructed two opposite poles: what they saw as a standardized and general evaluative judgment and what they considered to be part of the very nature of school practice and their work, which took place in the real world.

In general, teachers and the leadership teams of the three schools we studied asserted that the characteristics of their students and families were an important handicap that affected the school’s performance. Teachers added that the evaluation system is more attuned to the type of student found in other kinds of schools, such as private ones. As one teacher remarked: “One should not use the same SIMCE test for children with different educational backgrounds” (Interview, Ignacia, Teacher, Escuela Sur, 2013).

For some of the teachers and leadership teams, the key social function of public schools serving disadvantaged populations was to ensure that students completed their schooling rather than struggle to improve their learning results. Teachers believed that students turn their feelings of neglect into behaviors that challenge the kind of environment that is appropriate for teaching and learning:

Here we can’t cover all school contents. Children often don’t come to school. You can’t do the same lesson for them and that puts them at a disadvantage. You can have a very organized lesson, but it is not useful, but you still cover the content. Then you must reinforce and keep working on it. I don’t know how much of this happens in other types of schools. But, deep down, I feel our students are at a disadvantage (Interview, Marlene, Teacher, Escuela Nova, 2015).

This description of students as abandoned and greatly deprived individuals has its correlate with the concept of a school left to its own resources and lacking the external support needed to work in these conditions. In the words of one principal: “They have neither visited, encouraged, or motivated us – they have left us alone” (Interview, principal, Liceo Rosa, 2013). In sum, the feeling was that by focusing only on test scores, educational authorities failed to see what was really needed to improve educational quality.

The school leadership teams’ discourse exhibited the strong belief that what they did on a daily basis was to further learning and education, that they did so with a clear moral purpose and in tune with the school’s history of commitment to the local community, and that all this was marginal to the SIMCE test:

We are more than the SIMCE results. We are the opportunity for many young people wanting to get out of the circle of poverty, even if they don’t regard themselves as poor. Many put in a lot of personal effort and show it. They study hard, get good grades, come to school and are responsive to the opportunity given to them. Others see us as a school that is open to the community. They thanked us for being a school that works in solidarity with the community (Workshop, principal, Pedro, Liceo Rosa, 2012).

In sum, the main issue identified by teachers and leadership teams was that the SIMCE test did not consider the real conditions under which their schools work. The discourse of the school actors was built on the conviction that they had a subordinate place in the system and that they were at a disadvantage in relation to other schools. In the current state-of-affairs, the public schools that we studied felt that they occupied a special niche in the education market—they provided care and attention for the poorest students, while the subsidized private schools systematically rid themselves of them.

4 Discussion and conclusions

Using an ethnographic study of three Chilean public schools operating in disadvantaged neighborhoods of Santiago, we answered two questions regarding the lived experiences of key school actors with the national assessment system known as SIMCE. The study’s conclusions showed similarities and differences in schools’ translation and interpretation based on their contexts.

First, we asked how key school actors re-contextualize (Ball et al., 2012) educational policies that affect them, and more specifically, how they translate their meanings in the schools’ daily life. Even though we observed some differences in some specific school practices, on broader terms, strategies used before, during, and after the test was taken were remarkably similar among schools despite their official classification. We found that, given the market system under which the schools operated, they used similar strategies to retain or improve their schools’ classification and avoid the loss of students. Thiel et al. (2017) argue that adaptative and evasive strategies could be found in accountability contexts with low or high stakes; nevertheless, their frequency is less where the consequences are lower.

This could explain the similarities we found, regardless of their official classification and external pressure. Also, commonalities could be related to a similar context where schools work. Falabella (2015) reported how low-performing and marginalized schools’ teachers and leadership teams responded to the pressure they feel to perform in SIMCE by focusing on those contents that will be evaluated instead of other concerns teachers deemed as or even more important. Our findings show how similarities in schools’ situated contexts (Ball et al., 2012), regarding students’ intakes and disadvantaged school contexts, affected teachers and school leadership teams’ policy translation. Except for the few weeks at the end of the year when testing is over and other matters can be tackled, all three schools move through the school year with the shadow of SIMCE hanging over them. SIMCE produces constant anxiety related to student’s performance, as Rojas and Leyton’s (2014) study also showed. While the official justification for the SIMCE system was that it would encourage schools to improve their teaching and learning quality, teachers and leadership teams translate this objective into forms of avoiding low SIMCE results. This stance explains why, when faced with adverse results, there is mutual recrimination rather than a decision to focus on how to improve the teaching/learning process.

The pressures to which the school communities felt subjected marked their policy translation. Although, at first glance, participants seemed to experience contradiction regarding the SIMCE’s results demands and what is possible in their schools, as Falabella (2015) and Thrupp (2017) have suggested, under more careful analysis, these contradictions showed a greater complexity. For example, as we learned in one of the cases, the SIMCE situation was exemplified as a battlefield in which students must valiantly encounter the enemy and fight for their school by responding as expected on the test. Thus, it was the students who must soldier on and face the challenge, perceived by some as a threat. The school leadership contributed to the struggle by working on the design of a strategy for success. This strategy may include practice in the formalities of the test, allocation of extra time and resources to the subjects measured by the test, or putting the teacher considered most effective in charge of classes and subjects evaluated. As we observed, this mode of translating the policy contrasts with policymakers’ assumption that SIMCE results will stimulate the school community to review their teaching and learning strategies. Following Carrasco and From (2016), this study showed that school leadership teams displayed specific strategies to respond to external demands or “the rules of the game.” In contrast to Carrasco and From (2016), this study showed that, in disadvantaged school contexts, strategies aimed to improve students’ performance on the test to achieve a better categorization and avoid school closure rather than on increasing student enrollment and market competition.

The above leads to the second question formulated for this study: How do teachers and school authorities interpret the accountability policies that affect their everyday school life in the light of conditions derived from market policies and school competitiveness? We identified similarities in policy interpretation but also some nuances among schools based on their categorization and achievement on the test. If results were not good, especially in in-recovery schools, they blamed the contexts in which they worked and the policymakers for not taking these into consideration, like what Falabella (2020) found on her research. If results were positive, especially in the autonomous and the emerging schools, they congratulated themselves and saw results as the product of the school staff’s hard work or of the teacher in charge of the successful class. In this way, external contexts (Ball et al., 2012) based on schools’ evaluations, ratings, and direct consequences affected policy interpretation by teachers and school leadership teams. Contrary to what research on the impact of SIMCE in other school contexts has observed (CIDE, 2012), participants argued they did not “teach to the test” but acknowledge they worked hard on their students learning to respond to the test format. This aligns with the findings of Verger et al. (2020) about how some schools, especially low performing and vulnerable ones, when faced with performance pressure, predominantly adopt one of two strategies: either ignoring this pressure, as they believe the school has a different focus, or fabricating results to avoid negative consequences without altering their values and approach. Along these lines, even though we observed pressure across all schools, there were differences in its perception. While high performing schools concerned about maintaining their status to preserve their autonomy, low performing ones worried about the possibility of closure. This aligns with Parcerisa’s (2021) findings on how the perception of policy determines how the pressure to perform is experienced.

Despite these nuances, teachers and school leadership teams in all schools we studied did not perceive the SIMCE testing system in line with its publicly declared objectives of providing information on curricular coverage and school quality and making it accessible to parents to inform their school choice. Nor did they see the information delivered by SIMCE as an input to improve their educational strategies in line with the diagnosis provided. On the contrary, it was perceived as a menacing policy hanging over the schools that targeted and held responsible those who work with very disadvantaged populations with little consideration for the difficulties that come along with that work. In the context of the market system to which these schools were subject, SIMCE was seen as a threatening instrument, suggesting the loss of students to private subsidized schools that operate under more favorable conditions (Falabella & Opazo, 2014) and therefore possible school closure. Thus, for schools in disadvantaged socio-economic contexts, SIMCE operated as a visible threat and part of an evaluation system that did not consider school particularities (Verger et al., 2016) and crashed with their professional culture (Ball et al., 2012) and their values and commitments to work on disadvantaged contexts. The SIMCE test as an evaluation device is perceived as conflicting and distant from schools’ contextual conditions and as a pressure tool over schools.

Schools’ daily life, institutional strategies, and the contradictions of teachers and principals regarding the test serve as evidence as to the hypertrophied nature of the accountability rhetoric as perceived in educational contexts that operate under market mechanisms. What emerges, among other issues, is the contrast between standardization and the real world as well as between the general and quantitative nature of the test and the specific context of singular schools. More to the point, it lays open the contrast between the schools’ concepts of responsibility and the unfairness of being made accountable in structurally unequal contexts, which are out of the control of teachers and school authorities.

This study contributes to the expansion of the literature related to accountability policies by showing that the external pressure over schools is interpreted similarly by participants regardless of their classification. The Chilean accountability system assumes that schools should experience varying levels of pressure based on their classification. However, teachers and leadership teams did not experience these different levels of pressure. All participants, including those on autonomous schools, feel threatened by dropping on the official classification levels since they do not feel they can control the factors that influence the classification. This feeling of threat could also be related to the “war” metaphors used by participants. Teachers and leadership teams refer to the examiners as strangers and enemies that invade schools. They also use harangues invoking students’ courage to face the test as if it was a battle. These findings also suggest that educational policies based on standardization and direct consequences over schools located in disadvantaged contexts are not translated in practices related to improvement and students’ learning, nor are they interpreted as fair and pertinent to these school contexts. In contrast to these high-stakes accountability system, evaluation policies based on trust on teacher professionalism, local control, democratic values, and complex concepts of teaching and learning could be more effective promoting student learning (Cochran-Smith et al., 2018) and teacher wellbeing (Rojas & Leyton, 2014).

However, this research has some limitations, as it was conducted solely in the metropolitan area before the pandemic, when SIMCE was suspended and a new assessment was introduced, which was like SIMCE but without consequences. As a result, we lack insight into how this policy is recontextualized in rural settings, other regions of the country, pandemic, and post-pandemic scenarios. Also, since ethnography’s aim is to reach a deeper understanding of the phenomenon, it does not allow us to observe its full extent or quantify its variations, so there could be ways of recontextualizing accountability that could not be observed with our method. Further research on the impact of accountability instruments needs to delve into how disadvantaged students and families translate and interpret their purposes and how these policies are seen to affect their hopes and aspirations. Previous studies (Corvalán & Román, 2012) have reported that SIMCE results were not relevant for families’ school satisfaction and choice; however, families’ translation and interpretation of SIMCE after the intensification of its results’ consequences, regarding school categorization and closing, since 2017, remain unexplored.