Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Context

Costa Rica is known to be a country that values education. Education was made constitutionally universal and free in 1847, at a much earlier date than most of the, at that time, more developed countries in the region. In 1987, the country took the decision to introduce computers in every primary school. In 2011, the Constitution was reformed so as to establish as mandatory for the State an investment in education of a minimum of 8 % of the Gross Domestic Product.

The Costa Rican education system has 940,000 students, 73,616 teachers, and 4,523 schools (MEP, 2014). It is managed nationally by the Ministry of Public Education. The Ministry implements the policies and the curriculum approved by the National Board of Education (Consejo Superior de Educación), a high level constitutional body responsible for defining the country’s educational policy and thus above the Ministry, although at the same time chaired by the Minister. The Ministry decides on the curricula and the projects that have national scope, as well as the strategies and guidelines that shape the national education system.

Regional offices, of which there are 27, help the Ministry to organize and supervise the delivery of the educational services through schools, and to contextualize the educational policies. Locally, the schools have the option of developing an annual working plan, on the basis of their specific strengths and needs, as part of a continuous improvement process.

The Costa Rican education system’s structure comprises three major levels:

  1. 1.

    Preschool (2 years, 5–6 years of age)

  2. 2.

    General Basic Education:

    • Primary school (6 years, 6–12 years of age)

    • Lower Secondary school (3 years, 13–15 years of age) taught by teachers of different specialties

  3. 3.

    Diversified Education (2 or 3 years, depending on the branch; 16–18 years of age); the main branches are Academic (2 years) and Technical (3 years), with other branch options such as artistic, scientific, etc.

Since 1987, the Ministry and the Omar Dengo Foundation (ODF) have run together a national program that has led the introduction of digital technologies in the schools, first in the form of computer laboratories, now in combination with mobile technologies in the classroom (with several modalities: the one to one model in some cases, and mobile labs in others). In 2014 the National Program of Educational Informatics reaches 71 % of the student population. It plans to reach 100 % by 2017.

One particular feature of this program has been a constructivist vision of the potential of computers in education as tools to think and to create, and thus to promote higher order skills such as problem solving, creativity, collaboration, etc. This approach to technology in education has inspired a set of projects that have as their basis the learning of programming.

More recently, the Ministry of Education launched a series of curriculum reforms directed towards making the learning more relevant and attractive for the students and to develop social and personal competencies that have traditionally not been part of the education system’s objectives. A particular aim for Costa Rica, through its active participation in the ATC21S project, was to strengthen the assessment component of these recent curriculum reforms implemented during recent years, in addition to providing Latin American countries with educational resources for assessing and teaching 21st century skills through the promotion of efficient use of the Information and Communications Technologies (ICTs). In this context, the Costa Rican activities in adaptation of tasks provide a state-of-the-art model for countries wishing to modify these complex online tasks for local use. The following provides information on the adaptation process for the tasks.

ATC21S Development in Costa Rica

The Latin American Chapter of ATC21S was implemented in Costa Rica by the Ministry of Public Education and the Omar Dengo Foundation, with the support of the Inter-American Development Bank (IDB), Intel and Microsoft Latin America, and the Costa Rica-United States Foundation for Cooperation (CRUSA).

The Inter-American Development Bank played a major role in bringing Costa Rica into the project. Its argument about the value of having a Latin American country to help contextualize and validate the assessment instruments in the context of the developing world convinced the ATC21S consortium of the relevance of Costa Rica’s participation in the project.

The Costa Rican Ministry of Education was also quick to accept the invitation to be part in ATC21S, with the technical support of ODF, as it saw the initiative as a great opportunity to understand, through new approaches to assessment, how students progress in their learning of these crucial skills and how to support them in this process. The IDB’s proposal also obtained an enthusiastic response from other partners that were invited to join the initiative: Intel Latin America, Microsoft Latin America, and CRUSA Foundation.

The Ministry of Education assigned a National Project Manager to lead the country’s participation and conducted the necessary arrangements with the IDB and the rest of the supporting partners to officially represent Costa Rica as a member of the international project. Other roles that the Ministry took on were to invite the participating schools to be part of the project and to provide the necessary approval for the teachers to participate in the training workshops. A group of curriculum specialists from the Ministry helped in the process of localizing some of the assessment tasks. In the latest phase of the project, it also took a leading role in the production of digital resources aimed at helping teachers and students explore new ways of teaching and learning key competencies.

The Omar Dengo Foundation designated a National Project Coordinator and took on the responsibility for the technical implementation and for the administrative and financial management the project. A team of three researchers, including the National Project Coordinator, conducted the process needed to localize the assessment system, as well as the field research activities and reporting to the University of Melbourne. Finally, the Omar Dengo Foundation contributed with the creation of dissemination materials, including a website, a digital interactive booklet, and a series of videos showing examples of best teaching practices identified among participating teachers.

Localisation

As a Spanish speaking country, one of the first needs that Costa Rica encountered when it became a member of ATC21S was to translate the assessment instruments into the local language. Unlike countries such as Singapore or even Finland, where higher levels of English literacy can be assumed, the Costa Rican student population, in general, does not have an English proficiency level to guarantee its valid participation in tests written in that language.Footnote 1 In addition, the role assigned to the country as representative of the Latin American region was to facilitate the development and implementation of a set of assessment tools translated into standard Spanish.

Beyond the translation of all tasks to standard Spanish, the localisation of two tasks developed to assess the skill Learning through Digital Networks (LDN), involved additional work. These two tasks, The Arctic Trek and Webspiration (Poetry), both require students to explore web resources that are external to the tasks. This proved to be a major challenge for the team, due to the difficulty of finding Spanish language web resources similar enough to those used in the English version of the tasks. Therefore, the team produced from scratch a set of web resources in Spanish language to substitute the original ones linked to the tasks in English.

Normally, it would not be much of an issue for students to be able to access Spanish language sites for their school-based research, because of the growing body of good quality content in this language on the Web. But in this case, it was required that the resources would match as closely as possible those used in the original tasks, so as to not risk the equivalence of the level of difficulty of the items included in the tasks.

The Arctic Trek

The adaptation process for the Arctic Trek was divided into three stages:

  1. 1.

    The search for a theme for which online resources exist in Spanish, and which are equivalent to those used in the original task

  2. 2.

    Selection of appropriate web resources and modification of the text within the task, according to information for the new theme

  3. 3.

    Creation of web resources that could not be found online, as well as new design elements.

In the case of Arctic Trek, the major challenge was to find external web resources that were the equivalent of those used in the original English-language task, in large part high-quality scientific sources on the Arctic. Due to this limitation, it was necessary to identify a theme that was closer to the Latin American context. A panel composed of the research team, the National Project Manager and science advisors, considered as its first option the rain forest, a bioclimatic landscape present in an important part of Latin America. However, the panel ultimately chose to work with the Antarctic, another Latin American landscape that would tie in with the Arctic theme in the original task.

The next step was to locate online resources with information on the Antarctic that met the following requirements:

  • Belong to recognized educational-scientific organizations that could be considered trustworthy sources of information; preferably not personal websites or blogs

  • Discuss topics relating to the Antarctic

  • Have maps, tables or numerical data that could be used in different parts of the task, as was done in the original task in English

  • Be interesting and engaging for children and adolescents

  • Demonstrate a level of complexity appropriate for each age group (including aspects of content and language)

  • If possible, be ad-free.

In the third stage, the text for the task was modified to correspond to the new theme and the selected web resources – mainly the clues for each age group that would be included in the developed task. In the same way, some parts of the task that contained graphics had to be redesigned to match the updated theme. Following the example of the polar bears and Arctic wolves in the original task, the team compiled information on animal populations in the Antarctic – specifically, penguins – and this information was used to create the new graphics. For the final creation of the measurement-based graphics, an expert in mathematics was called upon to define the ranges that the lines should trace.

The final phase of adaptation consisted of the ad hoc creation of a Spanish-language web resource to substitute the “Tagxedo” website, a resource used as filler in the original task. Tagxedo is a web resource that enables creativity by building word clouds. Its role in the task is inclusion in the list of websites that are available to students and from which they can obtain the information needed to answer the questions that the task poses to them. Due to the lack of a similar resource in Spanish language, in its place a website was designed that would allow visitors to draw like the artist Jackson Pollock, with information on this famous American painter and on expressionism (visit http://atc21s.mep.go.cr/jackson-pollock/).

The contextualization work concluded with the creation of a new graphic design for the whole task. The original task in English language used a graphic design based on Arctic wolves which in the Spanish version were substituted with Antarctic penguins.

Webspiration (Poetry)

In the case of this task, the adaptation process consisted of four major steps:

  1. 1.

    The selection of three poems in Spanish to replace those used in the original English-language task

  2. 2.

    Modification of texts within the task, according to the new poems

  3. 3.

    Compilation of web resources to replace those used in the original task

  4. 4.

    Creation of web resources that could not be found online.

Adaptation began with the selection of Spanish-language poems to replace those used in the original English-language task. For this, Spanish literature experts were consulted, who made recommendations about themes, language and figures that were appropriate for the ages of the students who would be doing the tasks. Cognitive laboratories with students allowed the team to identify that one of the three poems originally selected presented too high a level of difficulty, and it was replaced. The final selection includes a poem by José Martí, a Cuban poet from the nineteenth century; another from Amado Nervo, a Mexican poet who wrote at the beginning of the twentieth century; and finally one from Raúl Aceves, a contemporary Mexican poet who kindly granted author rights to his poem. With the poems selected, the team proceeded to modify the text concerning the poems to mirror the logical rules employed and the desired difficulty level for each age group from the original English language based task.

The next stage in the adaptation process for this task consisted of the substitution of web resources used in the task by their equivalents in Spanish. It was possible to find existing resources or websites for:

  • Poetry terminology

  • Authors

  • Dictionary

  • Glossary

The criteria for their selection were the educational content of each and their relationship to a trustworthy and serious institution. For example, for the “Dictionary” resource, the online Dictionary of the Real Academia de la Lengua Española was chosen – the official organization in the Spanish speaking countries in charge of protecting and promoting the good use of the language.

Finally, as it was not possible to find an equivalent resource in Spanish for the website FavoritePoem.org, it was necessary to create an ad hoc resource. The consequent website in Spanish is called Our Favorite Poems (visit http://www.atc21s.mep.go.cr/nuestrospoemasfavoritos/) and presents 18 videos of people with different backgrounds reciting their favorite poems and explaining the significance the poems have in their lives.Footnote 2 In the production of this resource, particular attention was paid to representing the ethnic and cultural diversity that exists in Latin America. For example, the participants in the videos come from different countries of the region (Argentina, Chile, Costa Rica, Nicaragua, Perú, etc.), and the nationalities of the poets they discuss also vary. In addition, an Afro-American person and an indigenous person participated, talking about poetry in their respective cultural backgrounds and reciting a poem representative of them.

Adapting these tasks into Spanish required consideration of issues that were both relevant and significant with regard to their application, such as cultural diversity, language, varying geographical scenes, and some limitation on the use of existing online resources. Throughout these adaptations, the primary goals for the Costa Rican team were for the assessment tools to be valid for the rest of the region, as well as responding to the curriculum reforms that are occurring in Costa Rica.

Method

Task Concept Checks, Cognitive Laboratories, Pilot Programs and Trials

In 2011, the team’s work efforts were concentrated on the fieldwork for Collaborative Problem Solving (CPS) and the technical feedback for the international research team. In two high schools, the research team conducted concept checks with eleven Science, Mathematics and English teachers, and cognitive laboratories with twenty 15-year-old students.

Pilots were conducted in two elementary and two high schools, with four groups of 11-year-old students, two groups of 13-year-old students and two groups of 15-year-old students. Eight Science, Mathematics and Educational InformaticsFootnote 3 teachers took part in a four-hour training session, where they were familiarized with the project, its objectives, the tasks and the administration guidelines. Then the teachers administered the tasks. The results of these activities were reported to the team at the University of Melbourne, along with suggestions for the task administration procedures to be used in Costa Rica.

In the trials sixteen elementary and high schools collaborated throughout the country, with the participation of 90 11-year-old students, 222 13-year-olds, and 188 15-year-olds. Forty teachers of various subjects, including Mathematics, Science, Spanish and Educational Informatics (and the principals of some of these schools) participated in a four-hour training session in preparation for the administration of the tasks.

In 2012, new trials for the CPS tasks were conducted, this time in eleven high schools, with 593 13 and 15-year-old students. A two-day workshop was organized with the participation of 30 teachers in order to allow them to learn not only about the tasks and how to administer them, but also about the 21st century skills movement, the Collaborative Problem Solving skill and the assessment rationale behind ATC21S. When the trials were concluded, the teachers shared their experience with the assessment tasks in a 2-day briefing workshop, providing insights about how to use these tasks and develop didactic interventions that could help the students develop the skills.

In 2012, we conducted cognitive laboratories for the Learning through Digital Networks (LDN) tasks with twelve students (four 11-year-olds, four 13-year-olds and four 15-year-olds) in one elementary school and three high schools. This activity provided valuable information about students’ reactions to the localised tasks.

In 2013, another set of trials was conducted, this time combining CPS tasks and, for the first time in Costa Rica, LDN tasks. The goal was to provide additional CPS data to complete the psychometric calibration of the tasks, and also to test the Spanish-language version LDN tasks technically – both in terms of task-specific bugs as well as issues associated with the assessment system and its platform including registration and reporting. Of the 776 13-year-old and 15-year-old students from nine high schools who participated in these new trials, 575 students provided full scorable data, the loss due in part to school connectivity issues and task non-completion.

The 24 teachers who took part in these 2013 trials had also participated in the project in 2012 and were well familiarized with the CPS tasks, but as this was not the case with the LDN tasks and new features of the assessment system, additional training was organized – again a two-day workshop prior to the task administration and a two-day workshop after it. In this last workshop, the teachers had the chance to review the students’ CPS reports, compare them with what they would have expected of their students, and reflect on how to use the information provided by the reports to make decisions about didactic interventions with their students.

In every stage of the field work, the sample of students was opportunistic. The participating schools were public in all cases except for one. This means that the socio-economic background of the participating students can be assumed to be mainly middle and middle–low. The schools were selected on the basis of their technical infrastructure (computer labs in good condition and good connectivity levels) and the positive disposition of the principal and the teachers towards the project. These two conditions provided, the research team tried to ensure that the school sample was varied enough regarding their geographical localization (rural and urban) and their record of student performance.

Responses to ATC21S Tasks

The teachers’ and students’ responses to the tasks were generally very positive. Though there were some technical difficulties – mainly due to Internet connectivity in the schools – the students enjoyed the tasks and the teachers highly valued the opportunity to get to know and use such innovative tools to assess skills that, though regarded as very important, have not been part of their explicit teaching objectives.

The students found the tasks both engaging and challenging. In the words of a student at Naranjo Bilingual High School: “You have to think, it makes sense, and it is fun”. They worked on them with persistence and interest, though at times they felt helpless and frustrated, especially those who faced technical difficulties. A frequent comment made by the teachers was how striking it had been for them to see their students concentrating so deeply for a period of over two hours – the average duration of the sessions. As one of the teachers at Palmares Bilingual High School put it: “It drove the students’ interest because it is a change of routine and because they like to be challenged” (Greivin Calderón, personal communication, 2011).

At the same time, something that was evident for the research team from classroom observations was that in general the students do not read thoroughly what is handed to them. Also, for many students, it was shocking, and even disturbing in some cases, not to be told precisely what to do. This may be due to the fact that they are accustomed to receiving complete and specific indications, so ATC21S tasks represent a big change from the way instructions are given in the daily life of Costa Rican classrooms. Another classroom observation of interest was the degree to which some students limited their exploration of the elements being displayed on their screen, as if they were unaccustomed to such lack of constraints, or had not had the opportunities to explore digital environments.

The teachers showed great interest in learning how to incorporate the assessment tasks into their subjects. They appreciated their value as a tool to assess the students’ skills and to inform their teaching practice, but also as learning resources in their own right. The Principal of Presbítero Manuel Bernardo Gómez Elementary School expressed “What I like of this assessment is that it allows students to explore, without being afraid to fail, something that they hardly experiment with in their classes” (Katerine Ramírez, personal Communication, 2011).

The schools managed the task administration sessions well, with some exceptions in which teachers did not follow guidelines precisely. These schools provided the research team with the opportunity to develop examples of how to do the break-out session with the students and how to engage them for the tasks using additional materials (such as an introductory presentation, for example) with background information about the project and the assessment. Some teachers found it difficult to restrain themselves from helping the students when the students experienced difficulties. As a result, teachers were encouraged to provide generic help to the students by giving advice such as:

  • To read very carefully the instructions and identify vital pieces of information in them

  • To consider the fact that partners may have different things on their screens and different resources

  • To persist despite possible difficulties

  • To trust the power of the collaboration among the team members to resolve the tasks

  • To be focused on the work being done by the partner, not the person next to them, and to keep constant communication with the partner through the chat

  • Not to leave the task before informing their partner and hopefully not before having reached an acceptable solution

  • To keep silence.

Based on its experience, the Costa Rican research team recommended that:

  • Schools need to have good internet connectivity (minimum of 4Mbps for a group of 18 students). In order to lessen connectivity burden, a good strategy is to have the student pairs do the task bundle in a different order, as this can avoid having all of them using the same resources at the same time.

  • Teachers can make use of a presentation in order to introduce students to the goal of the assessment tasks and the importance of the skills being measured. This presentation needs to be written in a simple language, so that it can be easily understood by children. It can also make use of eye-catching images and motivational phrases so that students become engaged with the activity.

  • Before starting the tasks with the computers, it is important to seat students around in a circle in order to explain the activity but mostly to motivate them, emphasizing the basic instructions the students should take into account.

  • Students whose assigned student is ‘A’ must sit side by side and, if possible, away from students whose assigned student is ‘B’, because even though partners don’t sit nearby, students tend to look at the screen of their neighbours so they can see what the complementary students have on their screens. If possible, a good idea is to have the pairs split up in different rooms.

  • The teachers need to study well the administration procedure before initiating the process.

  • It is important to verify that students know how to type their logins correctly and that they choose their assigned student (‘A’ or ‘B’) accordingly.

  • Teachers must not tell students what to do, and they must not help them solve the tasks or answer the surveys. Students must discover what should be done and decide, together as a team, what they want to do, as well as when to move on to the next page or when to finish the task.

  • Teachers can do a closing at the end, in a circle, so that students can comment about their experience with the tasks. They can be asked about what they liked the most, what they think these tasks assess, how they collaborated and what they think was the most difficult part. The learning gained through the process should be highlighted: the relevance of good communication skills, of knowing how to express their ideas in a clear way so that people understand them, of thinking with their partner about the problem and what is it about, and of trying different solutions and strategies, etc. One main thing to remind the students of is that these tasks are not measuring their intelligence but the strategies they use to solve problems, their persistence, their willingness to collaborate and their skill at it. This is important, as many students may see the assessment tools as very challenging and only achievable by the brightest.

Challenges

Strong Internet connectivity is vital for the effective use of the ATC21S system. Considering that the tasks were conducted with relatively small groups of students (18 on average) and that most of the participating schools had 4 Mbps connectivity, there are concerns in countries such as Costa Rica that there is not sufficient technical infrastructure capacity to support such initiatives at large scale.

Learning how to use technology effectively in their classrooms was something that the teachers considered to be vital for them to be able to teach and assess 21st century skills. Recognizing themselves as digital immigrants, their main concerns were, first, that they are falling behind their students in terms of knowledge of digital tools and digital practices and, second, that they do not have appropriate equipment and resources for incorporating digital technologies in the classroom. Despite this, they showed willingness to learn new things, asking for ongoing training and support.

Teachers expressed the need to better know and understand what each of the skills involves in terms of components and specific behaviours, as well as about the ways these can be learned. This is closely related to knowing how to translate this information into specific didactic interventions.

Beyond the incorporation of new didactic strategies and digital technologies in their teaching practice, Costa Rican teachers talked about the need to make more profound changes in the education system. They talked about the need for a shift of paradigm: to change our current approach to education as an individual action and begin to understand it as a collective and social action. They believe that this must be accompanied by classes that encourage autonomy and responsibility of students for their own learning.

Such a change requires a whole-school commitment. That’s why the teachers recommended that the plans to generalize the use of ATC21S tools and approaches in our education system should include training opportunities available for every interested teacher, online resources to support their work, but also a school-centred professional development strategy oriented to promote collective efforts towards the development of these key skills for our children and young people.

In order to provide a starting point for these future developments, the ATC21S Latin American Chapter complemented its work in support of the localisation and validation of the assessment system with the following dissemination and production activities:

Additionally, the International Forum ATC21S: Assessment and Teaching XXI Century Skills was organized in San José, Costa Rica, on 2 and 3 April 2014 (visit http://www.fod.ac.cr/competencias21/index.php/areas-de-recursos/comunidad/foro-internacional-atc21s#.U5NkM3KwaQk). The goal was to create a space for analysing and reflecting on the teaching, learning and assessment of 21st century skills, linking with educational reforms promoting MEP. It was attended by international experts such as Claire Scoular, Lead Researcher at ATC21S from the University of Melbourne, Australia; Eugenio Severin, Chilean consultant in education and technology; Moritz Bilagher, regional coordinator for Evaluation and Analysis of Trends in the Regional Bureau of Education for Latin America and the Caribbean of UNESCO; and Alberto Cañas, associate director of the Institute for Human and Machine Cognition.

Conclusion

In addition to participating with other partner countries in the fieldwork aimed at validating the assessment tools, the Costa Rican team has achieved the rigorous translation, adaptation and, in one case, creation of these instruments. This component of the project has contributed to understanding how linguistic and cultural differences affect the assessment of skills and competencies in the 21st century.

Within the strategy of the Costa Rican Ministry of Education, participation in the ATC21S project was part of a program to promote the reform of teaching and assessment practices through the use of technology and the development of better teaching and learning tools and approaches. These efforts join a policy that aims to make the most of the use of mobile technologies, the educational informatics laboratories in schools and the application of innovative, more meaningful and participatory learning methodologies in the curriculum.

The set of skills defined by the ATC21S project closely relates to the programs of study developed within the Costa Rican curriculum under the “Ethics, aesthetics, and citizenship” program (Civic education, Fine arts, Education for life, etc.). They also coincide with an emphasis on key skills in the reform of subjects like Mathematics and Spanish. For example, into the Spanish program of study in Secondary Education there were introduced elements of logic, in order to develop the ability to detect reasoning, build arguments and engage in debates in different contexts, and to improve students’ reading comprehension as well. In Primary Education, the “Think Art/Piensa en Arte” program was implemented in Spanish classes as a way to strengthen critical and rational thinking via the employment of art as a pedagogical resource.

Given the preceding conditions, the Assessment and Teaching of 21st Century Skills project helps to consolidate and advance important efforts to improve the quality and relevance of our education. The further work that still needs to be done will allow our education system to complete the process of migrating from a traditional to a more innovative system. That transformation will be a reality, thanks to what innovative teachers and committed school leaders, pushed by the strong impulse of their students, will build collaboratively to better respond to our society’s most challenging needs and expectations.