The Contributions of International Large-Scale Studies in Civic Education and Engagement

Abstract

Mathematic, scientific, and literacy skills have taken center stage in the field of international large-scale assessments, and country rankings have become ubiquitous. However, three studies since the 1970s have addressed the topic of measuring civic engagement and citizenship and have examined patterns of student achievement in attitudes and skills as well as knowledge. Because of the complexity of preparation for citizenship and workplace readiness in different democratic systems, these civic education projects have had an innovative edge in both assessment development and the analysis undertaken. Results from these studies have led to insights into political events, such as the difficulty of establishing civic education after a dictatorship, the rise of anti-immigrant parties, and changes in the political participation of young adults in Europe and the United States. These studies provide information about how students are able to get along with others in society, acquire norms, and participate via democratic means to implement change. In addition to considering civic studies in an international perspective, this chapter will present results of secondary analysis of Civic Education Study data to illustrate the utility of these studies, and will discuss analysis relevant for policy and for researchers in political science and psychology.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

International large-scale assessments (ILSAs) have the potential to make contributions at least at three levels. They can contribute at the macro level (to a general understanding of countries’ educational goals), at the meso level (to the needs of education policy specialists and those such as journalists who communicate information about education to the public), and at the micro level (to improve processes of teaching and learning). Examining a range of these international studies and their interpretation at the beginning of the second decade of the twenty-first century suggests that these potential contributions to constructive change are not being fully realized. The enhancement of studies of civic education and engagement in the portfolio of ILSAs can play a unique positive role in this process. A short history of one of the organizations deeply involved in this area, the International Association for the Evaluation of Educational Achievement (IEA)—which launched its studies in civic education in the early 1970s during the Cold War, a time of broad disagreement on what constituted legitimate civic education—provides a context for these issues in general. It also provides a context for discussing education (and assessment) relating to civic engagement and citizenship in particular.

History of ILSAs and of IEA in Relation to Civic Education

A small group of scholars in the late 1950s envisioned an empirically based science of comparative education research. Some Europeans were interested in recently broadened access to upper secondary education; in the United States, competition with the Soviet Union due to the launch of Sputnik had raised concern. Mathematics was chosen as the topic for the first IEA cross-national study because it was universally valued (Husen 1967).

In the late 1960s, IEA conducted surveys in six subject areas, including civic education. The Nation at Risk commission in the United States in the early 1980s requested one paper on the civic values learned in schools (Torney-Purta and Schwille 1986). Its full report, however, only used data from the IEA science and mathematics studies as it lamented the performance of students in the United States. Raising an alarm seemed to be the only role for international studies of achievement in debates on excellence (Torney-Purta 1987). This missed a potential contribution of international studies in identifying effective practices, the context of these practices, and the extent to which they might be adapted for use internationally (Torney-Purta 1990), a topic that is still a matter of debate (Luke 2011).

In the mid-1980s, the National Education Goal of making the United States first in the world in science and mathematics was announced, and IEA’s leadership was asked to move quickly into international large-scale assessments of these two subjects. At about this time, a program officer at the National Academy of Sciences called together a small interdisciplinary group to discuss how to ensure that the planned studies were rigorous enough to be persuasive to policymakers. The Board on International Comparative Studies in Education’s (BICSE) first report, Framework and Principles for International Comparative Studies in Education (Bradburn and Gilford 1990), called for international studies “addressing a range of content areas and grade levels” and “encompassing quantitative survey research studies as well as more intensive studies that use a range of qualitative methods” (p. 9–10).

BICSE encouraged attempts to unpackage the processes of science and math education in participating countries through video studies and case studies associated with the Trends in International Mathematics and Science Study (TIMSS). Thousands of hours of observations of mathematics classrooms and interviews with students, teachers, and parents took place in Germany, Japan, and the United States, often showing inconsistencies between intended curricula and observed educational practices (LeTendre 1998). Although these studies were consistent with IEA’s original aims to understand processes of education, beginning in about 2002, trend studies in science and mathematics began to garner the major attention. This focus has been predicated on the belief of many economists and educational policymakers that high science and mathematics achievement is a major precondition for economic success.

The “hand-wringing response” by US journalists when TIMSS results are released is predictable. It used to be Japan that anchored the top spot; now it is often Finland. However, a few scholars are beginning to raise questions. Some argue that there are substantial groups of students in the US who show excellent performance, but pervasive inequity in schooling means that many other students have low performance (often related to lack of home or neighborhood educational resources as well). Recently some are arguing that what are variously called noncognitive attributes influenced by schooling (Levin, Chap. 5) or Twenty-first Century Competencies (National Research Council 2010) are products of schooling that are important to economic growth.

Perhaps the focus in the policy debate on testing in order to produce country rankings in science and mathematics has resulted in looking at a relatively easily measured but too narrowly defined domain. The old joke about the man who looked under the lamp post for his keys may be a relevant analogy. It was easier to see in the strong light, even though his keys were lost blocks away. Large-scale assessments in mathematics may be easier to design than assessments about engagement in one’s studies (Brophy 2008; Marsh et al. 2000) and less expensive to administer than assessments of the ability to participate effectively with others in cooperative groups (Johnson and Johnson 2009). Attending to outcomes beyond cognitive achievement could be useful in predicting economic productivity and might also send a signal to school leaders that it is safe to move some attention to activities more motivating than test preparation (see a recent Rand Corporation report by Schwartz et al. 2011). In this context, what is the place of civic education studies, and what do they have to offer?

International Studies in Civic Education and Engagement

Knowledge of one’s government and how it works, attitudes conducive to engagement in democracy and positive intergroup attitudes have been themes in research for several decades. Studies in the field known as political socialization research began in the 1960s when political scientists and psychologists began to study the ways in which young people become involved in the political systems and communities in which they live (reviewed by Jennings 2007). Within the last 15 years, increasing use of the term “civic engagement” recognizes the multiple ways citizens can participate in the civic sphere (Sherrod et al. 2010). The name for the construct changed because there were problems both with the term “political” (emphasizing partisan politics) and with “socialization” (emphasizing a top-down process). In international comparative research that includes a cognitive component, “civic education” is the term that has been used to describe many studies in this area, and more recently, “civics and citizenship education.”

In general civic education studies conceptualize competent democratic citizenship as encompassing knowledge, skills, attitudes, values, and behaviors (current and expected). Responsible citizens have fundamental knowledge of democratic processes, an awareness of issues in their nations and communities, and an understanding of ways to obtain and analyze information. They also participate in their communities (including volunteer activity) and in organizations (at school and in their neighborhoods), possess basic civic-related skills (such as cooperation in groups and effective and respectful communication), are concerned for the rights of others (as well as themselves), and are predisposed to find democratic methods to bring about change.

Why and How the Civic Domain Is Important to ILSAs

There are a number of reasons that studies in the civic domain are valuable for inclusion in ILSAs. First and most obviously, studies of civic-related topics make a unique contribution to understanding young people’s preparation to live in democracy and willingness to respect human rights (including their attitudes toward ethnic minorities and immigrants in their societies) and their willingness to vote and to participate in activities that further democratic processes (Coley and Sum 2012). It is no longer the case that only adults who are eligible to vote have extensive opportunities to participate in political and social action, however. Scholars’ and citizens’ interest in human rights has burgeoned over the last several decades in policy-related as well as education-related discourse (Ramirez et al. 2011) during the same period that studies of civic education have grown in importance. Daily headlines are reminders that political and civil rights are among the aspirations of young people as well as adults in many countries. The capacity to assess whether and how to be involved along with the motivation to fulfill roles as responsible and active citizens are usually acquired by the end of adolescence. Assessing the nature and quality of these capacities and motivations comparatively is an important contribution of ILSAs in civic education.

A second contribution of civic education studies is that outcomes broader than knowledge are conceptualized and measured, providing a more satisfying view of students and their learning than studies limited to cognitive outcomes. This is true of both the Civic Education Study (CIVED, conducted by IEA in 1999) and the International Civics and Citizenship Study (ICCS, conducted by IEA in 2009). Taking an even broader perspective, the DeSeCo project of the Organisation for Economic Co-operation and Development (OECD) formulated a wide-ranging set of competencies needed for a successful life (Rychen and Salganik 2001) with consultation from multiple disciplinary perspectives and involving scholars from several countries. Although these competencies were never instantiated in measures for a comparative study, the DeSeCo project together with IEA’s experience in civic education show that students’ outcomes broader than knowledge can be conceptualized and agreed upon.

Third, many of the competencies falling under the rubric of civic education are aspects of readiness for adapting to the workplace and participating in community life. These include the ability to decode and understand media presentation and experience in understanding and respecting individuals who have different ideas and perspectives. If one wished to raise a country’s reputation in the world using arguments about the excellence of the workforce, achievement of these competencies would be an advantage. In other words reliable measures developed for international civic education studies cover a wide a variety of competencies (corresponding to many of those Levin identifies in this volume). Many of these competencies are valuable in the workplace and can make an independent contribution to economic growth.

For twenty years groups of scholars and policy advisors who have looked in depth at international comparative assessments (for example, the National Academy’s BICSE Board) have been calling for a portfolio of international studies in different areas, defining a balanced range of competencies, and including studies of different types to look at ways of improving the educational process in a range of subject areas (Bradburn and Gilford 1990; Gilford 1993). Recently a workshop sponsored by the Board on Science Education at the Academy explored twenty-first century skills and called for further research and elaboration (National Research Council 2010). Studies in the civic domain have the potential to fill many of these needs.

A fourth contribution is the opportunity that studies in areas such as civic education provide to investigate the context in which learning takes place. This is especially true when analytic models that are able to investigate aspects of the social and cultural context are utilized. We will refer later to the process of unpacking of international findings, a concept used in cross-cultural psychology studies (Vaughn 2010). Analyses with a small grain size allow inferences about specific factors that might account for high (or low) performance, for scores with normal (or bimodal) distributions, or for strong (or weak) associations between variables. To put it in another way, what are the educational and developmental processes that lie behind different patterns of achievement? How does the national context (for example, historical political factors or current problems with government corruption) set parameters for these processes? To realize this contribution requires a period of time after the collection of data by IEA (or any comparable organization) to allow secondary analysis. This is the reason that most of the examples given in this chapter come from the CIVED study (testing in 1999), although we also consider a few ICCS results (testing in 2009). The fact that the United States did not participate in ICCS makes that study of somewhat less value for purposes of illustration, however.

To summarize, studies in the civic domain should remain a part of ILSAs (and even be strengthened) for the insights about global, national, economic, and community issues and the methodological innovations they can provide (especially in measuring attitudes and skills in large samples). The importance of studies such as these has been recognized by a number of groups (including the National Research Council) over a period of two decades. IEA’s CIVED and ICCS projects have established a strong foundation on which to build.

IEA’s Role in Civic Education

IEA has a history of work in civic education that extends over more than four decades. This entry in the International Encyclopedia of Education reflects on the atmosphere of the 1970s in IEA when this first civic education study was conducted:

It is difficult to recapture today the concerns that surrounded this domain in the midst of the Cold War. What counted as legitimate civic education in one country was not what counted in countries with different ideologies. Measurement was daunting in the civic education domain, where attitudes were important as desired outcomes of civic education, and where model standards for measurement of knowledge were rare. In other words, this study was a bold move with risks both for the researchers and for IEA as an organization (Torney-Purta et al. 2010b, p. 656).

The first civic education study, including measures of content knowledge, attitudes (anti-authoritarian, trust in government, support for women’s political rights) and participatory behavior (discussion of political issues) tested in 1971. The results from this survey of about 30,000 students were published in the six-subject series (Torney et al. 1975). The fact that endorsement of democratic values was high in what was then West Germany, 25 years after the end of World War II, or that students in the United States had relatively low scores on support for women’s rights, received little public attention, however. Results showed that an open classroom climate for discussion was one of the central predictors of both civic knowledge and civic engagement. This was an early and successful attempt to connect classroom processes to students’ outcomes in an ILSA. However, from the late 1970s until the mid 1980s, IEA did not repeat a study in civic education but began to focus on science and mathematics, and on reading literacy. A general classroom environment study was launched but did not attract as much attention as subject matter studies in these three areas (Anderson et al. 1989).

However, during this period, groups in the United States became interested in assessments of political socialization and civic education. During the late 1980s, the National Assessment of Educational Progress (NAEP) tested the civic knowledge of representative samples 4th, 8th, and 12th graders. Political scientists Niemi and Junn (1998) reanalyzed the 1988 NAEP data and suggested that current education was inadequate in this area. Similar discontent with the effectiveness of civic education and the level of youth participation was voiced in England and Australia during this time (reviewed in Arthur et al. 2008).

The most important event during this period was the fall of the Berlin Wall in 1989 and the collapse of Communism across Central and Eastern Europe. Questions about the extent to which the educational systems of these countries were prepared to teach young people about democracy and human rights were raised. Would it be possible to replace Marxism with democratic theory? Could teachers be asked to teach in a new way and cover material about political rights with which they had limited familiarity? What about young people who had been warned by their families never to state their views about social and political issues outside their homes?

In 1993 the General Assembly of IEA (its governing body) requested a proposal for a civic education study, with part of the impetus from Eastern European delegates who saw the relevance of the comparative methodology for studying how their next generation of citizens could be prepared for democracy. Declining levels of political interest and participation among young people motivated some delegates from Western Europe and the United States to support a civic education study. An innovation was a two-phased design in which the first phase of the study was a set of structured national case studies (a qualitative approach). The guidelines for the case studies recognized that conceptions of civic education could vary more across countries than in subjects like mathematics and science. Each participating country’s team wrote a case study responsive to a list of general and specific questions about the nature of civic education in the country, reviewed by the international steering committee. Although the materials in the case studies were diverse, it was possible to identify domains that could be addressed across countries (Torney-Purta et al. 1999). There was considerable consistency across the case studies about the challenges that schools face (Schwille and Amadeo 2002).

Developing plans for social science studies by consensus among researchers from different countries is often difficult, as a survey of 26 projects by Torney-Purta (2008) has documented. At the first meeting of the CIVED National Research Coordinators in the Netherlands in 1994, considerable mistrust existed and a collaborative atmosphere had to be nurtured. All those in attendance voted on a list of topics that might be covered in a test and survey. There was enough agreement that the group agreed to go ahead (with IEA support). To assure that no one country’s perspective was dominant, actual quotations from the case study documents of several countries were incorporated in test specifications covering the following domains and subdomains: “Democracy and its Defining Characteristics,” “Institutions and Practices in Democracy,” “Citizenship Rights and Duties” (including topics relating to human rights), “National Identity,” “International Relations,” and “Social Cohesion and Diversity.”

The information collected during this early phase of the study contributed to the design of the instruments. Approximately 90,000 14-year-old students from 28 countries were administered tests of civic knowledge and skills and surveys of civic attitudes, activities, and anticipated actions in 1999. Findings were released in 2001 and reported in Citizenship and Education in Twenty-eight Countries: Civic Knowledge and Engagement at Age Fourteen (Torney-Purta et al. 2001; see also Baldi et al. 2001). Fourteen-year-olds in the United States performed well in many areas. Overall civic knowledge scores and scores on several attitude scales placed them in the group of countries with scores above the international mean.

In the following year, over 50,000 upper secondary school students from 16 countries received a similar test of civic knowledge and skills (and also economic literacy items not given to the 14-year-olds) and the same survey of civic attitudes and behaviors (Amadeo et al. 2002). Details about the scales can be found in Husfeldt et al. (2005) and Schulz and Sibberns (2004).

The theoretical framework of this IEA Civic Education Study (CIVED) conceptualized the ways in which “the everyday lives of young people in homes, with peers and at school serve as a ‘nested’ context for young people’s thinking and action in the social and political world” (Torney-Purta et al. 2001, p. 20). This theoretical model has its roots in Bronfenbrenner’s ecological theory (covered in Wilkenfeld et al. 2010) and Lave and Wenger’s ideas about situated cognition (covered in Torney-Purta et al. 2010a). Recently, the idea of a developmental niche for emergent participatory citizenship has been employed (Torney-Purta and Amadeo 2011). In short, these models posit that adolescents’ engagement in the community and the development of an identity within the group, together with classroom instruction and the everyday experience of a climate for open and respectful discussion of issues, facilitate learning about citizenship and democratic processes.

In 2009, IEA conducted the International Civics and Citizenship Study (ICCS). A more elaborated conceptual framework guided this test’s development, including “Civic Society and Systems,” “Civic Principles,” “Civic Participation,” and “Civic Identities.” A larger pool of cognitive items was developed (many accompanied by introductory explanatory material) and matrix sampling was used. Data were collected in 38 countries, with the findings released in late 2010 (Schulz et al. 2010). Of the 17 countries that participated in both 1999 and 2009, only Slovenia showed a significant increase in civic knowledge (on a set of items developed in CIVED 1999 and reserved to test trends over time). An innovation in this study was the implementation of regional modules for Asia, Europe, and Latin America (Kerr et al. 2010).

In the period from 1990 to 2010, in summary, major steps were taken in IEA comparative studies, including those in civic education. At the same time it has become clear that science, mathematics, and literacy studies will continue to be repeated on a regular schedule. The preponderant method of presenting these results to the public and to educators is likely to remain rankings of countries, relying on the shock value of the relatively low position of countries like the United States to get attention for mathematics and science education. Civic education studies have not become part of official cycles and trend studies. In fact, these studies have enormous but unrecognized potential. The next two sections will illustrate specific contributions of studies in the civic education area by presenting some secondary analysis of material not included in the original reports (and therefore not well known internationally):

• Patterns in multiple aspects of student outcomes that include but go beyond knowledge

• Positive values and attitudes relating to democratic engagement

• Attitudes toward ethnic groups and immigrants

• Skills and attitudes important in the workplace

The emphasis will be on analyzing the data in order to unpack contexts and processes and on relating the findings to recently expressed interest in outcomes of education other than knowledge.

Studies of Civic Education as Opportunities to Study Multiple Aspects of Student Outcomes

Most of the reports of the comparative large-scale mathematics and science studies are devoted to cognitive achievement results. The civic education studies conducted under IEA auspices have had multiple outcome measures balanced between cognitive and noncognitive assessment items. In CIVED each student has a knowledge score based on 38 items. This knowledge score has strong psychometric properties across countries and was also designed to be decomposed into knowledge and skills items (Torney-Purta et al. 2001). Each student also answered a number of attitudinal items (formed into about 20 different scales with strong psychometric properties, ranging from support for different types of citizenship activities to attitudes toward women’s political rights or trust in government).

Because of the diversity of civic education experiences across the world, we did not expect that any country would perform uniformly well on all aspects of the test (cognitive) and survey (attitudes). In fact, it turned out to be appropriate to rank countries’ student performances only on total civic knowledge scores. The basic reports contain tables of the attitudinal scores with the countries in alphabetical order (Torney-Purta et al. 2001; Amadeo et al. 2002). No country or region appears to be superior on all potentially valuable aspects of civic engagement and citizenship. The ICCS findings are similar, and not markedly different on a regional basis from the findings of CIVED 10 years earlier (Schulz et al. 2010). The multidimensionality of the civic instruments means that there are many possibilities for secondary analysis. The next sections present three examples.

Cognitive Diagnostic Models of Conceptual Knowledge and Skills

To illustrate the power of conceptualizing educational outcomes in a multidimensional way, three types of analysis conducted with CIVED data will be described. The first example uses the cognitive test data to look at civic-related cognitive capacities in a single nation at a smaller grain size than the original reports. Zhang et al. (2012) took a model-based cognitive diagnosis approach to analyze IEA CIVED test items administered to US students. A distinction was made in CIVED between conceptual or content knowledge (for example, asking what is usually contained in a country’s constitution)¹ and skills items (for example, asking a student to interpret a political cartoon or to distinguish between a fact and an opinion). We decomposed targeted cognitive components into a still smaller grain size and analyzed four multidimensional components through an advanced psychometric mode called cognitive diagnostic modeling (CDM). This approach allows researchers to test hypotheses about the nature of students’ response processes when they answer assessment items. By using CDM, one can classify students into different profile groups based on their item responses. Cognitive diagnostic models have been used in the past in secondary analysis of data from large-scale assessments such as TIMSS, PIRLS, and NAEP to obtain information about students’ cognitive capacities (Chiu and Seo 2009; Tatsuoka et al. 2004; von Davier 2007; Xin et al. 2004; Xu and von Davier 2008).

The IEA CIVED data had not previously been analyzed to identify the abilities underlying students’ performance using the cognitive diagnosis approach. Matrix sampling was not used in CIVED, and each respondent answered all 38 test questions, making this analysis somewhat less complex than in ILSAs where test-lets and matrix sampling are used (Rutkowski et al. 2010). Four cognitive attributes describing the content and process skills underlying the CIVED test items were identified (Zhang et al. 2012): basic conceptual knowledge, advanced conceptual knowledge, media-based skills, and advanced interpretive skills. Based on mastery of each attribute, students were classified into four different cognitive profiles. Examining these cognitive profiles suggests that basic conceptual knowledge is prerequisite for more advanced conceptual knowledge. It appears that in the United States a substantial group of students acquires civic skills without having basic conceptual knowledge and some acquire these skills outside of school.

Then, using multilevel analysis contextual factors such as characteristics of civic education experience were linked to the specific cognitive profiles. Results showed that students’ possession of particular civic-related attributes are associated with their socioeconomic backgrounds, experience with an open discussion climate, and with conceptually based traditional teaching.

Zhang and Torney-Purta (2010) extended the model-based cognitive diagnosis approach to CIVED data from Australia and Hong Kong. The four cognitive attributes identified through the analysis described above were consistent across the three countries. However, Hong Kong students were strong in basic conceptual knowledge but weak in analyzing and synthesizing skills. A considerable proportion of U.S. students were strong in analyzing and synthesizing skills and deficient in conceptual knowledge. In general evidence from the analyses supported the hypothesis that basic conceptual knowledge of civic topics is prerequisite for more advanced conceptual knowledge but to a lesser extent for skills. This secondary analysis looking at the data in a relatively small grain size shows the value of cognitive modeling as a technique for understanding different aspects of performance on a test of cognitive civic knowledge and skills within and across nations.

Examining Countries’ Positions on Multiple Dimensions of Attitudes

Another illustration of the value of comparisons of multiple dimensions that are possible in studies in the civic domain is shown in Table 6.1, which contains about a dozen countries’ means on three attitudinal scales from CIVED. The first column contains means on support for the norms of conventional citizenship (e.g., voting), and in the second column on support for the norms of social movement citizenship (e.g., joining a human rights or volunteer organization). Finland, which is always strong in achievement test scores (well above the mean in international civic knowledge, for example), has an unaccustomed place at the bottom of the country rankings for two measures of attitudes toward civic engagement. This discrepancy between achievement and attitudes scores suggests that placing extensive emphasis on countries whose students excel in cognitive achievement can distort policy debates (as Takayama 2010, has suggested in the case of Japan).

Table 6.1 Twelve countries’ means on three attitudinal measures. (Sources: Torney-Purta et al. (2001) and Husfeldt et al. (2005))

Results on attitudes toward ethnic and minority groups are found in the third column of Table 6.1. The post-Communist countries, plus Germany and Italy, are toward the bottom, with the Nordic and Anglo-Saxon countries at or above the mean. Note that no country is substantially above the international mean on all three scales in Table 6.1 (though Portugal and the United States rank fairly high). Positive scores on these attitude scales are also associated with aspects of schooling outside the content of the formal curriculum, for example, positive contacts between immigrants and non-immigrants in the school setting and opportunities in the classroom to discuss issues on which individuals have different points of view (Torney-Purta et al. 2008).

Country means on these attitude scales from the study of upper secondary students in a smaller number of countries (Amadeo et al. 2002) show almost identical results. Furthermore, in the 2009 ICCS study, Finland once again scored significantly below the international average on both the norms of conventional citizenship and on social movement-related citizenship scales (Schulz et al. 2010).

The guidelines developed by the CIVED national representatives and other advisory groups in this subject area have always distinguished between different types of participation (here the more and the less conventionally political). Furthermore, young people’s attitudes toward immigrant groups and ethnic groups have been of nearly as much interest as cognitive outcomes in many countries. This is especially true because the testing of school-based samples allows the analysis of factors associated with positive or negative attitudes (for example, the proportion of immigrant students in the school or the extent to which intergroup relations or community issues are seen as an appropriate topic for discussion). In short, the ability to analyze noncognitive dimensions (including attitudes) has been especially important to educators who teach in increasingly diverse communities and in policy debates on how to prepare young people who will seek employment in increasingly diverse work settings.

Person-Centered Approaches to Understanding Patterns of Civic Attitudes

A second approach used in the CIVED analysis with potential for dealing with multiple dimensions is the person-centered approach to analysis. This contrasts with the variables-centered approach. Person-centered analysis has a relatively long history in developmental psychology, especially exemplified in the work of Swedish psychologists (Bergman et al. 2003; Mahoney et al. 2001). Person-centered analysis (in this case K-mean cluster analysis) is especially useful for large-scale studies where there are multidimensional outcomes. Instead of looking at mean differences on variables, in this approach one looks for clusters or groups of persons who have similar patterns or profiles of attitudes.

In this CIVED analysis about 30,000 students in ten countries were clustered using their responses on 12 attitudinal scales. One cluster analysis included the United States, Australia, and three Western Europe countries; the other analysis included five countries in Eastern Europe.² For details of the analysis and the results see Torney-Purta (2009) and Torney-Purta and Barber (2011). The purpose was to identify distinct clusters of individuals who differed in systematic ways in their civic and social attitudes. To decide the cluster names we looked at the pattern of means on the 12 attitudinal scales of each cluster group in comparison to those of other cluster groups. We also have suggested a label for each cluster in the form of a “motto” that expresses the particular characteristics of each profile of attitudes. This makes it easier for a general audience to interpret than designations of proficiency levels, especially when the method used to set the proficiency cut-points is often not transparent and sometimes arbitrary. This labeling of clusters can represent the results of sophisticated analysis of attitudinal data and produces a presentation that can be understood by audiences with little statistical expertise. Below is a description of the five clusters extracted in these ten countries and then the distributions of cluster membership by country.

Adolescents who are found in the Social Justice cluster in these ten countries endorse immigrants’ rights, the rights of minority/ethnic groups, and women’s rights (average of about one standard deviation above the mean). Students in this cluster have relatively low scores on scales measuring belief in the importance of citizens participating in action, either in the conventional political domain or through social action in communities or nongovernmental organizations, however. For the Social Justice cluster, the motto is, “I believe in rights for everyone but do not feel obligated to do much about it.”

Adolescents in the Conventionally Oriented cluster in these countries show high levels of trust in governmental institutions and are patriotic (both in the sense of having positive national feelings and protectionist attitudes toward their nation). Adolescents in this cluster have high levels of political self-efficacy and believe that adults should be active in socially oriented activities (e.g., volunteering to help the community or joining human rights organizations) as well as in conventional political activities such as voting. In Australia, the United States, and the three Western Europe countries, members of this cluster also have relatively high social justice attitudes. In describing this cluster for these countries (but not for the Eastern European countries), it is appropriate to use the term Conventional/Inclusive. For the Conventional cluster across the ten countries an appropriate motto is, “I believe in my country and will support the status quo with political and civic actions that are expected of me.”

The Indifferent cluster in both regions contains individuals who have attitudes very close to the mean on the large majority of the attitudinal scales. Adolescents in the Indifferent cluster are willing to do the minimum as citizens. They are inclined to obey the law and may vote, but there are many nonpolitical activities that interest them more. In the Nordic countries, they may correspond in some respect to the “stand-by citizens” identified by Amnå and Zetterberg (2010). The Disaffected cluster is similar to the Indifferent cluster but with more negative beliefs about norms of citizenship related to both conventional political activities and involvement in the community. The Indifferent and the Disaffected clusters can be described by the same motto: “I have better things to do with my time than be active in politics, but I won’t do anything rash.”

The fifth cluster in these ten countries shows an Alienated profile including negative attitudes almost uniformly across the scales. For this cluster group, scores on trust in government averaged between one and two standard deviations below the international mean. Their attitudes toward rights of immigrants, minorities, and women were also extremely negative when compared with those of students in the other clusters.

The proportion of students in this Alienated cluster who think it is “not important for citizens to obey the law” ranges from more than 30 % in Australia and the United States, to 23 % in Sweden, 16 % in Finland and 10 % in England. This compares with between 1 and 3 % of students in the Social Justice, Conventional and Indifferent clusters who do not believe in obeying the law. Willingness to protest by actions such as occupying buildings or blocking traffic were common only among the Alienated cluster, not among those supporting social justice, for example. The motto for the Alienated cluster is, “I’m angry about the immigrants and minority groups in my country, and I don’t trust the government. I have the right to do what I want.” The remainder of this section will devote attention to clusters of students in Australia, the United States, and the three Western European countries, with special attention to the alienated group of students.

This clustering suggested that in 1999 there was already a significant amount of anti-immigrant feeling and xenophobia among adolescents in Australia, the United States, and several European countries and that this group seemed willing to act on these attitudes in ways that may be against the law. This age cohort turned 26 years old in 2011. These findings shed some light on recent increases in the strength of anti-immigrant feeling among adults in these countries (Vertovec and Wessendorf 2010) and on recent instances of violent action against immigrants or those perceived to support immigrants’ rights.

Civic knowledge scores are highest among the Social Justice and Conventional clusters (Table 6.2). In the United States and Australia, Indifferent students have civic knowledge scores that are quite similar to Conventional cluster group members. In England, Finland and Sweden the Indifferent students have lower average knowledge scores than the Conventional students. In all countries the students in the Alienated cluster have the lowest knowledge scores.

Table 6.2 Average total civic knowledge score in each cluster (by country)

Distributions of the clusters in the CIVED data across the Western European countries (Fig. 6.1) shows that the most prevalent pattern is Disaffected, while about 7 % of the students are Alienated. In the United States there are approximately equal numbers of students in the Conventional/Inclusive and the Disaffected group, while about 7 % are Alienated. A study 20 years ago of attitudinal clusters among adults in the United States also found substantial proportions in categories such as Disaffected, Bystanders, and Followers (Ornstein et al. 1988).

Fig. 6.1

Distributions of cluster membership in five countries

The second most prevalent cluster among English and Swedish students (after Disaffected) is about a quarter of the students in the Social Justice group. Remember, however, that these students are not particularly interested in taking action in support of rights and justice. In Australia and Finland, in comparison, Disaffected and Indifferent students are frequent. Across all five countries focused on here, about 7–8 % of the students show the alienated pattern of negative attitudes toward immigrant and ethnic groups and also low trust in government institutions. The country patterns of clusters within the post-Communist countries show Disaffected and Indifferent students are a majority. The proportion of student in the Alienated cluster ranges from 7 % in the Czech Republic to 20 % in Bulgaria.

To summarize this secondary analysis of CIVED, there are many students who are positively disposed toward social justice (nearly half in the United States and more than 30 % in England and Sweden in the social justice or conventional/inclusive clusters). It is sobering to note, however, how few students seem motivated by positive social justice attitudes to join relevant organizations or participate in non-violent protests. Furthermore, in all ten of these countries substantial proportions of the adolescents surveyed are indifferent, disaffected, or alienated. The alienated group is a small but worrisome minority in both new and old democracies.

The school appears to have a role in reducing the likelihood of belonging to the cluster characterized by anti-immigrant attitudes and lack of trust in institutions (the Alienated Cluster). The IEA CIVED Study contained two school-based measures of the climate in school, one focused on adolescents’ sense that they can have an effect in the school setting (Confidence in the Value of School Participation) and the other measuring adolescents’ sense that their classrooms are places where respectful discussion of different opinions takes place (Openness of Classroom Climate for Discussion). Those adolescents who report that their school climate and classroom climate are positive are less likely to belong to the Alienated cluster. These results (from a logistic regression) are parallel in the two regions (Torney-Purta 2009 and Torney-Purta and Barber 2011).

In summary, after extensive variable-oriented analysis of the CIVED data (modeled on the ILSAs in other subject areas) this exploratory analysis attempted to identify groups of adolescents sharing similar attitudes and provided several insights. Adopting a cluster-based analytic strategy has theoretical roots and also a practical advantage for presentations to nonacademic audiences. Identifying profiles that characterize individuals within and across countries aids in interpreting the information gained from cross-national summary statistics. When it is possible to see a cluster of individual adolescents who remind them of young people they know, adults are much more likely to understand the strengths and weaknesses found in patterns of civic engagement than when they are told only about averages and statistical trends. Particularly when there is a problematic group, such as the alienated adolescents, it is important for educators and policymakers to know the extent of its membership and nature of their views.

Cluster analysis could be conducted with rating scales in ILSAs other than CIVED. Using the ICCS data from 2009, it would be informative to cluster individuals according to their responses on the items dealing with national identity, identity within Europe, and global identity. How many students identify with none of these groups, for example? Or how many are strongly nationalistic and lack a sense of membership in other levels of the system? This would be more informative for policymakers than reporting that the mean score on a national identity item is higher than on a European identity item (Kerr et al. 2010). Likewise, the clustering approach could be helpful in suggesting how young people relate to their communities and to associational groups within and outside of school. This could be more informative than comparing average rates of participation (Schulz et al. 2010). Finally, if psychometrically strong scales on classroom environment were included in all large-scale comparative studies it would be possible to cluster these perceptions of learning environments across subject areas (as Tapola and Niemivirta 2008, have done in their study of Finnish students).

In summary, person-centered analysis of data from civic engagement studies has the potential to provide researchers, educators, and policymakers with a wide range of information on students’ knowledge, attitudes, and actions. Cognitive diagnostic modeling enables us to better understand profiles of students’ learning in the cognitive domain of civic education with implications for classroom practice. Cognitive diagnostic modeling was facilitated in CIVED because every student answered all the items. The cluster analysis was facilitated because of the range of reliably scaled attitudinal items included in the CIVED instrument. This person-centric cluster analyses along multiple dimensions allowed us to examine trends and patterns both within and across several countries.

Studies in Civic Engagement as Opportunities to Study Contexts

International studies contribute to a variety of discourses and at a variety of levels. For example, studies producing country rankings in mathematics and science have contributed to debates about entire countries and their potential economic competitiveness as the next generation comes of age. Citing poor student performance on international tests is often a way to stimulate interest in educational reform. The results of international comparative studies can contribute much more than this, however. Studies in civic education provide evidence of this.

National and School Level Contexts in Relation to Civic Knowledge and Attitudes

What does it mean to unpack international comparative results at different levels (vom Hau 2009)? What characteristics of countries and their macro level policies are related to their strong or weak performances? Our first insights about the possibilities of looking at specific aspects of national context came when examining CIVED item responses from Chile, Colombia, Portugal and the United States for a report to the Organization of American States (Torney-Purta and Amadeo 2004). These countries showed strong support for several aspects of citizens’ action. However, in addition to attitudinal scales, the CIVED survey included a number of questions such as the following: “Is it good or bad for democracy when citizens have the right to elect leaders freely” or “when everyone has the right to express opinions?” There was little variation across the 28 countries in agreement with these positive statements. More interesting were the responses to more contentious statements like these: “Is it good or bad for democracy when courts and judges are influenced by politicians” or “when wealthy business people have more influence on the government than others?” Students in Chile (and to some extent in Portugal) were less likely than those in other countries to see these conditions as bad for democracy (Torney-Purta and Amadeo 2004). This raised the possibility that experiencing a dictatorship in the country’s recent past might be associated with young people being less alert to possible threats to democracy, or that some of the problematic characteristics of such governments are difficult to eradicate even after a democracy is established.

Once we began using HLM (Hierarchical Linear Modeling), a new set of possibilities opened for analyzing characteristics of countries in relation to young people’s responses contained in the CIVED dataset. This type of secondary analysis would be appropriate for ICCS and other ILSAs as well. Analytic methods such as hierarchical linear modeling analysis are well designed for IEA data because of the nested nature of the sample (students sampled within schools). These methods are especially useful for looking at how countries’ contexts and practices influence achievement or attitudes. Torney-Purta et al. (2008) examined knowledge, support, and practice of human rights among adolescents in relation to national policies and conditions. As the previous section indicated, the CIVED study has data from 14 year olds drawn from nationally representative samples of schools. We collected several pieces of information from other databases about citizenship and human rights policies at the country level: first, the extent to which the country referred to human rights in its international discourse on education (Suarez and Ramirez 2007) and second, its ratings by the Freedom House (a New York-based organization that serves as an independent watchdog on issues of human rights) indicating support for civil rights. These were available in 27 of the 28 countries (not Hong Kong).

The outcome variables at the student level included responses on two CIVED knowledge items that dealt explicitly with international human rights and on 36 other items dealing with other civic topics. We also used scores for each student on support for the norms of social movement-related citizenship and positive attitudes toward immigrants’ rights. In addition we had information from students about their home literacy resources, about the extent to which the classroom climate was open for respectful discussion of different opinions, whether the student believed that student participation made a difference in school, how often the teacher discussed international issues, and how often the student read international news (this could include online reading, though that was not frequent in 1999) (Amadeo 2007).

We looked at two analyses, one of knowledge (two items) and one of attitudes (three scales). These were carefully controlled HLM analyses (Torney-Purta et al. 2008). Here we present results for one knowledge item and two attitude scales, first looking at the country level. For knowledge, students who were more likely to correctly answer the question about the U.N. Declaration on the Rights of the Child than one would predict on the basis of their overall civic knowledge were especially likely to come from countries where the government frequently referred to human rights in intergovernmental discourse (Suarez and Ramirez’s count of mentions of human rights in governments’ submission to the International Bureau of Education published in 2007). We also looked at individual level predictors and found that those who correctly answered the children’s rights question were more likely to read international news and more likely to have experience with student democracy at school.

For attitudes we found that accurate knowledge demonstrated in answering the item about the Declaration of the Rights of the Child was a positive predictor of both positive attitudes toward immigrants’ rights and of the belief that citizens should be active in social movement organizations (such as environmental or human rights groups). Reading the international news, confidence in student democratic participation at school, and an open climate for discussion in the classroom were also positively related to immigrant rights attitudes and social movement support at the individual level. Home literacy resources were not significantly related to either attitude scale. We did not find country-level policy effects on these two attitude scales.

This analysis is a first step in unpacking aspects of the country and school level contexts, in particular what it means for a country to have a favorable climate for teaching about human rights and what it means for a school to give students an everyday experience of democracy, embodied in an open climate for classroom discussion and opportunities for students to form groups to take action on school problems.

Similar analysis of CIVED data has been undertaken of support for immigrants’ rights with country level predictors such as policies regarding how many years an immigrant must wait before applying for citizenship. Intergroup attitudes are an especially important area for analysis in depth because, as the previous section showed, there are substantial proportions of young people characterized by a pattern of negative attitudes. Further, the sense of national identity seems to be based on exclusion of ethnic groups or immigrants for many young people. Finally, there is evidence from the ICCS testing in 2009 that few teachers think that anti-racism education is part of their responsibility (Schulz et al. 2010). In short, there are a number of opportunities for investigating the relation of characteristics of national context to students’ attitudes (see also Amnå 2011).

Classroom and School Contexts in Relation to Civic and Workplace Competencies

Another aim of CIVED secondary analysis has been to unpack the meaning of specific factors within the school context, in particular the climate for respectful discussion in the classroom (see also Hess 2009). Beginning with the first civic education study in the 1970s and continuing with the basic reports from CIVED and from ICCS, open classroom climate for discussion has been a powerful predictor of both knowledge and participation outcomes. For example, across countries, Barber and Torney-Purta (2009) showed the extent to which having an open classroom climate for discussion was especially effective in promoting male students’ support for women’s rights. Few of the other ILSAs have such a thread of common findings about aspects of classroom processes extending across several decades. Trzesniewski et al. (2011) argue in their book about secondary analysis for psychologists that constructs such as these are especially fruitful areas for study.

Another part of recent secondary analysis of CIVED has attempted to understand students’ preparation for the workplace as well as for citizenship. In the United States, this area has recently been called twenty-first century competencies (or noncognitive skills) and includes outcomes such as the ability to understand communications in a variety of media (media literacy), ability to understand the economic system and global issues, skill in cooperating with diverse others, and innovative problem solving. These outcomes fit well into the general theme identified in a recent National Research Council Workshop (2010), which enumerated complex problem solving, self-management, and systems thinking as part of twenty-first century skills.

Educators have expressed concern that it will be several years before tests can be developed to assess these competencies internationally. But we realized that the CIVED instruments administered to students already included many of these outcomes. Beginning in 2008 we looked within the US dataset (and later within other countries) to see what aspects of social studies, history and civics classrooms were associated with the achievement of several twenty-first century workplace competencies.

First, we focused on two dimensions of the students’ perceptions of educational activities within their classrooms. One was the extent to which there was an open and respectful climate for class discussions, measured by a five-item scale including items such as: “the teacher encourages us to discuss issues about which there are different opinions.” A parallel scale, with four items, assessed the extent to which students’ classrooms were characterized by traditional teaching activities such as lectures and textbook use.

Four groups of students were identified. The group above the median on both the open class climate scale and the traditional teaching scale was called the Both group; the group below the median on the open class climate and the traditional teaching scale was called the Neither group. The group above the median on open classroom climate but below the median on the traditional teaching scale was called the Interactive group. The group above the median on the traditional teaching scale but below the median on the open classroom climate scale was called the Lecture group. In the United States although the Neither and Both groups were large (700–850), there were also substantial numbers of students found in the Interactive and Lecture groups (400–550).

Comparing these four education groups on mean levels of workplace competencies is another way to unpack the CIVED findings. Details of the analysis are in Torney-Purta and Wilkenfeld (2009), and Table 6.3 summarizes these findings. There are significant differences between the four educational groups on media literacy skills, with the interactive group the highest and the group who reported receiving neither interactive nor traditional education the lowest (Fig. 6.2). A similar pattern appears for economic knowledge (not shown). For positive attitudes toward ethnic groups, the Interactive group and the Both group are the highest (and not significantly different from each other). A similar pattern appears for a self-efficacy scale (not shown). In summary, for skills and intergroup attitudes, the interactive experience of an open classroom climate for discussion appears to be vital either by itself or in combination with traditional teaching.

Fig. 6.2

Media literacy skills by educational experience

Table 6.3 Twenty-first century competencies: summary of students’ scores based on type of civic education instruction

CIVED also has measures of the kinds of experience students report in learning to understand others who hold different opinions and in cooperative groups. In this set of outcomes we observe a “stair step” pattern. The group with both interactive and traditional experience is the highest, the interactive group the second highest, followed by the lecture and the neither groups. A similar pattern characterized the students’ endorsement of responsibility, for example, willingness to obey the law and work hard (Fig. 6.3).

Fig. 6.3

Norms of responsibility: mean scores by educational experience

It was striking that the interactive and lecture-based experiences were compatible and together led to the achievement of both civic and workplace competencies. On none of the measures was the group that only experienced Lectures (and other traditional activities) superior to the Interactive or to the Both group. In all of the comparisons, the group with Neither kind of experience had the poorest outcomes.

A civic educator who was designing programs for post-Communist countries asked whether the same results could be expected in those contexts. Using the CIVED data from Estonia, Latvia, and the Russian Federation, the same four groups were formed (within each country) based on their perceptions of whether their classrooms had a positive classroom climate for discussion (above the median) or a low climate for discussion (Torney-Purta and Wilkenfeld 2009, 2010). Also these groups were divided according to whether they were above or below the median in reporting that traditional teaching (lecture, textbook use) was characteristic of their classrooms. The results in these Post-Communist countries were almost identical to those in the United States (previously shown in Table 6.3 and Figs. 6.2 and 6.3). Again the group that had only the traditional/lecture-based experience was not superior to the Both or Interactive group on any of the outcomes examined. In other words, interactive classroom climate seems to be effective for civic and workplace outcomes both in the United States and in newer democracies. The group receiving neither type of civic education always had the lowest scores.

There is increasing interest in research about students’ perceptions of the learning conditions and climates of their classrooms. In a study of six countries (four of which overlap with those in the IEA studies), Gorard and Smith (2010) arrive at much the same conclusions as the CIVED study (Torney-Purta et al. 2001; Amadeo et al. 2002) and ICCS (Schulz et al. 2010). For example, students surveyed and interviewed by Gorard and Smith in Italy and England were more likely than those in Belgium (French) to perceive that teachers encouraged them to make up their own mind and respected their opinions even if they disagreed. Students in the Czech Republic were least likely to hold this point of view. Gorard and his colleagues looked broadly at the practice of equity in the classroom. They wanted to know how students themselves experienced injustice, for example, whether they ever felt humiliated by a teacher, whether they felt that less able students or students who put forth less effort deserved extra help. According to these authors, equity can be addressed convincingly only if student perceptions as well as actual socioeconomic gaps are taken into account. The IEA data in civic education include items that would allow this issue to be more fully addressed in secondary analysis.

Both the IEA studies in the civic education area and research conducted outside of IEA suggest that strong scales measuring students’ perceptions of classroom climate and instructional processes ought to be part of ILSAs across subject areas. There is a growing consensus that documenting social gradients of achievement and proposing generalized solutions (such as requiring more qualified teachers to teach in schools with poor students) are unlikely to be successful unless the perspectives of the students about quality and equity in their education are also taken into account.

Conclusions

Should civic and citizenship subjects be included in the cycle of comparative studies? We answer an emphatic yes for several reasons. Because both cognitive and noncognitive variables are assessed, civic education ILSAs can contribute to an in-depth understanding of young people’s current civic knowledge, attitudes, and activities as well as their expectations for the future. Stated another way, civic and citizenship studies go beyond country rankings of achievement scores and examine citizenship from multiple dimensions and viewpoints, providing information that is important to strengthening democracies and national economie. Further, studies in civic education and engagement can provide balance to the predominantly negative view of adolescents in the United States (and elsewhere). For example, in CIVED in 1999, US students performed well on the test of civic knowledge and civic skills. In the person-oriented cluster analysis, the distribution of US students showed a high proportion of students in the Conventional/Inclusive Citizen cluster. These findings counter the view that all adolescents are apathetic, disengaged, and unreliable—negative portrayals of adolescents that can become self-fulfilling prophecies. That is an important lesson for adults (parents, teachers, and politicians), who may underestimate what most young people are capable of as citizens and members of their communities.

Advocacy for a cycle of studies in civic education (or as part of a broader study including noncognitive outcomes in general) and widespread participation in these studies would help to track the extent to which students continue to understand and support democratic principles and practices in times of global change and uncertainty.

Also, there is some evidence that the often discussed (and also sometimes criticized) twenty-first century competencies believed to be necessary in the workforce bear considerable similarity to civic knowledge, attitudes, and skills (an argument supported by analysis in Torney-Purta and Wilkenfeld 2009). Measuring these competencies will help educators and employers know where to focus teaching and teachers’ professional development. We cannot expect young people to use effective workforce strategies and skills if we do not help them learn to do so.

More broadly, if democracies are to survive and thrive, all citizens need to reach a minimum threshold of knowledge and participation in each succeeding generation. Everyone, of course, also needs basic literacy and numeracy; math and science knowledge is essential in everyday life. But relatively few young people go on to mathematics- or science-related careers. Decisions to specialize in these subjects are based on a variety of factors that include but are not limited to cognitive achievement. Some longitudinal studies in the United States in fact show the importance of attitudes developed in the early years of college rather than in high school in solidifying the choice of mathematically related careers (Musu-Gillette 2010). Efforts to prepare citizens begin in middle childhood, should involve every young person, and can be tested reliably by age 14.

In addition, researchers in the civic and citizenship area have been innovative in their modes of testing and have been early adopters of new methodologies of analysis (especially HLM). This area of inquiry (and also the way the measures were designed) is especially well suited to studying the effects of national contexts and also the effects of classroom or school contexts. These studies have a record of contributing to thoughtful unpacking of context issues, for example, the value of participation in civil deliberative discussion. The civic-related studies have provided empirical evidence about controversial but important topics such as attitudes toward equality for women and immigrants.

To give one example relating to the study of contexts, although the World Values Survey and the European Social Survey of adults each deal with attitudes toward immigrants and immigration, these data collections are based on limited sets of questions and do not assess the everyday contexts in which attitudes are acquired and expressed in behavior. Using nationally representative samples of schools, it is feasible to investigate the contexts in which adolescents’ attitudes about immigrants’ rights and civic engagement develop. Imagine the situation if we believed that attitudes toward immigrants were only acquired after the age of 20. In order to examine the context in which these attitudes developed, we would need representative samples of places of work as well as college and universities. Because we know that these attitudes have roots in adolescence, having nationally representative samples of schools (as international comparative studies do) provides a feasible way to study the process.

Ensuring that civic-related studies appear in the cycle of ILSAs at approximately 10-year intervals (either on their own or as part of studies of noncognitive factors) is necessary but not sufficient to capitalize on the strengths of these studies. Greater reflection is also needed, about how we collaborate most effectively to address country differences and policy-related issues, about how we bring the next generation of researchers into the process, about how to look simultaneously at national and international perspectives, and about how to incorporate new methodologies and constructs. This would mean maintaining an innovative edge in the system of international studies by reestablishing a committee, perhaps with foundation support and certainly with interdisciplinary membership and individuals from other nations as well as the United States.

This committee might resemble the Board on International and Comparative Studies in Education (BICSE). Oversight is not needed now (as was BICSE’s role two decades ago). What is needed is a group of scholars and policymakers charged with reflecting on the potential for secondary analysis and innovative measurement of ILSAs of different types and in different subject areas. These studies need to become less expensive to conduct. Better ways to present their results to policy-oriented audiences and journalists (as well as those who are more academically oriented) need to be developed. Suggestions made earlier about the use of cluster analysis and labels that are transparent are relevant here. Further there is a need to articulate the practical implications of these findings to teachers and others who work directly with youth. This would require more attention to both educational processes and contexts than is currently found in reports that focus on country rankings.

A redesigned BICSE could also encourage and suggest support for further secondary analysis, especially of projects like ICCS. This is needed for several reasons. The reports issued by IEA are inherently complex and are not always easy for those outside the projects to interpret. The IEA organization is not allowed by its policies to make recommendations about educational reform to its member countries. Many researchers lack the resources to use the existing data fully to investigate important hypotheses having to do, for example, with process, with equity, and with context (topics to which the CIVED study has made particular contributions).

When ILSAs have been completed, they should be examined regarding their implications for future design issues. For example, ICCS developed a cognitive test of civics that had a great deal more introductory reading material than was used in CIVED. Females outperformed males on the ICCS cognitive test, perhaps because females tend to be better readers than males. The advisability of retaining or changing this approach to cognitive civics measure in future cycles should be examined. Further investigation of a topic such as intolerance or anti-immigrant feeling, measured in both CIVED and ICCS, could mobilize the interest of many researchers from a cross-section of disciplines and could suggest the collection of additional information to aid in the interpretation of results (Barber and Torney-Purta 2012). Examples could be found in other subject matter assessments as well.

Finally, classroom process measures with both common items and items tailored to specific subjects could be developed based on the successful model in the civic studies. Mixed methods, for example, case studies and videos that are integrated and appropriately sequenced into large-scale data collections, would make enormous contributions.

In short, civic-related subjects should remain in the regular cycle of ILSA studies. CIVED and ICCS are not optional niche studies but central in showing an appropriately multidimensional picture of adolescents and their preparation for adulthood at a time when the large proportion are still in school. These studies have the potential to help policymakers understand the development of civic-related skills and attitudes (both positive and negative). For example, secondary analysis of the CIVED uncovered the roots in adolescence of xenophobic attitudes of 14-year-olds and a small group of deeply alienated youth. Studies of adults’ attitudes in this area such as the World Values Survey can never give the rich contextual information about the everyday settings in which intergroup relations occur that is available in studies sampling schools, as ILSAs in education do. Opportunities for extensive interactive civil discussions of issues about which people disagree have been shown to have a positive role to play. One of the first places that most young people encounter opportunities for such discussion is in schools (Hess 2009).

In conclusion, the profile and contributions of large-scale international studies in education would be seriously diminished if assessments in fields such as civic education, civic learning, and civic engagement were no longer included. These studies make contributions at the macro level (to a fuller picture of countries’ education system), and at the micro level (to understanding classroom climate and practices). Studies in this area meet a number of the criteria set out by Bogenschneider and Corbett (2010) in their discussion of ways to further evidence-based policy. Civic-related research is salient to the democratic process to which elected officials and those who work with them are committed. It is possible to make the results in this area accessible in everyday language and transparent in a way that complex statistical presentations of educational data often are not. Many policymakers or members of their staffs are ready to hear information that counteracts the superficial analysis of country rankings that often appears in the media, and they are interested in innovative ways of looking at educational achievement that unpacks its implications for educators.