Keywords

1 Introduction to Science-Related Noncognitive Outcomes and Their Importance for Lifelong Learning Processes

What are the important goals of science education? As science provides the most profound explanations that we have about our material world, this knowledge is one of the major cultural achievements of modern societies (Kind and Osborne in press). Science affects everybody—in everyday as well as in professional life (Bybee 1997; Millar and Osborne 1998). However, besides acquiring a knowledge about science, students also need to come to recognize the importance and significance of science for their daily life and for society, in order to engage with and address the political and moral dilemmas posed by issues such as environmental deterioration, or the need to deal with rapidly advancing technologies. Hence, students need to begin to value science, develop an interest in science, and establish positive science-related self-views. These so-called noncognitive outcomes are of increasing importance for educational policies and labor markets because they influence not only scholastic performance, but also career decisions (e.g., Parker et al. 2014), educational attainment, and labor market success (e.g., Heckman et al. 2006; Almlund et al. 2011).

Heckman and colleagues broadly define noncognitive outcomes as “personal attributes not thought to be measured by IQ tests or achievement tests” (Kautz et al. 2014, p. 13). However, the term “noncognitive” potentially demarcates the difference between cognitive and noncognitive outcomes in a way that is not valid, as many of the so-called “noncognitive outcomes” are not devoid of cognition. Hence, sometimes other terms are used, such as character skills, soft skills, life skills, twenty-first century skills, or socio-emotional skills.

In the research tradition of science education, these concepts are mostly summarized under the term attitudes. Attitudes are an individual’s affective, cognitive, and behavioral reactions towards an object or phenomenon (Rosenberg and Hovland 1960). They can be differentiated into attitudes towards science and scientific attitudes (Gardner 1975; Klopfer 1971; Osborne et al. 2003). However, to date, for both facets, there is no consensus about how many sub-constructs exist, how these can be classified, or how they can be labeled and interpreted (see Kerr and Murphy 2012). They include constructs such as interest in and enjoyment of science, perceived value of science, or attitudes of peers and friends towards science (see also Tytler and Osborne 2012).

Attitudes influence whether students will actively and of their own accord engage in situations where science competencies are necessary. Hence, they are closely associated with science knowledge (i.e., content, procedural, and epistemic knowledge; OECD 2016). However, this relationship is reciprocal: Attitudes can be a consequence of science performance as well as affecting science performance (e.g., Köller et al. 2000). When students are interested in science, and feel positive about their competencies, they can be more confident when they deal with science topics and may use more effective learning strategies (Schneider et al. 2015). These can, in turn, facilitate learning gains. On the other hand, students’ knowledge about science influences whether they feel confident about and become interested in science. For example, a student who repeatedly experiences failure in science will probably not enjoy sciences and may not see the personal value of science for him or herself.

Independent of their relationship with science knowledge, attitudes are also important stand-alone outcomes. They shape the identity and personality of students (e.g., Krapp and Prenzel 2011) and influence decisions about selecting science-specific courses at school, science-related studies, or jobs (e.g., Parker et al. 2014). Dedicated and engaged science learners are more likely to choose science courses at school and to pursue careers in science (Bøe 2012; OECD 2007; Renninger et al. 2015). Such an outcome is of high relevance because the shortage of skilled workers in specific technical and science professions such as engineering and physical science—especially among females—has become a concern in recent years, and is expected to worsen in the future (European Commission 2006; OECD 2008; National Center for Education Statistics 2009).

However, students with high levels of science competencies may additionally show high levels of competencies in mathematics and reading (OECD 2009). Thus, they are able to study a wider range of subjects and have a wider choice of job opportunities. For example, Wang et al. (2013) showed that students with high math/high verbal skills were less likely to pursue a STEM (science, technology, engineering, and mathematics) career than were students with high math but only moderate verbal skills. In addition, this relationship was moderated by students’ self-concept, which is an important noncognitive outcome. Hence, these outcomes can tip the scales in decisions about future educational pathways. Unfortunately, the Programme for International Student Assessment (PISA) has shown that in some countries, top performers in science show comparatively low levels of interest and future-oriented motivation to engage in science (OECD 2009; Drechsel et al. 2011). Moreover, positive science attitudes decline over the course of school life (e.g., Breakwell and Beardsell 1992).

In sum, educational systems are confronted with two challenges. First, how can they support students to become enthusiastic about science in order to foster high levels of science literacy and participation in everyday as well as in professional life? Second, how can particularly talented students who show high levels of science competencies be inspired and supported in order to ensure the next generation of STEM specialists? Moreover, it is not only policy makers who have become aware of the fact that attitudes are instrumental for both personal and societal growth. In some countries, the public has also started to discuss the importance of noncognitive outcomes for a fulfilling life, participation in the labour market, and a society that values more than achievement (e.g., Aktionsrat Bildung 2015).

To summarize, science education pursues multiple educational goals: It should foster science achievement AND science-related attitudes—something which is reflected in the framework for PISA 2015 (see Fig. 12.1) which incorporates both facets. Furthermore, it differentiates science attitudes into more domain-general attitudes towards science and scientific attitudes, and attitudes towards domain-specific science topics. Moreover, attitudes also include the social meaning of a science domain and aspirations towards science. As Fig. 12.1 indicates, both outcomes are shaped by encountering science at school and by out-of-school experiences (see Kuger 2016, Chap. 16, this volume; Müller et al. 2016, Chap. 17, this volume). Such out-of-school experiences also include daily encounters with families, peers, and the media. Fostering multidimensional goals for science education promotes a broad science education for everyone, and is also more likely to engage those students who will become the next generation of scientists. In particular, for decisions about educational and professional career choices, science attitudes may be even more important than prior science performance (indicated by a dotted line from achievement to career choices; e.g., Renninger et al. 2015).

Fig. 12.1
figure 1

Framework model for science attitudes and their relationship with other variables in PISA 2015

Full size image

In some countries, the importance of multidimensional goals has already been acknowledged and explicitly stated in educational acts and school curricula (e.g., Aktionsrat Bildung 2015; European Commission 2011). Notably, the international PISA Science Framework also defines scientific literacy on the basis of cognitive outcomes and attitudes (OECD 2016).

In what follows, the goal is to identify and discuss the important science-related attitudes that would need to be assessed in a large-scale assessment study such as PISA. We will provide an overview of the current state of research with regard to science-related outcomes, in order to introduce a framework model of how this topic could be defined in the PISA 2015 field trial (Fig. 12.1). We will identify important indicators that are crucial to inform policy, research, and practice and that, therefore, would need to be assessed in an ideal study. To do this, we will distinguish between general attitudes towards science, scientific attitudes, domain-specific attitudes towards technology and the environment, and discuss the social meaning students attribute to science, and their career aspirations.

For each of the constructs we briefly explain the theoretical background and refer to the corresponding literature. In addition, we discuss the relationship with science performance, whether the outcomes are stand-alone in their own right, and whether they affect future engagement in science. Also, the relationships with other science attitudes are addressed. Where possible, we also refer to results from earlier PISA cycles.

However, when selecting a set of indicators for a large-scale assessment such as PISA, there are some limitations and selection criteria that arise from the characteristic features of international large-scale studies of students at the end of compulsory school. These criteria and limitations also guide the selection process and thus, are discussed in this chapter (see also Kuger et al. 2016, Chap. 4, this volume). Moreover, we describe how previous international large-scale assessments (ILSAs) have covered the topic of science-related attitudes. At the end of the chapter, based on the research background, selection criteria, limitations, and the coverage in other large-scale studies, we identify and present relevant constructs that were realized for this topic in the PISA 2015 field trial.Footnote 1

2 Attitudes Towards Science

“Attitudes towards science” refers to the affects, beliefs, and values that students hold about an object such as school science, specific science topics, the impact of science on their daily lives, on society, or scientists themselves (Tytler and Osborne 2012). Attitudes embody different psychological concepts with diverse theoretical backgrounds, such as emotional and motivational orientations, self-related cognitions, strategies, and value beliefs, which are discussed separately below.

2.1 Emotional Orientations

Based on the control-value theory of achievement emotions (Pekrun 2006), achievement emotions are emotions that are directly linked to achievement activities or outcomes. They influence students’ general effort, learning and performance in the classroom, as well as their willingness to engage in science in particular. Achievement emotions can be further classified with regard to their valence (positive vs. negative) and degree of activation (activating vs. deactivating). At a proximal level, the experience of specific emotions is influenced by the feelings of perceived control (i.e., control appraisals) and perceived importance of the activity or outcome (i.e., value appraisals).

Students with positive emotions such as enjoyment may develop a more stable disposition to interest in science, and use more elaborated metacognitive strategies, whereas students with negative emotions, such as anxiety, may avoid engagement in science (Pekrun and Linnenbrink-Garcia 2014). To date, enjoyment and anxiety have been the achievement emotions studied most intensely.

Enjoyment of science reflects students’ attachment to learning science and experiencing it as a meaningful activity (Laukenmann et al. 2003). As a result, students better regulate their learning and solve problems more creatively (Pekrun et al. 2002). It is closely related to interest in science (Krapp and Prenzel 2011), develops out of previous experiences in learning science, and influences expectations about future science experiences, as well as students’ participation in science activities (Ainley and Ainley 2011a, b). PISA 2006 showed that, in the majority of countries, and without gender differences, students enjoyed learning science. In addition, in most countries, enjoyment was positively related to achievement. Only some countries showed a negative association between enjoyment and science performance (OECD 2007).

Anxiety about science describes a fearful emotional state about science achievement activities or outcomes. Therefore, it is related to poorer performance (Gungor et al. 2007) and differs between boys and girls, with girls being more anxious about science than are boys (see Mallow et al. 2010). It can draw students’ attention away from focusing on a task and thus, can undermine their self-regulation and learning (Pekrun 2014). In addition, anxiety can be content-specific. For chemistry, Eddy (2000) found that, besides learning chemical equations and taking final exams, students showed high levels of anxiety with regard to having adequate laboratory skills (handling chemicals). Moreover, they were anxious about the potentially harmful effects of the chemicals.

2.2 Motivational Orientations

Closely associated with emotions are motivational orientations. They can be classified into more intrinsic (e.g., interest), or instrumental forms of motivation. These orientations shape the personality of students and foster their willingness to engage in science, over and above the things students must do (Krapp and Prenzel 2011). Hence, they are important stand-alone outcomes and affect students’ course selections and career choices.

Interest in science is a multidimensional construct that is composed of acognitive, affective, and value-related component (Krapp and Prenzel 2011). According to the person-object theory of interest (Krapp 2002a, b), interest arises when people establish a relationship with a specific science object. Interest-based activities are characterized as self-intentional and thus, they have intrinsic qualities (see Rheinberg 2008 for different conceptualizations of intrinsic motivation). In addition, interest has an epistemic component: It is directed towards knowing and learning more, in order to get a deeper understanding of the object (Krapp and Prenzel 2011). Interests can refer to a certain disposition of a person (i.e., traits) or to current engagements with an object of interest (i.e., states).

Highly interested students acquire more new domain-specific as well as meta-cognitive knowledge (Krapp and Prenzel 2011). Analysis of data from PISA 2006 showed that female students and students from non-European countries reported higher interest in living/health topics. In contrast, boys and students from European/Western countries reported higher interest in physical/technology topics (Drechsel et al. 2011; Olsen and Lie 2011). At the individual level, for the majority of countries, students who reported a high interest in science also performed better in science. However, at the country level, students in high-performing countries showed relatively low levels of interest (Bybee and McCrae 2011; see also Sjøberg and Schreiner 2007; Shen and Tam 2008 for similar results).

Instrumental motivation to learn science is directly related to the desired outcomes of specific actions and its consequences, rather than to the learning activity itself. It refers to whether students think learning science is useful for their later life. According to self-determination theory (Deci and Ryan 1985, 2002), students sometimes need to internalize and integrate the external demands of school, teachers, and parents into their own values. If they do this successfully, no external control will be necessary (i.e., autonomous motivation). Nevertheless, activities will be accomplished because of their instrumental value, rather than for the intrinsic enjoyment of the activity itself. The theory distinguishes four developmental forms of internalization that vary in their degree of autonomy and the perceived locus of causality (external vs. internal): External regulation, introjection, identification, and integration (see Ryan and Deci 2000 for a more detailed description of these forms). Instrumental motivation also shows some similarities with the construct of utility values in Eccles’ Expectancy Value Model (1983, 2011).

PISA 2006 showed that students perceived science to be useful for themselves, their future work, and career prospects (OECD 2007). Students from more-advantaged socio-economic backgrounds showed higher instrumental motivation compared to students from more disadvantaged backgrounds. However, the relationship of instrumental motivation with achievement at the country level was mixed—some countries showed a positive, others a negative relationship. However, in longitudinal studies, instrumental motivation was positively related to performance; moreover, it has been found to be one of the most important predictors of course selection and career choices (Eccles 1994; Eccles and Wigfield 1995; Wigfield et al. 1998).

2.3 Self-Related Cognitions

One of the general educational goals of attending school is to establish positive, realistic self-views and confidence in one’s own abilities. Regardless of their reciprocal relationship with achievement, positive self-related cognitions are desirable outcomes of education themselves (Helmke and van Aken 1995). They are related to goal setting and strategy use (Zimmerman 1999) and to university entry and course selections (Parker et al. 2014).

Science self-concept means to what extent students in general believe in their academic abilities in science. It can be conceptualized as a uni- or multi-dimensional construct, depending on how science (integrated vs. different subjects) is taught at school (Jansen et al. 2014). According to the internal/external frame of reference model (Marsh 2007; see Jansen et al. 2015a for an application to the science domain), on one hand, self-concepts develop through social comparisons of one’s achievement with the achievement of other students (external frame). On the other hand, science self-concept is influenced by comparisons with one’s achievement in other domains (internal frame) that are distinct from science, such as the verbal domain (contrast effect), or similar—such as mathematics (assimilation effect). However, although a small assimilation effect exists—mathematics abilities are particularly related to a higher physics self-concept—the contrast effect is more important for the development of science-self concepts (Jansen et al. 2015b). It has been found that, for example, achievement in German language classes is negatively related to self-concept in physics.

Science self-concept is related to future-oriented motivation (Jansen et al. 2015a) and predicts undertaking studies in STEM fields after school (Parker et al. 2014). In PISA 2006, many students reported a high positive self-concept, with only small to moderate gender differences (OECD 2007). However, further analyses showed that female high performers in science had a lower self-concept in science than did their male peers (Buccheri et al. 2011).

Science self-efficacy refers to subjective beliefs in one’s own capabilities to successfully accomplish specific tasks in science (Bandura 1977). In contrast to self-concept, it is less stable and more context- and content-specific (Bong and Skaalvik 2003). However, both are moderately correlated with each other (e.g., Bong et al. 2012; Jansen et al. 2015b). Four major sources of self-efficacy can be distinguished (Bandura 1997): Previous mastery experiences with similar tasks (“If I did it before, I can do it again.”), vicarious experiences (“If someone else can do it, I may also be able to do it.”), social persuasion (“If my friends think I can do it, I’m confident that I can do it myself.”), and physiological arousal.

Self-efficacy is also related to inquiry-based learning opportunities (Jansen et al. 2015a) and to activity choices during learning processes, efforts, perseverance, goal orientations, goal setting, and achievement (e.g., Bandura 1993; Pajares et al. 2000). Research suggests that students who do not believe that they have the skills to solve a problem generally show lesser effort while solving it and, in particular, when they face difficulties. As a self-fulfilling prophecy, the probability of solving the task declines, which negatively predicts future achievement, and in turn negatively affects subsequent self-efficacy (Parker et al. 2014; see also Valentine et al. 2004).

PISA 2006 showed that students felt most confident in explaining why earthquakes occur more frequently in some areas than in others. Students felt least confident, however, in discussing how new evidence could lead to a change of understanding about the possibility of life in Mars (OECD 2007). Within each country, self-efficacy was positively associated with students’ achievement (see also Parker et al. 2014 for longitudinal effects).

2.4 Self-Regulation Strategies

Self-regulated learning refers to the self-generated thoughts, affects, and actions that students systematically orient towards attaining their goals (Zimmerman 1989). Based on the three-layered model of self-regulated learning (Boekaerts 1999), it is a dynamic interplay between cognitive strategies, metacognitive strategies, and motivational-volitional control (the latter refers to motivational orientations and perseverance/self-control, which are described in Sect. 12.2.2 and Bertling et al. 2016; Chap. 10, in this volume). These strategies develop in relation to specific tasks that are not necessarily generalizable across domains (Zohar and Barzilai 2013). They are important stand-alone outcomes, as well as a prerequisite for successful science performance, as they support the transfer of knowledge to new contexts and tasks.

Cognitive strategies of information processing and the knowledge about their value are the basis for enabling students to shape their own learning actively. They can be distinguished according to their depth of information processing, into more surface-oriented strategies (e.g., rehearsal/memorization) or strategies that are oriented towards a deeper understanding of the learning content (e.g., elaboration, organization; Mandl and Friedrich 2006). Memorization refers to learning key terms, repeat learning of material etc., in order to save the information in long-term memory. Elaboration means to connect new information to existing knowledge structures, in order to facilitate retrieval. Organization is related to constructing internal learning content relations.

Students should be equipped with different processing modes that they can choose from, depending on the learning situation. However, students are not necessarily aware of the cognitive strategies they use. This may be one reason why the relation between frequency of use and outcomes such as competence or motivation, are rather weak (Boekaerts 1999).

Metacognitive strategies have been broadly defined as reflecting on one’s own thinking (Flavell 1979). Although classifications of metacognition differ (Artelt and Neuenhaus 2010; Zohar and Barzilai 2013), most of them distinguish between two major components: Declarative metacognitive knowledge and procedural metacognition. Declarative metacognitive knowledge refers to verbalized knowledge about prerequisites for and processes of understanding, remembering, and learning required by individuals, and the tasks and strategies they deploy (Efklides 2008). Procedural metacognition can be further divided into metacognitive skills and metacognitive experiences. Metacognitive skills are activities of cognitive regulation such as planning, monitoring, evaluating (Veenman 2011; Flavell et al. 2002), and controlling (Whitebread et al. 2009). Metacognitive experiences are cognitive and affective perceptions in specific problem-solving situations such as puzzlement or “aha moments” (Efklides 2006).

In the case of learning science, metacognitive strategies have mostly been studied in the context of specific topics and how they influence conceptual knowledge and inquiry-based learning (Zohar and Barzilai 2013) in particular. In addition, they are mutually related to other attitudes towards science (Akerson and Donnelly 2008). Students’ metacognition can be influenced by teaching practices such as embedding metacognitive prompts—however, research would suggest that teachers themselves seem to have difficulties in switching between teaching subject matter and teaching higher-order thinking, and placing more emphasis on the use of such strategies (Thomas 2012; Zohar and Barzilai 2013).

2.5 Value Beliefs About Science

Value beliefs about science refer to students’ appreciation of the role and contribution of science for understanding phenomena and the constructed world. Students need to learn to value the contributions of scientific and technological development for their personal life, as well as for society as a whole. Value beliefs are particularly related to scientific attitudes. However, not only students’ own value beliefs but also those of their peers and parents, influence their attitudes towards science, their scientific attitudes, and their science performance.

General value of science means appreciating the contribution of science to understanding the natural world and improving social and economic living conditions, independently of how students value science for their personal life. Students should appreciate the contributions of researchers from different societies and cultural backgrounds to the progress of science and technology.

PISA 2006 showed that the majority of students valued science in general without any gender differences (OECD 2007). They agreed that science can improve peoples’ living conditions and that it is valuable to the society. However, a significant proportion of students distinguished between science contributing to technical understanding and productivity on the one hand, and on the other hand a broader concept of science encompassing economic and social benefits, which was valued less. The construct of general value of science was also positively related to science performance (OECD 2007).

An additional indicator of the general value of science is the prestige of STEM-related occupations. Occupational prestige refers to the perceived social value of an occupation relative to others and thus, it reflects perceptions of labor market success (Treiman 1977). Prestige can stem from different sources, such as power and privilege, social distinction and recognition, or income and education (Zhou 2005). For example, the International Standard Classification of Occupations (ISCO-08; International Labour Office 2012) distinguishes the prestige of occupations on the basis of their requisite skill level and skill specialization.

Personal values of science are fundamental antecedents of emotional feelings about science, such as enjoyment, motivation to learn science, and motivation for a long-term engagement in science (Heckhausen 1991). With regard to the person-object theory of interest (Krapp 2002a, b), they can be conceptualized as the value-related valence of an object of interest that is particularly important for students’ identification with the object (see Sect. 12.4.2).

Personal value beliefs are related to greater intentions to engage in science activities (Pekrun 2000) and to better science performance (OECD 2007). PISA 2006 showed in addition that personal values are distinct from general values of science, and that students do not necessarily relate general values to their own life (OECD 2007).

2.6 Attitudes of Parents and Peers Towards Science

Not only students’ own attitudes, but also those of their parents and peers, are related to their scientific attitudes and their science performance.

Parents’ attitudes are directly related to their children’s science attitudes (Tenenbaum and Leaper 2003), their educational and occupational science aspirations (e.g., DeWitt et al. 2013) and their science performance (Perera 2014; Sun et al. 2012). In particular, the fathers’ attitudes seem to be related to students’ science performance (Simpson and Oliver 1990). Moreover, parents having a science-related job themselves, is related to stronger science aspirations (DeWitt and Archer 2015).

Peers’ attitudes also shape students’ attitudes, their engagement in science activities, and their career choices (Duncan 1993; Eccles 2011; Patrick et al. 1999). They are particularly relevant during mid-adolescence, when students seek the approval and support of their peers (Furman and Buhrmester 1992). For example, Stake and Nickens (2005) have shown that students’ expectations of becoming a scientist in the future were positively related to having peers who were engaged in and supportive of science activities.

3 Scientific Attitudes

Scientific attitudes refer to how students think about science. They display dispositions to look for material explanations and to being skeptical about many of these explanations. Scientific attitudes are meaningful for valuing empirical evidence as the foundation of beliefs and the knowledge that we have about our material world (Osborne et al. 2003). Developing an informed, in-depth understanding of the notion of science, scientific concepts, and scientific methods is an important goal of science education (e.g., National Research Council, US 2012). In addition, students’ scientific attitudes influence their perception of the learning environment, which in turn affect their learning processes and performance (Hofer and Pintrich 1997).

Valuing scientific approaches to inquiry refers to valuing scientific ways of gathering evidence, the importance of considering alternative ideas, the use of facts and rational explanations, a logical and careful process for drawing conclusions, and communicating with others. It is related to students’ ability to respond critically when confronted with science- and technology-related situations.

Recognizing the value of scientific approaches to inquiry can be independent of using such methods themselves, or being positively disposed towards all aspects of science (OECD 2006). In PISA 2006, students showed strong levels of support for scientific inquiry, which was positively associated with science performance in all countries (OECD 2007).

Epistemological beliefs are beliefs about the nature of knowledge and knowing (Hofer and Pintrich 2002), and are closely related to students’ general values of science and scientific inquiry (Fleener 1996; Hofer and Pintrich 2002). Students with more sophisticated epistemological beliefs, such as regarding knowledge as uncertain and evolving, also use higher-level learning strategies (Yang and Tsai 2012).

Sandoval (2005) distinguishes four aspects of epistemological beliefs students need to understand. First, scientific knowledge is socially constructed by people in cooperation, collaboration, or competition. Hence, it depends on the underlying theories, methods used, and interpretations of the outcomes. Secondly, there are different forms of scientific knowledge (laws, theories, models, hypotheses), which vary in their power to explain the world. Third, scientific knowledge varies in its certainty, and current scientific ideas may change with upcoming new methods or competing ideas. Fourth, scientific methods are diverse.

Dealing with uncertainty and ambiguity—sometimes also referred to as intolerance of uncertainty (Chen and Hong 2010) or the need for cognitive closure (Webster and Kruglanski 1997)—involves student attitudes when they need to deal with unexpected or challenging events or topics that offer only little information about the future development of the situation (Dalbert 1999). In science contexts, these situations are common because of the discursive and evolving nature of producing scientific knowledge.

However, students may differ in how they perceive unexpected and ambiguous situations: They can perceive them as challenges or threats (Dalbert 1999). Depending on this perception, students show different ways of processing information, use different coping strategies, and deal with the situation differently. A positive tendency to deal with uncertainty and ambiguity is related to higher well-being (Thielsch et al. 2015). Moreover, it may also influence students’ ability to handle and integrate conflicting information from different texts (Stadtler et al. 2013).

4 Attitudes Towards Domain-Specific Science Topics

When looking at public discussions and media awareness today, two science-related topics seem to be of particular interest: Fast advancing technologies, and environmental challenges. Hence, students’ attitudes towards these topics attract increasing attention by researchers and policy makers.

4.1 Technology

Technology has always played a major role in the progression of human societies: People use technologies to expand their possibilities, to develop new products, systems, and environments. Hence, promoting interest in and engagement with engineering in able students, is one of the major goals of societies (European Commission 2013). However, although students show high levels of technology acceptance and use of technical devices in their daily lives (“digital natives”), they are not necessarily interested in an increased engagement with technology or in considering it for their future career plans. They use modern technologies in a pragmatic, purposeful, and object-oriented way; how these technologies work is not important to them (Jakobs et al. 2009; Ziefle and Jakobs 2009).

Attitudes towards and the use of products and everyday technologies are shaped by early experiences in dealing with technology, and are particularly influenced by the father (Jakobs et al. 2009). Also, schools provide opportunities to engage with technology, which can additionally enhance students’ interest in underlying scientific concepts (Jones and Kirk 1990). Interest in, and acceptance of technology, is related to students’ perceived competence (i.e., self-efficacy and self-concept) in using specific technologies and to the feeling of being able to control technologies and their consequences for their own life (Ziefle and Jakobs 2009; Neyer et al. 2012).

Attitudes towards external technologies (e.g., nuclear and renewable energy, application of genetic engineering in agriculture and reproduction medicine, cell phone electromagnetic waves) are of great importance for modern societies (Renn 2008). For these technologies (also sometimes framed as socio-scientific issues; e.g., Morin et al. 2014), students may show more ambivalent attitudes and evaluate the opportunities and the risks for the development of a society. The underlying question is: Which technological developments may be appropriate for the design of a desirable future? In the 1980s/1990s, computers also were part of these external technologies. However, they no longer have an acceptance problem; the trend has, if anything, been reversed. They are often used without critical evaluation of which data are collected and what can be done with them.

4.2 Environment

In view of pressing global environmental problems such as global climate warming or air pollution, the promotion of pro-environmental behavior receives increasing attention (e.g., UNEP 2015). The relationship between different environment-related dispositions and pro-environmental behavior is most commonly analyzed in the framework of the theory of planned behavior (for an application in the field of environmental behavior see Harland et al. 1999), the value-belief-norm theory (Stern 2000), and the norm activation theory (Schwartz 1973). In the field of education, this relationship has been incorporated into competence models (e.g., Seeber and Michaelis 2014; Corral-Verdugo 2002). The following have been repeatedly proven to be significant predictors (either within the abovementioned models or as single predictors) of pro-environmental behavior: attitudes (Kaiser et al. 1999), moral norms and values (Kals 1996; Corral-Verdugo 2002, connectedness with nature (e.g., Nisbet et al. 2009) and environmental knowledge (Frick et al. 2004).

Environmental awareness is often characterized as a multidimensional construct that encompasses an affective-motivational, a cognitive (i.e. knowledge regarding environmental issues and the impact of human behavior on the environment), and a behavioral component (Szagun and Pavlov 1995; Homburg and Matthies 1998). However, some approaches focus on one dimension only (e.g., Kollmuss and Agyeman 2002). PISA 2006 defined environmental awareness as a unidimensional construct, conceptualizing it as metacognitive awareness of one’s knowledge regarding environmental issues (OECD 2007).

The results of PISA 2006 showed that the majority of students reported knowing something about environmental issues (OECD 2009). However, students’ awareness varied significantly across different issues. For example, on average, 73 % of students knew something about clearing forests for other land use, but only 53 % reported knowing something about genetically modified organisms. Student environmental awareness was positively associated with their science performance. In general, male and female students had similar levels of environmental awareness. However, in countries that belong to the Organization for Economic Co-operation and Development (OECD), males tended to be more aware of environmental issues than were females.

Environmental optimism can be divided into spatial and temporal optimism (Gifford et al. 2009). Temporal optimism refers to the individual’s assessment of how far environmental problems will—over a certain period of time—improve or worsen (Dunlap et al. 1993). Spatial optimism refers to students’ assessment of the seriousness of certain environmental problems in their own surroundings, compared to other places in the world (Uzzel 2000).

PISA 2006 assessed temporal optimism and revealed that most students were not optimistic about improvements in environmental problems over the next two decades (OECD 2009). In addition, students’ optimism was negatively associated with their performance in environmental science. In general, girls and boys reported similar levels of optimism. However, in OECD countries, males tended to be more optimistic.

5 The Social Meaning of Science

The social meaning attached to a particular science domain involves common assumptions about the typical characteristics of a science subject (i.e., the image/stereotype of science), as well as about persons who prefer a science subject (i.e., prototype). The interests as identity regulation model (Kessels and Hannover 2007) proposes that the image of science and the perception of prototypical scientists affect whether students identify with science. Moreover, these features are related to students’ personal, professional and social aspirations (Christidou 2011).

Image of science refers to common assumptions about the characteristics, contents, and scripts of a science domain. It can be constructed using three dimensions: The perceived difficulty of the domain, the perceived masculinity or femininity, and the perceived opportunities for self-realization and expressing one’s own ideas (Kessels and Hannover 2007).

Students perceive school science as difficult, masculine, and with only few opportunities for self-realization; this leads to lower popularity, particularly among girls (Kessels and Hannover 2004). When students encounter science, this image is activated automatically and in turn affects students’ attitudes towards science. Because of this automatic activation, the image of science can be assessed primarily using implicit measures (Kessels et al. 2006).

Prototypes refer to specific characteristics of persons who favor science (Kessels and Hannover 2007). When a scientist is perceived as badly dressed and without any friends, students often do not want such characteristics to be part of their own identity. Hannover and Kessels (2004) showed that students who favored math or science were perceived as less physically and socially attractive, less socially competent, and less integrated. However, in contrast, they were also perceived as more intelligent and motivated. The smaller the differences between the self-image and the prototype, the more students liked the subject (Hannover and Kessels 2004; Kessels 2005; Kessels and Taconis 2012) and made science-related educational and professional choices (Taconis and Kessels 2009; Kessels and Taconis 2012).

6 Aspirations Towards a Science Career

The positive relationship between high aspirations and educational and occupational choices in science has been well documented in longitudinal studies (e.g., Croll 2008; Tai et al. 2006). In addition, studies from different theoretical backgrounds have confirmed a relationship with attitudes towards science—in particular with interest (e.g., Taskinen et al. 2013), parental science capital (e.g., DeWitt and Archer 2015), and self-related cognitions (e.g., Wang et al. 2013).

Future-oriented motivation to study science is particularly relevant for predicting how many students will opt for science in their future career choices. In general, students who show positive attitudes towards school and report greater persistence in times of difficulty, also show higher levels of future-oriented motivation (de Bilde et al. 2011).

PISA 2006 showed that although most of the students are interested in and enjoy learning science, only few students see themselves as using science professionally in the future (OECD 2007; see also Archer et al. 2010). Further analyses indicated that home background factors and science performance had only little predictive power for students’ motivation to study science in the future. Science teaching that is focused on applications or models, hands-on activities, interaction, and students’ investigations, or on receiving information about science careers, were better predictors (Kjærnsli and Lie 2011). Moreover, future-oriented motivation was moderately associated with instrumental motivation.

Career aspirations refer to whether students expect to have a science-related job at the age of 30. PISA 2006 showed that on average 27 % of the girls expected to pursue a science-related career compared to 23.5 % of the boys. These aspirations were not related to parents’ actual occupations. However, the best science performers were students who expected to work in a science-related job and had at least one parent working in science. Nevertheless, the most important predictor for high science-related career aspirations was instrumental motivation for a career in science (e.g., Taskinen et al. 2008).

7 Selection Criteria and Limitations of Large-Scale Assessments

Large-scale assessments such as PISA have special formats that guide the selection of possible constructs. The focus of PISA is on outcomes that are important for educational contexts—they need to be either important for explaining relations with cognitive outcomes, related to future willingness to engage in science, and/or to be stand-alone outcomes in their own right in assessing students’ personality or identity. Each construct considered for assessment needs to have a strong theoretical background. In order to achieve a short testing time—one of the most important constraints of large-scale assessments—the constructs need to be defined broadly enough to allow for content validity and, at the same time, narrow enough to produce reliable measures under strong time constraints.

Science-related outcomes are mostly assessed within the student questionnaire (see Sect. 12.8). Hence, statements with different types of answering options can be presented. These answering options can be more general agreement/importance scales (e.g., strongly agree to strongly disagree) or more specific event- or behavior-related scales such as frequency ratings (e.g., never or almost never to every day).

Within the questionnaire, it is important to state what is meant by the term science. Does it refer to the science subject itself as a discipline, the learning of a school science subject, specific science topics, or to the methods used in science? This clarification is crucial because, for example, interest in specific science topics can be different from interest in school science subjects (Haeussler and Hoffmann 2000). From an intercultural perspective, asking students about their attitudes towards school science also needs to take into account that, in some countries, secondary school science is further divided into biology, chemistry, and physics (sometimes earth science in addition; OECD 2007). The use of a questionnaire for assessment raises methodological challenges, particularly when the cultural backgrounds of the students differ so greatly. Response styles such as acquiescence or extreme response style patterns, social desirability, and the reference group students refer to in answering the questions, can influence “true” answers (van de Gaer et al. 2012; Heine et al. 2002; van de Vijver and He 2016, Chap. 9, in this volume).

In addition, PISA assesses outcomes at the end of compulsory school when students are 15 years old. Therefore, only relatively stable, established attitudes can be measured.

As the population of PISA is defined by age (15 years old) and students are sampled within schools but not classrooms, PISA does not provide a grade-based sample. Hence, no analyses at the classroom level can be undertaken and, consequently, the relation between the shared perception of opportunities to learn in a classroom and attitudes cannot be examined. However, some countries that are interested in such research questions implement additional grade-based samples.

PISA is also limited to cross-sectional data and thus, developmental research questions cannot be answered. However, some countries have undertaken an additional national longitudinal study in order to understand how cognitive outcomes and attitudes develop over time. For example, in Germany, in addition to implementing a grade-based sample (9th grade) in PISA 2003 and 2012, students were tested again one year later, when they were in 10th grade. This research is important for German schools and policy makers, as in most schools and states, compulsory education ends after 10th grade. Also, Australia and Canada implemented a longitudinal design and followed their students from the PISA 2003 cohort till adulthood, in order to examine their development across a longer time span (see for example Parker et al. 2014 for an analysis of the Australian data).

Although the international data are cross-sectional, with PISA 2015 the second cycle of reading, math, and science assessment is completed. Hence, trend data will be available that can tell us something about the development of attitudes and cognitive outcomes at the country level. For science-related outcomes, the first trend data will be available with PISA 2015. For most of the countries, these trend data are of particular importance and thus, the selection of constructs for PISA 2015 also needs to be not too dissimilar from what was assessed in PISA 2006, when science was last the major domain.

8 The Assessment of Science Attitudes in Previous Large-Scale Assessments

PISA 2006 followed two approaches for assessing students’ engagement in science: The traditional student questionnaire and the embedded item approach. With the traditional student questionnaire, students’ interest in and enjoyment of science, instrumental motivation, value beliefs about science, scientific attitudes, self-related cognitions, environment-related dispositions, and career aspirations were assessed (OECD 2007).

With the embedded item approach, PISA additionally assessed interest in science, support for scientific inquiry, and responsibility towards resources and environment within the PISA cognitive assessment (OECD 2006; Drechsel et al. 2011). To do this, directly after some of the science items, students were asked whether they would be interested in “knowing/learning/understanding more” about the specific contexts and domains the items were related to. Hence, students’ interest was related to specific science contexts, which allowed them to have a clear idea about what is meant by the term “science”. However, only a small number of interest items could be distributed within the complex booklet design of PISA; this limited the reliability of the assessment. Moreover, the approach needed additional testing time in the cognitive assessment, and may also have influenced students’ general test motivation. Hence, the decision was taken to drop such assessment of attitudes in the cognitive test administered in 2015.

Also, it is worth noting that the Trends in International Mathematics and Science Study (TIMSS) has assessed science-related attitudes. It has focused on students’ interest, instrumental motivation, students’ value of science, and their science self-concept in 4th and 8th grades (Martin et al. 2012). Eighth graders were asked about their general or subject-specific science attitudes, depending on how science was taught at school. Results showed for example that students who reported higher perceived values of science also showed higher levels of science achievement (see Martin et al. 2012 for more detailed results); this is in line with the PISA results for 15-year-old students (OECD 2007).

9 Identifying Important Constructs for PISA 2015

Aside from fostering science achievement, developing positive science attitudes is an important educational goal. Achieving these multidimensional goals enables students to participate in society on the basis of scientific reasoning, and influences their educational and professional career choices (Fig. 12.1). Therefore, the science-related outcomes module was rated as high priority for the PISA 2015 field trial (see Jude 2016, Chap. 2, this volume). Based on policy importance and the current state of research summarized in this chapter, and keeping the selection criteria and limitations of large-scale assessments that we have discussed above in mind, the constructs for this module in the PISA 2015 field trial were carefully selected.

Table 12.1 gives an overview and summarizes the theoretical path presented in the literature review above. The PISA 2015 field trial questionnaire focuses on attitudes towards science, scientific attitudes, attitudes towards domain-specific science topics, and aspirations towards science. For attitudes towards science, students were asked about their positive emotions towards science (enjoyment), as well as their interest in broad science topics, in contrast to their interest in school subjects. Moreover, their instrumental motivation, self-related cognitions (self-efficacy, self-concept), and value beliefs (general and personal value of science, value of science in the labor market, occupational prestige) were assessed. Also, items about the attitudes of peers and parents were included. For scientific attitudes, the student questionnaire contained questions about epistemological beliefs dealing with uncertainty and ambiguity, and valuing scientific approaches to inquiry. Attitudes towards domain-specific science topics were covered by questions about technology (technology commitment, the benefits and disadvantages of technologies, the use of technical devices, competence regarding the use of technology), and the environment (awareness, optimism). Finally, students’ aspirations towards science were operationalized by future-oriented motivation and specific career aspirations at age 30.

Table 12.1 List of constructs included in the PISA 2015 field trial to assess science-related outcomes: attitudes, motivation, value beliefs, strategies in the PISA 2015 field trial
Full size table

For these constructs, data were collected in the field trial using a rotated questionnaire design (four non-overlapping booklets; Kuger et al. 2016, Chap. 4, this volume). The results had to match specific criteria, such as requiring comparability across countries, or displaying an inter-culturally consistent low rate of missing data (see Kuger et al. 2016, Chap. 4, this volume for a complete list of criteria). Based on these results, it was then decided whether the proposed field trial constructs should also be considered for the main study (Table 12.1, Column 5).