Keywords

1 Introduction

Collaborative problem solving (CPS) was introduced to the Programme for International Student Assessment (PISA) for the first time in 2015. We start by introducing some collaborative domain concepts. In the next section of this chapter, we describe how assessment of the collaborative domain in the 2015 cycle drew on the work and measures of individual problem solving assessed in PISA 2012. We then focus on the literature basis of the 2015 developments and their extensions from 2012. Finally, this is followed by a discussion of the 2015 questionnaire measures used to examine some aspects of collaborative student attitudes and teacher practices in PISA 2015.

The conception of collaborative, or group problem solving had its origin in the types of problems and collaborative interactions that 15-year-old students face in and out of the classroom, as well as the issue of preparedness for future life in the workplace and in their studies.

The CPS construct was built on PISA’s 2012 conception of individual problem solving. PISA instrumentation often seeks to add new information and contexts to those of prior years. Since the PISA 2015 CPS domain included a substantial new component—collaboration—the decision was made to use the 2015 noncognitive questionnaire questions to further probe this element. CPS performance may be affected by dispositions. For questionnaire measures, dispositions were considered that may differ between countries and thus help to explain differences in CPS performance. Useful information might include potential cultural and practice differences across countries, including students’ experiences with and attitudes and strategies for CPS.

Regarding the 2015 PISA framework (OECD 2013), it was recognized that the ability of an individual to be successful in many modern situations involves participating in a group: to communicate, manage conflict, organize a team, and build consensus as well as manage progress. The PISA framework described the importance of improving collaboration skills for students and fostering CPS (Rummel and Spada 2005; Vogel et al. 2016). Thus, the measurement of collaboration skills is at the heart of problem solving competencies in the new PISA CPS framework. For 2015, clarifications were offered in three aspects of the collaborative context. First, the competency being described remained the capacity of an individual, not the group. Secondly, he or she must effectively engage in a process whereby two or more agents attempt to solve a problem, where the agents can be people or simulations. Finally, the collaborators had to show efficacy by sharing the understanding and effort required to come to a solution, such as pooling knowledge to reach solutions.

For the cognitive CPS assessment, collaboration in the context of problem solving formed the reporting scales for the assessment. For noncognitive data, this chapter describes the corollary noncognitive measures, or “constructs”, developed to support our understanding of student performance in CPS.

Such constructs are useful to accompany a cognitive instrument, because students’ actions in the cognitive test are influenced by affective factors, which can have an impact on the claims we make about scores. So by anticipating or measuring these factors, we may find potential causes and explanations for patterns in the test results.

Here, measuring dispositions toward the cognitive construct under study—CPS—took the form of student experiences with and attitudes toward collaboration, as well as teacher reports on their CPS-related activities and the support they provided to students in the classroom.

The student and teacher questionnaires for the PISA 2015 field trial reported here incorporated nine constructs intended to capture a snapshot of student dispositions toward collaboration. Two of these constructs involved student reports on their collaboration experiences within and outside the school context. One involved student self-perceptions of their efficacy and experience in collaboration. The remaining six constructs provided reflections from teachers on the support and experiences they provided to their students during collaboration.

2 Related Measures in Previous PISA Cycles

In PISA, the domain area of CPS builds off prior efforts to measure individual problem solving competencies. In PISA 2012, some student individual characteristics related to individual problem solving were measured: openness to learning, perseverance, and problem solving strategies. In addition, some problem solving experience questions were included. A description of the instrumentation for each of the 2012 attitudinal construct areas is shown in Table 11.1.

Table 11.1 PISA 2012 noncognitive attitudinal and strategy constructs designed as related to problem solving domain
Full size table

Both openness to learning and perseverance in problem solving were originally composed of 15 Likert scale items that formed the “Big Five”-based personality scales for the field trial. These were reduced to five items each for the main study. Other Big Five attributes, such as extroversion and agreeableness, were considered less related to individual problem solving for the 2012 context, so were not included in the questionnaires (McGivney et al. 2008).

Problem solving strategies for the 2012 questionnaire, by contrast, consisted of vignettes with subsections associated with the problem solving process. The vignettes posed situations in which students could report on problem solving strategies, such as seeking help from friends or consulting information sources to obtain solution approaches. The results of the 2012 questionnaires were seen as quite helpful for understanding the problem solving context.

3 Theoretical Background on Collaborative Problem Solving

This section focuses on assessment of the collaborative domain in the 2015 cycle. The 2015 CPS framework (OECD 2013) describes how the most important international previous and current discussions in research have used a number of different methods to measure the quality of problem-solving processes and products. Approaches cited by the CPS framework (OECD 2013) range from assessing actions during collaboration to products. Measures include solution success, as well as objects generated during the collaboration (Avouris et al. 2003). In-situ observables include analyses of log files in which the computer keeps a record of student activities, sets of intermediate results, and paths taken along the way (Adejumo et al. 2008). Team interactions also offer relevant information (O’Neil et al. 1997), including quality and type of communication (Cooke et al. 2003; Foltz and Martin 2008; Graesser et al. 2008) and judgments (McDaniel et al. 2001).

The international Assessment and Teaching for twenty-first century Skills (ATC21S) project examined the literature on disposition to collaboration and to problem solving in online environments. ATC21S described how interface design feature issues and the evaluation of CPS processes interact in the online collaboration setting (Binkley et al. 2010, 2012).

The ATC21S KSAVE framework described sets of knowledge, skills, attitudes, values and ethics that contributed to dispositions to CPS (Binkley et al. 2012). Measuring collaboration was also explored in the domain area of digital literacy (Wilson and Scalise 2012a, b; Wilson et al. 2015). Collaboration scholars often make a distinction between CPS and collaborative learning (Dillenbourg et al. 1996). Griffin and colleagues discuss the first (Griffin et al. 2012), while the second is a main focus of Laurillard’s work. In both cases, collaborators may organize activities to share group discourse and reflection, and to come to a shared understanding (Cakır et al. 2009). When a student is involved in explaining and justifying, this type of collaboration can become quite productive for learning (Baker and Lund 1997).

Key to understanding the PISA 2015 definition of CPS are three extensions of or clarifications to the original problem solving definition of 2012. First, the 2015 framework clarifies that the CPS competency being described remains the capacity of the individual. Secondly, to demonstrate the collaborative component in 2015, the individual must effectively engage in a process whereby two or more agents, such as fellow students, attempt to solve a problem. Finally, the collaborators must show efficacy by sharing the understanding and effort required to come to a solution. In other words, they must show that they can pool their knowledge, skills and efforts to reach a solution, in order to exhibit the full CPS competency.

To operationalize this definition, the four problem-solving processes of the original 2012 CPS framework were retained, and crossed with some new focal areas in collaboration.

Since individual problem-solving processes were already defined by the PISA 2012 framework and are strongly connected to the research literature in problem solving, they were retained:

  • Exploring and understanding

  • Representing and formulating

  • Planning and executing

  • Monitoring and reflecting

The CPS framework developers further pointed out that CPS competencies are influenced by factors such as the task, the team’s composition, the medium in which the task is applied, and the overall background context of the problem solving task (OECD 2013). At the dispositions level, results can be influenced by student experiences, opportunities and attitudes; these are the focus of Sect. 11.5 PISA 2015 CPS Constructs and Measures of this chapter.

4 Contextual Issues Related to the Need for Measuring CPS and Related Dispositions

The PISA framework (OECD 2013) indicates how student background variables such as experience and characteristics, combine with core skills in both collaboration and problem solving to create an outcome space. As described on the PISA 2015 CPS framework, CPS is not considered a traditional domain, because in most countries little or no explicit teaching in this area is done as a school subject. Rather, it is included as a practice in the classroom and embedded in student work in some contexts—often to a greater degree in some classrooms than others.

The PISA CPS framework (OECD 2013) describes how the extent of students’ familiarity with collaboration may differ across different PISA participating countries. Therefore, contextual questionnaires should include supporting data on CPS familiarity for students within:

  • Educational contexts: e.g., classroom and assessment experiences

  • Out-of-school contexts: e.g., home life and hobbies

  • Technology-specific contexts: e.g., gaming

  • Disposition to CPS: The way in which students perceive CPS and, in particular, their self-efficacy, can also affect their performance. Therefore, the following areas are of interest:

    • Interest in and enjoyment of collaboration

    • Value of collaboration skills

    • Self-perception of CPS ability

Due to logistical and space constraints in the background questionnaire, it was possible to measure only some of these areas in 2015, and the choice needed to be prioritized on the basis, for instance, of the advanced thinking of the Questionnaire Experts’ Group and the results of the field trial. In addition, questionnaires were developed so that some information could be gathered through the optional questionnaires taken by some countries, such as the ICT familiarity, teacher and parent questionnaires (see Kuger et al. 2016, Chap. 4; Jude 2016, Chap. 2, in this volume for more details on these questionnaires).

A key issue in the PISA 2015 CPS framework is who can serve as a partner—or agent—in a collaboration. The agent-basis of the collaboration is not a focus of this chapter, because it was not a focus of the noncognitive questionnaires, but computer agents and not human agents were employed for PISA 2015 CPS (OECD 2013).

5 PISA 2015 CPS Constructs and Measures

The PISA 2015 CPS Framework (OECD 2013) noted that some areas of interest for questionnaire constructs and measures included individual student characteristics, such as prior experiences, while others involved attitudes and self-perceptions in regard to collaboration. Key components of prior experiences include previous opportunities to work in teams, participate in problem solving, and engage in communication with peers in team efforts.

Student attitudes and self-perceptions included interest in and enjoyment of shared work, the valuing or not of collaborative skills, and student self-confidence in their ability to work with others.

For the field trial, these areas of interest were organized into three constructs. Sets of questions related to each construct. Two of the constructs involved student in-school and out-of-school experiences in collaboration-related activities (ST083, nine items; ST084, four items). An additional construct asked about student self-perceptions in collaboration (ST082, 18 items). As well, six constructs on the teacher questionnaire may be of interest regarding the CPS framework. The constructs collected information on some types of collaborative activities and support available in the classroom, including team and group efforts.

5.1 Student In-School and Out-of-School Experience of Collaboration

Student in-school and out-of-school experience in collaboration-related activities were covered in the field trial by two questions with several items each, associated with the ID numbers ST083 and ST084 respectively.

Both item sets were developed specifically for PISA 2015, through questionnaire development in collaboration with commentary from the CPS experts’ group. The questions had not been used in this form previously, and the questions were designed to be included in the student questionnaire.

Student experiences in collaboration were assessed using a scale incorporating nine items and four items (see additional online material at https://daqs.fachportal-paedagogik.de/). The first set examined student in-school collaboration experiences and the second set examined student out-of-school experiences. The two item sets were developed in part because the frequency of collaboration opportunities was considered to have potential to vary considerably between the two settings, at least for some students. The answer format was a 4-step Likert scale on frequency of the described experience (from 1: “always or almost always” to 4: “never or rarely”).

As described above, the PISA 2015 CPS framework notes that the extent to which students in different PISA participating countries may be familiar with collaboration may differ; therefore, it is important to have supporting data on their CPS familiarity with certain essential contexts. The essential contexts were described in the framework as

  • Educational: e.g., classroom and assessment experiences

  • Out-of-school: e.g., home life and hobbies

  • Technology-specific: e.g., gaming

The first two essential contexts are in-school and out-of-school respectively. The underlying theme of “technology specific” was worked into each of the two contexts, with some technology-related questions being addressed in each.

In addition, the out-of-school item set showed some quality issues, limited variation across countries, and perhaps an overly directed focus on technology. A suggestion for future work may be to combine the two question sets to describe a single construct, with less focus on the exact context of the collaboration and more focus on the frequency, range and opportunities for overall collaboration that the student generally experiences.

5.2 Student Self-Perceptions in Collaboration

In 2015, for the first time, PISA incorporated a question measuring students’ self-perceived collaboration and teamwork dispositions. The self-perceived dispositions were assessed using a scale incorporating 18 items, (see additional online material at https://daqs.fachportal-paedagogik.de/), testing the dimensions cooperate, advocate/guide, and negotiate. The answer format was a 4-step Likert scale (from 1: “strongly disagree” to 4: “strongly agree”).

The scale was based on one initially constructed by Wang et al. (2009), and was adapted and extended for PISA purposes. The scale construction by Wang et al. (2009) was based on a literature review of several previous conceptualizations of collaboration in the areas of educational and work psychology that differ in detail regarding their proposed dimensions (e.g., Loughry et al. 2007; Stevens and Campion 1994). For example, one of the collaboration conceptualizations in work psychology (Stevens and Campion 1994) proposes that collaboration skills are constituted by conflict resolution, CPS, communication, goal setting and performance management, as well as planning and task coordination abilities. Similarly, a collaboration conceptualization tailored to college students encompasses abilities such as, for example, contributing to the team’s work, interacting with teammates, and keeping the team on track (Loughry et al. 2007). Based on such conceptualizations, Wang et al. (2009) identified the four theoretically most relevant dimensions for collaboration in high school students: cooperation with team members (cooperate), influencing team members (advocate), guidance and mentorship (guide), and resolution of conflicts via negotiation (negotiate; Wang et al. 2009). Empirical analyses confirmed three basic dimensions of collaboration (cooperate, advocate/guide, and negotiate). According to Wang et al. (2009), to cooperate is reflected in “bringing ideas together, seeking solutions, and providing feedback to team members”; advocate/guide includes actions to “direct others, provide appropriate suggestions and criticism, and persuade others”; while to negotiate is the “tendency to listen, to adapt to change while there are conflicts and the ability to change conflicts”. All three dimensions were validated by a situational judgment test of collaboration, in which students rated the degree of teamwork behavior described in a scenario. The two dimensions cooperation and guide were additionally validated by teacher ratings of students’ collaboration behavior.

Compared to Wang et al.’s original scale (2009), the scale used for PISA purposes was adapted slightly (see additional online material at https://daqs.fachportal-paedagogik.de/). Three items from the original scale were added, two were adapted, and three were omitted. The three items that were added extended the collaboration dimension (“I find that working as a member of a team increases my ability to perform effectively”, “I prefer working as part of a team to working alone”, and “Teams make better decisions than individuals”). The two items that were adapted were also related to assessing collaboration (“I enjoy assisting in a team” and “I enjoy seeing my classmates be successful”). Three other items from the advocate/guide and negotiate dimension were omitted: “I am comfortable with providing criticism” (advocate/guide), “I can argue constructively” (advocate/guide), “I dislike people with challenging views” (negotiate).

The PISA questionnaire was constructed in such a way that such terms as “groups”, “classmates”, and “team” are interchangeable, and the questionnaire could be answered even if factual collaboration did not happen or was rare at a particular school. As the questionnaire had been adjusted to a wide range of group activities, it was aimed for use in any kind of group work or collaboration situation. The questionnaire items themselves were not meant to require the respondents to make an overt distinction between the use of collaboration and cooperation. Often, young adults in school and their teachers do not distinguish the two terms, although a distinction is made in some of the research literature (Dillenbourg 1999; Dillenbourg et al. 1996). In cooperative tasks, individuals may work on subtasks that are brought together afterwards, whereas collaboration may involve more fully completing the entire task together. Collaboration is the focus of the cognitive assessments for the CPS framework. However, many individual responses to assessment subtasks in the assessment are completed by a single respondent or a computer avatar, so an element of cooperation can be seen.

The questionnaire items can be theoretically aligned to the PISA CPS framework (OECD 2013). The PISA CPS framework defines collaborative problem solving as “the capacity of an individual to effectively engage in a process whereby two or more agents attempt to solve a problem by sharing the understanding and effort required to come to a solution and pooling their knowledge, skills, and effort to reach that solution”.

This definition covers three CPS competencies: “establishing and maintaining shared understanding”, “taking appropriate action to solve the problem”, and “establishing and maintaining team organization”.

These PISA competencies are reflected in behavioral indicators (e.g., OECD 2013). Behavioral indicators of the first competency, “establishing and maintaining shared understanding”, are all actions to discover others’ abilities, as well as to communicate about the problem and the collaboration process. The second competency, “taking appropriate action to solve the problem” is reflected in actions to communicate and discuss the task assignment, to enact plans, and to perform actions according to these plans. Every action to monitor the task is also part of this competency. Finally, the third competency, “establishing and maintaining team organization” is characterized by actions to follow the collaboration plan and to motivate others to do so. Actions that reflect recognizing failures in collaboration and suggesting ways to fix them—for example, recognizing which information needs to be addressed to whom—are also part of this competency.

Similarly to behavioral indicators, the PISA competencies are also reflected in the questionnaire subscales (see Table 11.2). The items of the cooperate subscale cover all three competencies, as the items “I enjoy sharing ideas” and “I enjoy providing feedback” reflect the underlying motivation behind all three competencies. In contrast, the advocate/guide and negotiate subscales are only aligned to one competency. As negotiation skills are necessary when people enact plans with others, and to monitor and evaluate others’ work, the negotiation subscale is aligned to the competency “taking appropriate actions to solve the problem”. In comparison, guidance skills such as listening well to others, convincing others about particular plans, are a core proficiency in establishing collaborative work: therefore, the subscale guide can be aligned on the competency “establishing and maintaining team organization”. Taken together, the questionnaire subscale cooperate covers all three PISA competencies underlying the CPS framework; in contrast, the subscales advocate/guide and negotiate are dimensions underlying the competencies “taking appropriate actions to solve the problem” and “establishing and maintaining team organization”.

Table 11.2 Hypothetical mapping of the three collaboration competencies with the proficient behavior indicators (obtained from the PISA CPS framework, OECD 2013) and with the subscales of the self-perceptions questionnaire
Full size table

As the importance of collaboration skills is quite established for school, as well as for work (Klein et al. 2006), to be able to foster these skills in the long run, knowledge on the preconditions and correlates of good collaboration skills is necessary. One of these prerequisite skills is communication. Communication skills are important when students need to build a shared understanding of a problem and the abilities of individual group members (OECD 2013). Communication skills help students to organize their work effectively and to establish common rules within a group. Communication is also essential for “staying on track”, giving each other feedback and mutually monitoring the progress towards a solution. Items such as “I enjoy sharing ideas”, “I enjoy providing feedback” or “I am a good listener” reflect communication skills.

Beside communication skills, different personality traits are associated with good collaboration. Specifically, personality traits that are associated with positive emotional regulation contribute to successful collaboration. For example, emotional stability is associated with task performance, and agreeableness predicts cohesion and interpersonal teamwork behavior (O’Neill and Kline 2008). Emotional intelligence is positively associated with team performance and conflict resolution methods (Jordan and Troth 2004). Individual agreeableness is associated with good cooperative behavior (LePine and Van Dyne 2001).

Personality variables at the team level, such as team agreeableness and team mean conscientiousness, openness to experience, collectivism, and preference for teamwork, predict team performance in the field (Bell 2007). Other variables, such as self-efficacy during teamwork, need for social approval, and positive past experience with working in teams, are related to self-report measures of collectivistic orientation and a self-rated orientation towards team activities and goals. Research investigating collaborative behavior confirms that direct assistance and direction of team members is perceived to be most effective (Harris and Banes-Farrell 1997). Confirming these results, a recent review of questionnaires assessing teamwork found that the most frequently assessed collaboration dimensions in questionnaires are communication, coordination, and respect (Valentine et al. 2015).

Taken together, the effectiveness of collaboration depends on the sum of individual as well as group characteristics and competencies in collaborating and prioritizing group success over individual success. Therefore, apart from cognitive skills, interpersonal skills are also essential for CPS. All three CPS context assessment dimensions, cooperate, negotiate, and advocate/guide, reflect both self-perceived cognitive and interpersonal skills (OECD 2013).

5.3 Some Potential Limitations of the Measures

Methodologically, there are several sources of error when investigating relationships between variables using self-report (e.g., memory biases when people need to infer their traits from their past behavior). Particularly, two sources of bias related to the psychology of survey responses may be present when collecting self-report data that might play an important role in assessing students’ perceptions of their collaborative skills. The first one is social desirability bias: that is, the tendency of persons to present themselves in a favorable light, regardless of their true thinking and feelings about a particular issue (Podsakoff et al. 2003). As collaboration skills are socially favorable, students might have a tendency to present themselves more collaboratively than they truly think they are.

A second bias that might be related to the first bias is acquiescence, the tendency to confirm and say “yes” to a statement without really reading or thinking about that statement. Both biases produce spurious relationships between variables; the social desirability bias can even suppress or moderate relationships between variables (Podsakoff et al. 2003).

However, social desirability and acquiescence biases have been widely neglected in scale construction, evaluation, and implementation (King and Bruner 2000) as, for example, some researchers claim that social desirability bias is a validation criterion of a scale, given that it reflects the value of a trait in a particular culture (Fisher and Katz 2000). As social desirability bias has been shown to be a stable, multi-dimensional trait, rather than a situationally specific response set (Furnham 1986), it can be relatively easily controlled for, using statistical techniques. Research investigating predictors and consequences of collaboration skills using self-report needs to consider these biases (for a more detailed discussion of potential sources of answering biases in the context of international context assessment see He and van de Vijver 2016) and, in the best case, validate these self-report measures using teacher ratings or real collaboration tasks. In this case, data are available at the country level, both from the teacher question and from student performance on the 2015 CPS collaborative tasks. Please consult Chap. 4 of this book, by Kuger et al. (2016) for more details on methodological considerations.

5.4 Emerging Insights on Student Collaboration Factors

For the student self-perceptions in collaboration construct, analyses suggested a three-factor solution might offer a theoretical approach, as follows:

  1. 1.

    Participation in collaboration

  2. 2.

    Taking charge of a team, or leadership

  3. 3.

    Efficiency or usefulness beliefs regarding efficacy of collaboration

Following results on timing issues and the measurement aspects of the factors, however, it was decided to keep only the first collaboration dimension and to shorten the number of items in this dimension. Due to this reduction, eight items from the first collaboration dimension were used for the assessment in the main study. However, the other two factors would be useful to consider in future research, as well as in practice, for training and intervention studies.

Potential impacts on education processes and outcome(s) based on these high-level ideas underscore that it seems possible both to collect and to scale noncognitive information on disposition to collaboration, in useful ways. Measures may help provide intervention support, since in today’s world especially, teams with good collaborative skills are necessary in any group, from families to corporations, public institutions, organizations, and government agencies (OECD 2013). Thus, indicators of dispositions to collaborate, such as those described above, may be needed to create adequate interventions to train collaboration skills and to change current levels of individual collaboration.

The questionnaire dimensions collaborate, negotiate, and advocate/guide might be useful starting points for creating such interventions. Alternatively, the factor structure identified may be a more interesting starting point. It may also be helpful to consider how participatory a student is disposed to be in collaboration, along with his or her team leadership inclinations, and beliefs in the value or efficacy of collaboration.

Such an intervention in collaborative skills might consist of three basic elements: the first element might be to show the relevance of possessing collaborative skills, the second one might target communication skills, and the third element might target students’ group and task managing skills. With the first element, to create motivation to develop collaboration skills, students need to be shown the relevance of collaboration for task success. For the second, an intervention might focus on communication and negotiation skills: i.e., how to listen well to others, how to provide feedback etc. Finally, the third and last part of the intervention might focus on how to use communication skills to manage a group, how to create common goals and track the pursuit of these goals. Practical exercises would constitute an important part of interventions, as they are accompanied by social feedback that strengthens individual collaborative competencies in role play and feedback sessions—a practice successfully implemented in other social competency trainings (e.g., Personal Effectiveness Training; Liberman et al. 1975).

5.5 Supporting CPS Constructs: Information from Teachers

As context can either facilitate or inhibit collaborative behavior in students, in the PISA assessment a teacher questionnaire was administered to assess the school context of students. As shown in Table 11.3, four questions on different constructs were included in the final PISA main survey questionnaire, because they were strongly theoretically associated with the CPS framework. These teacher questions had been validated in previous studies, and were expected to provide important information on how to change contexts to change or to improve students’ CPS skills. Two questions on additional constructs were dropped, due to time restrictions, and also because they were somewhat narrow in conception. Although teacher information is not a direct measure, as is the student report, it may contribute to interpreting student experiences, so the four retained questions will be briefly discussed here.

Table 11.3 List of constructs included in the PISA 2015 field trial to assess dispositions for collaborative problem solving
Full size table

The first question of this questionnaire was very generally related to collaborative behavior in the class. The question assessed how often a teacher assigns particular activities to students.

The teacher evaluated how often he assigned such activities to the students. An example of a 6-point Likert scale for the first question ranged from 1: “never or almost never” to 6: “once a week or more”.

The second question assessed the level of teachers’ appreciation for students’ collaborative activities. On a 4-point Likert scale (1: “never or almost never”, 4: “always or almost always”) the teachers evaluated the frequency with which they showed appreciation for students’ collaborative activities. Appreciation could be shown by, for example, “individual appreciation for individual performance”, “individual appreciation for group performance”, etc.

The third question in this teacher questionnaire related to particular collaboration activities, assessed on a 4-point Likert scale (1: “never or almost never” to 4: “always or almost always”). Exemplary activities were “members of each group work according to the specialization of each member” or “members of a group work on a collective outcome”.

The last question assessed teachers’ grouping practices for collaboration activities. Grouping practices included “groups with a mix of abilities”, “groups of students with similar abilities”, “and groups as students chose them” on a 4-point Likert scale (1: “never or almost never” to 4: “always to almost always”).

It can be argued that these question sets help to inform about student self-perceptions, as they may help establish a theoretical tie between teacher work and student perceptions, at least to some degree in aggregate at the country level. Much future scientific work is needed, to establish whether teacher variables, such as fostering collaborative activities, grouping practices for collaboration activities, and showing appreciation for these activities, are associated with the development of students’ collaborative skills. Also, research is needed on whether teacher interventions are effective moderators of the association between students’ collaborative skills and students’ performance. More reflection on this topic may be possible from the main study results when both cognitive and noncognitive data are available.

6 Conclusion

In this chapter we have discussed student dispositions toward the emerging domain of “collaborative problem solving” (CPS), recently assessed by the PISA 2015 field trial and main study. In respect of the noncognitive measures in the field trial, nine CPS-related constructs were developed for the student and teacher questionnaires. The description of the constructs in this chapter shows how the questionnaire data explores student experiences of collaboration, as well as student self-reporting on perceptions of and attitudes toward collaboration. Questions for teachers, on classroom collaboration activities and approaches, were also developed.

Questions for the field trial separately explored in-school and out-of-school activities that involved student self-report on their collaboration experiences. Both question sets were dropped from the main study, primarily due to time restrictions, but also because the items were narrow in scope. Future questionnaire efforts in this area may choose to explore a combined question, and a reduced scale, to allow for evidence gathering while consuming fewer questionnaire minutes.

Comparing the PISA items to exemplary items stemming from the work psychology literature, the theoretical dimensions on which the items are based are quite similar: these include contributing to the team’s work, interacting with teammates, keeping the team on track, expecting quality, and having relevant knowledge, skills, and abilities (Loughry et al. 2007). The original collaboration, negotiation and advocate/guiding items were seen to have a three factor structure in field trial results at the international level, yielding three dimensions that might be interpreted as follows: collaboration experience, efforts to lead collaboration, and beliefs in the efficacy or utility of collaboration. The scale was reduced, due to time constraints, and primarily retained the first and largest factor.

Productive efforts were required to develop the items and constructs for a self-perception of collaboration questionnaire. This questionnaire might stimulate further research, as well as training and interventions. Similarly, results out of the research might lead to policy changes, better introducing collaboration skills to the school curricula, initially to enhance students’ collaboration skills at school and subsequently, to enable these students to be productive coworkers and teammates in later educational efforts, in their career, and in everyday activities. However, to get a more complete picture, especially across domains, much more scientific work needs to be done. The initial picture arising from the PISA 2015 questionnaire items on CPS paints a tantalizing prospect for this emerging educational domain.