Argumentation, considered essential for effective participation in twenty-first century democratic societies (Asterhan & Schwarz, 2016), refers to an interpersonal or intrapersonal process in which claims are made, supported, and evaluated by reasons and evidence (Nussbaum & Edwards, 2011). University students are expected to be able to argue rationally and critically (Andrews, 2010; Voss & Van Dicke, 2001). When writing requires argumentation, students are encouraged to accept the existence of different points of view in order to integrate the arguments and counterarguments of opposing positions (Kuhn, 2005; Leitão, 2003). Thus, when argumentative writing is based on multiple and contradictory sources, students must organise, select, and connect information from these sources (Spivey, 1997), a task that is often quite challenging but has been shown to be effective for learning (Nussbaum, 2020). This is because, during argumentation, new knowledge can result through conceptual change and the need for increased specificity (Baker, 2004, 2016).

This study is part of a line of research aimed at improving undergraduate students' argumentative synthesis writing from multiple contradictory source texts. As with our previous related inquiries (Granado-Peinado et al., 2019; Mateos et al., 2018, 2020), the present intervention employed explicit instruction in argumentation and collaboration as well as practice in pairs to facilitate students' performance. Particular foci were students' identification and interrelation of contrasting claims, or arguments, made by source texts. The most noteworthy findings highlighted that, in order to achieve this, it was not enough to explicitly teach writing strategies, but it was also necessary to explicitly teach students to collaborate. In the following, we first provide theoretical and empirical background for the present study, which was conducted with a specific subset of university students. We then describe the design, which involved four versions of the program characterized by progressive reduction of support. In this way, we could analyze facets of the intervention separately as well as in combination. We report results and then conclude by discussing the study and its contributions with respect to theoretical considerations and previous empirical work.

The construction of argumentative syntheses from multiple texts

Argumentation has been approached and defined from different disciplines, but the dominant theoretical framework has been based on the model developed by Stephen Toulmin. Important here is Toulmin’s (1958) model of persuasive argumentation, which identified elements of an argumentative text and the ways in which argumentative claims are supported. Another important theory of argument, called “new rhetoric,” was developed by Perelman and Olbrechts-Tyteca (1969), who considered, in great detail, the means that speakers or writers use to obtain “adherence” of an audience. Furthermore, the pragma-dialectical theory of Van Eemeren and Grootendorst (1992), which is a consensual conception, portrays the aim of argumentation as the resolution of differences of opinion held by speaker (or writer) and listener (or reader).

These diverse definitions of argumentation converge in the way they conceive the pragmatic criterion of goals: they assume that argumentation is a verbal activity for which the aim is to generate a change of attitude in the addressee with respect to a viewpoint. It is here that some theories of argumentation explicitly or implicitly encounter the issue of persuading others of one’s position. However, not every argumentative activity has a persuasive aim. Thus, Nussbaum (2008a) distinguished two goals when elaborating argumentative texts: a persuasive goal, or a reflective goal. Reflective writing focuses on “exploring and integrating various sides of an issue in order to reach a reasoned conclusion” (p. 551).Footnote 1

Argument-counterargument integration is a complex process since individuals tend to reason in biased ways, specifically by searching for evidence that supports their prior beliefs and by ignoring counterevidence (Nickerson, 1998; Villarroel et al., 2016). According to Nussbaum and Schraw (2007), “effective argumentation involves not only considering counterarguments but also evaluating, weighing, and combining the arguments and counterarguments into support for an overall final position” (p. 60). There are several strategies to achieve argument-counterargument integration. These authors identified as a ‘weighing’ strategy when the arguer considers both sides of a controversial issue, weighs the advantages and disadvantages and ends up supporting the position with stronger arguments. Another strategy defined by these authors is the ‘synthesis’ strategy, which consists of finding a creative solution, or a compromise solution, that brings together the best of the two opposing points of view. The third and final strategy for argument-counterargument integration, identified by these authors, is a ‘refutation’ strategy. Through the refutation strategy, the individual tries to show that some arguments related to the topic are false, irrelevant, or insufficiently supported. When the goal of argumentation is persuasion, refuting the arguments of the opposing position is considered an appropriate strategy. However, if the purpose is to explore different perspectives on a topic, reconcile positions and reach a collaborative, reasoned and informed conclusion, although refutation can be employed selectively, it may be more appropriate to use weighing and synthesising strategies.

Considering this background, this study focuses on argumentative writing in order to investigate different perspectives on a controversial topic and reach an integrative conclusion by making use of the three strategies considered above: weighing, synthesising, and refutation. Given that the different perspectives are presented to the student through the reading of two source texts that defend different or contradictory positions, the writing of an argumentative text with these characteristics could be understood as a task that involves the synthesis of different sources.

Writing a synthesis from multiple sources is a task that requires organizing, selecting, and connecting information from more than one text (Nelson & King, 2022; Spivey, 1997). This involves the ability to connect information within each text (intratextual integration), as well as connecting ideas from different texts (intertextual integration) (Segev-Miller, 2007). These sources may offer complementary or contradictory information, which the learner has to contrast and, ideally, integrate. When sources offer conflicting positions on a controversial issue, and the task requires arguing a conclusion, students not only have to identify and contrast them but are encouraged to integrate them, which could promote more constructive learning (Mateos et al., 2011, 2018; Wiley et al., 2014).

In addition, as studies such as that of Wiley et al. (2014) showed, the goal or purpose of synthesising multiple texts that present conflicting information on a topic influences the way in which these sources are integrated. Their results suggested that a position-based writing prompt (asking writers to take a position and support it) could lead students to take a perspective early on and to attend only to information that supported it. By contrast, when given a contributing-factors writing prompt, requiring them to construct an explanation, they considered the information provided by the different documents, favouring the inclusion of more varied perspectives than a position-based question.

The interest of our line of research has focused on synthesis writing tasks that require arguing from multiple texts that present conflicting positions on an issue with a reflective or deliberative, rather than a persuasive, goal. This type of task is necessary when students are faced with academic and/or scientific work, which is closely linked to the university context. These tasks require students to read a variety of documents that often present opposing evidence. The ability to identify arguments in one or several texts, to evaluate the relevance or quality of the reasons that support them, and to contrast and integrate them is a challenge of academic literacy (Britt et al., 2014).

A review by Barzilai et al. (2018) revealed the problems associated with the integration of information from different sources. Among the difficulties, reading and contrasting information from more than one text could increase the complexity of the task. When a synthesis is written from multiple sources it is necessary to make decisions about the structure in order to integrate the information from the different texts, unlike writing a summary where the same pattern used in the original text can be repeated (Mateos & Solé, 2009). In addition, when sources present conflicting and contradictory information on a topic, the understanding of the texts can also be affected, since students usually have difficulties including in their texts references to the arguments that defend positions contrary to their own, tending to ignore or exclude evidence against their own position. This is what has been termed in the literature as ‘my side bias’ (Wolf et al., 2009). This tendency, which has been shown in research into the writing of argumentative texts, (Kuhn, 1991; Nussbaum & Schraw, 2007; Schwarz, 2009) is also reflected in the writing of argumentative syntheses from sources that present contradictory positions on a topic (Cuevas et al., 2016; Mateos et al., 2018). In this way, students often ignore relevant information that is inconsistent with their own perspective and that supports another point of view. As a result students develop little argumentation and are insensitive to alternative perspectives (Song & Ferretti, 2013).

All the recurrent problems mentioned above illustrate a low-performing writer profile that is frequent among university students who could greatly benefit from the aid offered. Among all the difficulties the one that is particularly interesting to teach in this study is that of integrating information from multiple contradictory sources. Although college students are expected to develop proficiency in communication, including the ability to write at an appropriate academic level, and to create well-structured arguments, research indicates that they are not adequately prepared (Butler & Britt, 2011).

To the best of our knowledge there are no previous studies that have examined the differences between university students with different levels of performance in argumentative synthesis-writing tasks from multiple texts. Nor are there any studies regarding the differential impact of interventions for the improvement of argumentative synthesis writing depending on the skills of the university writer. Nevertheless, there is some prior research that shows some differences in writing in general and in writing persuasive essays among students with different levels of proficiency. Several studies, based on classical models defining the different strategies used by expert and novice writers, suggest that expert writers or high-achieving university students employed strategies for planning, setting goals, monitoring, textualizing and revising more frequently and thoroughly than novices or low achievers (Chien, 2012; Rahmawati et al, 2019; Rijlaarsdam et al., 2012). In the context of persuasive essay writing, Perin et al. (2017) found low levels of literacy skills in a sample of pre-university students. These authors suggest that an improvement in general reading skills may promote a better text-based summary, while improvement in general writing skills can be key to better persuasive essay-writing ability. Although research in primary and secondary education has found that intervention to improve writing is not equally effective for all types of writers, suggesting that those with lower skills may benefit more than those more skilled (see Rijlaarsdam et al., 2012), the literature in higher education is scarce. Since most of the studies reviewed do not take into account how expert writers differ from novice writers, differences in writing processes between these groups may determine many of the results found. Van den Bergh et al. (2015) suggested that to avoid this fallacy, only one group should be considered. Thus, more research is needed to support low achieving writers.

Therefore, in response to the need to help university students deal with such tasks, interventions have been developed to teach the processes of identifying, contrasting, and integrating arguments and counterarguments from sources that present opposing positions on a controversial topic.

Previous intervention studies on argumentative synthesis tasks with university students

Previous research has shown that both explicit instruction on strategies and collaborative writing tasks were effective in improving argumentative synthesis writing. The following are the studies that, from these two fields, have supported our subsequent research.

Explicit instruction on writing strategies

Instructional strategies aimed at improving argumentative synthesis writing have been influenced by instructional research related to argumentative writing as well as interventions into synthesis from multiple sources. In the former, from a cognitive approach this task is viewed as a problem-solving process in which students learn to write an argument through processes of self-regulation and/or explicit instruction. Authors such as Ferretti and Lewis (2013), stated that the key is to use explicit instruction, along with discussion, modelling and practice, to scaffold the process of acquiring and applying writing strategies, especially for struggling writers. Some research has emphasized self-regulation strategies. In this context, Song and Ferretti (2013) carried out a study to analyse the effect of instruction in self-regulation strategies in the use of critical questions and argumentation schemes. The latter are defined by the authors as “conventionalised ways of representing the relationship between what is asserted in the viewpoint and its supporting justificatory structure” (p. 69). They concluded that teaching students to answer critical questions about argumentation schemes was necessary for them to consider perspectives contrary to their own and to include more counterarguments and rebuttals in their writing.

Teaching self-regulation strategies has not been the only tool that has proven useful in improving argumentative writing. Argumentation schemas (Wolf et al., 2009) is a way of organising knowledge that helps in argumentative writing by answering questions evoked by source texts. It have been used to help students graphically represent arguments, reasons, and counterarguments, facilitating the refutation of points of view opposed to their own and consequently improving the quality of their writing. In the same vein, Butler and Britt (2011) designed instruction to scaffold the revision of argumentative essays. These authors found that argument schemas, which encouraged students to not necessarily accept a single position, and which demonstrated the importance of including counterarguments and refutations, could help them to make changes at a higher level by including more argumentative content and consequently improving their essay structure.

In a review study in the field of learning to write source-based synthesis texts, Van Ockenburg et al. (2019) stated that in order to acquire synthesizing skills it was necessary to involve students in a set of learning activities that promote all three transformational processes: selecting information from sources, organizing, and connecting that information. Indeed, this third process of connecting information between texts, known as intertextual integration, is an important aspect of synthesis writing. In this regard, a review by Barzilai et al. (2018) revealed that the most frequent instructional strategies and practices were to engage students in collaborative discussions and practices, facilitate explicit instruction on integration, provide guidelines for the integration process, manage graphic organizers, model the integration process, provide individual practice, and give feedback to students.

Collaborative writing

Among the instructional components and strategies that have been mentioned, collaboration is of special interest in this work, since it could contribute to the improvement of writing, in the framework of socially-regulated learning, by combining these two constructs: the construction of shared meanings and co-regulation (Volet et al., 2009). This is because during collaborative writing a series of exchanges takes place that facilitates the elaboration of thought, the relation of information with prior knowledge, and/or the understanding of the content (Nykopp et al., 2014). It is common that during this negotiation process, previous representations and ideas are made explicit, contrasted, and modified (Coll & Onrubia, 2001). This can promote a deeper understanding of content and constructive learning due to the inherently dialogical nature of argumentation (Ferretti & Lewis, 2013; Nussbaum, 2008b). Although discussions and collaborative practices may be positive for dealing with this task, it should not be assumed that students already have the skills to work effectively with others (Thomas, 2014).

Dillenbourg (2002) stated that it is important to teach students how to work together. This instruction can be directed to structuring the collaborative process and to regulating interactions, which includes dealing with problems that may arise. To achieve this, explanations or scripts that provide instructions on how to interact and solve problems can be administered (Scheuer et al., 2014). Representations of the process can also be provided to students through modelling or observational learning (Dale, 1994). A relevant study by Scheuer et al. (2014) incorporated a tool that helped regulate collaborative strategies (collaboration scripts) together with another aimed at supporting argumentative writing (argumentation diagramming). The results showed that combining task-focused and collaboration-focused aids was more effective than using argumentation diagrams exclusively. Although offering collaborative aids can be beneficial to the writing process, Kimmerle et al. (2017) pointed out that each specific collaborative stage of the writing process (i.e., knowledge introduction, information restructuring, and shared opinion), could require specific support.

Previous studies within our line of research

In line with the previous research reviewed, a study by Mateos et al. (2018) designed two intervention programmes to improve the teaching of argumentative syntheses from contradictory sources. The quality of the argumentative syntheses was measured using two variables: the coverage of arguments from different sources and the level of integration of the information. The first intervention included collaborative practice in pairs with the support of a guide (CPG) that intended to lead the students towards writing an argumentative synthesis. The second condition (CPG + EI) also added the explicit instruction and modelling in the strategies used in writing an argumentative synthesis text. Only those students who received additional explicit instruction improved their ability to integrate conflicting information and increased the number of arguments they selected from the sources. These findings suggested that explicit instruction was the key to improving argumentative synthesis writing; however it was shown that the learning path was not the same for the different processes involved in the task: argument identification and integration level of arguments and counterarguments (Mateos et al., 2020). This study showed that in the case of argument identification, the practice in pairs sessions provided an additional learning pathway that positively affected the results from the preceding explicit instruction. That is, explicit instruction showed an indirect effect through practice in pairs to improve argument identification. By contrast, for the other learning outcome (i.e., integrating arguments from various sources), a direct effect of explicit instruction was found, indicating that practice in pairs did not contribute to the final scores.

In these previous studies, one issue that has remained unresolved is that students’ difficulties with the integration task might partly be due to the difficulties they encounter in collaborative writing. This is why we saw a need to analyze the specific role of explicit instruction through explanations and modelling in collaborative processes during synthesis writing was relevant. For this purpose, we conducted a new study (Granado-Peinado et al., 2019) to analyse differential effects of support for collaboration and support for writing argumentative syntheses when the two were combined. The results showed that the most effective way to teach how to write collaborative argumentative syntheses from multiple and contradictory sources required explicit instruction. When explicit instruction was designed to improve both the writing process and collaboration, students achieved higher levels of integration. However, to identify a larger number of arguments from the source texts, explicit instruction that focuses solely on helping students to write argumentative syntheses has proved to be as effective as help directed at collaboration. This work showed that collaboration can contribute to improving the quality of synthesis writing as long as there is an instruction that models and unravels the processes that underlie it. However, this study raised questions that we intend to answer in the current study.

The present study

Our previous results were obtained from a study comprising a heterogeneous sample of students, who started from different levels of initial expertise in writing collaborative argumentative synthesis. Since the students were not selected based on their initial proficiency level, the final sample did not enable the distribution of the students into two homogeneous groups of either high or low level students. Therefore, in this study we have analysed the differential effect of combining explicit instruction in both writing and collaborative processes when all pairs started from a low level of initial proficiency in the construction of argumentative syntheses. On the other hand and based on the evidence found that there are two different learning paths for the two elements of argumentative synthesis writing: argument identification and integration level of arguments and counterarguments (Mateos et al., 2020). In this current study we wanted to corroborate whether there could also be any direct and/or indirect effects of explicit instruction (mediated through practice in pairs) during the construction of argumentative synthesis written in collaboration. For this purpose, it was necessary to additionally analyse intermediate syntheses written during the practice in pairs sessions.

Therefore, in this context, the following question arises: What will be the effect of explicit instruction and collaboration practice on the quality of the syntheses written collaboratively by pairs of students who start from a low level of expertise (i.e., previous writing of non-integrative syntheses that argued from a single position and ignored a contradictory position)? Explicit instruction included instruction in synthesis writing and in the collaboration process. Furthermore, and since it was not considered in the previous study (Granado-Peinado et al., 2019), it would also be interesting to answer the question: Is the effect of the explicit instruction mediated by the practice in pairs?

To answer these questions, we sought to analyze the following:

  1. 1.

    The differential impact of an intervention programme which combined explicit instruction with video-modelling of the writing and collaboration processes together with a guide and collaborative practice in pairs (CPG + EICSFootnote 2), when compared with three other programmes: (1) Collaborative practice in pairs with a written guide supported by explicit instruction about writing synthesis (CPG + EISFootnote 3); (2) Collaborative practice in pairs with a written guide (CPGFootnote 4); and (3) Collaborative practice in pairs (CPFootnote 5), when all pairs of students start from a low initial proficiency level (non-integrative syntheses).

  2. 2.

    The extent to which the impact of the explicit instruction in each of the dimensions of the product analysed, affects directly (based on the initial instruction session) and/or indirectly (through the practice in pairs sessions) the quality of the final collaborative syntheses (post-test).

In keeping with these objectives, this study addressed the following hypotheses.

As in the related previous study (Granado-Peinado et al., 2019), we expected that:

  1. 1.1

    After intervention, the pairs that received explicit instruction, the support of a guide and practice in pairs (CPG + EICS and CPG + EIS) would write better quality syntheses in terms of argument identification than those completing the other programmes (CPG and CP).

  2. 1.2

    Regarding the level of integration, the pairs of students that received explicit instruction on both writing and collaborative processes, the support of a guide and practice in pairs (CPG + EICS) would write better quality syntheses than those completing the other three programmes (CPG + EIS, CPG and CP).

Additionally, in line with previous study (Mateos et al., 2020), we expected that:

  1. 2.1

    There will be a direct and an indirect effect of explicit instruction via the practice in pairs on the quality of final collaborative synthesis, in terms of argument identification.

  2. 2.2

    By contrast, in relation to the level of integration, there will only be a direct effect of explicit instruction on the quality of final collaborative synthesis.

Method

Participants

Participants were 112 third-year psychology students comprising (13,6% male and 86,4% female) who were randomly assigned into 56 pairs that were maintained during the four sessions of the programme. This sample was selected from 160 students who voluntarily signed up to participate in this research, which was part of a writing workshop and for which they would receive some academic credit. Those who did not participate in this activity could earn the credits through other assignments set by the teachers of the different subjects.

The criterion for inclusion in this sample was to present low levels of argument integration. The students were encouraged to carry out a first collaborative synthesis (pre-test) where a mean of 2.39 points (SD = 0.87) out of a maximum of 6 points was obtained. Therefore, the mean level of argumentative integration was low although some pairs of students did produce integrative syntheses, scoring a maximum of 5 in our coding system.

For this reason in order to achieve a homogeneous sample, and to be able to verify whether the designed intervention was beneficial only for low proficiency pairs of students, we only selected those pairs who failed to overcome the my side bias effect. That is, someone who defends a position with reasons and arguments from only one source text, tending to ignore evidence against their own position (Wolf et al., 2009). Thus, this final sample of 56 pairs of students, which represented 69% of the potential participants at baseline, showed a low initial proficiency to write a collaborative argumentative synthesis.

All the pairs of students were randomly distributed into the four intervention programmes, depending on the aid offered, as mentioned above but repeated here: collaborative practice in pairs with a written guide supported by explicit instruction about collaborative writing synthesis (CPG + EICS); collaborative practice in pairs with a written guide supported by explicit instruction about writing synthesis (CPG + EIS); collaborative practice in pairs with a written guide (CPG); and collaborative practice in pairs (CP). The final distribution of the four intervention programmes is shown in Table 1.

Table 1 Distribution of the 112 students in the four intervention programmes
Full size table

Instruments and material

Intervention programmes

Four intervention programmes were designed with the same objective of improving the quality of argumentative syntheses of pairs of students. Each programme included different components (CPG + EICS; CPG + EIS; CPG; and CP), as mentioned above. The components are briefly detailed below (for more details, see Granado-Peinado, 2019).

Explicit instruction with video-modelling

In the programmes that included explicit instruction (CPG + EICS and CPG + EIS), one of the researchers explained the activities and processes involved in the synthesis writing task, emphasising the acquisition of knowledge by integrating information from different perspectives. These processes, which were instructed with an emphasis on their recursivity, concerned: (1) Reading source texts; (2) Identifying arguments from each position on the topic; (3) Contrasting arguments from both positions; (4) Developing an integrative conclusion that considers both positions; (5) Organising ideas to transfer to the written text; and (6) Revising the written text. This instruction was accompanied by a video modelling in which a pair of students performed the synthesis-writing task with a high level of proficiency.

In addition, the instruction for CPG + EICS participants also included the potential benefits of collaboration. To this end, the researcher explained different strategies for: (1) Constructively resolving controversies arising from the topic of discussion; (2) Actively listening to each other; (3) Adopt each other’s point of view, without imposing one’s own, and being open to change one’s opinion if necessary; (4) Mutually regulate both the processes of the writing task and the roles one might adopt. In this case, the video accompanying the instruction on effective collaboration showed how a pair of students, while producing a written synthesis, collaborated effectively and put the instructed strategies into action. Thus, the video model also showed examples of strategies that could help a difficult collaboration.

Guide

A guide adapted from previous studies by Mateos et al. (2018) was given to the participants in the CPG + EICS, CPG + EIS and CPG programmes (see online Appendix 1). This guide, with a graphic format, comprised a table to identify and interrelate the arguments in both positions. In addition, it included three blocks of questions to guide the reflection of the pairs of students towards the writing of an integrative conclusion, the organising and textualizing ideas and revising the final text. In the case of the most complete programme (CPG + EICS), the guide also included a list of suggestions on how to work constructively as a pair and regulate their interactions.

Collaborative practice in pairs

All the pairs of students participated in two practice in pairs sessions, writing an argumentative synthesis in pairs in each of these sessions.

Texts for collaborative writing synthesis tasks

We construct four pairs of argumentative source texts, one for each session of the programme. The texts were created by the researchers using topical issues in the field of educational psychology, a subject that all students had taken. This ensured that the content was comprehensible to all and ensured that, for design purposes, all texts contained the same parameters, controlling for structure, number of arguments, length, and reliability.

All the pairs of students in the four intervention conditions received the same texts. The four pairs of texts provided conflicting information on a controversial topic in education, representing one position for, and one position against, the debate in question: (1) School Day; (2) Teacher Evaluation; (3) External Student Evaluations; (4) System of Access to the Teaching Profession. As an example, in the first of the topics, the source advocating divided time for school states that it is necessary for children to have enough time in the middle of their day to rest, regain their attention, and develop informal learning. By contrast, the source advocating continuous time states that it is necessary to have afternoons free for homework and other leisure activities. A table with the arguments on this topic (school day) is included in online Appendix 2.

All texts were written in Spanish and had a similar argumentative structure, with an equivalent number of arguments and counter-arguments (between 8 and 9), a length ranging from 609 to 867 words, and good readability (Szigriszt-Pazos (2001) index between a 44.5–55.5, range which is suitable for high school and university students).

Procedure

Design and implementation

This study involved a total of four 90-min sessions per week for four consecutive weeks. The same researcher conducted all the sessions. In all four sessions, each time the pairs of students were asked to write a collaborative argumentative synthesis, the instruction was the same: “You are going to read two texts on a controversial topic in the field of education that present different positions. When you have finished you have to write a conclusion arguing from what you have read and from your own ideas on the topic”.

The first session focused on assessing the pairs of students’ initial skills in preparing collaborative syntheses before participating in the intervention (prior collaborative synthesis). As mentioned above, from the total number of students who initially participated in the research, only those pairs who in this first session showed a low initial proficiency to write a collaborative argumentative synthesis were selected.

After the initial assessment, the second session was aimed at receiving the intervention according to the programme to which they had been assigned. To summarise, the CPG + EICS and CPG + EIS programmes received explicit instruction with video modelling accompanied by a guide aimed at improving the quality of the syntheses. The difference between these two programmes was that the former also included instruction on how to collaborate effectively. By contrast, the students in the CPG programme only received the guide in this session, without any instruction about its purpose or how to use it. Finally, those in the CP programme received no instruction or support material. The pairs of students assigned to the programme that did not include any explicit instruction (CPG and CP) were asked to complete a reading comprehension task to ensure that the instruction time was the same across the four programmes. Subsequently, after receiving the explicit instruction or reading comprehension activity, the pairs of students in all programmes completed a collaborative synthesis task (first intermediate collaborative synthesis) using the help obtained in each programme.

In the third session, the pairs carried out a new collaborative synthesis writing task with the aids received in each programme (second intermediate collaborative synthesis).

Finally, in the fourth session, the pairs of students wrote the final collaborative synthesis (final collaborative synthesis), in this case without using the aids offered.

Treatment fidelity

In order to ensure the fidelity of the implementation of the intervention for all the pairs of students, a script was prepared with the content to be covered in each intervention programme. This script was used by the researcher in charge of each session, thus ensuring that the same order was followed in the explanation of each component. The participants were not allowed to intervene during the intervention session to ensure that the information conveyed by the researcher was the same in all four-intervention programmes. Also, to ensure that the syntheses produced were completed as instructed the participants were required to hand them in before leaving the classroom, and to sign an attendance sheet. On average, the syntheses produced included a mean of 429 words (SD = 83.54).

Coding system

Two independent judges evaluated 28,57% of the syntheses out of a total of 224. In cases where there was no consensus, the judges discussed the presence of different argumentative strategies and agreed on the most appropriate score for the synthesis. Inter-judge agreement was calculated using the Intraclass Correlation Coefficient (ICC). In particular, we used a Two-Way Mixed-Effect Model absolute agreement for the mean of the two raters (Koo & Li, 2016). In addition, Cronbach’s Alpha coefficient for consistency was also calculated. A good degree of reliability was achieved for the two dimensions. For the proportion of identified arguments an ICC = 0.890, CI 95% [0.818–0.933] and Alpha = 0.888 were found. For the level of integration an ICC = 0.831, CI 95% [0.721–0.898] and Alpha = 0.835 were found. Once consensus was reached, one of the researchers evaluated the remaining syntheses following the criteria established after inter-judge agreement.

The quality of the four collaborative argumentative syntheses was assessed in two dimensions: argument identification (the proportion of arguments identified in the source texts) and integration level (using a coding system developed by Mateos et al., 2018). The scale for the latter ranged from 0 to 6 points (see Table 2).

Table 2 Levels of the coding system for the quality of argumentative synthesis regarding the level of integration.
Full size table

On this scale, a score between 0 and 2 indicated that the pair of students did not relate the two positions in the debate indicating that integration had not been achieved, while scores above 2 showed different levels of integration: A score of 3 implied integration of arguments and counter-arguments through a refutation strategy, taking a single position. Scores 4 to 6 implied the use of weighing and synthesising strategies, representing a text with a higher level of integration. Therefore, the students were able to produce syntheses defending only one position (0–2) or including arguments and counter-arguments from the two source texts (3–6). A description of the categories of analysis of the level of integration and examples of the strategies employed by the pairs of students are shown in online Appendix 3.

Data analysis

Although the final sample was composed of 112 students, in 56 pairs, the unit of observation was each pair score because the written synthesis was unique for each pair. In the following analysis, a sample size of 56 cases (pairs) was then used.

Firstly, to analyse the effect of the programmes on the quality of the argumentative syntheses written in collaboration, two ANOVA models were conducted (one for each dependent variable): the four sessions form the within-subject factor and the programme form the between-subject factor. To test the hypotheses formulated previously (see 1.1. and 1.2 hypotheses) two planned comparisons were carried out: firstly, we compared the population means between the programmes that included explicit instruction (CPG + EICS and CPG + EIS) vs. programmes that did not include it. The second comparison was made between the population means of CPG + EICS (i.e. the programme including explicit instruction on both writing and collaborative processes) vs. CPG + EIS (i.e. the programme including explicit instruction only for the writing processes). For the planned comparison we used one-tailed tests. Secondly, to test the 2.1 and 2.2 hypotheses, we analysed whether the effect of the programme on the quality of the argumentative syntheses could be mediated by the practice in pairs, or if in the absence of any direct effects, indirect effects emerged, two mediation analyses were carried out, one for each dependent variable (i.e. argument identification and level of integration).

The Mplus v7.0 structural equation modelling software was used with a bootstrapping estimation method (10,000 bootstrap samples). In addition, to perform both mediation models the programme (independent variable) was treated as ordinal variable (coded as CP = 0, CPG = 1, CPG + EIS = 2, and CPG + EICS = 3). This was because the four instructions examined included components in a hierarchical/cumulative way (CP < CPG < CPG + EIS < CPG + EICS). This assumption was supported empirically (see below Tables 3 and 4, and Figs. 1 and 2) as the quality of the written syntheses increased according to the complexity of the instruction.

Table 3 Sample size, mean proportion and standard deviation of “arguments identification” in the four collaborative syntheses, based on the four-intervention programme
Full size table
Table 4 Means and standard deviation of “level of integration“ in the four collaborative syntheses, based on the four-intervention programme
Full size table
Fig. 1
figure 1

Means and error bars (CI 95%) for “arguments identification” in the four collaborative syntheses, based on the four-intervention programme. Note: CPG + EICS: Collaborative practice with a written guide supported by explicit instruction about collaborative writing synthesis; CPG + EIS: Collaborative practice with a written guide supported by explicit instruction about writing synthesis; CPG: Collaborative practice with a written guide; CP: Collaborative practice

Full size image
Fig. 2
figure 2

Means and error bars (CI 95%) for “level of integration“ in the four collaborative syntheses, based on the four-intervention programme. Note: CPG + EICS: Collaborative practice with a written guide supported by explicit instruction about collaborative writing synthesis; CPG + EIS: Collaborative practice with a written guide supported by explicit instruction about writing synthesis; CPG: Collaborative practice with a written guide; CP: Collaborative practice

Full size image

Results

Analysis of the impact of the programmes on the quality of the collaborative synthesis (argument identification and level of integration) generated after the intervention

Argument identification

A first ANOVA was carried out with the programme as the between-subject factor (four programmes) and session as the within-subject factor (four moments). The results showed a main effect of the intervention programme factor (F(3,52) = 6.433; MSe = 0.216; p = 0.001; η2 = 0.271), a main effect of the within-subject factor (session) (F(3,156) = 35.889; MSe = 0.483; p < 0.001; η2 = 0.408), and a marginal interaction effect (F(9,156) = 1.681; MSe = 0.023; p = 0.098; η2 = 0.088). The means and standard deviations for the variable ‘argument identification‘ are shown in Table 3.

Figure 1 shows the means and error bars (CI 95%) for the argument identification. This figure shows how the change in argument identification means is not the same for the four programmes, (i.e., the marginal interaction effect). A more pronounced increase in the integration levels can be seen with the CPG + EICS and CPG + EIS programmes than with the other two programmes from prior collaborative to first intermediate collaborative moments. The first planned comparison considers the CPG + EIS and CPG + EICS programmes together vs CPG and CP together. Significant mean differences were found between the two groups (the difference was 0.098 points higher for the explicit instruction groups, se = 0.025, p < 0.001). The second planned comparison analyzed whether there were any significant differences between the CPG + EICS vs CPG + EIS groups. The difference was marginal with 0.052 points greater for the CPG + EICS than for the CPG + EIS group (se = 0.036, p = 0.075). Non-significant differences were found in the means of the CPG and CP groups (0.004, se = 0.034, p = 0.451).

Level of integration

In this case, because of the pairs of students were selected with the same prior level of integration (‘argues in support’, level 2), the ANOVA model has only a descriptive value. The results showed a main effect of the intervention programme factor (F(3,52) = 17.033; MSe = 24.61; p < 0.001; η2 = 0.496), a main effect of the session factor (F(2,121) = 13.786; MSe = 25.50; p < 0.001; η2 = 0.210), and an interaction effect (F(7,121) = 2.61; MSe = 4.90; p = 0.014; η2 = 0.133). The descriptive data are presented in Table 4 (Pearson correlations between level of integration variables and argument identification variables can be seen in Table A1 in the online Appendix 4).

Figure 2 shows the means and error bars (CI 95%) for the level of integration for all the programmes in all the study moments. Again, it was compared CPG + EIS and CPG + EICS programmes together vs CPG and CP together (first planned comparison). The results showed significant mean differences between the two groups (the difference was 1.06 points higher than for the explicit instruction groups, se = 0.162, p < 0.001). In addition, the population means of the CPG + EICS and CPG + EIS programmes were compared. Significant mean differences were found (the difference was 0.442 points higher for the CPG + EICS instruction group programme with respect to the CPG + EIS group, se = 0.235, p = 0.033). Non-significant differences were found in the means of the CPG and CP groups (0.124, se = 0.223, p = 0.286).

In summary, these results show that the pairs of students in the explicit instruction programmes (CPG + EICS and CPG + EIS) significantly outperformed the quality of the syntheses than the non-explicit programmes (CPG and CP). In addition, the most complete programme, which included the explicit instruction on both writing and collaboration processes, the support of a guide, and practice in pairs (CPG + EICS), improved the quality of the syntheses in the two dimensions of analysis to a greater extent than CPG + EIS (marginally in the case of the identification dimension and significantly in the case of the integration dimension).

Mediation analysis to examine the effect of the intervention programmes on the quality of collaborative argumentative synthesis through practice in pairs

With regard to the second objective, we carried out a mediation analysis with structural equation modelling using the Mplus 7.0 program, with 10,000 bootstrap samples (95% confidence interval). As explained in the data analysis, programme (the independent variable) was treated as an ordinal variable (coded as CP = 0, CPG = 1, CPG + EIS = 2, and CPG + EICS = 3).

Argument identification

See Fig. 3 for a depiction of the mediation analysis. The results indicated that there was a direct effect of the programme on the 1st Intermediate Collaborative Synthesis written during the practice in pairs (a1 = 0.064, se = 0.012, p < 0.001); a marginal direct effect on 2nd (a2 = 0.036, se = 0.021, p = 0.081), but a non-significant direct effect of the programme on the final collaborative synthesis (a3 = 0.010, se = 0.023, p = 0.674). In addition, a total indirect effect of the programme on the final collaborative synthesis mediated by the first and second intermediate collaborative synthesis was found (total indirect effect = 0.050, se = 0.021, p = 0.014, CI 95% [0.010–0.091]).Footnote 6

Fig. 3
figure 3

Summary of the mediation analysis model when the dependent variable is “argument identification“

Full size image

In summary, the mediation analysis indicated that for argument identification, the effect of the programme on the final collaborative synthesis is completely mediated by the quality of synthesis written on practice in pairs sessions.

Level of integration

See Fig. 4 for a depiction of the mediation analysis. The results indicated that there was a direct effect of the programme on the quality of both the first and second intermediate syntheses (a1 = 0.796, se = 0.146, p < 0.001; a2 = 0.650, se = 0.204, p = 0.001) and a marginally significant direct effect of the programme on the final quality of the syntheses (a3 = 0.413, se = 0.235, p = 0.079). Contrary to the argument identification dependent variable, no statistically significant indirect effect was detected for level of integration and the (total indirect effect = 0.153, se = 0.189, p = 0.419, CI 95% [− 0.218–0.523]).

Fig. 4
figure 4

Summary of the mediation analysis model when the dependent variable is “level of integration“. Note: Programme variable is coded as ordinal variable: 0 = Collaborative practice; 1 = Collaborative practice with a written guide; 2 = Collaborative practice with a written guide supported by explicit instruction about writing synthesis; 3 = Collaborative practice with a written guide supported by explicit instruction about collaborative writing synthesis. The score in parentheses is the total indirect effect of the program on the final collaborative synthesis.t Significance level of p < .10, * p < .05, ** p < .01, and *** p < .001

Full size image

The mediation analysis for level of integration showed a direct effect of the programme on the integration quality of the syntheses in the two intermediate and final moments. In this case, synthesis written on practice in pairs sessions did not reinforce the effect of the programme in the final synthesis.

Taking this result into account, and in addition to those obtained in the descriptive analysis, we concluded that explicit instruction on both the writing and collaboration processes could be important to develop higher integrative collaborative syntheses.

Discussion

Contributions of the study

The present study was designed to meet two main objectives. The first was to test the effectiveness of an intervention programme (CPG + EICS) in comparison with three other intervention programmes in which the help provided to the students was progressively reduced, on the final quality of collaborative written syntheses, in a sub-sample of pairs of students who showed a low proficiency level in the collaborative writing of argumentative synthesis. The second objective was to evaluate the direct and indirect effects via the practice in pairs of the explicit instruction on the quality of the final syntheses.

Regarding the first objective, the results corroborated the first and second hypotheses. In accordance with our expectations, differences were found between the effects of the two programmes that included explicit instruction (CPG + ECIS and CPG + EIS) versus those that did not, in either of the two dimensions of analysis (CPG and CP). In addition, the pairs with low proficiency in the more complete programme (CPG + EICS) identified a higher proportion of arguments and achieved a higher level of integration than those who did not receive explicit instruction on collaboration processes (CPG + EIS). In summary, offering explicit instruction focused on the processes of writing and collaboration jointly was the most effective aid to improving synthesis in the two dimensions of analysis. Thus, explicit instruction helped students overcome the “my-side bias” (Wolf et al., 2009) by including relevant arguments and information from the two contradictory sources in their syntheses.

Concerning argument identification, these results corroborate the importance of providing explicit instruction. However, although in previous studies to achieve a synthesis with a higher proportion of arguments identified from sources, explicit instruction was effective in the writing processes, or in both writing and collaboration processes (Granado-Peinado et al., 2019; Mateos et al., 2018), in the present research a slight difference has been found. Our findings show that argument identification was higher when explicit instruction addressed the process of writing and the process of collaboration.

Thus, explicit instruction in collaborative processes has proven to be important in teaching students with low initial proficiency in synthesis writing to identify arguments. Similar results were found for the level of integration, supporting the previous evidence found (Granado-Peinado et al., 2019). Improving the level of integration of arguments and counterarguments required explicit instruction, compared with the programmes that did not include any explicit instruction. Again, the pairs that received explicit instruction focused on both writing and collaborative processes achieved higher levels compared with those who were only instructed in the writing processes.

The process of integrating information from multiple sources has proven highly challenging for students (Barzilai et al., 2018). Reading and contrasting information from multiple texts, especially when they are contradictory, increases the complexity of the task (Mateos & Solé, 2009; Wolf et al., 2009). Furthermore, although there is little evidence in this field, research suggests that low-proficiency university students may use fewer strategies than high achievers (Rahmawati et al., 2019; Rijlaarsdam et al., 2012). Therefore, explicit instruction aimed at unravelling both the writing and collaboration processes could benefit pairs of students with low synthesis writing skills to a greater extent in the two dimensions of the analysis, compared to that which was found in the previous study, where the explicit instruction of collaborative processes only contributed significantly to the level of integration dimension. In this sense, the explicit instruction of the more comprehensive programme helped, on the one hand, students to use in their syntheses strategies involving balanced-reasoning, such as “weighing” and “synthesising”, to achieve argument-counterargument integration (Nussbaum & Schraw, 2007). On the other hand, it also allowed benefiting from collaboration during the practice in pairs. These results support the idea that it is important to teach collaborative strategies explicitly, since it cannot be assumed that students know how to work together (Thomas, 2014).

Continuing with the hypothesis testing, the third and fourth hypotheses linked to the second main objective were confirmed. In terms of argument identification, when the two intermediate syntheses were analysed, practice in pairs mediated the effect of explicit instruction. This implies that practice in pairs reinforced the effect of explicit instruction on both the writing and collaborative processes (CPG + EICS). However, when the analysis focused on the level of integration, we found a direct impact of the explicit instruction. In this case, the practice in pairs did not mediate the effect of the explicit instruction on the level of integration.

These results support the evidence found and offer a better understanding of the different interventions. When explicit instruction focuses on both the writing and the collaboration processes/or the writing processes alone, a direct effect on the quality of the intermediate and final syntheses is observed regarding to level of integration. In addition, synthesis written during the practice in pairs fully mediated the effect of the explicit instruction only on the dimension of argument identification. These findings are similar to those obtained in the previous study by Mateos et al., (2020), where it was found two learning paths based on the two variables analysed: argument identification and level of integration. These two learning paths were found in the same direction: explicit instruction was a direct effect on both variables, while practice in pairs only had an additional indirect effect on argument coverage. These results could be indicating that, in line with the research by Kimmerle et al. (2007), students need different aids to learn to write argumentative syntheses depending on the stage of the collaborative writing process that they are in. In this sense, the programme that included explicit instruction on collaborative processes had the greatest effect on the two dimensions of the argument synthesis quality: argument identification and level of integration. This more complete instruction helped directly in the integration dimensions of quality of the syntheses and, in the case of the argument identification; the practice in pairs reinforced this effect of the instruction on the quality of the final synthesis.

In this way, a tentative explanation could be that the help that explicit instruction on collaborative processes can offer to improve the quality of argumentative syntheses is different depending on the nature of these two processes. Identifying arguments is a simpler and less demanding process than integrating arguments. As a result, explicit instruction on the writing process was reinforced through the synthesis written during the practice in pairs. However, and as derived from our findings, this effect could be maximised when the pair of students share an opinion on the topic of discussion. That is, in the contexts that minimise controversy between the pair and that could be a facilitator in the identification and selection process included in the syntheses. However, the complexity of the integration process would lead to the need to collaborate in a more strategic way, thus requiring explicit instruction of the collaboration processes as well. This time, the practice in pairs did not reinforce this effect. It is possible that unravelling collaborative processes helped the pairs of students to become aware of the benefits of exchanging perspectives (Nykopp et al., 2014) and making explicit and contrasting ideas (Johnson & Johnson, 2009) when working collaboratively. However, the practice in pairs was not effective in reinforcing what was learned. More support may be needed to consolidate this complex process of integration.

Despite the evidence found in favour of explicit instruction as a necessary component when learning to identify and integrate arguments, this paper raises new questions. The sample selection did not allow high- and low-level pairs of students to compare their ability to learn to write argumentative syntheses. The evidence in this field is scarce and in future studies, it would be necessary to include students with different initial levels. This would allow an investigation into whether weaker writers benefited more from the aids offered and to analyse whether the complexity of the task implies that only those who have low prior synthesis writing skills take advantage of the benefits of collaboration. Furthermore, it would be necessary to have a heterogeneous sample of students based on their previous opinion of the topics. This would allow organizing pairs of both a high and low level of controversy before the debates. This would enable an observation of the effects of any possible controversy between the members of the pairs and to elucidate whether it promotes the more effective use of collaboration strategies, benefiting the process of writing argumentative synthesis.

Conclusions

This paper contributes to the previous research by providing new evidence that explicit instruction is necessary to teach students to integrate arguments from contradictory sources. In this study, even with a homogeneous sample of low-level student pairs, this result is confirmed. Furthermore, the mediation analyses carried out in this work also support the idea that there are two different learning paths for the two dependent variables: explicit instruction in both the writing and collaboration processes is effective in identifying and integrating arguments and in addition, practice in pairs mediates this effect only for argument identification.

These results give rise to some reflections. The evidence provided in this study supports the idea that university students are not adequately prepared to face complex writing tasks that involve high levels of thought processing (Butler & Britt, 2011). Despite this, our study shows that it is possible to teach them how to write argumentative syntheses in collaboration. To this end, it is not enough to propose activities without a specific objective or to administer support such as a guide without instruction. These practices need to be supported by explicit teaching that ideally should not only focus on the writing itself but also on how to collaborate effectively. University teachers could be trained on the importance of including this type of academic activity, in line with that proposed by Newell et al. (2011). It is necessary to insist on the idea that this type of writing task must be put into practice after explicit teaching, considering the characteristics of the classroom, the students, and their curriculum.

On the other hand, it appears that the specific task of writing argumentative syntheses involves two distinct processes: the identification and integration of arguments. Therefore, the way to teach each of them may vary according to their complexity, so it is essential to unravel this and provide adequate aids. Complex and highly demanding processes such as argument integration would require explicit teaching. However, the identification of arguments from texts, being a simpler process, could be mechanized and automated through practice in pairs.

Limitations and future directions

Regardless of the findings of this work, our study has some limitations. Firstly it should be noted that argumentation is a complex task that involves different facets and processes (e.g., Kuhn, 1991; Nussbaum & Schraw, 2007; Nussbaum, 2008a; Perelman & Olbrechts-Tyteca, 1969; Toulmin, 1958; Walton, 1996). However, in this study only some of these aspects have been addressed. Given the interest of our line of research in the task of synthesising multiple sources, this study focuses on the identification of the arguments provided by sources that present opposing positions on controversial topics and on the strategies used to contrast and integrate them. Therefore, the results found are limited only to the intervention on these processes when students produce argumentative syntheses. Future studies could take into account the incorporation of other facets of argumentation into the design and development of interventions in collaborative writing synthesis.

On the other hand, the instruction is framed in a single teaching session, accompanied by two extra sessions of practice in pairs. Despite the brevity of the intervention, it appears to be effective in improving the quality of collaboratively written argumentative syntheses, showing low to moderate effect sizes. Therefore, in future studies, it would be useful to increase the number of instruction and/or practice sessions to see if the improvement is even greater. Also, the results enable us to obtain information on the quality of the collaboratively written product, but not on the processes. Analysing how students collaborate, the contribution each one makes to the task, as well as the regulation strategies they put in place would be imperative to obtain a deeper insight into how collaboration can be beneficial for synthesis writing.

In terms of sample configuration, it would not only be necessary to increase the sample size, but also to have a diverse sample of students with high and low proficiency in collaborative argumentative synthesis writing. For this reason, in the future, it would be interesting to be able to purposively select a diverse sample of high and low proficiency students. In this way, we could examine whether, as we expect, students with low initial proficiency benefit to a greater extent than the other students.

In short, this study offers suggestions on how to design effective interventions to improve argumentative synthesis writing in collaboration in the university context and provides an opportunity for further research considering the characteristics of the study population and the context in which these types of writing tasks are performed.