Introduction

Peer review, a reciprocal activity in which students respond to one another’s writing and produce oral and/or written feedback to express their opinions and suggestions, has long been acknowledged for its multifarious benefits for students. Recently, writing scholars have started examining the relative beneficial effects of giving versus receiving comments on students’ writing. Researchers who proposed the learning-writing-by-reviewing hypothesis have argued that giving peer review comments is more beneficial because assessing and commenting on peers’ writing can potentially enhance givers’ ability to detect, diagnose, and solve similar problems in their own work [6, 7]. However, findings are mixed. While some studies that examined students’ writing about a similar topic support this argument [6, 25], others that investigated students’ writing about different topics do not [20, 36]. On the other hand, scholars that stressed the benefits of receiving comments have argued that peer comments can also enhance receivers’ ability to reflect on their work [28], critically judge the validity of givers’ feedback [35], compare it against their writing to detect, diagnose, and fix potential problems in their writing. Classroom-based research has also reported cases that receivers’ awareness of the differences between their writing and givers’ suggestions prompted them to evaluate those suggestions [38] and to make more corrections to enhance their writing [37].

Given the respective arguments and the mixed findings of the relative beneficial effects of giving versus receiving peer review comments on students’ writing, further research is needed. Informed by the socio-cognitive writing process theory [12, 13] and learning-writing-by-reviewing hypothesis [7], the purpose of this exploratory classroom-based study is to examine a missing link in extant literature—the role of topic difference in the actual revisions that students made in their writing according to the written comments they give versus receive, and how these revisions contribute to their revised draft in an EFL writing class. Understanding the role of topic difference in the actual revisions that students make in their own writing has both research and pedagogical significance. Research wise, writing researchers can test to see if the dominant learning-writing-by-reviewing hypothesis, derived from disciplinary writing in English-as-the-first-language (L1) classes, can be extended to the EFL writing context where students write about self-selected (and thus different) topics. Pedagogically, writing teachers can understand how giving versus receiving peer comments on different topics influences students’ subsequent revisions. This information can help writing instructors make informed decisions on how to devise and structure instructional activities to help students reap maximal benefits when they play a dual role as peer feedback giver and receiver.

Literature Review

Socio-cognitive Writing Process Theory and Learning-Writing-by-Reviewing Hypothesis

The socio-cognitive process writing theory emphasizes the recursive composing and revising processes whereby writers attempt to make their meanings clear to the audience in the academic community [12, 13]. Writing is deemed as a result of the writer’s interaction with audience, text, context, and language [10] and as recursive processes of planning, composing, responding, and revising/editing [21]. Peer review can offer writers opportunities to discover the audience’s perspectives, explore effective ways to express intended meanings, notice and resolve undetected problems in their work. These benefits were initially posited for writers. However, some researchers argued that reviewers could reap similar benefits. Cho and MacArthur (7, p. 74) postulated a learning-writing-by-reviewing hypothesis, maintaining that reviewing peers’ writing provides givers with an opportunity to act as readers and help them better understand how readers would interpret their writing. Giving peer review comments also requires givers to explain their opinions and make suggestions. Cho and MacArthur [7] argued that these requirements enhance givers’ knowledge about what constitutes good writing.

A common assumption underlying the social-cognitive process writing theory [12] and the learning-writing-by-reviewing hypothesis [7] is that both writers and reviewers are self-regulated learners who constantly plan, set goals, monitor their progress, interpret and use internal and external feedback to reflect on their work to improve it [3]. Both giving and receiving peer comments can help givers and receivers understand audience perspectives and expectations, notice and resolve undetected issues in their work, and reflect on how to revise them.

Studies on Giving Versus Receiving Peer Comments on Students’ Writing

Recent research has witnessed a surge of studies on the relative effects of giving versus receiving comments on students’ perceptions and their writing in higher education contexts. The following review first presents L1 studies and then L2 studies.

L1 Studies in Discipline Writing Context

Research on L1 students’ perceptions about giving versus receiving peer comments has shown that students found both receiving and giving feedback valuable [14, 27, 32]. They perceived different benefits in receiving versus giving comments. For example, Nicol et al. [30] reported that students perceived having benefitted from receiving comments in “subject content” and learning about different reader perspectives. In contrast, they perceived having benefitted from giving comments in “learning processes,” including “critical thinking…and transfer of learning” (p. 112). Despite these self-reported perceptions, no further investigations were conducted to check students’ perceptions against their actual revisions.

The other strand of L1 research focuses on examining the relative effect of receiving versus giving peer comments on students’ writing. For example, Cho and Cho [6] examined the relationship between the comments L1 students received and their revision quality versus that between the comments they gave and their revision quality in reciprocal online peer review where students both gave and received comments. The results show that the comments students gave on “the strength of macro-meaning and the weakness of micro-meaning” (p. 637) of peers’ writing could significantly predict their revision quality. However, most comments they received from peers failed to do so. Cho and Cho [6] acknowledged that their correlation results did not indicate any causal relationship between the comments students gave versus received and their respective revision quality. They suggested that researchers employ other methods to further examine these relationships.

However, subsequent L1 researchers mostly used the correlation analysis method to examine the relative effects of giving versus receiving peer comments on givers’ and receivers’ writing. Some reported similar findings [23, 24], but others reported conflicting findings. For example, Trautmann [36], examining this issue in a class where students collaboratively conducted their experiments but individually wrote their lab report on a topic of their choice, found that receivers made more substantial revisions than givers. In a subsequent study, she found that only “aspects of receiving” (p. 700) could significantly predict students’ enhanced revision quality. This finding suggests that students’ revision quality was more related to the comments they received when they produced and reviewed different topics. Student perceptions also supported the empirical data. Another study by Huisman, Saab, van Driel, van den Broek [20] found that when writing about a self-selected topic, peer feedback givers and receivers did not differ significantly in their respective gain scores in content, structure, and style from draft to revision.

L2 Studies in General Writing Context

Fewer researchers have examined students’ perceptions about benefits of receiving versus giving peer feedback [4] and the relative effect of receiving versus giving peer comments on students’ writing in L2 writing contexts [5, 25, 33]. Regarding students’ perceptions, Cao et al. [4] reported that about half of the participants perceived having benefited from both giving and receiving peer comments. The perceived benefits for reviewing included recognizing the importance of using connectives to link ideas and learning different ways to structure the summary. The perceived benefits for receiving peer comments were obtaining valuable insights on improving self-writing, similar to those reported in L1 studies [30].

Lundstrom and Baker [25] conducted a quasi-experimental study to examine the relative effect of giving versus receiving peer feedback on givers’ and receivers’ performance in an ESL program. They divided four ESL classes taught by different writing instructors into a high-beginning and a high-intermediate group with givers and receivers in each proficiency group. The givers received peer review training and receivers received revising strategy training. During training, the givers practiced applying the taught reviewing strategies to review former students’ writing, whereas the receivers employed the taught revising strategies to revise those students’ writing based on the givers’ comments. No interaction occurred between groups. The pre- and post-test results show that high-beginning givers significantly outperformed receivers in almost all aspects of writing. For the high-intermediate givers, only those without peer review experience outperformed their counterparts in overall writing quality. Controlling for the instructor factor in Lundstrom and Baker’s [25] study, Sotoudehnama and Pilehvair [33] conducted a replication study where they taught and trained givers and receivers. The pre- and post-test show that givers outperformed receivers at both proficiency levels. The low-level givers’ gain scores were higher than the high-level givers’.

Despite the careful research design in both studies, the receivers’ tasks were more cognitively demanding than the givers’ because they needed to decipher the writers’ ideas and givers’ comments to revise the essays, whereas the givers needed to comprehend only the writers’ ideas to produce reviews. The “ambiguity and insufficiency” of givers’ comments [33, p. 43], along with problems in the writers’ essays, rendered the receivers’ practice tasks more difficult. Moreover, the receivers’ tasks were less authentic because they could not apply the taught revision strategy to their own writing [20]. The differences in task complexity and authenticity during the instructional training for givers versus receivers may have affected the effectiveness of respective training for each group and their post-test writing performance.

Finally, Chang [5] conducted a classroom-based study to understand how givers revised their writing according to the comments they gave. She examined, among other things, the revision source that influenced givers’ revisions. The results show that only around 2% of givers’ revisions were triggered by the comments they gave, but 24% of the revisions were triggered by those they received. Despite Chang’s detailed descriptions of the type and size of revisions triggered by the comments givers gave, the much larger number of revisions triggered by the comments they received was unexamined.

Research Gaps

The previous review revealed two research gaps. First, no researchers have examined the role of topic difference in the relative effect of giving versus receiving comments on students’ revision. Perception studies have shown that, when producing and reviewing the same topic, givers could avoid making similar mistakes in their writing and emulate different ways to construct their own writing [4, 30]. However, when givers produced and reviewed different topics, the detected mistakes and learned approaches might be less relevant and applicable to improving givers’ writing. The beneficial effects of reviewing the same topic on givers’ writing might disappear when givers review a topic different from theirs. Receivers, in contrast, would not be affected in the same way in this situation because they would still receive helpful comments and different perspectives relevant to the content of their writing [4, 30].

Producing and reviewing different topics imply that students can choose to write about topics that are interesting and relevant to them. Recent task-based writing research has shown that students writing about self-selected topics outperformed those writing about teacher-assigned topics in fluency [9] and text quality [2]. They also use significantly more sophisticated words and phrases when writing about a topic more relevant to their lives [39]. Extending these findings to the context of peer review, it seems safe to assume that students tend to produce more fluent and better writing when they write about self-selected topics. Besides, they also receive peer comments to improve their writing [4, 30]. However, writing about self-selected topics means that givers are likely to review a topic different from their own, potentially reducing the chances for givers to apply or avoid what they observe in peers’ writing while revising their own work. It follows that students are likely to benefit more from receiving than from giving comments in this situation. The previous argument is based on logical inferencing rather than empirical evidence. Given that previous research has not considered the role of topic difference in the relative effect of receiving versus giving peer comments on students’ revision, the aim of this classroom-based study is to explore this issue.

In addition, existing studies have not examined the actual revisions triggered by the written comments students give versus they receive and how these revisions contribute to the revision quality. Previous research has examined the association between the comments students give versus receive and their revision quality [6, 23, 24, 36], givers’ and receivers’ respective writing performance after training [20, 25, 33], or the revisions triggered by givers’ comments in their own writing [5]. If constant revision is a prerequisite to better writing, then examining how students actually revise according to the comments they receive versus give can help us understand how they progress in their writing.

The Study

Given the two research gaps, the purpose of this classroom-based study was to explore how producing and reviewing different topics affect the relative effect of giving versus receiving comments on students’ revision in three essay assignments in an EFL writing class over a semester. Specifically, we examined respective effects of the comments students gave versus received on their subsequent revision regarding revision amount, type, and score. The research questions are as below.

In each assignment, is there a difference between

  1. 1.

    the number of revisions triggered by the comments that students gave and that triggered by the comments they received?

  2. 2.

    the type of revisions triggered by the comments that students gave and that triggered by the comments they received?

  3. 3.

    the score of revisions triggered by the comments that students gave and that triggered by the comments they received?

The reason for examining three essay assignments was to avoid collecting data from one-shot writing session and report findings based on that particular session only to obtain a more holistic perspective on the three questions.

Methodology

Context and Participants

This study was conducted at the first author’s EFL writing class at a Taiwanese university. Sixteen sophomore English majors, one male and fifteen females, with age average of 19.6, enrolled for this class and participated in this study. All were Taiwanese learners of English and had learned English more than 7 years prior to this study. More than half had passed the intermediate level of a local standardized language proficiency test (General English Proficiency Test). Most of their language proficiency levels were between B1and B2 in the Common European Framework of Reference for Languages. None had experience with peer review before this study.

Writing Class

The writing class, English Composition III, was part of a 2-year writing program for English majors. Scheduled for three hours per week for 18 weeks, this course was aimed to introduce different academic essay types. The teacher-researcher divided the semester into two stages–preparation and three writing cycles. During the preparation stage, she trained students to perform peer review both in and after class. The in-class training included teacher demonstration of how to provide revision-oriented comments and student practice by following Min’s (2005) 4-step peer review procedure (1) clarifying writers’ intentions, (2) identifying, (3) explaining, and (4) solving perceived issues in peers’ writing. During the after-class individual coaching session, the teacher-researcher went over the comments that each student produced regarding focus (global or local issues) and procedure and discussed with them how to make the comments more revision-oriented by following the 4-step procedure. For example, most givers identified problems in peers’ writing (step 2) but failed to offer explanations (step 3). The teacher-researcher would remind them to add an explanation to help writers better understand the reason(s) underlying their comments.

For each writing cycle, students were required to compose a 500-word essay in MS Word on a self-selected topic in each essay assignment–narrative, comparison/contrast, cause/effect–at home. The reason for allowing students to write about self-chosen topics was to enhance their interest in the writing assignment. Students were required to print out two copies of their drafts and bring them to class for face-to-face reciprocal peer review where they played a dual role of giver and receiver. Each student reviewed two randomly-assigned peers’ writing and received two reviews from them during two 50-min class periods for each assignment. As feedback givers, they were provided with specific guiding questions for each assignment during peer review (Appendix 1). As feedback receivers, they were encouraged to respond to peer comments in writing while revising their drafts at home, explaining whether they agreed/disagreed with peers’ advice and adopted/disregard it in their revision. This ‘‘a posteriori reply’’ task [15, p. 308] was to promote students’ "mindful reception" [1, p. 213] of peer feedback and awareness of text ownership. Students were required to write comments and response in English but allowed to discuss these in their preferred language.

Students were also required to write reflection journals in English in less than 250 words after revising their first drafts of each assignment. The reason for asking them to write reflection journals was to help the teacher-researcher understand their perceptions about the instructional activities in the writing class. Altogether each student composed three multiple-draft academic essays, 6 reviews, and 4 reflection journals during the semester.

Data Collection Procedure and Data Type

As shown in Fig. 1, once students completed peer review in class for each assignment, they were required to revise their first drafts at home and upload their first and second drafts, peers' comments, their written response to peer review comments, and reflection journals onto Moodle for the teacher-researcher's comments within four days. Altogether 94 drafts and 64 journal reflections were collected. In addition, 16 written responses to an open-ended questionnaire were collected at the end of the semester (See Table 1 for data collection procedure and data type).

Fig. 1
figure 1

Sample writing cycle

Full size image
Table 1 Data collection procedure and data type
Full size table

Students’ First and Second Drafts

Students’ first drafts were used as a comparison base for detecting changes that students made in their second drafts triggered by the comments they received versus they gave.

Peer Review Comments

Peer review comments that students received and those they gave were used as references for identifying the source of students' revisions.

Students’ Written Response to Peer Comments

Students’ written response served as a reference to verify the source of revision, especially those revisions triggered by the comments they received.

Reflection Journal

Reflection journals were used to glean information on their revising process and perceptions about receiving versus giving comments during peer review. No specific writing prompts were given. Students were encouraged to write about their perceptions of the peer review training, comments they gave and received during each peer review.

Open-Ended Questionnaire

The open-ended questionnaire comprised six questions, three of which were related to this study. These three questions asked students to recall (1) the most and (2) the least beneficial comments they received as writers, and (3) the comments they both gave to peers and used in their own writing (Appendix 2). Students’ written responses to these questions were used as another reference to the source of their revisions, especially those triggered by the comments they gave to peers. Students were provided with copies of comments they received and gave to facilitate their recall of a specific example when they filled in the questionnaire. The response rate was 100%.

Data Coding

Type of Peer Comments

Peer comments were coded according to Cho et al.’s [8] classification scheme as three types-directives, praise, and others (see Table 2 for definition and example of type of peer comments). The second author and a 4th-year doctoral student majoring in writing coded all data. The interrater reliability for comment types was 0.95.

Table 2 Definition and example of type of peer comments
Full size table

Revision Source

The revisions were coded for their sources: self- or peer-triggered. Students were required to highlight the changes they made in different colors according to different sources: self revisions in black color and peer-triggered revisions in non-black color. The following excerpt illustrates how a student revised. The blue part is a change in response to a peer’s comment and the underlined words in black are her self-revision.

“My dad stepped in  his bike with him.”

Once a self-revision was indicated in black color, it was cross-examined against the comments students gave to others, their reflections about giving comments, and their written responses to the question on the questionnaire about using the comments they gave in their own revision. If a similar comment or suggestion was also present in the comments they gave to peers and reported in their reflection journals or questionnaires, this revision was counted as a case based on the comment they gave. Otherwise, it was classified as “other self-initiated revision” (e.g., “with him”). Given the focus of this study, these self-initiated revisions were not counted. In cases where a student received and gave a similar comment and also enacted a revision, the source of revision was determined by the students’ use of color in that revision.Footnote 1

Revision type

Revision type was coded according to Paulus’s [31] taxonomy–surface changes and meaning changes. Surface changes include formal changes (e.g., punctuation) and meaning-preserving changes (e.g., vocabulary/grammar). “Meaning changes” include revisions at “microstructure and macrostructure levels.” Microstructure changes are “simple adjustments or elaborations of existing text,” which “[d]o not affect the overall summary, gist, or direction of the ideas in the text,” and “[m]ay involve the use of cohesive ties, causing sentence sequences to be understood as consistent and parallel connected discourse.” Macrostructure changes “alter the text’s overall direction and gist” [11], cited in [31, p. 273]. The interrater reliability for revision types is 97% for surface-level changes and 89% for meaning changes, respectively.

Revision Quality

To assess the revisions students made in response to the comments they gave versus received, the teacher-researcher used the analytical grading rubric employed by Lundstrom and Baker [25]. Constructed on a 10-point scale for 6 dimensions, this rubric covers three global aspects–organization, development, coherence–and three local ones–structure, vocabulary, and mechanics, nicely corresponding to Paulus’s [31] revision types of meaning changes (macro-level and micro-level changes) and surface-level changes (meaning-preserving and formal changes).

The teacher-researcher explained the rubrics to the 2nd author and the other research assistant, demonstrated how to score a student’s revised draft, let them score two students’ revisions individually, and discussed the results with them to ensure that both understood how to use the rubrics. After training, they separately scored revisions that students made in their writing based on the comments they gave to peers. One week later, the two raters scored the revisions students made in their writing based on the comments they received.

A difference larger than 1 point in each score was discussed among the two raters first. If they could not reduce the score difference to less than one point, the teacher-researcher, blind to each rater's score, served as the third rater. An average of the two scores that differed in less than 1 point was used as the final score for that item. For revision score, the interrater reliability was 0.93 for organization, 0.89 for development, 0.87 for cohesion/coherence, 0.90 for structure, 0.85 for vocabulary, 0.92 for mechanics, and 0.93 for overall quality.

Data Analysis

After each type of data was coded and checked for inter-rater reliability, descriptive statistics was performed. Paired-samples t-tests were also run by using SPSS to analyze differences in amount of revision, revision type, and revision score. The significance level was set at 0.05.

Students’ reflection journal entries and written responses to the questionnaire, along with writers’ response to reviewers’ comments, were analyzed via the content analysis method to understand the self-reported problem-solving processes (e.g., problem detecting, diagnosing, evaluating, solving, etc.) and to validate the revision source indicated in students’ revised texts.

Written Response to Reviewers’ Comments

Students’ written responses were examined for receivers’ agreement or disagreement with peers’ comments. Regarding those they agreed with, further checking was performed to see if they revised according to these comments. Care was taken with comments that students disagreed with by checking if any self-revision was made. The interrater reliability for agreement was 97%.

Reflection Journals

Each reflection journal entry was coded for topics (e.g., give or receive reviews) and identified for their perceptions about receiving or giving comments. Attention was paid to specific instances in students' reported cognitive benefits of receiving or giving comments. These specific instances were cross-checked with the peer review comments they received or gave and the revisions they made.

Questionnaire

Attention was paid to the specific instances that students reported for giving and using the same comment in revision. Further cross-checking was done to see if they did revise accordingly. The interrater reliability was 95%.

Results

Table 3 shows the descriptive statistics of types of comments for each assignment. The total number of comments for each assignment was 256, 277, and 284 respectively. The majority were directives, taking up 94%, 85%, and 81% of the total comments in each assignment. Praise ranked the second, comprising 5%, 10%, and 9% of the total comments for each assignment. Only 1%, 5%, and 10% of the total comments for each assignment were reader responses. Given that (1) students did not change the parts that received praise in their revisions except expressing appreciation and (2) that no observable effect of their giving praise to peers was found in their revisions, the following analysis was focused on the effect of directives that students gave versus received for each assignment.

Q1: Amount of Revision Triggered by Directives Students Gave versus Received

Table 3 Type of peer comments
Full size table

Although each student gave and received different directives in their first drafts, the number of directives received by versus given to each draft was the same. The only difference lay in the givers or receivers of these directives. Table 4 shows that each student received and gave, respectively, an average of 15.06 directives on their first draft for the first assignment, 14.96 directives for the second assignment, and 15.33 directives for the third assignment.

Table 4 Amount of directives and revisions triggered by received versus given directives
Full size table

In general, students enacted significantly more revisions based on the directives they received than on the directives they gave in each assignment. In the first assignment, students enacted more revisions based on the directives they received (M = 11.75, SD = 3.36) than on those they gave (M = 0.69, SD = 0.95). The difference is statistically significant (t(15) = 13.51, p = 0.000). In the second assignment, they enacted more revisions based on the directives they received (M = 10.13, SD = 5.41) than on those they gave (M = 1.25, SD = 0.86). The difference is statistically significant (t(15) = 6.33, p = 0.000). In the third assignment, they enacted more revisions based on the directives they received (M = 10.67, SD = 5.41) than on those they gave (M = 1.33, SD = 1.68). The difference is statistically significant (t(14) = 5.62, p = 0.000).

Excerpts 1 and 2 illustrate how students enacted revisions based on the directives they gave versus received.

Giver's 1st draft

2nd draft

Comment to peers

Giver's Reflection

1. "…recalling that horrible and alone night…

Mom and Dad were not at home…"

"…recalling that horrible and alone night…

It was one night when I was in fifth grade. I came home from the cram school to find Mom and Dad…"

The transition sentences are not very clear in each paragraph. Maybe you want to add some sentences to make this essay more flowing

I pointed out the flow problem in Jessie's essay. When…revising…, I found I just told the story suddenly. So I added a sentence to make it smooth

Receiver's 1st draft

2nd draft

Comment from peers

Receiver's Reflection

2. …take the risk of running up against such an audience in the same cinema…"

"…take the risk of running up against such a viewer in the same cinema…"

“audience” is a collective noun. I suggest you change “an audience” to “such a person”

I checked this in a dictionary. I found “audience” refers to a group of people. Perhaps I can change it to “a viewer”

  1. aPseudonyms were used in all excerpts

Excerpt 1 is from Kelly's first assignment about her most terrifying experience–staying home alone in a typhoon day. She found a flow issue in her partner's writing and advised adding transitional sentences to make it smoother. In reflection of her own writing, she “found” that she narrated her “story suddenly” and solved this problem by adding “a transitional sentence.”

Excerpt 2 is from Elise’s second assignment on watching films at home versus at the cinema. She discussed the possibility of sitting next to an annoying talker at the theater and used “such an audience” to refer to that person. Her reviewer commented on the mismatched singular/plural issue and suggested a solution. Elise evaluated the accuracy of this opinion (“checking a dictionary”) and resolved this issue via using her own word choice.

Q2: Type of Revision Triggered by Directives Students Gave Versus Received

The second research question addressed the types of revision triggered by the comments they gave versus they received–meaning changes and surface changes. Meaning changes were further divided to macrostructure meaning changes and microstructure ones. Surface changes were divided to meaning-preserving changes and formal changes.

Table 5 shows that students’ revision types were more influenced by the directives they received than by the directives they gave. Regarding meaning changes, significantly more macrostructure meaning changes were triggered by the directives students received than by those they gave for the first (p = 0.000) and third (p = 0.003) assignments. Furthermore, significantly more microstructure meaning changes were triggered by the directives students received than by those they gave for each assignment (p = 0.000 for the first and third assignments; p = 0.003 for the second assignment).

Table 5 Type of revision triggered by received versus given directives
Full size table

Regarding surface changes, significantly more meaning-preserving changes were triggered by the directives students received than by those they gave for each assignment (p = 0.000 for each assignment). Finally, significantly more formal changes were triggered by the directives students received than by those they gave for the second assignment (p = 0.002).

In Table 6, Excerpts 3 to 5 illustrate a macrostructure, a microstructure, and a meaning-preserving change that students made according to the directives they received.

Table 6 Macro-structure, micro-structure and meaning-preserving revision based on received directives
Full size table

Excerpt 3 is from Angel’s second assignment. Her peer pointed out her mismatched focus between the introductory and body paragraphs. This inconsistency confused her reader. After rereading the first two paragraphs, Angel noticed the inconsistency, and took the advice by changing the focus of her thesis to differences in the introductory paragraph.

Excerpt 4 is from Andrea’s third assignment where she discussed three causes of fake news and finally attributed them in the concluding paragraph to the underlying culprit: the audience. In the introduction, however, she hastily juxtaposed “who is to blame” and “three causes to this social phenomenon…” without a smooth transition. Her reviewer pointed out a lack of “strong connection” between the two sentences. Attempting to maintain her original intention and address this transition issue, Andrea revised according to her reviewer’s suggestion (in blue color) by moving the who-question to the end.

Excerpt 5 is from Jim’s third assignment where he discussed causes of hiccups. His peer suggested that he change the expression of “muscle tensions” to “tense muscles.” He thought that this suggestion better expressed his idea and revised the expression according to the peer’s comment.

Table 7 details two students’ microstructure meaning changes (Excerpts 6 and 7) and one student’s meaning-preserving change (Excerpt 8) based on the directives they gave to peers.

Table 7 Microstructure and meaning-preserving revision based on given directives
Full size table

Excerpt 6 is from Joanne’s second assignment. She made a comment on the tone in her partner’s sentences and softened her own by adding “some say” in her second draft. Commenting on the tone in her partner’s writing had prompted her to notice (“found”) the same issue in her writing and to revise it in a more acceptable manner.

Excerpt 7 is from the second paragraph of Connie’s third assignment. As a reviewer, Connie suggested that her partner add a signpost “as for the negative effect” to clearly show the reader that the ensuing description was related to the negative effect. In her 1st draft, Connie first discussed the negative effects of smoking on smokers’ internal body change and then those on their appearances. In her second draft, she acted on her partners’ suggestions by reordering the sequencing of the internal and external changes, a microstructure meaning change. She also self-added a signpost “speaking of the external change” in the 2nd paragraph due to her comment to peers. As shown in her written response to the questionnaire, Connie noticed a similar lack of clear cues to guide readers to follow her ideas in her 1st draft after reviewing her partner’s writing and solved this issue via a similar strategy.

Excerpt 8 is from the last paragraph of Lillian’s first essay. As a reviewer, Lillian suggested that her partner, Lily, describe how she could not fall into sleep by using “tossing and turning in bed.” In Lillian’s own revision, she used the same vivid expression to describe her own situation.

Q3: Score of Revision Triggered by Directives that Students Gave Versus Received

Table 8 shows students’ revision scores based on the directives they gave versus they received on six dimensions–organization, development, cohesion/coherence, structure, vocabulary, and mechanics. The scores of revisions triggered by the directives they received were higher than those of revisions triggered by the directives they gave in almost all dimensions for each assignment (except for “vocabulary” and “mechanics” for the second assignment and “vocabulary” for the third assignment).

Table 8 Score of revision triggered by received versus given directives
Full size table

A series of paired-samples t tests were performed to see if the observed score differences were statistically significant. To avoid the inflated Type I error, a corrected Bonferroni alpha 0.007 (0.05/7) was adopted. The results show that for the first assignment, the scores of idea development (p = 0.003), cohesion/coherence (p = 0.001), and overall mean score (p = 0.001) of students’ revisions triggered by the directives they received were significantly higher than those triggered by the directives they gave. So were the scores of organization (p = 0.000), idea development (p = 0.001), and the overall mean score (p = 0.001) of revisions of the third assignment. For the second assignment, none of the score differences reached significance.

Discussion

Informed by the socio-cognitive writing process theory and learning-writing-by-reviewing hypothesis, this study aimed to complement the current knowledge base of the respective effects of giving versus receiving peer comments on students’ subsequent revision by exploring the role of writing topic difference in their actual revisions. The results show that these EFL students, when producing and reviewing different topics in three assignments, enacted significantly more revisions according to the directives they received than according to those they gave. Regarding revision type, they made significantly more macrostructure meaning changes according to the directives they received for the first and third assignments. In addition, they also enacted significantly more microstructure and meaning-preserving changes according to the directives they received than they gave for all assignments. The average revision scores based on the directives they received were significantly higher than those scores based on the directives they gave for the first and third assignments.

Topic Effect on Number and Type of Revision

Current research on the relative effects of giving versus receiving comments on students’ subsequent revision has yet to examine the influence of topic on this issue. This classroom-based study found that students’ revision did not benefit as much from the comments they gave as from those they received when they produced and reviewed different topics. This finding is similar to that of Chang [5] and Trautmann [36]. Regarding the amount of revision, Chang [5] found that students made 2.5 times more revisions based on the comments they received than on those they gave. Similarly, this study found that students made significantly more revisions based on the comments they received than on those they gave. Regarding revision type, Trautmann [36] found that receivers made more global revisions on content than givers when all wrote a lab report on self-selected topics. Similarly, this study found that students enacted significantly more macrostructure revisions for the first and third assignments and significantly more microstructure revisions for all assignments based on the comments they received.

One probable explanation for the similarity of these findings is that the comments givers gave to peers were less relevant to the content of their writing. This interpretation is supported by the evidence in this study. Students, when playing the dual role of giver and receiver, made significantly fewer microstructure and meaning-preserving changes based on the comments they gave to peers in each assignment (Table 5). These changes were mostly about particular sentence structures, vocabulary, and grammar issues related to the specific content of their own writing, which is different from their peers’. In contrast, the comments that students received from peers were more relevant to their own writing because their peers were trained to give comments to help improve their writing (e.g., Excerpts 2, 4–5). Thus they enacted significantly more microstructure and meaning-preserving revisions according to the comments they received.

It should be noted that when playing the role of givers in this study, students were required to use the same teacher-supplied guiding questions to comment on peers’ writing. The comments they gave to peers on the basic elements of an academic essay (e.g., hook, thesis, cohesion) were also relevant check points for their own writing. Reading a different topic probably made it more difficult for some givers to see that the insufficient aspects of peers’ work pertaining to these issues were also lacking in their own work. In fact, data from journal reflections (e.g., excerpts 1, 6, 7 and 8) show that only a few reflective students (e.g., Kelly, Joanne, Connie, Lillian) could detect issues that were common to peers’ and their writing despite topic differences, monitor and remedy similar problems in their own writing.

One interesting finding is about the second assignment—the comparison/contrast essay. Despite reviewing a different topic, students enacted a similar number of macrostructure changes based on the comments they gave versus on those they received (see Table 5), suggesting that they could amend an equal amount of global issues in self-writing after detecting and fixing similar ones in peers’ writing in this assignment. The revision scores based on given versus received comments consequently did not differ significantly in all aspects (see Table 6). A speculation is that it might be due to the clear point-by-point organizational pattern which students employed to contrast different aspects of two objects/events/positions in a symmetrical manner and the easy selection of comparison points. This facility might have helped students more easily see the relevance of peers’ global issues to theirs in self writing. Although this speculation can find support in previous reading research on text comprehension, recall, and production that readers experienced less difficulty in processing and summarizing ideas in comparison/contrast texts [26], as cited in [34], more research is needed to examine the validity of this speculation in writing.

Revision Type and Student Writing Ability

The finding that significantly more macrostructure and microstructure changes were triggered by the directives that students received seemed to suggest that they were still at a stage of needing peers’ assistance to detect global issues across paragraphs and to make their intentions clearer, more organized and cohesive within paragraphs. Research on students’ perceptions has also shown that students need peer comments to identify their blind spots [4, 30]. On the other hand, the finding that comparatively fewer microstructure meaning changes were triggered by the directives that students gave to peers seemed to suggest that some students were also able to detect, monitor, and remedy issues common to both peers’ and their writing despite topic differences. Previous research on students’ perceptions has shown that giving comments could help givers transfer what they learned to their own writing [30]. The manifestations of this learning, as revealed in students’ actual revisions in this study, were mostly related to sentence structure changes within paragraphs and vocabulary and grammar changes within sentences.

Revision Type and Revision Quality

Research has shown that macrostructure revisions are correlated with overall revision quality [31]. Although this study did not run a correlation analysis of the relationship between revision types and the overall revision quality, the findings seemed to be in line with Paulus’s research. This interpretation is based on the results of macrostructure and microstructure changes (Table 5) and the revision quality of those two types of changes and that of overall revision quality (Table 8).

For the first and third assignments, significantly more macrostructure and microstructure changes were triggered by directives that students received than by those they gave. The overall revision quality and the quality of revision of two global issues triggered by the comments students received for the first (i.e., development and coherence) and third assignments (i.e., organization and development) were significantly better than the overall revision quality and that of the same issues triggered by the comments they gave (Table 8).

In contrast, for the second assignment, no significant difference was found in the amount of macrostructure changes triggered by the directives that students received versus they gave, although a significant difference was found in the amount of microstructure changes. Similarly, no significant difference was found for the overall revision quality and the revision quality of the global issues. Taken together, the amount of macrostructure changes seemed to play a more important role than that of microstructure changes in enhancing the overall revision quality and that of global issues.

Authentic Revising Task and Receivers’ Revision

Unlike Lundstrom and Baker [25] and Sotoudehnama and Pilehvair [33] that required receivers to revise others’ writing based on givers’ comments during the experimental treatment, this study required receivers to revise their own writing based on givers’ comments. These more authentic writing tasks may have engaged receivers more in revising their own work based on givers’ comments. In fact, Huisman et al. [20] found that, when receivers revised their writing rather than others’ writing based on givers’ comments, receivers made similar improvements in their revision as givers when all students were given some freedom to choose a writing topic. Similarly, this study also found that the quality of receivers’ revision based on givers’ comments was improved. These findings suggest that authenticity of revising tasks, in addition to topic difference, might have played a role in receivers’ revision quality.

Self-reported Cognitive Operations During Revision as Givers and Receivers

Previous research on the relative effects on giving versus receiving feedback on student writing has been primarily based on arguments for potential cognitive skill learning while giving peer feedback—detecting, diagnosing, and solving problems [6] versus potential cognitive skill learning while receiving peer feedback—evaluating feedback information, detecting the gap between one's own current performance and intentions, and bridging the gap [18]. Both arguments are premised on a common assumption that students are constantly planning, monitoring, and revising their writing [12, 13].

Although students’ cognitive processes are not a major focus of this study, students’ reflection journals indicate that some reported that, after spotting problems in peers’ writing, they reflected on their own writing, found and solved various similar writing issues such as flow (excerpt 1), tone (excerpt 6), transition (excerpt 7), and language (excerpt 8). These self-reports seemed to confirm the problem detecting and solving processes predicted by the learning-writing-by-reviewing hypothesis.

However, writers’ reflection journals also show that they seemed to undergo some similar problem-solving processes while revising their drafts based on received comments. Upon receiving comments, they reported reexamining (e.g., Angel “reread” her writing) their writing, noticing and understanding the problems identified by peers. They reported using their knowledge to evaluate the appropriateness and accuracy of peers’ comments (Elise “checked the dictionary” and Jim’s evaluation of his peer’s suggestion as “more clear”). They also reported compromising between expressing their intentions and satisfying the audience’s expectation (excerpt 4). If they agreed with peers’ directives and perspectives, they needed to generate a revising strategy for incorporating peers’ directives in their revised drafts (excerpts 3 and 4). These reported cognitive skills were similar in nature to those they reported using when reviewing peers’ writing—to identify, explain, and resolve peers’ writing problems and to those described in the socio-cognitive writing process theory [12, 13].

Although a more systematic examination of givers’ and writers’ online cognitive processes is needed, a recognition that givers and receivers probably undergo some similar problem-solving processes should be in place for writing research and instruction. Research wise, it acknowledges the writer’s active role in interpreting, evaluating, and using feedback, a phenomenon less explored in extant research on this issue. Pedagogically, it seems more fruitful to divert writing researchers’ attention from arguing which cognitive learning mechanisms associated with giving versus receiving are more conducive to improving students’ problem-solving skills to examining which instructional activities can enhance their engagement and skills for advancing their revision and writing while they are playing each role.

Conclusion

This study explored the role of writing topic difference in the relative effects of giving and receiving peer comments on self revision in three essay assignments in an EFL writing class over a semester. The results overall demonstrate that receiving peer feedback was more beneficial than giving peer feedback on students’ revision. The findings show a possible limit of the more beneficial effect of giving than receiving comments on self-writing to similar topics. Due to the specific context of this classroom-based study involving only sixteen participants with similar language proficiency from the same writing class, the quantitative results cannot be generalized to other contexts. Despite the limitation, the findings of this exploratory study have the following implications.

Research Implication

In terms of research, most previous studies comparing the relative effect of receiving and giving reviews have focused on similar topics rather than different ones. Since research has shown that reading a similar topic can also help readers improve their own writing [7], it is unclear if previously-reported beneficial effects of giving feedback to others on self revision and writing are an effect of producing reviews only or a combined effect reading a similar topic and producing reviews. Further experimental research is needed to compare and contrast how producing, reading, and reviewing different and similar topics help students improve their revision in the short-term and their writing in the long term.

The other research implication is that extant research has inferred and interpreted the process of students’ revising from their written product and their post-writing reflection. Although the ‘‘a posteriori reply’’ [15, 16, p. 308] task in this study gives a glimpse into how students revised their texts in response to received comments, and the reflection journals and end-of-semester questionnaires help validate our content analysis of how students revised their own writing in response to given comments, a missing link still existed between the actual cognitive processes they underwent and their textual products triggered by the comments they gave to others. Further research needs to employ other methods such as think-aloud to explore students’ on-line revising processes to better understand how they revise according to received and given comments.

Pedagogical Implication

One pedagogical implication is that writing teachers need to design and implement writer-related activities to help students actively engage in peers’ feedback after receiving it. The “a posteriori reply” task, asking students to indicate and explain their (dis)agreement with peers’ feedback can be one such activity to enhance students’ reflection about their writing, motivation, and revision performance [22]. It not only helps promote the concept of writers’ ownership of their text but also provides them with a chance to further explain the misunderstood intentions in their writing. It also prompts writers to reexamine their writing to identify readers’ sources of confusion and to evaluate the accuracy and appropriateness of peers’ feedback for further revision.

The other pedagogical implication is that writing teachers should inform students of the benefits of reviewing both similar and different topics for enhancing their own writing. Reviewing similar topics can offer students opportunities to observe good models and apply similar approaches to and strategies for their own writing [30] as well as avoid problems observed in poor models in their own writing, and these benefits are mostly related to the specific writing task at hand. Reviewing topics unrelated to students’ own writing, on the other hand, can offer them opportunities to apply newly-learned evaluation criteria to different topics, helping them consolidate their developing knowledge about these quality standards that transcend topics and finally internalize this knowledge. These practices are an interim stage that students need to undergo before they can transfer the internalized criteria to evaluating their and others’ writing about any topic. Students usually express disinterest and discomfort in reviewing peers’ writing unrelated to theirs. Informing them of the interim and ultimate goals of reviewing similar and different topics and directing their attention from the specific content to general standards of quality paper can lessen their unease about their insufficient content knowledge and help sustain their engagement in producing quality reviews in case they need to review different topics. Careful sequencing of the topics for peer review from similar ones to different ones in the same genre can also help achieve afore-mentioned goals.

Interested writing researchers and instructors are encouraged to incorporate these and similar writer- and reviewer-related instructional interventions into their peer review activities so that students, regardless which role they enact during peer review, can be actively engaged in receiving and producing feedback and reap some benefits in their writing.