Introduction

Providing written corrective feedback (WCF) to students in writing classes is a common practice and has been shown to be effective in improving writing quality (e.g., [9, 13, 15, 20]). After a feedback is given, the correction job can be designed for second language (L2) or English as a foreign language (EFL) students to revise, and they may rely on some reference tools to help the revision process particularly when they do not have enough linguistic knowledge about the problem. Concordance programs, with dictionaries, grammar references, and online search engines like Google, are common learning references. The development of using corpora for language learning as a new reference tool comes from a renewed interest in authentic data offered to second language learners, and the belief that learners can adopt new and more active roles in their learning process [1, 31] via inductive strategies of observing corpora carefully. Learners’ direct access to corpora, also called data-driven learning (DDL, [14]), has been shown to be beneficial to various aspects of L2 writing: collocation [18, 25], genre knowledge [6], proofreading [4], or error correction [7]. The effects of using learner concordance with other electronic tools for writing have also been addressed [16, 30, 32].

The current study contributed to our understanding of using corpus tools to correct errors in writing for one semester by keeping the ecological landscape of a classroom [12, 20], unlike that in a strictly controlled experiment setting. That is, we conducted the project by keeping what an EFL writing teacher would do regularly to promote a written communication atmosphere and meet curricular requirements of the class. Specifically, we documented a group of college students’ feedback uptake augmented by corpus consultation with several assignments in a writing course. We showed how the learners engaged in such a process by exploring the extent to which they benefited from processing corpus-aided written corrective feedback (CAWCF) and how they felt about such an approach. We also illustrated two learners’ engagement with corpus-informed feedback processing with their lookup processes and detailed data triangulation.

Review of the Literature

Written Corrective Feedback in L2 Classrooms

Various dimensions of written corrective feedback (WCF) have been well-documented in its rich literature of traditional L2 writing classrooms. We show a collective picture from two recent meta-analyses and more updated individual studies [15, 20]. Kang and Han [15] confirmed that feedback is generally effective in increasing grammatical accuracy (with an effect size of 0.54), given focused or unfocused (or comprehensive) feedback. With a focus on genre types and mediating variables, they found that both direct and focused feedback has durable effects; while genre types do make a difference, journal writing is not amenable to correction but composition is, which is more effective than letters. In Liu and Brown’s meta-analytic report [20], four methodological recommendations for future WCF research are raised: (a) to improve design of low ecological validity in most studies where one-shot treatment in timed in-class writing tasks was adopted; (b) to provide qualitative data with mixed-methods on learners’ processing of feedback, strategies used to handle feedback, and feedback retention; (c) to give clearer error categorization; and (d) to conduct more longitudinal studies (also suggested in [15]). For a study to be ecologically valid, the design needs to ensure the research process equipped with the routines and materials commonly and regularly applied in the language classroom. The current study responds to the call for improving the research design of ecologically valid CAWCF studies in classrooms by involving students in multiple writing assignments over one semester.

To complement quantitative WCF findings discussed above, two recent case studies of L2 writing focused on student-writers’ individual differences by offering qualitative insights. Ferris et al. [9] explored ten L2 college freshman students’ uptake of WCF, by asking the students to write four timed persuasive essays in class over a semester. A 15-week longitudinal multi-case classroom research design was adopted. They marked three to four error patterns on the student essays using indirect feedback based on an agreed-upon list of error codes. Through retrospective interviews, they found that most learners used their intuition to see what sounded right. They suggested that writing teachers should not look at learners’ written product only. Individual and contextual factors influenced these learners’ writing development, and writing teachers should take “a more finely tuned approach to correct feedback” ([9], p. 307) by reducing students’ anxiety and raising their motivation for revision. Through a naturalistic multiple-case study approach, Han and Hyland [13] examined four Chinese college students’ cognitive, behavioral, and affective engagements with WCF after they finished a take-home writing assignment with two drafts and went through feedback processing for error correction. The depth of feedback processing in their four cases (out of 25 students) was not related to the effectiveness of revision. They argue that learner beliefs and goals, and the interactional context where WCF was received and assimilated into their L2, may result from individual differences and various engagement amounts and types.

Learner Concordancing for Language Learning

New kinds of affordance and learning potential are available as a recent meta-analysis of 64 studies has been conducted on applying DDL to effectively learn various aspects of a second language including writing [1]. Learners may interact with corpora, inducing patterns of use with observation of concentrated examples in context [27], in order to trigger writing development [22]. Recent advances on corpora and online tools such as “British National Corpus” [2] and “Corpus of Contemporary American English” (COCA, [8]) bring students more reference choices for assistance.

DDL studies conducted in Taiwan EFL contexts also demonstrate successes as various applications have been innovated (e.g., [4, 16, 17, 19, 25, 29, 30]). In college contexts, Lin [19] examined how English teachers used concordancing to teach grammar as compared with the traditional grammar approach and found favorable results of concordancing. Lai and Chen [16] explored how college students used dictionaries or concordancers for different writing purposes. Wu [29] investigated the detailed effects (e.g., collocation accuracy or complexity) of DDL for learning collocations in writing. Yeh et al. examined the combined use of a concordancer and automatic writing assessment tool on error correction in a college writing context and found positive effects. In high school contexts, Chang and Sun [4] show the better proofreading performance of a group of high school students, augmented with concordance searches, under the scaffolding condition than another without scaffolds. Their design of prompts also increased students’ certainty level while performing the task. Lee and Liou [17] applied DDL in a high school context for vocabulary learning purposes. Sun and Wang [25] compared two groups of high school students using either inductive or deductive approaches to learn collocations of different difficulty levels and found the inductive group performed better on easy collocations than the deductive group. From this brief review, using concordancing for error correction in a college writing context warrants more research in Taiwan as limited studies were concerned about this dimension of applying learner concordancing for writing purposes.

Corpus-Enhanced Writing Revision

Compared with studies examining general corpus-assisted writing issues, much fewer have addressed how corpora can scaffold students’ correction process and uptake of WCF at the revision stage. Meanwhile, Cobb and Boulton [5] draw corpus scholars’ attention to longer term effects of students’ concordance work. Most existing learner concordancing literature examined general L2 writing contexts for graduate students (e.g., [32]) or looked into learner perceptions only without examining their writing performance (e.g., [24]). Some examined students’ performance at the decontextualized sentence level or (particular) collocations in isolated sentences [17, 25, 29]. The current study focused on changes made onto the undergraduate students’ essay-level texts after corpora were incorporated to assist the WCF processing process. Six relevant CAWCF studies involving data of students’ essays were reviewed. Todd [27] examined whether Thai post-graduate students could induce word patterns from corpora for self-correction in their English writing. They found adjective errors marked by their instructor on the essays were the easiest compared with those of verbs and nouns. Thirty-two doctoral students from Hong Kong were examined in Crosthwaite [7]. They composed a piece of English text as part of their graduate study, received teacher feedback, and rectified errors using both Sketch Engine for Language Learning (SKELL) and BNCWEB. The participants went through five 2.5-h sessions for corpus training followed by revisions. Crosthwaite found that the number of corpus-led revisions reached an average of 83% reduced errors (of all types) from the drafts, and the students felt mostly positive about introducing the corpora into their revision process. Collocation errors were more successfully rectified than morphosyntactic errors.

Two studies looked into how French learners used corpora to rectify errors in texts. On graduate students, Chambers and O’Sullivan [3] documented how DDL can help French learners correct errors in their texts in a 2-h lab session. Similarly, O’Sullivan and Chambers [21] examined an undergraduate student group in a similar context. Both studies found encouraging students’ performance and graduate students’ attitudes toward DDL was slightly more positive. Two studies investigated how undergraduate students could use corpora to correct errors in their English essays. Tono et al. [28] recruited 93 Japanese college students to use corpora to correct errors in their timed essays composed in the lab. Students were divided into two groups of different proficiency. It was found that the revisions of omission and addition errors were more than those of misformation errors (i.e., verb tense) (e.g., “my parents talk him very often”) in student essays. The revisions from the less proficient group yielded a higher accuracy rate than the more proficient due to the former’s shorter papers with easier vocabulary and structures accompanied by fewer errors. Gaskell and Cobb [11] was the only study that was designed like a regular writing class with ten assignments over 15 weeks. They used Lextutor (http://www.lextutor.ca) to revise errors in the essays of lower intermediate EFL learners over a semester. They found three error types were decreased in later assignments, word order, capitals/punctuation, and pronoun, but errors increased. Although providing concordance lines for learners was a way to give learners directions in the consultation process, it deprived the chance for learners to figure out the key words to search for themselves, which is an essential step of induction. They suggested that future research will use a sharpened research instrument for recording lookup behaviors.

Among sharpened research instruments, video recording is argued to be able to complement existing research procedures and data [16, 22, 23]. Using video recording to capture learners’ moment-in-moment behaviors while performing online corpus consultation promises to provide insights into L2 writing development. Through a one-semester longitudinal study, Park [22] triangulated screen recording, student writing assignments, oral and written reflections, and corpus-query analysis of undergraduate ESL students in order to show the learners’ language development. Park demonstrated his three learners’ writing development by providing improved writing performance and language awareness.

From the review above, research of corpus-aided error correction seems to yield effective outcome, although the number of such studies is still quite limited. Two of them were on French learning students, instead of English learners. Half of the studies reviewed did not investigate undergraduate students like the target population in the current project. More importantly, only one study [11] proceeded like regular coursework in a writing class using multiple assignments. The other studies were conducted with one-shot feedback provision, timed in-class writing tasks, or targeting one single or limited focused error types. Such design has been criticized as the lack of classroom ecological validity [20] in regular L2 writing classes particularly with only one-shot treatment of feedback. These facts indicate more longer term classroom-based research with ecological validity on undergraduate learners who use corpora to correct errors in writing are sorely needed to better inform teachers and researchers.

The current study bridges research gaps in several aspects with an aim to investigate how a group of college students used the WCF to improve writing when various references including concordancers were available. With a research design to ensure ecological validity that was augmented by video recording of learners’ corpus consultation process, the study documented a semester-long writing course and investigated how corpus consultation (and other e-reference resources) may play a pivotal role in EFL students’ self-correction process and outcome. The reason why students’ use of other e-reference tools was also observed was that tools such as English–Chinese dictionaries or Google were common to EFL students in Taiwan and to exclude their use is against their habitual referencing practice. With various considerations of classroom ecology, the current study made corpus integration a normalized practice for CALL teachers and researchers by having corpus consultation an integral part of the writing classroom activity for an extended period of time. Research questions included the following:

  1. 1.

    How did the college students improve writing inaccuracies when corpus tools were integrated into their revision over one semester?

  2. 2.

    How did the students affectively respond to DDL-aided error correction? Why so?

  3. 3.

    What might further account for the students’ writing performance and perceptions when corpus tools came into play for their revision, given two case studies?

Method

Context of the Current Project

The study was conducted in a university of central part of Taiwan. Fourteen second-year college students participated, who majored in English. Based on our prior experiences of teaching in the institution, a very limited number of the participants had studied abroad before they were admitted into college. Built around a process pedagogy where each writing assignment went through three drafts until it obtained a final grade, the English writing program required all its students to take the course from the first to the third year (14 credit-hours in total). The second-year program focused on expository essay practice. Fourteen participants (11 females, 3 males) came from an intact class which was arranged by the department office. The students whose first language was Chinese had studied English as a school subject for at least 8 years in schools before they were admitted to the college. Although writing was required as part of their college entrance examination, limited high school writing instruction was provided. Their self-reported TOEIC scores showed an average of 637. They all signed a consent form giving permission to use their essays and relevant data. Before the project started, a background questionnaire was given to understand the participants’ prior English learning experiences and their habits of using reference tools. The results indicated these learners never or rarely used corpora but relied mainly on online Chinese–English bilingual dictionaries or Google for reference purposes. Two average students of writing performance (based on the instructor’s classroom-based assessment on their essay assignments) were chosen out of the 14 students in the group for closer examination as case studies.

Instructional Design

For the writing course where the project took place, the instructional design considered how to meet the requirements of ecological validity, the goal of our writing program, good WCF practice [20], and the teacher–researcher’s teaching philosophies. In the course, genre knowledge and writing skills were given through the textbook with relevant writing practice. Three relevant writing assignments of different genre/topics were designed: narrative, analysis, and comparison/contrast. For each assignment, three drafts were required for practice and completed at home. Students wrote the first draft to obtain peer review comments and the instructor’s feedback on content and organization in order to work out the second draft. On the second draft, the instructor marked lexical and grammatical errors based on an agreed-upon list of 20 error codes (see Appendix Table 2, adopted from [9]), as indirect feedback with neither corrections nor metalinguistic explanations provided. By adopting a mid-focused strategy, the teacher provided roughly two to six coded errors of lexical or grammatical types in order not to intimidate students from further revision (facing too many to handle at a time) but be informative enough to improve writing accuracy (by focusing on selected types). The number of marked errors depended on different qualities of each student’s draft. Feedback of content and organization (goal of a particular genre writing) was also addressed in practice (as both the students and the course required such feedback) but not within the scope of the current study.

After the participants obtained the marked second draft, they revised it in a computer lab using the corpus tools or other e-references (dictionaries, Google, etc.), wrote a record of revision (to be illustrated shortly), and had the revision process be video-recorded at the same time. The resulted version turned into the third draft for the instructor to give a grade based on their accumulated efforts on the three versions. To ensure good reporting practice in the project, the time interval between writing assignments was 1 month and feedback turnaround time among drafts was 1 week. One additional writing task given in class which took place in the beginning of the semester was error-marked by the teacher and served as students’ homework practice of corpus consultation after she demonstrated how a concordancer could be used for error correction. This task served a diagnostic purpose for the instructor to understand students’ writing weaknesses.

DDL Training

From the background questionnaire, we found that the participants already had adequate computer literacy when they entered the college as computer education had been provided from the primary school. We introduced three concordance programs to the participants by giving orientation of how to use two bilingual concordance programs, TANGO and TOTALrecall (http://candle.cs.nthu.edu. tw/totalrecall/totalrecall/totalrecall.aspx?funcID=1), and simple functions of COCA [8] such as key-word-in-context (KWIC, see Fig. 1) because their interface and functions were user-friendly.

Fig. 1
figure 1

Concordance lines from TANGO and illustration as part of DDL tutorial

Full size image

Three tools, TANGO, TOTALrecall, and COCA, have their different functions and purposes for students’ consultation in the study. TANGO (http://candle.cs. nthu.edu.tw/collocation/ webform2.aspx?funcID=9) targets verb-noun and adjective noun collocations (with extensions of verb-noun-preposition and verb-preposition-noun ones). The concordancer includes corpora of (a) Sinorama 1990~2000 (a Chinese–English bilingual online magazine, https://www.taiwan-panorama.com/en), (b) Voice of America (VOA), (c) BNC [2], (d) Hong Kong News & Laws, and (e) Academic collocation from BNC listed. When the user chooses Sinorama or Hong Kong News & Laws, they can view the bilingual output of English–Chinese aligned texts. For the other three corpora, they view English output only. To use TANGO (http://candle.cs.nthu.edu.tw/collocation/webform2.aspx?funcID=9), a user provides a key word and specifies its word category (a noun or a verb), and then chooses one of the collocation types (e.g., VN-verb and noun or AN-adjective and noun, VPN, or VNP-preposition) as consulting steps. Upon submission of the key word and collocation category, a group of frequent collocations from the computer program is shown and the user can click on the phrases to study a sample of sentences. The user can study the output sentences which comprise chunks of texts with collocated words.

TOTALrecall is a Chinese–English bilingual concordancer where its corpora include Sinorama 1990~2000 and records of Hong Kong Legislative Council. Corpus of Contemporary American English (http://corpus.byu.edu/coca/, [8], COCA) is a recently developed monolingual English concordancer with a much larger corpus than TANGO and TOTALrecall. Designed for English users, COCA is a recently developed English monolingual program with the largest corpus of 14 billion words of today. It has a wide variety of sophisticated functions such as List, Chart, Collocates, or Compare, in addition to KWIC. KWIC is the basic display function in all concordancers.

No matter how big the corpus of the concordancer is, there is the limitation of not being able to locate output for a user’s key word. We introduced the three concordance tools so as to reduce the likelihood of “not found” when the students learned to look up entries for error correction. Bilingual texts in TANGO and TOTALrecall can help students understand the concordance lines when the English texts may contain unfamiliar words. TANGO provides adjective-noun and verb-noun collocations as DDL often helps learners to observe the context of target words and acquire word patterns for their writing. COCA has a corpus much larger than those in the other two tools; it is most useful after the student attempts TANGO and TOTALrecall and cannot find any result. When the target is not related to collocations of AN or VN (VNP, VPN), then it makes sense to try either TOTALrecall or COCA.

Based on suggestions in prior studies [7, 10, 11, 24], the DDL tutorial was designed with, first, an introduction of the program’s functions. This was followed by a paper-based session by the teacher’s selected output concordance lines with errors made in the students’ writing with guidance of how to induce word usage patterns together in class (see Fig. 1 for the illustration) when the group of students first learned to use these concordance programs.

Illustration of Induction

The number in the parenthesis of the five concordance lines in Fig. 1 indicates the frequency of the collocation VN (devote+noun in this case) extracted from the monolingual BNC corpus such as 46, 18, or 10: those with a large number mean more common collocations. In the orientation for students, semantic features of nouns as the right collocate following devote (time, himself, life, energy, themselves) were called attention and so was the preposition to with the instructor’s explanation. Some students may mistakenly take “to” as an infinite; thus, assembling in the first incident was highlighted and so was social life in the second incident. Active voice of devote was also pointed out when students were trained to look at the left collocate. Possible patterns for “devote” from students’ induction are the following:

  1. 1.

    “Devote” + oneself: “Devote” can be followed by oneself as in concordance lines of (2) and (5) above.

  2. 2.

    “devote” + time, energy, or life as in (1), (3), or (4).

  3. 3.

    “to” is a preposition, not an infinitive (to + verb) so it is followed by “assembling” in (1), “the social life” in (2), “the use of …children” in (3), or “…types of literary study” in (5).

For learner training, we sometimes used examples of mistakes which appeared in the students’ essays; this made them feel a strong sense of relevance and vision clearly regarding the usefulness of these concordance programs. One example is given below to show how students’ attention was brought to examination of the target linguistic feature:

[in a writing topic on ghosts/God] I do notbelieve [VF, marked by instructor with the error code VF and underlined key word believe] them. [TANGO was chosen to look up the key word believe]

Patterns of believe that and believe in were highlighted after concordance lines were examined with the two usage patterns induced.

After the first-round orientation of the concordance program’s basic functions, four to five in-class sessions were arranged with 30 to 40 min each depending on how many illustrated examples were and how complicated the consultation/induction process was. Through in-class group discussion, the students and the teacher examined concordance lines and left/right collocates together to find patterns while some students were asked to reflect why certain patterns were succinct and applicable to certain errors in their own texts or other contexts.

Research Materials

An evaluation questionnaire, students’ texts of drafts, their records of revision, video recording, and a retrospective interview were research materials used in order to collect data in the study. An 18-item questionnaire in Chinese was designed and used to elicit the participants’ perceptions concerning using corpus consultation for error correction in their assignments with a 5-point Likert scale of agreement. Over the 18-week semester, we collected four different texts from each participant: one diagnostic writing task and three different genre/topic assignments. A student’s written record of revision was designed as illustrated in Fig. 2 by Terry (a student’s pseudonym). In the record, students provided error types, corrections, reference sources (corpora, their own knowledge, or other reference tools), and patterns found or their comments (particular segment of concordance example taken down for comparison). Suzuki [26] demonstrated the positive effect of written languaging (reflection) on uptake of written corrective feedback in the L2 writing class, and we took this written record in our study as one form of written languaging. When the student indicated that they used their own knowledge to correct a particular error, it meant an incident where errors were marked by the instructor and the student soon realized, with the error highlighted and the student’s own awareness, how to correct it by themselves. It might be out of carelessness while composing the second drafts but the student already had the linguistic knowledge for a correct form.

Fig. 2
figure 2

Illustration of the record of revision by one participant on the draft 2 of the third essay assignment

Full size image

Two errors of (3) and (4) were illustrated via errors taken from Terry’s essay:

(3) Although I hadlistened [VF] different types of music…. [marked by the instructor]

  1. (4)

    This song made a tremendous response when it released.

The other research materials included screen recording of the error correction process where a screen capture program (Icecream App, http://icecreamapps.com/Screen-Recorder/) was adopted for the learners to record their lookup process. Last, one one-to-one after-class retrospective interview in Chinese was conducted. We referred to Ferris et al. [9] for items in the protocol.

Data Collection

The data collected included all 14 students’ questionnaires, four texts per student with error markings and corrections in drafts, written revision records, a retrospective interview, and video recording of revision sessions in the 18-week semester. In the beginning of the semester, the first students’ written text, an in-class diagnostic essay, was produced taken as the entry level of their writing performance. It was then marked by the instructor with the error list, and corrected by students at home with their written record of revision after the three concordancers were introduced in class to them.

In a computer lab, while the students were correcting marked errors on the second draft of each of the three writing assignments across the semester, video recording was conducted and they were also asked to write a written record of revision. Ample time was given for lookups, revisions, video recording, and writing their revision records in the computer lab. Close to the end of the semester, the interview was conducted to understand how the participants felt about correcting errors in their essays using concordancers and other e-reference resources. Students’ own essay drafts and revision records were prepared for relevant questions.

Data Analysis

To analyze and triangulate various data, we first tallied, categorized, and aggregated the number (frequency) and type of errors in all the student texts. We followed the method used in Han and Hyland [13] to calculate error rates, which was the number of errors per hundred words in each participant’s second draft of the three assignments. Questionnaire data were analyzed mainly through means of the group. For interview data, qualitative methods were adopted for content analyses mainly to complement the questionnaire findings. Video recording of how learners actually consulted corpora was used to corroborate their written records of revision. Each “incident” of lookups in the video files was viewed by the researcher and read against the students’ written records of revision to examine (a) which word (or phrase) was consulted in their second draft of each essay assignment, (b) whether it was due to an error marking by the instructor with which linguistic feature (e.g., VF, verb formation) or self-initiated queries, (c) what pattern or comment was taken on the written record, and (d) whether the particular error was successfully corrected (against their third draft) or not.

Results and Discussion

Group patterns of the 14 students’ error correction in texts aided by corpora and their perceptions about using corpora to revise essays are shown first, followed by detailed analyses of two focal students in order to respond to the three research questions, followed by discussion respectively.

Error Rates and Types in Writing

The essay lengths across all the drafts under investigation ranged from 233 to 366 words long. Two hundred and seventy-three errors were coded on all the second drafts of the three assignments. This means each student made 6.5 errors per essay on average of the group across the semester. To adjust the factor of different essay lengths, we adopted the measure of error rates per 100 words [13]. Each raw number of total errors per student’s draft was divided by their hundredth unit of essay (2.33 if s/he wrote 233 words), and this came to an error rate of the student. All students’ error rates were added and divided by the total student number and came to an average error rate. For the 14 participants, across the second drafts of the three multi-draft essay assignments, the average error rate in the first essays (essay 1) was 2.54% (the narrative genre), 3.08% in essay 2 (analysis), and 2.37% in essay 3 (comparison/contrast) with an average of 2.60% of the three. Concerning the error types, the participants made a great variety of error types across their three assignments in the semester. The dominant error type in the group was verb tense (VT), on which 11 of the 14 participants made errors, followed by noun plural marking and verb phrase formation.

Error Reduction Across Three Essay Tasks

We compared marked errors in the second drafts of all the three essay assignments with the corrections in their third drafts within the same assignments after the students used various reference resources. The number of the errors which still stayed in draft 3 was divided by the number of the total marked errors in the second drafts (error reduction). The error reduction value was timed by 100% and this came to an error reduction rate. All reduction rates were added and divided by the total number of students. This yielded an average error reduction rate for each essay task. Rates of error reduction across the three major essay assignments were 58.80%, 71.10%, and 81.90% (70.60% on average of the three assignments). This means that the students could correct marked errors successfully after they noticed problematic areas noted by the instructor, and they used various knowledge sources or references including corpus tools for assistance to revise, similar to earlier research [7, 28]. While error rates and types as well as error reduction gave a window of students’ writing performance, error reduction shows their efforts of improving inaccuracies when corpora came into play for error correction.

Tool Choice or Knowledge Sources Chosen for Error Correction

Through the participants’ written records of revision while they were working on correcting errors on their second drafts of the three assignments and the recorded video files of their revision process, we tallied and aggregated the sum of each tool choice or knowledge source they drew to rectify errors. The distributions of proportions under each are shown in Table 1 (in % unit as group average per essay assignment). Using the percentage was because the total incidents across the three essays were different, making comparison of raw numbers impossible. Regarding the first diagnostic essay, the frequency distributions for each knowledge or tool source (from high to low) were the students’ own knowledge (59.20%), using electronic dictionaries (10.60%), using Google (4.50%), and the last category asking the instructor (1.5%), or using other resources (3.00%). Only on 21.20% of errors in the diagnostic essays did the students use TANGO, COCA, or TOTALrecall to correct. However, we found an obvious increase of later corpus use for essays 1, 2, and 3 as shown in Table 1. The frequency of corpus use for correction was 32.70% in essay 1 and 65.40% of essay 2, but went down to 44.20% for essay 3 (average of the three, 47.43%). Given one semester, corpora became the most frequently used reference of all tools when the students sought assistance to correct errors, 47.43%, followed by the participants’ own knowledge, 42.93%. It is obvious that the students’ proportions of corpus consultation increased as they worked from essays 1, 2, to 3 in the 18-week semester. The distribution patterns among the frequency usage rates of the three corpus tools were that TANGO was used the most often (58% of all corpus queries), followed by TOTALrecall (20%) and COCA (22%, with the three amounting to 100%).

Table 1 Distribution of tool choice or knowledge source (%) across 4 essays
Full size table

With 47.43% error correction rates using corpora (129 out of 273 errors), we would know on what linguistic features of the consulted headwords and with what success this group chose corpora to correct errors. Altogether, there were 195 corpus queries recorded (after comparing those in the written revision records and the video files) while revising lexicogrammatical inaccuracies on the second drafts of the three assignments. The frequency and error types of corpus consultation did not show a clear pattern among the three genre types (assignments) composed at different times but fluctuated very much. The learners’ needs of corpus consultation mostly came from the instructor’s coded errors (129 out of 195, 66.15%) and, additionally, from some self-defined or desired targets for lookups (66, 66/195 = 33.85%). That is, 66.15% of corpus queries came from the instructor’s marked errors on student essays, and about one-third of all (33.85%) were self-initiated queries along the line of applying concordance programs for improving writing inaccuracies. The latter can be regarded as an initial sign of emerging learner autonomy on corpus use, expanding its function for general learning (not limited to correction only). To initiate a query, mostly the participants figured one target word as input; occasionally, they chose two adjacent words as the key in order to find the desired phrasal usage or word use in context. As for their decision making for revision on essay drafts after the query process, 91% of the queries were found to be correct cases as we compared their second and third drafts. The others were classified into incorrect cases (7% misinterpretation) and non-use (2% “defaulting to a safer option” in Park’s term, [22], p. 378). Non-use cases were those the participants consulted but did not apply into error correction.

We found 11 linguistic features of the consulted headwords the 14 participants relied on corpora for error correction. Among the 11 error types, verb formation (18%) was ranked on the top, followed by verb tense and word formation (both 14%), and then word choice (10%). They also looked up preposition, chunks, noun plural marking, and word choice, and self-desired verbs, nouns, or adjectives (see examples in Appendix Table 3). The 11 features showed what was feasible for corpora to help error corrections as far as these participants’ consultation literacy was concerned. Although error types corrected through corpora were diversified, they were slightly different from the 20 codes we used to mark most errors that occurred on students’ essay drafts [7, 11]. The participants tended to use their own knowledge (41.60% of all sources, average of those in the three assignments) to correct errors of subject–verb agreement, comma splice, article, plural marking, verb tense, or run-on types.

To answer the first research question, we found that the participants could reduce errors in their later essay drafts after they received the instructor’s written corrective feedback and attempted corrections using the three corpus tools and various knowledge sources or other tools, similar to precursor research [7]. They made use of various knowledge sources and tools to correct errors successfully with over 70% marked errors being reduced. Over 47% of errors were corrected with the assistance of the three corpus tools, and the frequency of use of corpus tools increased across the four essays in the semester with 91% of the consulted incidents successfully applied to correct errors. Additionally, TANGO was used the most frequently among the three corpus tools.

Students’ Perceptions on Corpus Tools Through the Questionnaire

Based on the results of means of items in the evaluation questionnaire (see Appendix Table 4), most students reported that they knew how to look up words through the corpus tools (mean = 4.4 out of the total, 5.0, in item 1) and DDL was helpful to them about learning word usage (3.9, item 2). Without surprise, while using corpus tools, they used a great variety of other tools for correction (4.1, item 14) such as online bilingual dictionaries or Google. If given more time, they were confident that they would become more skillful in correction and corpus consultation next semester (3.9, item 15). Most of them would continue to use corpus tools in order to keep improving their writing (3.8, item 11). Compared with dictionaries, corpus consultation better helped them correct errors (3.6, item 4) and made their sentences and vocabulary use more natural (3.6, item 5). With slightly reduced agreement, they reported their vocabulary use and grammar improved due to corpus use (3.5, item 10), which allowed them to memorize grammar and word usage (3.5, item 12). The rating of whether learner autonomy was encouraged through DDL was not high enough (3.4, item 7). The other items which were rated lower were those concerning corpus use with writing confidence or fun (items 8 and 13, both 3.3), serving as future e-reference tools (item 9, 3.4), or assisting writing development (item 3, 3.4). The coded errors from the instructor’s indirect feedback were regarded as helpful (item 6, 4.1), but the agreement scale on whether the practice with corpus consultation was enough was only 3.5 (item 16). The overall mean ranking was 3.67, slightly higher than 3.00 (which stood for “unsure”).

Item 17 and item 18 of the questionnaire probed more into the reasons for advantages and disadvantages of using corpora for error correction (cf., [24]). To show their reservations on CAWCF, the participants indicated that corpus consultation was time consuming, compared with using dictionaries. Sometimes, it was difficult to understand examples in the corpus, to find a proper key word as lookup headwords, or to use induced patterns to correct errors. These were also found in prior studies [7]. Some of them were simply not used to corpus tools. In spite of these challenges, two-thirds of them felt they had improved consultation skills later in the semester as they worked on DDL to correct errors across the three assignments. For benefits of corpus tools, the students indicated that they could see many more example sentences than those in a dictionary, the target sentence in real use, the context of target words, and more frequent example sentences. They could also search for and learn target sentences independently and started to think more about word class while making correction. DDL may serve as either affordances or constraints [32] in the WCF processing and its role depends on how learners can skillfully harness its learning potential into their revision needs. The answer to the second research question was affirmative (similar to [3, 7]) but with moderate satisfaction.

Two Cases

In an attempt to find possible explanation for the group findings, two students out of the 14 were examined in detail as case studies: Dan (a male student) and Vivian (a female one, both pseudonyms). Their writing performance was among the average in the class as observed by the instructor. We triangulated all of the two students’ texts plus their revision records, screen recording of corpus lookups, and interview and questionnaire data. In sharp contrast, Dan was a very limited corpus user for error correction in all the drafts across the semester (5 times for four assignments), while Vivian was a frequent user (21 times), based on their self-reported revision records and our video recordings. Dan preferred using the cell phone for necessary lookups; he seemed underengaged with self-correction using corpora to query but would choose to ask the instructor’s advice in class for immediate corrections. In private, he revealed in the interview that he tended to be impatient about details in writing accuracy or revisions. Dan complained that corpus consultation for error correction at the revision stage of writing was time consuming and he encountered Internet lag often with the concordance programs (other interviewees indicated they did not have problems with corpus use on their smartphones). Yet, for DDL benefits, Dan indicated he could see the target sentence in real use, the context of target words, and more frequent example sentences.

On the other hand, Vivian endorsed all listed seven DDL benefits elicited in item 18 of the questionnaire, and she looked up not only words concerning her own mistakes in the essay drafts but also additional words of her choice. She further looked up some other words so as to learn more. In one record whose draft did not show any marked errors (essay 3-draft 2), she looked up seven target words (imagination, relationship, doubt, carry, enemy, forcible, escape) and their VN or AN combinations. With such self-initiated queries, Vivian shows a clear sign of developing learner autonomy in adopting corpus consultation into the revision process and phrase learning for writing.

When we examined the video recording, Dan sometimes looked randomly at example sentences and ended up not using the search result, leaving the error to stay between drafts. Vivian consistently clicked on the VN, VNP, and VPN options (using TANGO) to look through example sentences on verbs such as draw or change in order to observe word behaviors in different contexts when the pattern was specified. Potentially, she learned various phrases through such opportunities being exposed to additional examples which she took up and induced phrasal patterns for writing. Vivian strongly agreed that she would take corpus tools as the frequent references for learning in the future but Dan did not agree. Their different trust and preference of corpus tools resulted in at least succinctly contrastive frequency of corpus use and various learner–corpus interactions as illustrated in the two examples below.

(Dan’s second draft, text marked in italicized format)

[past tense marker shown in the beginning of this paragraph]…heteach[VT, marked by the instructor]me many things, like almost all the things. For example, heteach[VT, instructor’s marking]me that if you will not do your best on one thing, then do not do it…. And he alsoteach me do not mess up with[VF]the one who bother you.

(Vivian’s second draft)

We walked through [PREP] a variety of stands.

Without fully understanding what VT meant (as confirmed from our interview), Dan used TOTALrecall to check teach. The concordancer produced the following example sentences:

But our teachers never taught us what we ought to buy to eat.

They have taught me how to look at the world, how to look at humanity.

To teach a man to fish is better than to give him a fish.

After viewing the examples, Dan decided to change his original sentence into:

He said “to teach a man to fish is better than [omit ‘to’] give him a fish”.

He did not successfully correct this mistake as Dan had limited awareness of the error nature and ignored the affordance provided by corpus consultation. Thus, it is less likely for him to incorporate feedback into subsequent revision [26].

(Vivian’s earlier draft)

We walked through [PREP] a variety of stands.

Vivian checked walk using TANGO and clicked on the VPN option. She found the first VPN collocation (1) ~along road, having 5 instances, and the 30th collocation (30) ~through forest having 1 instance. Then, she examined their instances. At the end, she decided to change her original verb phrase from walk through to walk along, making a successful correction. She told us that she figured out the difference between through and along and in her sentence, along is appropriate.

figure a

We found that Vivian demonstrated full trust in DDL benefits and deep engagement with corpus-aided feedback processing at both the affective (feeling positive about corpus use for error correction and learning) and behavioral (more lookups and higher correction frequency) levels (cf., [13]). As time went by, she developed more mature consultation skills to help with writing revision. Dan was perhaps underengaged with feedback processing in his corpus-aided revision process due possibly to his unfamiliarity with corpus tools, less consultation literacy, and not recognizing corpus benefits for improving writing inaccuracies. The intricate individual differences which caused different learner–corpus interactions, lookup frequency, and revision outcome warrant more future investigation. From our case studies, it seems that learner engagement with corpus-aided feedback processing plays a crucial role (like that in [13]) because it may originate from positive affective responses from the learners to corpus consultation as a powerful tool for error correction, and as cognitive extension [32]. Deep engagement with more hands-on corpus consultation led to successful corrections and reinforces continued practice and sophisticated skills in the process over the semester.

Conclusion and Implications

In this semester-long classroom-based study, concordance programs were used to resolve lexicogrammatical inaccuracies in writing. To respond to the three research questions, the students’ attempts for correction with the instructor’s indirect feedback indicated that errors in their writing decreased by more than 70% concerning their three writing assignments over time in spite of fluctuation. For the role of corpus consultation, learner concordancing seemed to have made the major contribution (47.43%, on top of all sources of help, including own knowledge and tools). The usage rates of corpus tools increased during the semester from 21 to 65% though they later dropped to 44.20% with an average of 47.43%. The group of EFL students indicated that DDL was acceptable (3.67 out of 5.00) to them as a usable e-referencing tool, compared with dictionaries, for consultation to make their essays more accurate (similar to [3]).

The fact confirmed that with the instructor’s indirect feedback of coded errors, the learners could learn to use concordancers to correct errors in their writing (91% successful rate) over time with the students’ moderate satisfaction about the process. Before the participants joined this project, their dominant reference tools were online bilingual dictionaries. Now, they were familiar with and could use concordancers to make error corrections at the post-writing revision stage, realizing corpus tools were among their useful reference resources at hand.

Although not every participant found DDL very useful for their error correction, findings of our two cases show that one found it a novel and useful learning method of writing more accurately like the deeply engaged Vivian in our case study. Fully appreciating the merits of concordancers, these learners actively found concordancers for help to improve inaccuracies in their essays. The motivated autonomous learners went even further to initiate queries of other target words when the words were not marked by the instructor as errors, a sign of self-directed learning. On the other hand, technology adoption is a long learning process and may require a new habit adaptation (in another case). Given longer time with integration of corpora into this year-long writing course which we plan to continue in its second semester, we look forward to the learners’ full mastery of corpus consultation literacy and adoption of corpora into their regular writing references.

Research Implications

Liu and Brown [20] criticized the lack of classroom ecology of precursor WCF research with one-shot treatment on timed writing. To ensure classroom ecological validity, the current study managed the implementation of multi-drafts in three different assignments (topics/genres) over one semester as the design fully integrated corpus consultation into the students’ revision stage in a writing course. We found moderate success coming from the group’s corpus consultation as Yoon [32] argues that including corpus use into L2 writing can serve both a cognitive extension and a distraction. In other words, corpus use does not solve every issue concerning writing revision or improving accuracy on student writing. Judicial use of various electronic tools, learners’ trust on tool usefulness, and their adequate consultation skills plus learner engagement with corpus-aided feedback uptake ensure long-term practice of DDL for writing revision, and may ultimately lead to writing development.

Without a strict control over corpus use, it is difficult to argue all positive benefits we found in the current project came solely from learner concordancing for their writing revision. However, our classroom-based design shows to L2 researchers and writing teachers a feasible design with longer term integration of corpus use as done in regular classrooms or curriculum with confirmatory and satisfactory findings (e.g., [7, 27, 28]).

Pedagogical Implications

Like traditional WCF practice in L2 writing classrooms, our learner concordancing experiences indicate careful pedagogical planning and instructional design are crucial to make corpus-aided feedback processing to take effect. Learners must have adequate consultation literacy to make good use of the potential of concordancers in order to rectify errors in their essays, which leads to accurate writing and successful learning. Although most participants appreciated the assistance the corpus tools offered to their revision process, unsuccessful attempts and non-use from queries, or even being unwilling or unable to make effective corpus consultation for error correction, were also noted in the processes of some learners. We suggest that the students should be given long enough training with several DDL illustration sessions integrated into a writing course in order to provide continuous scaffolding when necessary. In our study, one semester with several rounds of illustration or teacher modeling plus practice in the students’ own essays is crucial or at least very helpful. Corpus tutorials with grammar explanation using cases in specific writing context of errors in the students’ own text are very informative. Written records of revision, a form of reflection (“languaging” in Suzuki’s term, [26]), are shown to be effective, as the design raised students’ language awareness about how they could induce patterns from concordance lines and apply them to improve writing inaccuracies.

The study contributes to our further understanding of corpus-aided written corrective feedback (CAWCF) research when a group of EFL learners was observed over one semester concerning reduced errors in various assignments, frequency and purposes of corpus consultation, and their perceptions of corpus integration into CAWCF processing. To maintain curricular standards (set by particular writing programs) and integrate corpus use into weekly practice of an EFL undergraduate class is a significant strength in the study design as it keeps the classroom ecology [12, 20]. We believe the findings of our study have wider applicability to other writing classes and the written communication needs of the real world than other studies with much stricter control for corpus use alone (and excluding other references such as dictionaries) or for one single concordancer.

Limitations and Further Studies

We were unable to show a clear picture of exactly what error types could be rectified using corpora and what could not, or what causes resulted in particular error types the students used their own knowledge or other tools to correct, unlike findings in Tono et al. [28] or Crosthwaite [7]. First, we tried to make an ecological valid classroom-based study by allowing students to choose tools among various options. This is different from a rigorous experiment with strict control over corpus use on target error types. Second, we believe that learners’ own differential lexicogrammatical knowledge on different genres makes it extremely challenging to predict particular error types and provide appropriate corpus-aided instruction accordingly. In helping learners rectify errors with corpora, Park [22] indicates a limitation on the tools themselves because not any error types can be rectified via corpora. Our learners showed that they relied on corpora for particular lexicogrammatical error types, but used their own knowledge to correct errors of comma splice, run-on, or subject–verb agreement because corpora have little to offer in correcting those error types and it seems that they knew well when to use corpora or not.

The quantitative picture in our group patterns may show to other writing teachers what they can reasonably expect if they desire to incorporate learner concordancing into their students’ revision process in a semester-long course context. The two focal cases further illustrate the important role of motivation and learner engagement with corpus consultation in the process [9, 13]. Our next steps are to keep the CAWCF practice in a year-long project by documenting whether more learner satisfaction may emerge after more students are convinced with usefulness of corpus tools for revision purposes with increased consultation literacy, and how learner autonomy with corpus use develops when the observation period is extended.