Abstract
As technologies advance, the use of corpora and concordance programs has shown their potential to help second language writing classes. Although some writing research has demonstrated usefulness of corpora in different ways, much less is known concerning how corpora can scaffold students’ correction and uptake of written corrective feedback for revision. The current study adopted a semester-long classroom-based research design keeping classroom ecology in order to examine how second-year EFL college students took up teacher-coded feedback by consulting corpus tools and other e-reference resources to rectify errors in their essays. Fourteen participants composed three multi-draft essay assignments plus a diagnostic essay in various genres. They used two Chinese–English concordance programs and “Corpus of Contemporary American English” to help their revisions. Questionnaires, interviews, students’ marked and revised essays, their written revision records, and video recording of learner–corpus interaction were analyzed and triangulated. Errors were found to decrease with increased frequency of corpus consultation over the semester. Although most participants appreciated the assistance of corpus tools, reservations were noted in the process. Two student cases with average writing performance point to learners’ engagement with corpora as one crucial factor interacting with perceptions and outcomes of lookups. Pedagogical implications and limitations are discussed.
摘要
隨著科技的進步,語料庫及其檢索系統之應用已展現對英文寫作教學之成效,但針對大學部非以英文為母語學習者,其使用語料檢索來更正英文作文錯誤之探索,則相對不足。本文植基於生態效度及課堂研究理念,觀察十四名大二學生在修讀一學期作文必修課中,如何利用老師標示作文中錯誤以及語料檢索工具以輔助改錯之過程。學生完成三份作文以及學期初診斷寫作,每篇作文經過三次撰稿修訂;使用之語料檢索系統含兩項中英檢索程式以及美國當代英文語料庫檢索系統。資料收集含問卷、訪談、學生被標錯之二稿及修正過之三稿、修正紀錄以及檢索改錯之螢幕錄影檔。我們發現學生在一學期當中,作文之錯誤隨三份作業越益變少,語料檢索則隨該三作業使用頻率增加。雖然問卷中,大部分學生對語料檢索輔助改錯持正面態度,他們對某些改錯效益持保留態度。兩項個案深入學生檢索改錯過程,發現學生投入程度是決定語料檢索頻率,成效及正負態度之關鍵性因素。文末提出教學應用以及本研究限制。
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
Introduction
Providing written corrective feedback (WCF) to students in writing classes is a common practice and has been shown to be effective in improving writing quality (e.g., [9, 13, 15, 20]). After a feedback is given, the correction job can be designed for second language (L2) or English as a foreign language (EFL) students to revise, and they may rely on some reference tools to help the revision process particularly when they do not have enough linguistic knowledge about the problem. Concordance programs, with dictionaries, grammar references, and online search engines like Google, are common learning references. The development of using corpora for language learning as a new reference tool comes from a renewed interest in authentic data offered to second language learners, and the belief that learners can adopt new and more active roles in their learning process [1, 31] via inductive strategies of observing corpora carefully. Learners’ direct access to corpora, also called data-driven learning (DDL, [14]), has been shown to be beneficial to various aspects of L2 writing: collocation [18, 25], genre knowledge [6], proofreading [4], or error correction [7]. The effects of using learner concordance with other electronic tools for writing have also been addressed [16, 30, 32].
The current study contributed to our understanding of using corpus tools to correct errors in writing for one semester by keeping the ecological landscape of a classroom [12, 20], unlike that in a strictly controlled experiment setting. That is, we conducted the project by keeping what an EFL writing teacher would do regularly to promote a written communication atmosphere and meet curricular requirements of the class. Specifically, we documented a group of college students’ feedback uptake augmented by corpus consultation with several assignments in a writing course. We showed how the learners engaged in such a process by exploring the extent to which they benefited from processing corpus-aided written corrective feedback (CAWCF) and how they felt about such an approach. We also illustrated two learners’ engagement with corpus-informed feedback processing with their lookup processes and detailed data triangulation.
Review of the Literature
Written Corrective Feedback in L2 Classrooms
Various dimensions of written corrective feedback (WCF) have been well-documented in its rich literature of traditional L2 writing classrooms. We show a collective picture from two recent meta-analyses and more updated individual studies [15, 20]. Kang and Han [15] confirmed that feedback is generally effective in increasing grammatical accuracy (with an effect size of 0.54), given focused or unfocused (or comprehensive) feedback. With a focus on genre types and mediating variables, they found that both direct and focused feedback has durable effects; while genre types do make a difference, journal writing is not amenable to correction but composition is, which is more effective than letters. In Liu and Brown’s meta-analytic report [20], four methodological recommendations for future WCF research are raised: (a) to improve design of low ecological validity in most studies where one-shot treatment in timed in-class writing tasks was adopted; (b) to provide qualitative data with mixed-methods on learners’ processing of feedback, strategies used to handle feedback, and feedback retention; (c) to give clearer error categorization; and (d) to conduct more longitudinal studies (also suggested in [15]). For a study to be ecologically valid, the design needs to ensure the research process equipped with the routines and materials commonly and regularly applied in the language classroom. The current study responds to the call for improving the research design of ecologically valid CAWCF studies in classrooms by involving students in multiple writing assignments over one semester.
To complement quantitative WCF findings discussed above, two recent case studies of L2 writing focused on student-writers’ individual differences by offering qualitative insights. Ferris et al. [9] explored ten L2 college freshman students’ uptake of WCF, by asking the students to write four timed persuasive essays in class over a semester. A 15-week longitudinal multi-case classroom research design was adopted. They marked three to four error patterns on the student essays using indirect feedback based on an agreed-upon list of error codes. Through retrospective interviews, they found that most learners used their intuition to see what sounded right. They suggested that writing teachers should not look at learners’ written product only. Individual and contextual factors influenced these learners’ writing development, and writing teachers should take “a more finely tuned approach to correct feedback” ([9], p. 307) by reducing students’ anxiety and raising their motivation for revision. Through a naturalistic multiple-case study approach, Han and Hyland [13] examined four Chinese college students’ cognitive, behavioral, and affective engagements with WCF after they finished a take-home writing assignment with two drafts and went through feedback processing for error correction. The depth of feedback processing in their four cases (out of 25 students) was not related to the effectiveness of revision. They argue that learner beliefs and goals, and the interactional context where WCF was received and assimilated into their L2, may result from individual differences and various engagement amounts and types.
Learner Concordancing for Language Learning
New kinds of affordance and learning potential are available as a recent meta-analysis of 64 studies has been conducted on applying DDL to effectively learn various aspects of a second language including writing [1]. Learners may interact with corpora, inducing patterns of use with observation of concentrated examples in context [27], in order to trigger writing development [22]. Recent advances on corpora and online tools such as “British National Corpus” [2] and “Corpus of Contemporary American English” (COCA, [8]) bring students more reference choices for assistance.
DDL studies conducted in Taiwan EFL contexts also demonstrate successes as various applications have been innovated (e.g., [4, 16, 17, 19, 25, 29, 30]). In college contexts, Lin [19] examined how English teachers used concordancing to teach grammar as compared with the traditional grammar approach and found favorable results of concordancing. Lai and Chen [16] explored how college students used dictionaries or concordancers for different writing purposes. Wu [29] investigated the detailed effects (e.g., collocation accuracy or complexity) of DDL for learning collocations in writing. Yeh et al. examined the combined use of a concordancer and automatic writing assessment tool on error correction in a college writing context and found positive effects. In high school contexts, Chang and Sun [4] show the better proofreading performance of a group of high school students, augmented with concordance searches, under the scaffolding condition than another without scaffolds. Their design of prompts also increased students’ certainty level while performing the task. Lee and Liou [17] applied DDL in a high school context for vocabulary learning purposes. Sun and Wang [25] compared two groups of high school students using either inductive or deductive approaches to learn collocations of different difficulty levels and found the inductive group performed better on easy collocations than the deductive group. From this brief review, using concordancing for error correction in a college writing context warrants more research in Taiwan as limited studies were concerned about this dimension of applying learner concordancing for writing purposes.
Corpus-Enhanced Writing Revision
Compared with studies examining general corpus-assisted writing issues, much fewer have addressed how corpora can scaffold students’ correction process and uptake of WCF at the revision stage. Meanwhile, Cobb and Boulton [5] draw corpus scholars’ attention to longer term effects of students’ concordance work. Most existing learner concordancing literature examined general L2 writing contexts for graduate students (e.g., [32]) or looked into learner perceptions only without examining their writing performance (e.g., [24]). Some examined students’ performance at the decontextualized sentence level or (particular) collocations in isolated sentences [17, 25, 29]. The current study focused on changes made onto the undergraduate students’ essay-level texts after corpora were incorporated to assist the WCF processing process. Six relevant CAWCF studies involving data of students’ essays were reviewed. Todd [27] examined whether Thai post-graduate students could induce word patterns from corpora for self-correction in their English writing. They found adjective errors marked by their instructor on the essays were the easiest compared with those of verbs and nouns. Thirty-two doctoral students from Hong Kong were examined in Crosthwaite [7]. They composed a piece of English text as part of their graduate study, received teacher feedback, and rectified errors using both Sketch Engine for Language Learning (SKELL) and BNCWEB. The participants went through five 2.5-h sessions for corpus training followed by revisions. Crosthwaite found that the number of corpus-led revisions reached an average of 83% reduced errors (of all types) from the drafts, and the students felt mostly positive about introducing the corpora into their revision process. Collocation errors were more successfully rectified than morphosyntactic errors.
Two studies looked into how French learners used corpora to rectify errors in texts. On graduate students, Chambers and O’Sullivan [3] documented how DDL can help French learners correct errors in their texts in a 2-h lab session. Similarly, O’Sullivan and Chambers [21] examined an undergraduate student group in a similar context. Both studies found encouraging students’ performance and graduate students’ attitudes toward DDL was slightly more positive. Two studies investigated how undergraduate students could use corpora to correct errors in their English essays. Tono et al. [28] recruited 93 Japanese college students to use corpora to correct errors in their timed essays composed in the lab. Students were divided into two groups of different proficiency. It was found that the revisions of omission and addition errors were more than those of misformation errors (i.e., verb tense) (e.g., “my parents talk him very often”) in student essays. The revisions from the less proficient group yielded a higher accuracy rate than the more proficient due to the former’s shorter papers with easier vocabulary and structures accompanied by fewer errors. Gaskell and Cobb [11] was the only study that was designed like a regular writing class with ten assignments over 15 weeks. They used Lextutor (http://www.lextutor.ca) to revise errors in the essays of lower intermediate EFL learners over a semester. They found three error types were decreased in later assignments, word order, capitals/punctuation, and pronoun, but errors increased. Although providing concordance lines for learners was a way to give learners directions in the consultation process, it deprived the chance for learners to figure out the key words to search for themselves, which is an essential step of induction. They suggested that future research will use a sharpened research instrument for recording lookup behaviors.
Among sharpened research instruments, video recording is argued to be able to complement existing research procedures and data [16, 22, 23]. Using video recording to capture learners’ moment-in-moment behaviors while performing online corpus consultation promises to provide insights into L2 writing development. Through a one-semester longitudinal study, Park [22] triangulated screen recording, student writing assignments, oral and written reflections, and corpus-query analysis of undergraduate ESL students in order to show the learners’ language development. Park demonstrated his three learners’ writing development by providing improved writing performance and language awareness.
From the review above, research of corpus-aided error correction seems to yield effective outcome, although the number of such studies is still quite limited. Two of them were on French learning students, instead of English learners. Half of the studies reviewed did not investigate undergraduate students like the target population in the current project. More importantly, only one study [11] proceeded like regular coursework in a writing class using multiple assignments. The other studies were conducted with one-shot feedback provision, timed in-class writing tasks, or targeting one single or limited focused error types. Such design has been criticized as the lack of classroom ecological validity [20] in regular L2 writing classes particularly with only one-shot treatment of feedback. These facts indicate more longer term classroom-based research with ecological validity on undergraduate learners who use corpora to correct errors in writing are sorely needed to better inform teachers and researchers.
The current study bridges research gaps in several aspects with an aim to investigate how a group of college students used the WCF to improve writing when various references including concordancers were available. With a research design to ensure ecological validity that was augmented by video recording of learners’ corpus consultation process, the study documented a semester-long writing course and investigated how corpus consultation (and other e-reference resources) may play a pivotal role in EFL students’ self-correction process and outcome. The reason why students’ use of other e-reference tools was also observed was that tools such as English–Chinese dictionaries or Google were common to EFL students in Taiwan and to exclude their use is against their habitual referencing practice. With various considerations of classroom ecology, the current study made corpus integration a normalized practice for CALL teachers and researchers by having corpus consultation an integral part of the writing classroom activity for an extended period of time. Research questions included the following:
-
1.
How did the college students improve writing inaccuracies when corpus tools were integrated into their revision over one semester?
-
2.
How did the students affectively respond to DDL-aided error correction? Why so?
-
3.
What might further account for the students’ writing performance and perceptions when corpus tools came into play for their revision, given two case studies?
Method
Context of the Current Project
The study was conducted in a university of central part of Taiwan. Fourteen second-year college students participated, who majored in English. Based on our prior experiences of teaching in the institution, a very limited number of the participants had studied abroad before they were admitted into college. Built around a process pedagogy where each writing assignment went through three drafts until it obtained a final grade, the English writing program required all its students to take the course from the first to the third year (14 credit-hours in total). The second-year program focused on expository essay practice. Fourteen participants (11 females, 3 males) came from an intact class which was arranged by the department office. The students whose first language was Chinese had studied English as a school subject for at least 8 years in schools before they were admitted to the college. Although writing was required as part of their college entrance examination, limited high school writing instruction was provided. Their self-reported TOEIC scores showed an average of 637. They all signed a consent form giving permission to use their essays and relevant data. Before the project started, a background questionnaire was given to understand the participants’ prior English learning experiences and their habits of using reference tools. The results indicated these learners never or rarely used corpora but relied mainly on online Chinese–English bilingual dictionaries or Google for reference purposes. Two average students of writing performance (based on the instructor’s classroom-based assessment on their essay assignments) were chosen out of the 14 students in the group for closer examination as case studies.
Instructional Design
For the writing course where the project took place, the instructional design considered how to meet the requirements of ecological validity, the goal of our writing program, good WCF practice [20], and the teacher–researcher’s teaching philosophies. In the course, genre knowledge and writing skills were given through the textbook with relevant writing practice. Three relevant writing assignments of different genre/topics were designed: narrative, analysis, and comparison/contrast. For each assignment, three drafts were required for practice and completed at home. Students wrote the first draft to obtain peer review comments and the instructor’s feedback on content and organization in order to work out the second draft. On the second draft, the instructor marked lexical and grammatical errors based on an agreed-upon list of 20 error codes (see Appendix Table 2, adopted from [9]), as indirect feedback with neither corrections nor metalinguistic explanations provided. By adopting a mid-focused strategy, the teacher provided roughly two to six coded errors of lexical or grammatical types in order not to intimidate students from further revision (facing too many to handle at a time) but be informative enough to improve writing accuracy (by focusing on selected types). The number of marked errors depended on different qualities of each student’s draft. Feedback of content and organization (goal of a particular genre writing) was also addressed in practice (as both the students and the course required such feedback) but not within the scope of the current study.
After the participants obtained the marked second draft, they revised it in a computer lab using the corpus tools or other e-references (dictionaries, Google, etc.), wrote a record of revision (to be illustrated shortly), and had the revision process be video-recorded at the same time. The resulted version turned into the third draft for the instructor to give a grade based on their accumulated efforts on the three versions. To ensure good reporting practice in the project, the time interval between writing assignments was 1 month and feedback turnaround time among drafts was 1 week. One additional writing task given in class which took place in the beginning of the semester was error-marked by the teacher and served as students’ homework practice of corpus consultation after she demonstrated how a concordancer could be used for error correction. This task served a diagnostic purpose for the instructor to understand students’ writing weaknesses.
DDL Training
From the background questionnaire, we found that the participants already had adequate computer literacy when they entered the college as computer education had been provided from the primary school. We introduced three concordance programs to the participants by giving orientation of how to use two bilingual concordance programs, TANGO and TOTALrecall (http://candle.cs.nthu.edu. tw/totalrecall/totalrecall/totalrecall.aspx?funcID=1), and simple functions of COCA [8] such as key-word-in-context (KWIC, see Fig. 1) because their interface and functions were user-friendly.
Three tools, TANGO, TOTALrecall, and COCA, have their different functions and purposes for students’ consultation in the study. TANGO (http://candle.cs. nthu.edu.tw/collocation/ webform2.aspx?funcID=9) targets verb-noun and adjective noun collocations (with extensions of verb-noun-preposition and verb-preposition-noun ones). The concordancer includes corpora of (a) Sinorama 1990~2000 (a Chinese–English bilingual online magazine, https://www.taiwan-panorama.com/en), (b) Voice of America (VOA), (c) BNC [2], (d) Hong Kong News & Laws, and (e) Academic collocation from BNC listed. When the user chooses Sinorama or Hong Kong News & Laws, they can view the bilingual output of English–Chinese aligned texts. For the other three corpora, they view English output only. To use TANGO (http://candle.cs.nthu.edu.tw/collocation/webform2.aspx?funcID=9), a user provides a key word and specifies its word category (a noun or a verb), and then chooses one of the collocation types (e.g., VN-verb and noun or AN-adjective and noun, VPN, or VNP-preposition) as consulting steps. Upon submission of the key word and collocation category, a group of frequent collocations from the computer program is shown and the user can click on the phrases to study a sample of sentences. The user can study the output sentences which comprise chunks of texts with collocated words.
TOTALrecall is a Chinese–English bilingual concordancer where its corpora include Sinorama 1990~2000 and records of Hong Kong Legislative Council. Corpus of Contemporary American English (http://corpus.byu.edu/coca/, [8], COCA) is a recently developed monolingual English concordancer with a much larger corpus than TANGO and TOTALrecall. Designed for English users, COCA is a recently developed English monolingual program with the largest corpus of 14 billion words of today. It has a wide variety of sophisticated functions such as List, Chart, Collocates, or Compare, in addition to KWIC. KWIC is the basic display function in all concordancers.
No matter how big the corpus of the concordancer is, there is the limitation of not being able to locate output for a user’s key word. We introduced the three concordance tools so as to reduce the likelihood of “not found” when the students learned to look up entries for error correction. Bilingual texts in TANGO and TOTALrecall can help students understand the concordance lines when the English texts may contain unfamiliar words. TANGO provides adjective-noun and verb-noun collocations as DDL often helps learners to observe the context of target words and acquire word patterns for their writing. COCA has a corpus much larger than those in the other two tools; it is most useful after the student attempts TANGO and TOTALrecall and cannot find any result. When the target is not related to collocations of AN or VN (VNP, VPN), then it makes sense to try either TOTALrecall or COCA.
Based on suggestions in prior studies [7, 10, 11, 24], the DDL tutorial was designed with, first, an introduction of the program’s functions. This was followed by a paper-based session by the teacher’s selected output concordance lines with errors made in the students’ writing with guidance of how to induce word usage patterns together in class (see Fig. 1 for the illustration) when the group of students first learned to use these concordance programs.
Illustration of Induction
The number in the parenthesis of the five concordance lines in Fig. 1 indicates the frequency of the collocation VN (devote+noun in this case) extracted from the monolingual BNC corpus such as 46, 18, or 10: those with a large number mean more common collocations. In the orientation for students, semantic features of nouns as the right collocate following devote (time, himself, life, energy, themselves) were called attention and so was the preposition to with the instructor’s explanation. Some students may mistakenly take “to” as an infinite; thus, assembling in the first incident was highlighted and so was social life in the second incident. Active voice of devote was also pointed out when students were trained to look at the left collocate. Possible patterns for “devote” from students’ induction are the following:
-
1.
“Devote” + oneself: “Devote” can be followed by oneself as in concordance lines of (2) and (5) above.
-
2.
“devote” + time, energy, or life as in (1), (3), or (4).
-
3.
“to” is a preposition, not an infinitive (to + verb) so it is followed by “assembling” in (1), “the social life” in (2), “the use of …children” in (3), or “…types of literary study” in (5).
For learner training, we sometimes used examples of mistakes which appeared in the students’ essays; this made them feel a strong sense of relevance and vision clearly regarding the usefulness of these concordance programs. One example is given below to show how students’ attention was brought to examination of the target linguistic feature:
[in a writing topic on ghosts/God] I do notbelieve [VF, marked by instructor with the error code VF and underlined key word believe] them. [TANGO was chosen to look up the key word believe]
Patterns of believe that and believe in were highlighted after concordance lines were examined with the two usage patterns induced.
After the first-round orientation of the concordance program’s basic functions, four to five in-class sessions were arranged with 30 to 40 min each depending on how many illustrated examples were and how complicated the consultation/induction process was. Through in-class group discussion, the students and the teacher examined concordance lines and left/right collocates together to find patterns while some students were asked to reflect why certain patterns were succinct and applicable to certain errors in their own texts or other contexts.
Research Materials
An evaluation questionnaire, students’ texts of drafts, their records of revision, video recording, and a retrospective interview were research materials used in order to collect data in the study. An 18-item questionnaire in Chinese was designed and used to elicit the participants’ perceptions concerning using corpus consultation for error correction in their assignments with a 5-point Likert scale of agreement. Over the 18-week semester, we collected four different texts from each participant: one diagnostic writing task and three different genre/topic assignments. A student’s written record of revision was designed as illustrated in Fig. 2 by Terry (a student’s pseudonym). In the record, students provided error types, corrections, reference sources (corpora, their own knowledge, or other reference tools), and patterns found or their comments (particular segment of concordance example taken down for comparison). Suzuki [26] demonstrated the positive effect of written languaging (reflection) on uptake of written corrective feedback in the L2 writing class, and we took this written record in our study as one form of written languaging. When the student indicated that they used their own knowledge to correct a particular error, it meant an incident where errors were marked by the instructor and the student soon realized, with the error highlighted and the student’s own awareness, how to correct it by themselves. It might be out of carelessness while composing the second drafts but the student already had the linguistic knowledge for a correct form.
Two errors of (3) and (4) were illustrated via errors taken from Terry’s essay:
(3) Although I hadlistened [VF] different types of music…. [marked by the instructor]
- (4)
This song made a tremendous response when it released.
The other research materials included screen recording of the error correction process where a screen capture program (Icecream App, http://icecreamapps.com/Screen-Recorder/) was adopted for the learners to record their lookup process. Last, one one-to-one after-class retrospective interview in Chinese was conducted. We referred to Ferris et al. [9] for items in the protocol.
Data Collection
The data collected included all 14 students’ questionnaires, four texts per student with error markings and corrections in drafts, written revision records, a retrospective interview, and video recording of revision sessions in the 18-week semester. In the beginning of the semester, the first students’ written text, an in-class diagnostic essay, was produced taken as the entry level of their writing performance. It was then marked by the instructor with the error list, and corrected by students at home with their written record of revision after the three concordancers were introduced in class to them.
In a computer lab, while the students were correcting marked errors on the second draft of each of the three writing assignments across the semester, video recording was conducted and they were also asked to write a written record of revision. Ample time was given for lookups, revisions, video recording, and writing their revision records in the computer lab. Close to the end of the semester, the interview was conducted to understand how the participants felt about correcting errors in their essays using concordancers and other e-reference resources. Students’ own essay drafts and revision records were prepared for relevant questions.
Data Analysis
To analyze and triangulate various data, we first tallied, categorized, and aggregated the number (frequency) and type of errors in all the student texts. We followed the method used in Han and Hyland [13] to calculate error rates, which was the number of errors per hundred words in each participant’s second draft of the three assignments. Questionnaire data were analyzed mainly through means of the group. For interview data, qualitative methods were adopted for content analyses mainly to complement the questionnaire findings. Video recording of how learners actually consulted corpora was used to corroborate their written records of revision. Each “incident” of lookups in the video files was viewed by the researcher and read against the students’ written records of revision to examine (a) which word (or phrase) was consulted in their second draft of each essay assignment, (b) whether it was due to an error marking by the instructor with which linguistic feature (e.g., VF, verb formation) or self-initiated queries, (c) what pattern or comment was taken on the written record, and (d) whether the particular error was successfully corrected (against their third draft) or not.
Results and Discussion
Group patterns of the 14 students’ error correction in texts aided by corpora and their perceptions about using corpora to revise essays are shown first, followed by detailed analyses of two focal students in order to respond to the three research questions, followed by discussion respectively.
Error Rates and Types in Writing
The essay lengths across all the drafts under investigation ranged from 233 to 366 words long. Two hundred and seventy-three errors were coded on all the second drafts of the three assignments. This means each student made 6.5 errors per essay on average of the group across the semester. To adjust the factor of different essay lengths, we adopted the measure of error rates per 100 words [13]. Each raw number of total errors per student’s draft was divided by their hundredth unit of essay (2.33 if s/he wrote 233 words), and this came to an error rate of the student. All students’ error rates were added and divided by the total student number and came to an average error rate. For the 14 participants, across the second drafts of the three multi-draft essay assignments, the average error rate in the first essays (essay 1) was 2.54% (the narrative genre), 3.08% in essay 2 (analysis), and 2.37% in essay 3 (comparison/contrast) with an average of 2.60% of the three. Concerning the error types, the participants made a great variety of error types across their three assignments in the semester. The dominant error type in the group was verb tense (VT), on which 11 of the 14 participants made errors, followed by noun plural marking and verb phrase formation.
Error Reduction Across Three Essay Tasks
We compared marked errors in the second drafts of all the three essay assignments with the corrections in their third drafts within the same assignments after the students used various reference resources. The number of the errors which still stayed in draft 3 was divided by the number of the total marked errors in the second drafts (error reduction). The error reduction value was timed by 100% and this came to an error reduction rate. All reduction rates were added and divided by the total number of students. This yielded an average error reduction rate for each essay task. Rates of error reduction across the three major essay assignments were 58.80%, 71.10%, and 81.90% (70.60% on average of the three assignments). This means that the students could correct marked errors successfully after they noticed problematic areas noted by the instructor, and they used various knowledge sources or references including corpus tools for assistance to revise, similar to earlier research [7, 28]. While error rates and types as well as error reduction gave a window of students’ writing performance, error reduction shows their efforts of improving inaccuracies when corpora came into play for error correction.
Tool Choice or Knowledge Sources Chosen for Error Correction
Through the participants’ written records of revision while they were working on correcting errors on their second drafts of the three assignments and the recorded video files of their revision process, we tallied and aggregated the sum of each tool choice or knowledge source they drew to rectify errors. The distributions of proportions under each are shown in Table 1 (in % unit as group average per essay assignment). Using the percentage was because the total incidents across the three essays were different, making comparison of raw numbers impossible. Regarding the first diagnostic essay, the frequency distributions for each knowledge or tool source (from high to low) were the students’ own knowledge (59.20%), using electronic dictionaries (10.60%), using Google (4.50%), and the last category asking the instructor (1.5%), or using other resources (3.00%). Only on 21.20% of errors in the diagnostic essays did the students use TANGO, COCA, or TOTALrecall to correct. However, we found an obvious increase of later corpus use for essays 1, 2, and 3 as shown in Table 1. The frequency of corpus use for correction was 32.70% in essay 1 and 65.40% of essay 2, but went down to 44.20% for essay 3 (average of the three, 47.43%). Given one semester, corpora became the most frequently used reference of all tools when the students sought assistance to correct errors, 47.43%, followed by the participants’ own knowledge, 42.93%. It is obvious that the students’ proportions of corpus consultation increased as they worked from essays 1, 2, to 3 in the 18-week semester. The distribution patterns among the frequency usage rates of the three corpus tools were that TANGO was used the most often (58% of all corpus queries), followed by TOTALrecall (20%) and COCA (22%, with the three amounting to 100%).
With 47.43% error correction rates using corpora (129 out of 273 errors), we would know on what linguistic features of the consulted headwords and with what success this group chose corpora to correct errors. Altogether, there were 195 corpus queries recorded (after comparing those in the written revision records and the video files) while revising lexicogrammatical inaccuracies on the second drafts of the three assignments. The frequency and error types of corpus consultation did not show a clear pattern among the three genre types (assignments) composed at different times but fluctuated very much. The learners’ needs of corpus consultation mostly came from the instructor’s coded errors (129 out of 195, 66.15%) and, additionally, from some self-defined or desired targets for lookups (66, 66/195 = 33.85%). That is, 66.15% of corpus queries came from the instructor’s marked errors on student essays, and about one-third of all (33.85%) were self-initiated queries along the line of applying concordance programs for improving writing inaccuracies. The latter can be regarded as an initial sign of emerging learner autonomy on corpus use, expanding its function for general learning (not limited to correction only). To initiate a query, mostly the participants figured one target word as input; occasionally, they chose two adjacent words as the key in order to find the desired phrasal usage or word use in context. As for their decision making for revision on essay drafts after the query process, 91% of the queries were found to be correct cases as we compared their second and third drafts. The others were classified into incorrect cases (7% misinterpretation) and non-use (2% “defaulting to a safer option” in Park’s term, [22], p. 378). Non-use cases were those the participants consulted but did not apply into error correction.
We found 11 linguistic features of the consulted headwords the 14 participants relied on corpora for error correction. Among the 11 error types, verb formation (18%) was ranked on the top, followed by verb tense and word formation (both 14%), and then word choice (10%). They also looked up preposition, chunks, noun plural marking, and word choice, and self-desired verbs, nouns, or adjectives (see examples in Appendix Table 3). The 11 features showed what was feasible for corpora to help error corrections as far as these participants’ consultation literacy was concerned. Although error types corrected through corpora were diversified, they were slightly different from the 20 codes we used to mark most errors that occurred on students’ essay drafts [7, 11]. The participants tended to use their own knowledge (41.60% of all sources, average of those in the three assignments) to correct errors of subject–verb agreement, comma splice, article, plural marking, verb tense, or run-on types.
To answer the first research question, we found that the participants could reduce errors in their later essay drafts after they received the instructor’s written corrective feedback and attempted corrections using the three corpus tools and various knowledge sources or other tools, similar to precursor research [7]. They made use of various knowledge sources and tools to correct errors successfully with over 70% marked errors being reduced. Over 47% of errors were corrected with the assistance of the three corpus tools, and the frequency of use of corpus tools increased across the four essays in the semester with 91% of the consulted incidents successfully applied to correct errors. Additionally, TANGO was used the most frequently among the three corpus tools.
Students’ Perceptions on Corpus Tools Through the Questionnaire
Based on the results of means of items in the evaluation questionnaire (see Appendix Table 4), most students reported that they knew how to look up words through the corpus tools (mean = 4.4 out of the total, 5.0, in item 1) and DDL was helpful to them about learning word usage (3.9, item 2). Without surprise, while using corpus tools, they used a great variety of other tools for correction (4.1, item 14) such as online bilingual dictionaries or Google. If given more time, they were confident that they would become more skillful in correction and corpus consultation next semester (3.9, item 15). Most of them would continue to use corpus tools in order to keep improving their writing (3.8, item 11). Compared with dictionaries, corpus consultation better helped them correct errors (3.6, item 4) and made their sentences and vocabulary use more natural (3.6, item 5). With slightly reduced agreement, they reported their vocabulary use and grammar improved due to corpus use (3.5, item 10), which allowed them to memorize grammar and word usage (3.5, item 12). The rating of whether learner autonomy was encouraged through DDL was not high enough (3.4, item 7). The other items which were rated lower were those concerning corpus use with writing confidence or fun (items 8 and 13, both 3.3), serving as future e-reference tools (item 9, 3.4), or assisting writing development (item 3, 3.4). The coded errors from the instructor’s indirect feedback were regarded as helpful (item 6, 4.1), but the agreement scale on whether the practice with corpus consultation was enough was only 3.5 (item 16). The overall mean ranking was 3.67, slightly higher than 3.00 (which stood for “unsure”).
Item 17 and item 18 of the questionnaire probed more into the reasons for advantages and disadvantages of using corpora for error correction (cf., [24]). To show their reservations on CAWCF, the participants indicated that corpus consultation was time consuming, compared with using dictionaries. Sometimes, it was difficult to understand examples in the corpus, to find a proper key word as lookup headwords, or to use induced patterns to correct errors. These were also found in prior studies [7]. Some of them were simply not used to corpus tools. In spite of these challenges, two-thirds of them felt they had improved consultation skills later in the semester as they worked on DDL to correct errors across the three assignments. For benefits of corpus tools, the students indicated that they could see many more example sentences than those in a dictionary, the target sentence in real use, the context of target words, and more frequent example sentences. They could also search for and learn target sentences independently and started to think more about word class while making correction. DDL may serve as either affordances or constraints [32] in the WCF processing and its role depends on how learners can skillfully harness its learning potential into their revision needs. The answer to the second research question was affirmative (similar to [3, 7]) but with moderate satisfaction.
Two Cases
In an attempt to find possible explanation for the group findings, two students out of the 14 were examined in detail as case studies: Dan (a male student) and Vivian (a female one, both pseudonyms). Their writing performance was among the average in the class as observed by the instructor. We triangulated all of the two students’ texts plus their revision records, screen recording of corpus lookups, and interview and questionnaire data. In sharp contrast, Dan was a very limited corpus user for error correction in all the drafts across the semester (5 times for four assignments), while Vivian was a frequent user (21 times), based on their self-reported revision records and our video recordings. Dan preferred using the cell phone for necessary lookups; he seemed underengaged with self-correction using corpora to query but would choose to ask the instructor’s advice in class for immediate corrections. In private, he revealed in the interview that he tended to be impatient about details in writing accuracy or revisions. Dan complained that corpus consultation for error correction at the revision stage of writing was time consuming and he encountered Internet lag often with the concordance programs (other interviewees indicated they did not have problems with corpus use on their smartphones). Yet, for DDL benefits, Dan indicated he could see the target sentence in real use, the context of target words, and more frequent example sentences.
On the other hand, Vivian endorsed all listed seven DDL benefits elicited in item 18 of the questionnaire, and she looked up not only words concerning her own mistakes in the essay drafts but also additional words of her choice. She further looked up some other words so as to learn more. In one record whose draft did not show any marked errors (essay 3-draft 2), she looked up seven target words (imagination, relationship, doubt, carry, enemy, forcible, escape) and their VN or AN combinations. With such self-initiated queries, Vivian shows a clear sign of developing learner autonomy in adopting corpus consultation into the revision process and phrase learning for writing.
When we examined the video recording, Dan sometimes looked randomly at example sentences and ended up not using the search result, leaving the error to stay between drafts. Vivian consistently clicked on the VN, VNP, and VPN options (using TANGO) to look through example sentences on verbs such as draw or change in order to observe word behaviors in different contexts when the pattern was specified. Potentially, she learned various phrases through such opportunities being exposed to additional examples which she took up and induced phrasal patterns for writing. Vivian strongly agreed that she would take corpus tools as the frequent references for learning in the future but Dan did not agree. Their different trust and preference of corpus tools resulted in at least succinctly contrastive frequency of corpus use and various learner–corpus interactions as illustrated in the two examples below.
(Dan’s second draft, text marked in italicized format)
[past tense marker shown in the beginning of this paragraph]…heteach[VT, marked by the instructor]me many things, like almost all the things. For example, heteach[VT, instructor’s marking]me that if you will not do your best on one thing, then do not do it…. And he alsoteach me do not mess up with[VF]the one who bother you.
(Vivian’s second draft)
We walked through [PREP] a variety of stands.
Without fully understanding what VT meant (as confirmed from our interview), Dan used TOTALrecall to check teach. The concordancer produced the following example sentences:
But our teachers never taught us what we ought to buy to eat.
They have taught me how to look at the world, how to look at humanity.
To teach a man to fish is better than to give him a fish.
After viewing the examples, Dan decided to change his original sentence into:
He said “to teach a man to fish is better than [omit ‘to’] give him a fish”.
He did not successfully correct this mistake as Dan had limited awareness of the error nature and ignored the affordance provided by corpus consultation. Thus, it is less likely for him to incorporate feedback into subsequent revision [26].
(Vivian’s earlier draft)
We walked through [PREP] a variety of stands.
Vivian checked walk using TANGO and clicked on the VPN option. She found the first VPN collocation (1) ~along road, having 5 instances, and the 30th collocation (30) ~through forest having 1 instance. Then, she examined their instances. At the end, she decided to change her original verb phrase from walk through to walk along, making a successful correction. She told us that she figured out the difference between through and along and in her sentence, along is appropriate.
We found that Vivian demonstrated full trust in DDL benefits and deep engagement with corpus-aided feedback processing at both the affective (feeling positive about corpus use for error correction and learning) and behavioral (more lookups and higher correction frequency) levels (cf., [13]). As time went by, she developed more mature consultation skills to help with writing revision. Dan was perhaps underengaged with feedback processing in his corpus-aided revision process due possibly to his unfamiliarity with corpus tools, less consultation literacy, and not recognizing corpus benefits for improving writing inaccuracies. The intricate individual differences which caused different learner–corpus interactions, lookup frequency, and revision outcome warrant more future investigation. From our case studies, it seems that learner engagement with corpus-aided feedback processing plays a crucial role (like that in [13]) because it may originate from positive affective responses from the learners to corpus consultation as a powerful tool for error correction, and as cognitive extension [32]. Deep engagement with more hands-on corpus consultation led to successful corrections and reinforces continued practice and sophisticated skills in the process over the semester.
Conclusion and Implications
In this semester-long classroom-based study, concordance programs were used to resolve lexicogrammatical inaccuracies in writing. To respond to the three research questions, the students’ attempts for correction with the instructor’s indirect feedback indicated that errors in their writing decreased by more than 70% concerning their three writing assignments over time in spite of fluctuation. For the role of corpus consultation, learner concordancing seemed to have made the major contribution (47.43%, on top of all sources of help, including own knowledge and tools). The usage rates of corpus tools increased during the semester from 21 to 65% though they later dropped to 44.20% with an average of 47.43%. The group of EFL students indicated that DDL was acceptable (3.67 out of 5.00) to them as a usable e-referencing tool, compared with dictionaries, for consultation to make their essays more accurate (similar to [3]).
The fact confirmed that with the instructor’s indirect feedback of coded errors, the learners could learn to use concordancers to correct errors in their writing (91% successful rate) over time with the students’ moderate satisfaction about the process. Before the participants joined this project, their dominant reference tools were online bilingual dictionaries. Now, they were familiar with and could use concordancers to make error corrections at the post-writing revision stage, realizing corpus tools were among their useful reference resources at hand.
Although not every participant found DDL very useful for their error correction, findings of our two cases show that one found it a novel and useful learning method of writing more accurately like the deeply engaged Vivian in our case study. Fully appreciating the merits of concordancers, these learners actively found concordancers for help to improve inaccuracies in their essays. The motivated autonomous learners went even further to initiate queries of other target words when the words were not marked by the instructor as errors, a sign of self-directed learning. On the other hand, technology adoption is a long learning process and may require a new habit adaptation (in another case). Given longer time with integration of corpora into this year-long writing course which we plan to continue in its second semester, we look forward to the learners’ full mastery of corpus consultation literacy and adoption of corpora into their regular writing references.
Research Implications
Liu and Brown [20] criticized the lack of classroom ecology of precursor WCF research with one-shot treatment on timed writing. To ensure classroom ecological validity, the current study managed the implementation of multi-drafts in three different assignments (topics/genres) over one semester as the design fully integrated corpus consultation into the students’ revision stage in a writing course. We found moderate success coming from the group’s corpus consultation as Yoon [32] argues that including corpus use into L2 writing can serve both a cognitive extension and a distraction. In other words, corpus use does not solve every issue concerning writing revision or improving accuracy on student writing. Judicial use of various electronic tools, learners’ trust on tool usefulness, and their adequate consultation skills plus learner engagement with corpus-aided feedback uptake ensure long-term practice of DDL for writing revision, and may ultimately lead to writing development.
Without a strict control over corpus use, it is difficult to argue all positive benefits we found in the current project came solely from learner concordancing for their writing revision. However, our classroom-based design shows to L2 researchers and writing teachers a feasible design with longer term integration of corpus use as done in regular classrooms or curriculum with confirmatory and satisfactory findings (e.g., [7, 27, 28]).
Pedagogical Implications
Like traditional WCF practice in L2 writing classrooms, our learner concordancing experiences indicate careful pedagogical planning and instructional design are crucial to make corpus-aided feedback processing to take effect. Learners must have adequate consultation literacy to make good use of the potential of concordancers in order to rectify errors in their essays, which leads to accurate writing and successful learning. Although most participants appreciated the assistance the corpus tools offered to their revision process, unsuccessful attempts and non-use from queries, or even being unwilling or unable to make effective corpus consultation for error correction, were also noted in the processes of some learners. We suggest that the students should be given long enough training with several DDL illustration sessions integrated into a writing course in order to provide continuous scaffolding when necessary. In our study, one semester with several rounds of illustration or teacher modeling plus practice in the students’ own essays is crucial or at least very helpful. Corpus tutorials with grammar explanation using cases in specific writing context of errors in the students’ own text are very informative. Written records of revision, a form of reflection (“languaging” in Suzuki’s term, [26]), are shown to be effective, as the design raised students’ language awareness about how they could induce patterns from concordance lines and apply them to improve writing inaccuracies.
The study contributes to our further understanding of corpus-aided written corrective feedback (CAWCF) research when a group of EFL learners was observed over one semester concerning reduced errors in various assignments, frequency and purposes of corpus consultation, and their perceptions of corpus integration into CAWCF processing. To maintain curricular standards (set by particular writing programs) and integrate corpus use into weekly practice of an EFL undergraduate class is a significant strength in the study design as it keeps the classroom ecology [12, 20]. We believe the findings of our study have wider applicability to other writing classes and the written communication needs of the real world than other studies with much stricter control for corpus use alone (and excluding other references such as dictionaries) or for one single concordancer.
Limitations and Further Studies
We were unable to show a clear picture of exactly what error types could be rectified using corpora and what could not, or what causes resulted in particular error types the students used their own knowledge or other tools to correct, unlike findings in Tono et al. [28] or Crosthwaite [7]. First, we tried to make an ecological valid classroom-based study by allowing students to choose tools among various options. This is different from a rigorous experiment with strict control over corpus use on target error types. Second, we believe that learners’ own differential lexicogrammatical knowledge on different genres makes it extremely challenging to predict particular error types and provide appropriate corpus-aided instruction accordingly. In helping learners rectify errors with corpora, Park [22] indicates a limitation on the tools themselves because not any error types can be rectified via corpora. Our learners showed that they relied on corpora for particular lexicogrammatical error types, but used their own knowledge to correct errors of comma splice, run-on, or subject–verb agreement because corpora have little to offer in correcting those error types and it seems that they knew well when to use corpora or not.
The quantitative picture in our group patterns may show to other writing teachers what they can reasonably expect if they desire to incorporate learner concordancing into their students’ revision process in a semester-long course context. The two focal cases further illustrate the important role of motivation and learner engagement with corpus consultation in the process [9, 13]. Our next steps are to keep the CAWCF practice in a year-long project by documenting whether more learner satisfaction may emerge after more students are convinced with usefulness of corpus tools for revision purposes with increased consultation literacy, and how learner autonomy with corpus use develops when the observation period is extended.
References
Boulton, A., & Cobb, T. (2017). Corpus use in language learning: a meta-analysis. Language Learning, 67(2), 348–393.
British National Corpus, the version 3 (BNC XML Edition). (2007). Distributed by Oxford University Computing Services on behalf of the BNC Consortium. Retrieved on 24 February 2019 from http://www.natcorp.ox.ac.uk/.
Chambers, A., & O’Sullivan, I. (2004). Corpus consultation and advanced learners’ writing skills in French. ReCALL, 16(1), 158–172.
Chang, W. L., & Sun, Y. C. (2009). Scaffolding and web concordancers as support for language learning. Computer Assisted Language Learning, 22(4), 283–302.
Cobb, T., & Boulton, A. (2015). Classroom applications of corpus analysis. In D. Biber & R. Reppen (Eds.), Cambridge handbook of English corpus linguistics (pp. 478–497). Cambridge: Cambridge University Press.
Cotos, E., Link, S., & Huffman, S. (2017). Effects of DDL technology on genre learning. Language Learning & Technology, 21(3), 104–130. Retrieved on 24 February 2019 from https://scholarspace.manoa.hawaii.edu/bitstream/10125/44623/1/21_03_cotoslinkhuffman.pdf.
Crosthwaite, P. (2017). Retesting the limits of data-driven learning: feedback and error correction. Computer Assisted Language Learning, 30(6), 447–473.
Davies, M. (2008). The corpus of contemporary American English: 560 million words, 1990-present. Retrieved on 24 February 2019 from http://corpus.byu.edu/coca/.
Ferris, D. R., Liu, H., Sinha, A., & Senna, M. (2013). Written corrective feedback for individual L2 writers. Journal of Second Language Writing, 22(3), 307–329.
Gabrielatos, C. (2005). Corpora and language teaching: just a fling, or wedding bells? TESL-EJ, 8(4), A1, 1–37.
Gaskell, D., & Cobb, T. (2004). Can learners use concordance feedback for writing errors? System, 32(3), 301–319.
Guerrettaz, A. M., & Johnston, B. (2013). Materials in the classroom ecology. The Modern Language Journal, 97(3), 779–796.
Han, Y., & Hyland, F. (2015). Exploring learner engagement with written corrective feedback in a Chinese tertiary EFL classroom. Journal of Second Language Writing, 30, 31–44.
Johns, T. (1991). Should you be persuaded: two samples of data-driven learning materials. English Language Research Journal, 4, 1–16.
Kang, E. Y., & Han, Z. H. (2015). The efficacy of written corrective feedback in improving L2 written accuracy: a meta-analysis. The Modern Language Journal, 99(1), 1–18.
Lai, S. L., & Chen, H. J. H. (2015). Dictionaries vs. concordancers: actual practice of the two different tools in EFL writing. Computer Assisted Language Learning, 28, 341–363.
Lee, C. Y., & Liou, H. C. (2003). A study of using web concordancing for English vocabulary learning in a Taiwanese high school context. English Teaching & Learning, 27(3), 35–56.
Li, S. (2017). Using corpora to develop learners’ collocational competence. Language Learning & Technology, 21(3), 153–171. Retrieved on 24 February 2019 from http://www.lltjournal.org/item/3018.
Lin, M. H. (2016). Effects of corpus-aided language learning in the EFL grammar classroom: a case study of students’ learning attitudes and teachers’ perceptions in Taiwan. TESOL Quarterly, 50(4), 871–893.
Liu, Q., & Brown, D. (2015). Methodological synthesis of research on the effectiveness of corrective feedback in L2 writing. Journal of Second Language Writing, 30, 66–81.
O’Sullivan, Í., & Chambers, A. (2006). Learners’ writing skills in French: corpus consultation and learner evaluation. Journal of Second Language Writing, 15, 49–68.
Park, K. (2012). Learner-corpus interaction: a locus of microgenesis in corpus-assisted L2 writing. Applied Linguistics, 33(4), 361–385.
Park, K., & Kinginger, C. (2010). Writing/thinking in real time: digital video and corpus-query analysis. Language Learning & Technology, 14(3), 30–49. Retrieved on 24 February 2019 from http://www.lltjournal.org/item/2699.
Quinn, C. (2015). Training L2 writers to reference corpora as a self-correction tool. ELT Journal, 69(2), 165–177.
Sun, Y. C., & Wang, L. Y. (2003). Concordancers in the EFL classroom: Cognitive approaches and collocation difficulty. Computer Assisted Language Learning, 16(1), 83–94.
Suzuki, W. (2016). The effect of quality of written languaging on L2 learning. Writing & Pedagogy, 8(3), 461–482.
Todd, W. R. (2001). Induction from self-selected concordances and self-correction. System, 29, 91–102.
Tono, Y., Satake, Y., & Miura, A. (2014). The effects of using corpora on revision tasks in L2 writing with coded error feedback. ReCALL, 26(2), 147–162.
Wu, Y. J. (2015). Utilizing corpus resources companied by other consultation resources in enhancing collocation accuracy and collocation richness in L2 writing. Dissertation in University of California, Santa Barbara, March.
Yeh, Y., Liou, H. C., & Yu, Y. T. (2007). The influence of computerized feedback and bilingual concordancing on EFL students’ writing. English Teaching and Learning, 31(1), 117–160. Retrieved on 24 February 2019 from http://www.etl.url.tw/issue_detail.php?bgid=100.
Yoon, C. (2011). Concordancing in L2 writing class: an overview of research and issues. Journal of English for Academic Purpose, 10(3), 130–139.
Yoon, C. (2016). Concordancers and dictionaries as problem-solving tools for ESL academic writing. Language Learning & Technology, 20(1), 209–229. Retrieved on 24 February 2019 from http://www.lltjournal.org/item/2939.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
They all signed a consent form giving permission to use their essays and relevant data.
Appendices
Appendix 1
Appendix 2
Appendix 3
Rights and permissions
About this article
Cite this article
Liou, HC. Learner Concordancing for EFL College Writing Accuracy. English Teaching & Learning 43, 165–188 (2019). https://doi.org/10.1007/s42321-019-00022-5
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s42321-019-00022-5