EFL College Students’ Concordancing for Error Correction

Abstract

With the advent of technology, many have shown the benefits of data-driven learning (DDL), i.e., using corpus data to improve second language writing accuracy. Despite the increase in DDL studies, some gaps still exist. Previous studies tended to examine learners’ corpus use without looking into other correction methods (e.g., Google, dictionaries, or personal knowledge). Little has been documented regarding whether an error was really corrected by corpus concordancing or if it was based on students’ linguistic knowledge. Moreover, although prior research has indicated that shorter class periods would be enough for basic corpus training, how learners are involved in DDL for error correction over an extended period of time has rarely been documented. This study aims to bridge the gaps by examining not only students’ use of corpus tools, but also how they use other methods for error correction over an 18-week semester. Four corpus tools were introduced: Corpus of Contemporary American English, Just the Word, Netspeak, and Google (targeting the use of quotation marks “ ”). Adopting a mixed-method approach, data sources included students’ essay drafts and revision logs, retrospective interviews, as well as a questionnaire. The findings showed that the majority of the corrections were based on learner knowledge, while corpus-based corrections remained limited. Nevertheless, the students were able to draw on various reference resources including the four tools for error correction, and 70% of marked errors were corrected successfully. Qualitative analysis of the survey and interview data revealed that the participants perceived DDL for error correction as beneficial, although some seemed to have difficulties using these tools. Among the tools, Netspeak and Google with quotation marks were consulted more frequently because they provide easy access to search results with the frequency of the target word/phrase, while COCA appeared to be more complicated. This paper concludes with a discussion of pedagogical implications and limitations of the study.

摘要

隨著科技日新月異, 許多研究顯示使用語料驅動學習, 亦即運用語料檢索來輔助學生改錯。僅管這方面的研究有增長的趨勢, 針對臺灣大學生如何使用語料庫工具改錯的研究相當有限。過去研究多半著重在學生如何使用語料庫改錯, 而缺乏探討學生是否並用其他方法或工具進行改錯(像是:Google、字典、學習者知識)。少有研究記錄是否學生是透過語料檢索進行改錯或透過學習者之語言知識。再者, 雖然過去有研究指出基礎語料檢索訓練並不需要佔用太多課堂時數, 針對大學生在學期間如何使用語料庫工具改錯, 研究記載仍相當不足。為補足上述文獻缺口, 本研究旨在探討一班大學英語主修生如何在一學期的時間內使用語料庫工具及其他方法進行作文改錯。本研究訓練了四種語料庫檢索工具, 分別為:Corpus of Contemporary American English、Just the Word、Netspeak、以及帶有雙括號的Google (“ ”)。本研究採混合研究法, 資料來源涵蓋:每位學生的作文、訂正記錄、回顧式訪談、及問卷。研究發現大多數的訂正都是基於學習者的知識, 而基於語料庫工具的改正, 十分有限。僅管如此, 學生知道如何運用各種字詞檢索工具, 包括本研究介紹的四個工具, 成功訂正了70%老師所標示的錯誤。透過問卷及訪談的質性分析, 本研究也發現, 儘管有些學生認為這些工具不容易上手, 大多數的學生認為使用語料庫工具檢索對改錯是有助益的。在四種工具中, 學生使用Netspeak和帶有雙括號的Google較為頻繁, 因為這兩項工具簡單好用、方便查詢字詞的使用頻率; 相較之下, COCA顯得比較複雜。本文最後提供教學建議及針對研究限制進行相關討論。

FormalPara 關鍵詞

訂正性回饋、以英語為外國語寫作、字詞檢索、語料庫工具

Introduction

In many ESL or EFL university writing classrooms, most students who are at either the lower or intermediate level constantly make errors in their writing compared with those who have higher proficiency levels [2]. Students’ errors are often lexically or grammatically incorrect expressions, which can influence readers’ understanding of the texts. To solve this problem, written corrective feedback (WCF) is commonly provided, and it has been shown to be successful in terms of promoting written accuracy [10, 13, 21, 36]. When learners receive corrective feedback, they may have insufficient lexical or grammatical knowledge to correct errors and may therefore turn to reference resources (e.g., Google, bilingual dictionaries, concordancers) or smartphone applications for help. Over these years, direct corpus use or learner concordancing has been incorporated into L2 writing [39]. Substantial research has shown that concordancing can enhance data-driven learning (DDL) [20] which can promote not only inductive learning (through self-correction) but also deductive learning (through self-confirmation) [14].

In second language (L2) writing research, learners’ use of corpora or online reference tools has been examined for various learning purposes: collocations [26], prepositions [6], paraphrasing [5], thesis writing [9], and error correction [16]. Of these, an increasing number of studies have shown the positive effects of concordancing on error correction [17, 18, 27, 31, 34].

This study contributes to the growing literature on DDL for L2 corrective feedback by investigating the effect of concordancing on error correction and learners’ attitudes toward such an approach. The current research was conducted over an 18-week semester to gain insights into how a class of intermediate EFL undergraduate students utilized four corpus tools for error correction: Corpus of Contemporary American English (COCA), Just the Word, Netspeak, and Google with quotation marks. In this study, these four tools are referred to as “corpus tools”¹ since each of them is based on a certain corpus and provides authentic linguistic data or concordancing output for users to query, compare, and identify suitable answers for their correction. It is hoped that this study will shed light on corpus-aided written corrective feedback for developing L2 learners’ ability to correct their errors.

Literature Review

In this section, studies related to two areas will be discussed. First, issues concerning how corpus tools have been utilized for facilitating error correction will be reviewed. Next, research regarding how Google has been applied for training students to correct errors will be presented.

Using Corpora for Error Correction

Previous studies on concordancing for L2 written corrective feedback have examined the use of corpora by advanced students such as graduate students [8, 11, 12, 23, 40, 42] as well as by lower or intermediate level undergraduate students [16,17,18, 22, 34, 37]. These studies showed that with guided training, corpora may be beneficial as reference tools in terms of solving specific linguistic problems for error correction and writing revision. How learners treat errors and utilize corpus tools for self-correction has received increasing attention.

Since the present study focuses on how EFL college students use teacher feedback and concordancing for error correction in their essay drafts, the following sections review studies that have adopted corpora as reference tools for lower- or intermediate-level undergraduate students, and discuss issues that are related to using corpora for error correction in four aspects: (a) corpus tools and training, (b) number and types of errors, (c) effectiveness of error correction with corpus consultation, and (d) learner responses to corpus use.

Corpus Tools and Training

For lower level or intermediate learners, corpus interfaces need to be user-friendly and freely accessible so that students can interact with the corpus and exploit the corpus data after training. Tono et al. [38] provided training in the use of a corpus (IntelliText) for approximately 15 to 20 min. Gilmore [18] introduced two corpora (British National Corpora and the COBUILD Corpus) and offered a 90-min training session to Japanese EFL university students. Mueller and Jacobsen [30] examined Japanese EFL students’ use of the Corpus of Contemporary American English (COCA) and provided training sessions that also lasted 90 min. Similarly, Bridle [3] provided a 90-min session to train Chinese ESL students in how to employ BYU-BNC (British National Corpus) to correct their errors in four short essays over a 6-week course. Quinn [34] used the first five class sessions to familiarize students with Collins Wordbanks and another 10 sessions to teach them how to correct coded errors. Liou [27] introduced three corpora (i.e., COCA, TOTALRecall, and TANGO) and offered four to five training sessions with a focus on the basic functions of these corpora (30 to 40 min each).

To conclude, while some training sessions were rather short [38], others lasted longer due to course availability and the range of corpus tools provided [27, 34]. Three studies [3, 18, 30], however, gave 90-min training sessions, which seem to suggest that this training duration should be enough to help learners understand the basic functions of these corpora and how to examine the output during training. In these three studies, 90-min sessions were arranged for training one or two kinds of corpora. Aiming to offer four tools for training, the present research thus considered it necessary to double the training duration (i.e., 180 min at least).

Number and Types of Errors

The number and types of errors marked in students’ essay drafts were indicated in previous research. Regarding the number of errors, Tono et al. [38] and Satake [35] provided each student with only two pieces of coded error feedback, while Geiller [17] offered up to five errors. Liou [27] marked two to six errors and provided a coded error list consisting of 20 error types for students to correct in three different essay genres across multiple drafts. These studies highlighted around five errors due to some possible reasons: time taken for teachers to mark errors, limited time for students to conduct classroom DDL, and the possibility that novice learners might be overwhelmed when provided with many errors. In contrast, others marked more than five with a range of error types and encouraged lower-intermediate learners to correct errors using corpus data. For instance, Bridle [3] marked 10 errors per essay based on a list of 20 error types in students’ four essay drafts. Mueller and Jacobsen [30] offered 15 errors with three error types (inappropriate collocates, prepositions, and word choice) in student essays. In a 2-h lab session, Gaskell and Cobb [16] encouraged students to use corpus output to correct as many errors as possible, not just focusing on the number and type of errors marked by the teacher. Altogether, these studies showed that depending on specific purposes and considerations, a different number of errors might be provided.

In addition, the above studies showed that error marking systems vary greatly. The number of error types ranged from two to 20. Most of the researchers seemed to create their own error codes, with a few adopting a well-established coded-error list (e.g., [13]). Liou [27] was the first to adopt the error list developed by Ferris et al. [13]. Using a comprehensive coded error system is important because it allows researchers to compare findings across different learner groups. Such a list may also increase novice teachers’ willingness to employ DDL in their classrooms since creating a coded error list may be a daunting task. More studies are needed to examine how a systematic coded error list can be applied to facilitate learner concordancing.

Based on the suggestions of previous research [16, 27, 37], the current study aimed to provide 10 to 15 errors in students’ essay drafts so as to encourage learners to concordance for more corrections. Moreover, Ferris et al.’s [13] error list was adopted for marking students’ errors in this study.

Effectiveness of DDL for Error Correction

Research has shown that corpus consultation could improve L2 writers’ ability to self-correct. Tono et al. [38] found that omission and addition errors were easily identified and corrected, whereas misformation errors (i.e., errors where one grammatical form is used in place of another) were low in correction accuracy. Certain errors are more suitable for checking against corpus data than others. This study also showed that the higher level students utilized corpus tools more frequently than the lower level students because they seemed to understand the corpus data better. Mueller and Jacobsen [30] revealed that preposition and verb choice were successfully corrected with corpus consultation, while dictionaries were more beneficial for items of academic register. Luo and Liao [29] showed that a higher accuracy rate was found with the use of corpora for error correction than when using an online dictionary. The corpus group showed positive attitudes toward corpus use for error correction, but they also indicated challenges in analyzing corpus data. Larsen-Walker [25] found that L2 writers were able to use linking adverbials correctly after receiving training on using the Michigan Corpus of Upper Level Student Papers (MICUSP). Bridle [3] found that while corpus-aided corrections were quite successful, application of the corpus was limited. Corpus consultation was largely limited to errors involving choice of synonyms. Use of the corpus dropped from the first revision cycle to the fourth. Among the four rounds of revisions, students primarily relied on their own knowledge for error correction. Liou [27] showed that students’ errors across three essays seemed to decrease with their frequent use of corpus tools. The usage rates of corpora increased from 21 to 65% but later dropped to 44%. The dominant error type was verb tense, followed by noun plural marking and verb phrase formation. Taken together, these studies suggest that with appropriate training, L2 writers can learn how to correct errors through the use of corpora.

Learner Responses

Learner reaction to corpus use has been documented across the literature, including both positive and negative responses. Mueller and Jacobsen [30] revealed that participants generally found it difficult to utilize COCA because its interface was entirely in English, and it was hard to formulate a search query. In spite of this, some learners still regarded it as a useful tool. Luo and Liao [29] found that students showed positive attitudes toward corpus use for error correction, but they also identified challenges in analyzing corpus data. In Liou’s [27] study, most participants found DDL useful, and commented that, compared with dictionaries, consulting corpora could make their essay revisions more accurate. Bridle [3] found that students resisted using BYU-BNC because corpus consultation was time-consuming and many considered it difficult to analyze the concordances. However, some learners still saw the potential of using particular interface features. Bridle concluded that corpus use seems to be related to learner type and the nature of the error being corrected. Primary concerns included the time taken to search, the process of exploring the massive amount of corpus data, and the challenge of determining the correct answer relevant to a certain error.

Google-Driven Learning for Error Correction

While the above studies utilized corpora or concordancers for error correction training, some indicated that these tools might not be suitable for all students. To make DDL acceptable to a wider learner population, an increasing number of researchers have argued for the use of the web, i.e., Google as a corpus for L2 writers [33, 40]. For example, Conroy [7] reported that students preferred using Google searches for error correction. During training, sometimes the students could not readily find concordance examples in the corpus they were using (i.e., Brown Corpus), but when they were directed to a Google search they were often able to resolve the difficulty. Acar et al. [1] found that the use of the quotation marks (“ ”) search technique allows beginner ESL writers to easily notice and improve their grammatical errors or unnatural expressions by checking the number of results each search query generates. Han and Shin [19] taught Korean students how to use Google targeting the use of quotation marks (“ ”) and a wildcard (*) in a 4-day workshop. Geiller [17] trained 17 French L1 students to formulate Google queries and use a customized Google search engine to correct up to 10 “untreatable” errors occurring in two essays. These findings showed that Google can serve as an alternative to corpora and can assist L2 writers in various ways. Google techniques such as quotation marks (“ ”) and a wildcard (*) can allow students to refine their queries for more advanced results.

Summary and Research Questions

Based on the above discussions, some gaps have become clear. First, previous studies tended to look at learners’ corpus use without looking into other correction methods (e.g., Google, dictionaries, or personal knowledge). Little has been documented regarding whether an error was really corrected by corpus concordancing or if it was based on students’ linguistic knowledge. In two studies [3, 27], the participants were allowed to use their knowledge or other reference tools such as dictionaries for error correction in addition to the use of corpora introduced in the course. As Liou [27] stated, it would be considered against learners’ consultation habits if their use of these reference tools were excluded. Similar statements were noted by the students in Yoon and Hirevla’s (2004) study regarding the fact that dictionaries and corpora can sometimes complement each other, depending on the given task. Overall, the existing literature mainly reported on corpus use, rather than corpus use in comparison with other correction methods such as learner knowledge, dictionaries, or Google.

Moreover, although studies have indicated that shorter class periods would be enough for basic corpus training, how learners are involved in DDL for error correction over an extended period of time has rarely been documented. Among all the studies reviewed above, only two were conducted over a longer period of time, providing different kinds of coded errors across multiple draft revisions: 18 weeks in Liou [27] and 15 weeks in Quinn [34]. The other studies were conducted within a short period of time, possibly with a single treatment of feedback based on in-class timed writing tasks only [28].

Extending from prior research, it is crucial to examine not only students’ use of corpus tools, but also how they use other methods for error correction over an extended period of time. This study adopted four tools for training: Corpus of Contemporary American English, Just the Word, Netspeak, and Google. Considering Taiwanese university students’ common use of Google, the use of quotation marks in the Google search engine was included. Of note is that, relevant to the current research, Yoon [40] also adopted three corpus tools (COCA, Just the Word, and Google) in his study, but not Netspeak. However, Yoon conducted a cross-case study with the focus on two ESL graduate writers, while the present study looked at how a class of EFL undergraduates utilized corpus tools for error correction.

Three research questions guided this study:

• 1. Over an 18-week course, in what way do the college students correct teacher coded errors when four corpus tools are introduced into the essay revision process?

• 2. What kinds of error types are corrected using the four tools?

• 3. What are the students’ perceptions of concordancing for error correction?

Methodology

Settings and Participants

This study was conducted in a university located in southern Taiwan. An intact class of 23 students (one male and 22 females), majoring in English, was invited to participate. The students all signed the informed consent form and voluntarily participated in this study. In the English department where this study was conducted, students are required to take English composition courses for 2 years. This study took place in the first semester of their 2nd year English composition course in the 2018 Fall semester. At the beginning of the semester, the students were asked to fill out a background survey. The survey showed that in addition to four students who had never taken any proficiency tests, 15 students had taken the Test of English for International Communication (TOEIC; scores: 768 on average), and four had passed the intermediate and upper-intermediate level General English Proficiency Test (GEPT).² All the participants reported that they had never consulted corpus tools before, but relied on Google, Google Translate, or Chinese-English bilingual dictionaries for most searches.

Course Design

Because a process pedagogy was adopted in this course, the students needed to revise their essay drafts three times. After completing their first draft, they conducted peer review and revised based on peer feedback. Next, they met the instructor during a writing conference to obtain the instructor’s written feedback on global errors (content and organization) as well as on local errors (lexical and grammatical coded errors). Finally, the students needed to revise their drafts based on the instructor’s feedback and keep a revision log while correcting coded errors. To facilitate the students’ error correction, two 100-min tool training sessions were provided, as detailed in the Tool Training section.

Five Essays

Over a period of one semester, the students wrote five essays (one timed essay; four as major essay assignments). These consisted of three genres: an argument essay (timed essay), process essays (essays 1 and 2), and comparison-contrast essays (essays 3 and 4). An argumentative essay was chosen for the timed essay because this genre is widely used in tests such as the Test of English as a Foreign Language (TOEFL) or the International English Language Testing System (IELTS) writing. This essay served as a diagnostic essay produced to understand students’ entry level writing performance. Process essays as well as comparison-contrast essays were included due to the content sequence in the course textbook (Great Writing 5 by [15]). The topics of the five essays are listed below:

• Timed essay (argumentative essay)

  • Topic: Do you agree that children should begin to learn English as early as possible?

• Essay 1 (process)

  • Topic: Think of something you know how to build, create, or make. What materials are necessary? What is the process you need to follow?

• Essay 2 (process)

  • Topic: Think of a growing/hunting process of an animal/insect (e.g., how a caterpillar becomes a butterfly). Describe the steps involved in this process.

• Essay 3 (comparison/contrast)

  • Topic: Compare and contrast two places that are popular travel destinations.

• Essay 4 (comparison/contrast)

  • Topic: Compare and contrast two marriage customs/traditions from two different cultures.

One might doubt whether certain types of genres would generate certain errors. For example, argumentative essays may be more challenging to compose, and thus, learners might produce more errors in these genres. However, since most studies on concordancing for error correction do not specifically indicate types of genres adopted [16, 30, 34, 37] or why certain text types were used (e.g., argumentative essays in [29]; narrative, analysis, or comparison/contrast in Liu, 2019), it would be difficult to discuss the connection between the selection of genre types and DDL effectiveness. Thus, in this study, no particular assumptions were made regarding whether certain types of genres would prompt more corrective feedback.

In terms of length, the students were required to write 300 to 400 words for the timed essays, and more for the four essay assignments (400–500 words for essays 1 and 2 and 500–600 words for essays 3 and 4). On average, the actual number of words that the students produced across the five essays were as follows: 237 (timed essay), 486 (essay 1), 501 (essay 2), 593 (essay 3), and 628 (essay 4).

Error Codes

After the students submitted their timed essay, the instructor offered at least five instances of error feedback on their draft marked with Track Changes in Microsoft Word. A research assistant was invited to cross-check the error codes. Those that were considered questionable were either discussed to achieve consistency or were removed from the following analyses. Then, the students were guided to correct the teacher-coded errors in their timed essay and fill out the first revision log. A list of error codes developed by Ferris et al. [13] was distributed to the class (see Fig. 1). The instructor provided explanations of these error codes and made sure that the students understood what each code meant. When the students received the teacher-coded errors, they were asked if they had any questions. The students were told to use the four tools for correction. In their later revision logs, however, the students were allowed to consult other reference tools and were not limited to the four tools.

Fig. 1

Error codes used for marking in the second drafts [13]

After the students’ initial practice with concordancing for five errors in the timed essay, from essays 1 to 4, the instructor increased the number of coded errors to 10 to 15 in the second drafts of the students’ essays so as to encourage them to concordance for more corrections. This decision was made based on the suggestions of prior studies [16, 27, 37] as well as the students’ writing. When students produced longer essays, they seemed to make more errors and thus it was considered necessary to provide 10 to 15 coded errors in the four essays.

Based on the error codes in Fig. 1, the instructor used the “Insert a Comment” feature in Microsoft Word to indicate the error types the students made. In each comment, only one error code was provided (e.g., AGR). Most of the students were able to identify the types of errors marked in their essays by referring to the list in Fig. 1. For students to practice concordancing for different types of errors, the instructor attempted to identify a range of errors rather than marking the same type of error repeatedly in the students’ writing. In summary, the number and type of errors were marked deliberately based on the research design rather than the actual number or type of errors found in the students’ drafts.

For six types of error codes (i.e., COM, AP, SS, RO, CS, and FRAG), however, students were told to keep records of these changes in their essay drafts, but they did not have to record them in their revision logs. These six error types were excluded for three reasons. First, most studies on corpus-aided error correction tend to focus on lexico-grammatical errors and thus do not include these six error types (e.g., [3, 11, 16, 34, 37]). Second, since these error codes do not represent a specific word or phrase like the others listed above, it may be difficult to generate a query for the search. Third, research such as Liou [27] showed that students tend to use their knowledge to correct errors such as comma splice (CS) or run-on errors (RO). In Liou’s study, little was also mentioned regarding learner concordancing for errors such as COM, AP, SS, and CS. Thus, these six codes were excluded from analysis, leaving only 14 in this study.

Five Revision Logs

During their error corrections, the students were required to fill out revision logs based on the number of error codes the instructor provided. For instance, if five error codes were provided in the essay, the learner would need to correct these errors and keep a record of their corrections in the log. An Excel file was created by the instructor to collect students’ correction records: error types, original (errors), corrected (errors), tools used, and screenshots of search results. In terms of the tools used, if the correction was based on the students’ own knowledge or consultation with the instructor, they should also indicate that in their logs. The first revision log for the Timed essay was completed in a computer lab after the second tool training session. The other four revision logs for essays 1 to 4 were take-home assignments. Table 1 shows how a student, Sandra (pseudonym), used various resources including COCA and Google with quotation marks (“ ”) for correcting her marked errors in her timed essay.

Table 1 Excerpts from Sandra’s revision log for the timed essay

Corpus Tools

Four corpus tools were introduced: Corpus of Contemporary American English, Just the Word, Netspeak, and Google targeting the use of quotation marks (“ ”). These tools were chosen because their interfaces and functions were handy and user-friendly. More importantly, all of them are free and readily available online. Based on Yoon’s [40] study, the rationales for adopting these tools in this study were based on their differences in terms of corpus size and interface features. Specific reasons regarding why these concordancers were selected are explained as follows.

Just the Word³ (JTW) was employed because students could simply enter a word or a short phrase in the search box and analyze word combinations of the target words or phrases clustered based on different parts of speech (JTW now consists of about 80 million words of the British National Corpus) (see Fig. 2).

Fig. 2

A screenshot of finding the word combination of the verb “suggest” in JTW

The Corpus of Contemporary American English (COCA),⁴ consisting of 1 billion words, was introduced because the large amount of language data could allow learners to examine their lexical choices through the default interface functions such as List, Collocates, and KWIC (see Fig. 3). Of note is that for personal use, COCA often requires registration for continued use of the corpus; however, for institutional use, a paid academic license is required. Personal registration⁵ is free and recommended because it allows users to see a history of their past queries. For more updated information, please refer to the web link of COCA below.

Fig. 3

Using the KWIC function in COCA to find out what verb form typically occurs after the phrase “the key to”

Netspeak⁶ was chosen because it provides examples for students to type in searches at ease and observe the frequency of the occurrence of the target word or phrase. The learner can then click on the outputs, study the sentences in context, and identify the best result. Netspeak allows users to explore functions such as finding one word, finding two or more words, finding any number of words, or finding the best synonym (see Fig. 4). Currently, Netspeak comprises approximately 3.8 billion phrases up to a length of five words based on the “Web 1T 5-gram Version 1” corpus.⁷

Fig. 4

Search examples in Netspeak

In addition, a Google searching technique, quotation marks (“ ”), was introduced in one of the training sessions. This was adopted because it “help[s] ensure a search for the phrase as a whole string rather than individual lexical words in any form (inflected or not) in any order” ([19], p. 177). According to the researcher’s observation, most college students in Taiwan know how to use Google but do not know about searching with quotation marks (“ ”) on Google. By providing training in the use of this technique, it was hoped that the students could learn to compare the search results with the use of quotation marks and thus retrieve more relevant information. Figure 5 shows that the use of “raise questions” (4,330,000) in Google generated a large number of search results compared to “rise questions” (49,300). Students can then be guided to compare and analyze the outputs closely.

Fig. 5

Using Google with quotation marks (“ ”) to search “raise/rise questions”

Altogether, the four tools were included in this study to allow learners to cross-check the results when correcting their errors. These tools have varying features and functions that should be sufficient for the students to query, compare, and identify suitable answers for their correction.

Tool Training

A total of two 100-min tool training sessions were provided over a period of 2 weeks in a computer lab with Internet access. In session 1, the students were first instructed to practice JTW and COCA. In session 2, COCA, Netspeak, as well as Google with quotation marks were introduced. Since COCA is the most complicated of the tools, to help students become familiar with it, COCA practice was included in both of the training sessions. PowerPoint slides were created to demonstrate how to use these tools. The features and interfaces of each tool were first introduced. Different queries printed in a tool training worksheet (see Appendix 1) were then provided for students to practice consulting the tools to find out the answers. In this worksheet, some of the examples of errors were from students’ timed essays. The reason for doing so was to focus their attention on how corpus tools can be used to correct their errors. During the training sessions, the students were encouraged to discuss in pairs or groups if they were unsure about how to use the tools to explore the target word/phrase. After each training session, the students were assigned homework practice.

Data Collection and Analysis

Adopting a mixed-method approach, both quantitative and qualitative data were collected during an 18-week semester. Data sets included the following: (1) drafts of five essay writing assignments (with the focus on the 2nd drafts of each essay) per student, (2) five revision logs per student, (3) transcripts from a retrospective interview with each student, and (4) students’ responses on an end-of-course questionnaire. In addition to the essay drafts and revision logs mentioned earlier, a retrospective individual face-to-face interview was conducted with all 23 students near the end of the semester (see the interview questions in Appendix 2). Each interview lasted approximately 10 to 15 min and was conducted in Mandarin Chinese. All the interviews were audio-recorded. During the interviews, students’ revision logs were used to elicit more responses, and the participants were encouraged to comment on: (1) their perceptions and difficulties of using the four tools introduced in this course, and (2) how they selected tools for correcting certain types of errors. The end-of-course questionnaire written in English was designed to gather students’ feedback on the corpus-based writing instruction (6 items) with a 6-point Likert scale of agreement (see Table 7 for the questionnaire items presented in the “Findings and Discussion” section).

Table 2 Correction methods across five essays

To answer the first question, the above data sets were triangulated and the analyses focused on the participants’ use of the four tools (Netspeak, JTW, COCA, and Google with quotation marks) for error correction. With regard to the second question, students’ number of errors and error types in all of their second drafts across the five essays and their five revision logs were first calculated and categorized. In the revision logs, incomplete records were removed from the analysis. In addition, the six error types noted earlier (i.e., COM, AP, SS, RO, CS, and FRAG) were not included in the analyses. If the students recorded changes of these error types in the revision log, these records would be removed from the analysis. All the log data were coded by the author and a research assistant; any disagreement was resolved after discussion. The log data were used to compare the original errors in the second drafts with changes made in the final drafts. Regarding the correction methods in the log data, if a learner reported that he/she used more than one correction method (e.g., Google and a dictionary), each would be counted for analysis. Questionnaire data were analyzed through means and standard deviations. Interview data were analyzed with content analysis and then compared with the questionnaire responses to answer the third research question.

Findings and Discussion

This section begins with the presentation of the findings, followed by a discussion of the results. In response to the three research questions, the participants’ error correction results will be shown first, followed by error types corrected through the four tools. Lastly, students’ perceptions of tool-aided error correction will be reported.

Analyses of corrections across the five revision logs revealed that a total of 1137 errors were marked among 23 students’ second drafts across the five essays. This means that each student made 9.89 errors per essay on average. Among the 1137 errors, 77% of them (n = 873) were corrected by the students. From our analyses, the majority of corrections were successful (accurate corrections: 789, accounting for 90.4%; inaccurate corrections: 84, accounting for 9.6%). With the total number of corrected errors (n = 873) divided by the number of essays and the number of students, each student corrected about 7.59 marked errors per essay indicating the effort the participants could make to correct errors in each essay. The average accurate correction rate was 74.7% in the timed essay, 76.3% in essay 1, 65.7% in essay 2, 65.2% in essay 3, and 65.9% in essay 4 (with an average of around 70% for the five essays). The accurate correction rates dropped when students produced longer texts for essays 3 and 4 (on average 593 and 628 words respectively) compared to essays 1 and 2 (486 and 501 words on average). The decrease can also be related to the increase of the marked errors (the total number of marked errors in the five essays: timed essay: 229; essay 1: 211; essay 2: 175; essay 3: 264; essay 4: 258). In my observation, some students did not correct all the marked errors in the last two logs (i.e., seven students in essay 3; eight students in essay 4). It is possible that they might have lost patience with error correction, resulting in a decrease in accuracy.

In What Way Do the Students Correct Errors When the Four Tools Are Introduced?

This section will address the correction methods the participants used for error correction including the four tools, knowledge sources, dictionaries, and online reference tools. Among the 873 errors corrected by the students, we found that in the Timed Essay, the frequency distributions for each correction method (from high to low) were: corpus/concordancing tools (44.6%), the participants’ own knowledge (28.2%), dictionaries (20.3%), others (6.2%), and asking the teacher or peers (0.6%). The tools were most frequently used in the timed essay (44.6%), followed by essay 1 (40.8%), indicating the participants’ attempts to consult these tools after training. However, in essays 2, 3, and 4, an obvious decrease in corpus use occurred, as shown in Table 2. The average of corpus corrections across the five essays was 26%, which was lower than knowledge corrections (54.3%).

Table 3 Use of the four tools and dictionaries across 5 essays

The total number of corrections based on the four tools was 210 queries (divided by 23 students giving an average of 9.1 times per person). As shown in Table 3, with the four tools and dictionaries amounting to 100%, the most frequently used was dictionaries (37% on average), followed by Netspeak (25%), Google with quotation marks (23%), COCA (9%), and then JTW (5%). On average, the number of times students used the four tools and dictionaries across each essay was as follows: dictionaries (22 times), Netspeak (16 times), Google with quotation marks (13 times), COCA (8 times), and JTW (5 times). Interview data also confirmed that among the four tools, Netspeak, and Google with quotation marks were consulted more frequently because they provide easy access to search results with frequency of the target word/phrase.

Table 4 Frequency of consulting the four tools

Based on the students’ consultation records in all the log data, further content analysis revealed that the majority of students (n = 14) consulted the four tools fewer than 10 times across the five essays, while three used these tools over 10 times and one more than 20 times. To our surprise, three students never utilized any tools to correct their errors. In contrast, two students exploited these tools over 30 times. See Table 5 for details. Findings regarding these individual differences will be discussed under research question three.

Table 5 Distribution of error types across 5 essays (accurate corpus use)

While the above revealed the overall correction results, the following examines the effectiveness of the corpus-aided corrections by taking a closer look at accurate corrections (n = 789). Figure 6 shows that the use of all the reference tools reduced (including the four tools and dictionaries) while knowledge-based corrections increased from the first to the last essay assignment (from 25.1 to 71.8%), consistent with the results in Table 3.

Fig. 6

Accurate corrections (by correction methods)

Moreover, accurate corpus corrections seemed to exceed those made with dictionary-based corrections over the five essays (although only slightly higher in the last three essays). Altogether, these findings might mean that the students attempted to consult the four tools for the first few revisions and learned how to correct errors with corpus concordancing. Because of that, when they encountered similar errors again, they were able to self-correct the marked errors. It could also indicate that as the students gradually became aware of the marked errors, they were able to correct their errors successfully based on their linguistic knowledge, and thus their reliance on reference resources (i.e., corpus tools or dictionaries) would reduce. This finding is consistent with interview data reported by 20 students regarding their decreased corpus use due to their ability to make corrections on their own. For example, six of the 20 students (S5, S8, S9, S16, S20, and S21) noted that most of their errors lacked either the noun plural marker “s” or the third person singular verb ending “s.” Oftentimes when they encountered these errors, they just added “s” to the end of the word without turning to any reference tool for help, which was also reported in Liou’s [27] study.

To answer the first research question, the majority of the accurate corrections were based on learner knowledge, while corpus corrections were limited and reduced over time, consistent with Bridle’s [3] finding. However, this result was distinct from Liou’s [27] study showing learners’ increased corpus use in later essay revisions. This means that, while the design of this study is closer to Liou’s (i.e., learner backgrounds, coded error list adopted, and number of corpus tools provided), the result was more in line with Bridle’s research. This might be related to differences between learner preferences as well as the types of corpus tools. First, concerning learner preferences, Bridle investigated corpus use and learner types, and found that some learners (“reflectors”) might consider exploring the concordances too much work, while others (“pragmatists”) tended to make more use of the corpus. Although the current study and Liou’s research did not look into learner types and corpus use, it is possible that these learner differences may influence the results. Second, regarding the types of corpus tools, since Liou introduced two Chinese-English concordancer programs in her study, these might encourage her participants to exploit the corpus data over the entire semester in comparison to the English monolingual corpora introduced in Bridle’s (BNC) and the current study (JTW, Netspeak, COCA). In spite of these differences, this finding can be added to the literature by showing that when learners were allowed to use various correction methods, rather than being limited to only corpus tools, they tended to draw on a range of reference resources for error correction. In this study, with the use of numerous resources including the four tools, 70% of marked errors were corrected successfully. As Liou [27] noted, “technology adoption is a long learning process and may require a new habit adaptation” (p. 182). Learners’ use of corpora takes time and it may require additional classroom training to improve students’ corpus consultation.

What Kinds of Error Types Are Corrected Through the Four Tools?

In this section, results based on successful corpus corrections will be presented first, followed by unsuccessful corpus corrections. The total number of corrections based on the four tools was 210 queries, including 193 accurate corrections (92%) and 17 inaccurate corrections (8.1%), meaning that the majority of the corpus corrections were successful. Overall, we found that across the five essays, the learners used the four tools to correct 13 error types, but not one type (REF; pronoun reference vague or unclear). Among the 13 error types, verb formation was most frequently corrected with the four tools, followed by preposition, word choice, plural marking, and word form (see Table 6). This shows that corpus consultation is especially feasible when it is used for correcting these errors.

Table 6 Corpus-based error corrections

The reason that no student consulted the four tools to correct pronoun reference (REF) errors was that even if the learners made these errors, they tended to self-correct based on their knowledge rather than seeking help from any online reference tools. According to seven students who relied on their knowledge for self-corrections of REF errors, they indicated that they knew how to correct these errors and did not consider it necessary to consult any tools online. Table 7 shows successful examples of the 13 error types from students’ revision logs.

Table 7 End-of-course questionnaire¹

Figure 7 presents the frequency of the 13 error types corrected using the four tools. Differences were found regarding corpus selection for correcting particular error types. For instance, the students tended to use Google with quotation marks (“ ”) to correct errors such as verb form, word choice, and plural marker, similar to the interview results (n = 3). When correcting errors such as prepositions and word formation, they would choose JTW (consistent with the interviews; n = 5). Overall, the four error types (i.e., verb formation, preposition, word choice, plural marker) were frequently corrected through either Netspeak (also indicated in the interviews; n = 7) or Google with quotation marks. This implies that of the four tools, the students were more capable of utilizing these two concordancers for error correction. Based on this, it is suggested that L2 writing instructors consider employing these two corpus tools for training.

Fig. 7

Accurate corpus corrections: error types

While the above showed accurate corrections through corpus use, the following presents results of inaccurate corpus corrections. Among the 17 unsuccessful corpus-based corrections (out of a total of 210), four were “word choice” errors, whereas the others fell within diversified error types (i.e., articles, missing words, plural markers, prepositions, verb forms, verb tenses, wrong forms, and wrong words; each type had only one or two errors). The examples below show Taiwanese students’ typical “word choice” errors related to the word, “contact,” which should be replaced with “learn” in the following contexts:

• Taiwan’s children may only contact [learn; WC] Chinese when they were little. (S19; revision log for Timed Essay; COCA)

• A situation without testing lets children feel relaxed when they contact to [learn; WC] English. (S23; revision log for Timed Essay; COCA)

These unsuccessful corrections were mainly caused by L1 transfer. The phrase “jiē-chù” (接觸) in Chinese is often directly translated as “contact” in English, which resulted in the participants’ lack of awareness of how it should be used in the context of English. Even though the students learned how to use COCA to search for the word, “contact,” they were not able to find relevant results to correct such errors. In this example, the correction could be successful if noticing and guidance were provided. For future practice, the writing teacher can first inform the student about the error, and then guide the learner to explore the search results through the functions of COLLOCATES and LIST in COCA (see Fig. 8 for screenshots). Using the COLLOCATES function, the learner can try to input the word, “language” in the Word/Phrase box, and then select “verbs.all” in the Collocates box to find out what verbs frequently occur before “language.” Also, the learner can try to input the word, “contact” as a verb through the LIST function to verify whether “a language”can be used as an object when “contact” is used as a verb.

Fig. 8

The screenshots show the search boxes and search results of two queries (“language” and “contact”) performed on COCA

To answer the second question, the current study revealed that the students consulted the four tools to correct 13 types of errors (see Table 7), including verb formation, preposition, word choice, plural marking, as well as word form. This finding is similar to that of Liou [27] because she also used the same error code list (consisting of 20 error types) developed by Ferris et al. [13]. In her study, Liou [27] found that the participants used corpora to correct 11 error features, close to the 13 types found in this research. This finding can be added to the existing literature by indicating the availability of such a comprehensive list that can be utilized by novice teachers for DDL training with the focus on certain error types. The error types found in this study also corroborate Mueller and Jacobsen’s [30] finding regarding preposition and verb choice being able to be successfully corrected with corpus concordancing. These show that when students were provided with a range of correction methods, only certain types of errors seemed to promote corpus consultation [3, 4, 16, 27, 31, 38]. Of note is that in this study we also found that errors influenced by first language (L1) transfer can be included for more effective practice. The training can focus on how to evaluate and identify the relevant results from the corpus data in order to correct the error successfully.

Regarding the number of error codes marked in students’ essay drafts, this study provided approximately 10 to 15 error codes which might be too many compared to those marked in some previous studies [16, 27, 37]. While some researchers indicated that a wide range of error types may encourage learners to correct their errors with corpus data [4, 18, 30, 31], the question of whether a great number of error codes may discourage the participants from using corpus tools or concordancers for error correction remains. In this study, the analysis of revision logs revealed that some students did not complete their corrections in the last two logs (seven students in essay 3; eight students in essay 4). Thus, it is suggested that for novice corpus learners, three to five errors may be considered more feasible for corpus consultation practice.

What Are the Students’ Perceptions of Concordancing for Error Correction?

Learner perceptions of DDL for error correction were analyzed from the questionnaire and the interviews. Concerning the use of these tools, many seemed to agree that the tool training held at the beginning of the semester was enough for them to learn how to use the tools (item 1,m = 4.3) and using these tools helped them use English more naturally when writing essays (item 6, mean = 4.5) (see Table 2 for the questionnaire results). These responses seemed slightly positive (based on a 6-point Likert scale of agreement). However, in the interviews, nine students reported that corpus consultation was more complicated, time-consuming, and difficult to get used to compared with dictionaries:

• Student 4: “I didn’t know COCA and Netspeak before. I got mad when I used them because I’m not used to them.”

• Student 6: “Cambridge Dictionary is much easier than COCA.”

Eight participants reported that the interface of COCA was not easy to use and the massive amount of corpus data was distracting. They noted that it was challenging to observe the patterns from the examples in the corpus and to extract relevant information for error correction:

• Student 5: “I felt that the process of using COCA was quite complicated. Although there are many sentences, the colors make you dizzy and are distracting.”

• Student 6: “COCA is complex. It’s hard to know the answer. I have to read several sentences to figure out what I want.”

In contrast, five students commented on the interfaces of the other concordancers. For example, two students (S9, S12) indicated that Netspeak provides examples which demonstrate how users can utilize this tool to query phrases. Another student (S18) noted that the interface of JTW is similar to Netspeak which is also quite simple. See the excerpts of their comments:

• Student 9: “Netspeak is quite user-friendly. It gives you examples underneath the search box. If you haven’t used it for a long time, it tells you what to do [what you should type in the search box].”

• Student 18: “The interface of Netspeak is easier. I can find anything on it. I also use JTW. It’s similar to Netspeak. Both of them are simple. Sometimes I use Netspeak and Google with quotation marks interchangeably as well.”

Seven students seemed to benefit from Netspeak, while one student commented that the frequency shown on the search output is not always consistent with the sentences:

• Student 21: “I don’t really like Netspeak. Sometimes I clicked on the frequency, but there was no sentence in it. I can’t tell whether the frequency was based on the sentences.”

Regardless of these challenges, more than half of the students (n = 15) mentioned that they were able to use the four tools to correct errors across the five assignments. This also corroborates the responses in the questionnaire (item 3, Mean = 4.7). Some felt that the example sentences in COCA are more native-like than those in Google, and they could use COCA to find synonyms and collocations:

• Student 4: “If I want to find synonyms, I’d go to COCA. COCA has a lot of example sentences which are more native-like. Compared with Google, I tend to believe in the sentences from COCA because some examples in Google look strange.”

• Student 5: “It’s easier to find collocations in COCA. One of its special features is Collocates. There are many examples in it and it’s clear to understand the examples.”

• Student 20: “There are colors when I use it to find synonyms. It makes it easier.”

Nine students who preferred Netspeak and JTW also commented that these resources were easy to use and convenient to observe the frequency of the target words:

• Student 12: “There are instructions in Netspeak and they are pretty useful [for performing searches].”

• Student 16: “I often use these two tools because they are convenient.”

Three students who frequently used Google with quotation marks indicated its benefits:

• Student 18: “Compared with searching on Google, Google with quotation marks gives you more precise results because the words shown in the output are connected together [not separated like those presented in Google].”

In addition, two participants indicated that when they looked up prepositions, they would consult these two tools:

• Student 6: “When I look for prepositions, I would use Netspeak and JTW. I can then see the example sentences and frequency percentage.”

• Student 15: “For instance, when I have no idea about a certain preposition, I’d type a question mark and the target word. Several prepositions will pop up. I use Netspeak frequently for finding prepositions.”

Six students reported that they not only consulted the corpus tools in the revision stage but also in the drafting stage near the end of the semester. This implies that when students became familiar with these tools, they could apply corpus consultation skills from the revision stage to the drafting stage.

Individual differences were also found in terms of learners’ corpus use. We noticed that three students (S3, S11, S13) never consulted the tools throughout the essays, and they indicated that since they were able to self-correct based on their own knowledge, they seldom turned to other correction methods for help. In particular, one student (S3) noted her unfamiliarity with these tools which thus limited her further use of them: “I don’t use Netspeak, JTW, or COCA. I’m not familiar with them. I feel they are slower.” In contrast, two students (S9, S18) who exploited Netspeak over 30 times mentioned their preference for this concordancer and were able to find answers to correct a range of error types.

In answer to research question three, this study found both obstacles and opportunities while introducing corpus tools for students’ error correction practice. The majority of the students reported similar issues that were discussed in the literature: the time required to learn to use a corpus tool and to perform searches [27, 34, 41, 42], as well as the challenges of analyzing concordance lines and identifying a search result relevant to a particular error ([18]; Mueller & Jacobson, 2015; [31]). In this study, many students indicated that, compared with COCA, the other three were easier to use in terms of their simple design and presentation of search results. The abundant information presented in COCA made it difficult to observe the patterns from the examples and extract relevant information for error correction, while the other three tools are easy to use, prompting learners’ further use for error correction.

In this study, more than half of the students showed their preferences for corpus utilization and they found corpus-aided error correction beneficial. They saw the value of particular interface features over others, especially when they needed to look for synonyms, collocations, prepositions, and certain word classes while making corrections. This finding echoes those of other studies [3, 8], indicating that this area may deserve more attention while giving students corpus training. Finally, the students commented that the error codes helped them raise their awareness of the errors and determine appropriate tools for consultation. Similar results were discussed in prior research [30, 34]. This was also indicated in Chambers and O’Sullivan’s study (2004) regarding the fact that the errors needed to be highlighted; otherwise, it would be difficult to improve from error correction.

Conclusions and Implications

This study examined how 23 Taiwanese EFL undergraduate students utilized four tools, Netspeak, Just the Word, COCA, and Google (targeting the use of quotation marks), for correcting teacher-coded errors over an 18-week course. The findings showed that across the five revision logs, the students seemed to draw on their own knowledge for most of the error correction, while corpus consultation remained limited. Overall, most students reported that error feedback was beneficial in terms of noticing and concordancing for correction. Before they participated in this study, many only consulted online reference tools such as dictionaries, Google, or Google Translate. At the end of the semester, over half were familiar with the four tools and were able to use them interchangeably with other reference resources. Even though not every student found the four tools useful due to technical issues or unfamiliarity with these resources, some were motivated to use these tools not only for error correction but also during their drafting stage. We learned that the more user-friendly the corpus tools are, the more likely it is that EFL students will continue to use them. For instance, in this study, Netspeak and Google with quotation marks were consulted more frequently because they provide easy access to search results with frequency of the target word/phrase, while COCA appeared to be more complicated. We also found that if students treat error correction merely as a process to finish the work, they are less likely to consult reference tools and will continue to make numerous errors [27]. Some learners, however, considered the four tools as useful additional resources and were willing to engage in frequent corpus consultation.

Based on these findings, this study recommends that writing educators should consider introducing online corpus tools or concordancers to L2 students in order to encourage self-correction of errors and reduce teachers’ workload in the revision stage. For lower or intermediate level learners, three tools adopted in this study are highly suggested because of their simple and user-friendly interface designs: Just the Word, Netspeak, and Google (with the use of quotation marks “ ”). This study further suggests that when introducing corpus tools to EFL students for revision, it is crucial to pay attention to certain types of errors (e.g., verb formation, prepositions, word choice, word form, and plural marking) since they seem to prompt corpus consultation and thus more focused training on these aspects can be provided in L2 writing classrooms.

Two limitations of this study that might prompt future investigation should be indicated. First, the current research was only able to examine what students corrected and to what extent they chose to use corpus tools for error correction. Because the computer labs at the university under investigation did not allow users to download any screen capture software for practice after several trials, it would be of interest if this method could be included to obtain insights into “how” students go through the correction process [24, 32].

Moreover, technical issues were found when using JTW and COCA. Some students in this study mentioned that sometimes they could not enter JTW although log-in was not required. Others experienced difficulties using COCA because of its log-in failures and the wait time for results. Similar issues were also reported in Feng [12]. Of note is that COCA can be locked because institutional use often requires a paid license. To avoid this situation, teachers should be aware of this constraint so that they can plan accordingly.

Appendices

Appendix 1. Tool training worksheet

Appendix 2. Interview questions

• 1. Do you still remember what corpus tools were introduced in this course?

• 2. Among these tools, which one(s) do you use frequently? Which one(s) do you seldom use? Why? How often do you use the tool(s) you just mentioned?

• 3. What kinds of difficulties did you encounter when you use these tools? What did you do to solve the problems?

• 4. During the process when you correct errors marked in your second draft and keep records in your revision log, do you use the tools you just mentioned? If yes, Why? If no, Why not?

• 5. When you correct certain types of errors, do you turn to certain tools for help?

• 6. Which of the following stage do you consult online resources or corpus tools most frequently: Draft 1, Draft 2, or Draft 3? Why and why not?

• 7. Some students said that they tend to rely on their own knowledge for correcting errors in Draft 2. What about you? How do you usually correct errors in your draft 2? Could you talk about this process?

• 8. To what extent do you think you are familiar with these tools after tool training? Do you think the tool training was enough to help you understand how to use these corpora?

• 9. To what extent do you think the corpus tools introduced in this course benefit English writing?

• 10. Will you continue to use these corpus tools in the future?