Introduction

Metacognition has been recognized to play an important role in the reading process [2, 31] as it involves the planning, monitoring, and evaluating necessary to proficient reading [11]. Cohen [17] and Sheorey and Mokhtari [33] have shown that both skilled L1 and L2 readers are more metacognitively aware; in other words, while reading, they are capable of monitoring and reflecting on the cognitive process [6, 22], which leads to better reading comprehension. A metacognitive strategy is understood to be “a strategy in the cognitive dimension that helps the learner control (through planning, organizing, evaluating, etc.) his or her cognitive strategy use” ([27], p. 289). These metacognitive strategies are positively correlated to reading ability [10, 28, 33, 35]. Quite often, reading ability is evaluated through reading tests. Therefore, many studies have been done in order to understand the relationship between metacognitive strategies and L2 reading tests, in both the ESL [18, 21] and EFL contexts [5, 16, 29, 42, 48].

Metacognitive Strategy Use in Reading Tests

One major focus in this field is the study of the relationship between the number of metacognitive strategies used and L2 reading test performance. For example, in a study that focused on students’ strategy use in testing and non-testing situations, Chou [16] found that EFL learners tended to use more metacognitive strategies than cognitive strategies in testing contexts. Another study by Phakiti [29] examined the relationship between metacognitive strategy use and EFL learners’ reading test achievement. He found that metacognitive strategies are more frequently used by successful test-takers than their moderately successful and unsuccessful counterparts. Zhang and Seepho [47] further specified that EFL learners with higher reading test scores used twice as many metacognitive strategies as their peers having lower scores. In an attempt to test models hypothesized by structural equation modeling (SEM), Zhang and her colleagues examined the underlying structure of metacognitive strategies and test-takers’ reading test performance in test contexts [43,44,45, 49]. Their studies confirmed that for EFL learners, in the face of reading tests, “metacognitive strategies are the core of strategic competence and test takers’ focal attributes” ([43], p. 121).

However, metacognitive strategies that are used in completing a reading test differ between successful and less successful test-takers in terms of both quantity and quality. Purpura’s [32] structural equation modeling showed that low performers relied heavily on “metacognitive processing in retrieving information from long-term memory, an indication of lack of automaticity” (p. 176); on the other hand, high performers used metacognitive strategies to help them understand and remember. Drawing on O’Malley and Chamot’s [25] taxonomy, Zhang and Seepho [47] explicated the differences in metacognitive strategy use among high- and low-scorers of reading tests. They found that low-scorers tended to avoid certain strategies such as advance organizers and organizational planning, whereas students with higher reading scores used more monitoring strategies than their lower counterparts. Such qualitative differences were further supported by Lin and Yu [23], who concluded that it is the types and effectiveness of metacognitive strategies that is associated with L2 reading proficiency.

Although the crucial role of metacognitive strategies in successfully completing a reading test has been acknowledged, only a few of the studies were conducted in Taiwan where a high-stakes English proficiency test is of concern to many students [20, 23, 39]. Moreover, in the face of high-stakes reading tests, strategy instruction is recommended in the test preparation stage [19]. Cohen [17] also noted the importance of strategy instruction for language learners as it “…can enhance the students’ efforts to reach their own L2 goals…because it encourages them to find their own means to success” (p. 116). That is to say, for learners with a clear goal of passing a high-stakes English test, strategy instruction becomes crucial. Before any strategy instruction is implemented, understanding students’ metacognitive strategy use in reading tests, especially the differences between high reading scorers and low reading scorers in a testing context, is needed to help teachers devise a more suitable plan for instruction.

Method

The Study

This study investigated EFL university students’ use of metacognitive strategies to complete a TOEIC reading test, specifically focusing on the differences between learners with high and low reading scores. Differing from previous studies that mainly employed interviews as a data collection method accompanied by a reading test, this study uses think-aloud protocol. The findings showed the degree of effectiveness and flexibility of certain metacognitive strategies as used by high-scorers and low-scorers.

Context

Most universities in Taiwan offer compulsory English courses in their first or second year of study, and require students to meet an English requirement before graduation. This policy has been encouraged by Taiwan’s Ministry of Education [24], which may explain the popularity of standardized English proficiency tests, such as the one used in this study, the Test of English for International Communication (hereafter called TOEIC). Under the pressure of obtaining the required passing score, students in Taiwan may enroll in some courses aimed at helping them to prepare for the test.

Students in the university where this study was conducted were required to obtain a certain TOEIC score set by individual academic departments, ranging from 400 to 600. This study took place in an elective English course designed to help students prepare for standardized English proficiency tests; passing the TOEIC was the aim for most of the students. There were a total of 65 students from different majors enrolled in this 18-week course, and the majority of them were junior and senior students, most of whom were concerned with obtaining an English proficiency certificate for better job prospects. The class met 2 hours a week and required additional time to do the weekly exercises assigned as homework. Data collection began in week nine when midterm exams had just begun.

Participants

Four participants were selected from the 65 students enrolled in the course, among whom two were categorized as low-scorers (hereafter LS) and two as high-scorers (hereafter HS) based on their reported TOEIC reading scores from the previous 2 years. Apart from one female LS (a third year student), all the others were in their final year in university (one male LS and two female HS). At the time of data collection, their ages ranged from 20 to 22. The cut-off score between low and high proficiency in the reading section is 275, above which is equal to the intermediate level (B1: from 275 to 384) in the Common European Framework of Reference for Languages (CEFR) and below which is believed to be elementary level (A2: from 115 to 274). Participants’ demographic information and test scores are listed in Table 1:

Table 1 Participant Information
Full size table

Data Collection

The data collection methods used in this study included written records, think-aloud protocol, and semi-structured interviews.

Right after their midterm exam in the spring semester (for which a mock, short-version TOEIC test was administered), all the students were asked to write down how they dealt with the last reading passage. This written record of their strategy use served as a reference for the data collection that followed.

Around 2 weeks after the midterm exam, selected participants were invited for a think-aloud session individually, each lasting 35 to 55 minutes. Think-aloud data is believed to be able to reveal the actual thought processes more precisely than the other two types of verbal reports (self-observation and self-report) due to its introspective nature [18]. Before the think-aloud protocols, orienting the participants to retrieve quality verbal reports is considered an important step [17]. Therefore, each participant was given a brief explanation and a short warm-up practice where they were told to fetch an object near them and verbalize their thoughts while doing so. Participants were then asked to imagine themselves being in a testing situation while thinking aloud, with no time constraints imposed on them. This was because in a real TOEIC testing situation, test-takers are given a fixed amount of time for the entire reading section, but they have to manage their time and speed for each reading passage. Each participant was given the first short reading passage and asked to think aloud. During the course of the think-aloud session, whenever he or she stopped verbalizing, the researcher would remind him or her to keep talking. The think-aloud processes were video-taped so that they could serve as a source for stimulated recall and for the follow-up interviews. When the think-aloud session of the first passage was completed, the researchers would proceed to the stimulated recall stage by playing back the recording and having the participant watch it together with the researchers. Whenever there were unclear parts or longer pauses in the recording, the researchers would stop the recording and ask for clarification. The same procedure was repeated for the second passage. Furthermore, in order to access readers’ metacognitive processes employed during the reading tests, participants in this study were asked to verbalize their thoughts in their most convenient language, Mandarin Chinese.

Around 1 week after the think-aloud protocols, follow-up semi-structured interviews were administered for two purposes: first, for the researchers to elicit more information about the participants’ strategy use in a testing situation, such as how or under what circumstances they would use the strategy mentioned during the think-aloud session; second, for the researchers to clarify unclear parts that occurred during thinking aloud or stimulated recall. The interview guide in Appendix A served as a prompt to help participants reflect on their strategy use in the face of reading tests, while the majority of interview questions came from the data of their think-aloud and stimulated recall sessions. Each interview lasted from 20 to 60 minutes.

Reading Test Material

Two reading tasks were retrieved from the TOEIC Official Test-preparation Guide Vol. 4 (2013), Sample Test. The short passage was in the form of an invitation card with only two questions while the longer, more difficult material was a double-passage reading task composed of two emails related to each other. There were five questions participants had to answer to complete this task. These two tasks were chosen because each represented a different difficulty level (both in terms of length and number of questions).

Data Analysis

Data related to metacognitive strategies were analyzed based on those presented in O’Malley and Chamot [25]. Their taxonomy was selected for two reasons. First, in their development of the inventory of strategies used by language learners, strategies collected were those used by learners faced with tasks of language skills. Similarly, students in this study were faced with a language task—a reading test. Second, O’Malley and Chamot’s classification has been widely accepted in strategy literature. Many previous studies based their data analysis on this classification with some necessary modifications ([30, 37]; Zhang et al. [45]). The current study adopted a think-aloud protocol and semi-structured interviews, the metacognitive part of O’Malley and Chamot’s taxonomy should serve as a basic guideline to elicit students’ metacognitive strategy use, since follow-up questions would be proposed when the given information was not clear or not sufficient. In their taxonomy, metacognitive strategies are listed as follows (for detailed definitions, see [25], p. 137):

  1. 1.

    Planning

  2. 2.

    Directed attention

  3. 3.

    Selective attention

  4. 4.

    Self-management

  5. 5.

    Self-monitoring (including monitoring of comprehension, production, auditory perception, visual perception, style, strategy use, planning, and double-checking)

  6. 6.

    Problem identification

  7. 7.

    Self-evaluation (including evaluation of production, performance, ability, strategy, and language repertoire)

All the think-aloud, stimulated recall, and interview reports were conducted in participants’ native language, Mandarin Chinese, and then transcribed and translated into English. Each mention of the use of a metacognitive strategy was coded, after which detailed descriptions of how participants used these strategies were also analyzed. Finally, the comparison between the high-scorers and low-scorers in terms of their metacognitive strategy use was conducted in an attempt to investigate the qualitative differences in strategy use between the two groups.

Results and Discussion

Metacognitive Strategies Used by Both HS and LS

Planning

Planning before reading was a common strategy used among the four participants. All of them previewed the type of the passage they were about to read and the number of the questions for that passage. By doing so, they could make a decision on how detailed they should read and how much time they expected to allocate for a certain passage. For example, during the stimulated recall, May (HS) was asked about her reading speed, and her response is presented in Excerpt 01:

That article [the second one] is harder. Um, I would read faster for emails, but if it’s a report like what I just read, or two reports or announcement, I know they are harder, so I’ll read a bit more slowly. (May, HS, Excerpt 01)

Except for the time they planned to invest in the passage, May (HS) and Jane (LS) both mentioned the type of questions they determined to answer correctly. May (HS), for example, said that when working on the task, she would consider the questions she would not want to fail, and “questions about main ideas are quick to answer after reading the entire passage, so I will want to answer correctly without a doubt.”

Self-Monitoring

Both groups reported using self-monitoring strategies during the process, among which comprehension monitoring was the most prevalent strategy. During the reading process, all the participants mentioned checking whether they understood the part they had read. If not, most of them would go back and read the confusing part again. For instance, during thinking aloud, it was observed that when Jane (LS) experienced difficulty understanding the reading passage, she chose to keep on reading until she understood the meaning.

Directed Attention

Both high- and low-scorers tried to stay focused during the think-aloud process in order to locate the answer to each question. When they felt distracted, they would force themselves to stay focused and return to the sentence they just missed.

The results above echo findings revealed in previous studies examining EFL learners’ use of strategies for reading texts (e.g., [2, 36, 39, 42]). That is, both successful and unsuccessful readers employed a range of similar strategies. In the current study, it can be seen that both high- and low-scorers made use of planning, self-monitoring, and directed attention during task execution. With regard to the differences, some researchers (e.g., [4, 29]) pointed out that it was the frequency of strategy use that differentiated the levels of reading comprehension. For example, Phakiti’s [29] quantitative results showed successful EFL learners used significantly more metacognitive strategies than their less successful counterparts. Some qualitative studies, on the other hand, attributed such differences to the effectiveness of strategy use, textual element integration [39], and problem awareness [7].

Metacognitive Strategies Used Differently by HS and LS

In this research, major differences between the two groups can be found in strategies such as selective attention, self-monitoring, and self-evaluation.

Selective Attention

While both groups attempted to use selective attention, to deal with long and difficult passages, their use of this strategy appeared to be different. The HS group would quickly skim through the questions, determine the location of the answer to each question, and start reading the first passage. They were able to use the selective attention strategy to efficiently locate the answer in a long reading task. Rather than reading through all the questions at the same time, the HS group tackled the questions one after another. Moreover, their knowledge about the test helped them locate answers to a limited number of questions at a time. Excerpt 02 demonstrated Vera’s (HS) answering procedure:

I usually don’t read the questions first because there are too many of them and I will forget, so I will just start with the first question and go back to look for the answer, one after another. Then I know that the first two or three questions are usually for the first passage. (Vera, HS, Excerpt 02)

The LS group, on the other hand, also tried to use selective attention strategy when answering questions. However, when they used this strategy, they either predicted the location of the answer in an incorrect place or skipped passages that contained important information. For example, when reading a double-passage text, the LS were looking for the answer in the second passage when the answer was actually in the first one. In other words, even though they tried to incorporate the selective attention strategy, they were inept at using this strategy well enough to help them find the answer.

When it comes to dealing with the parts that are not related to answers, HS would skim through the unimportant parts to at least know the gist of the parts, whereas LS would skip the entire section and jump to the next paragraph. For example, if Allen (LS) thought that he had found the answer in the first paragraph, he would not read the following paragraphs.

The major difference in the use of the selective attention strategy as used by the HS and LS groups was in whether they could locate the answer correctly or not. This is in line with previous research comparing effective and less effective L2 readers [7, 39]. Wang [39] reported that, for the less successful readers, “each single textual element seemed to be isolated and unrelated in their reading”. It is probably due to this perception that the LS group had a very different approach to text reading. Tian’s ([36], p.218) study seemed to offer a possible explanation to the LS group’s approach. Her study with students enrolled in TOEFL coaching schools in Taiwan indicated that lower-proficiency learners skipped parts of the reading passages because of time pressure, which was a result of their lack of vocabulary. The low speed of reading could be attributed to the “lack of automaticity of vocabulary decoding”. Learners’ use of selective attention was presumably related to their vocabulary size and their ability in detecting coherence among sentences and textual elements. These may be the factors underlying the effectiveness of the selective attention strategy.

Self-Monitoring

Although both groups reported using self-monitoring strategies to check their understanding, the flexibility and variety of strategies employed appeared to differ between the two groups. While the LS group used almost only a comprehension monitoring strategy, the HS group employed a wider variety of self-monitoring strategies and used them more flexibly.

May from the HS group, for example, also used style monitoring strategy to check the writing style in a reading passage, which greatly facilitated her reading speed.

I believed that there is logic in an English composition. What an author said will be explained in the latter part. Or there might be some contradictory points, which will definitely be mentioned later…I will look for some key words such as “but” and “however” because I think these transitions are supposed to be important. (May, HS, Excerpt 03)

The LS group, on the other hand, read the text over and over again, or even decoded the text word-by-word. Allen (LS), for example, often attempted to monitor his own ability in translating every word. Furthermore, both Allen and Jane (LS) would just look for key words in the reading passage that also appear in the questions when dealing with long passages. They believed that by looking for key words without having to understand the meaning of the passage, they could answer questions in a shorter time. Such a reading style, in combination with the abovementioned habit of skipping parts that they believed to be unimportant, supports the findings of previous studies on low-proficiency readers’ strategy use [7, 39, 46]. In their studies, low-proficiency readers tended to focus on local, word-based processing and failed to monitor their comprehension of the entire text.

Another important difference found between the two groups is in regard to the flexibility of strategy use. For example, when Vera (HS) found that the strategy she employed did not produce a successful outcome during the task, she would adjust the strategy.

If I can’t find the answer suddenly, I would go back and read more carefully, but if I still can’t find the answer, I would follow what I have answered previously or based on the main idea to make a guess, and then skip the question…I think losing one or two questions is okay, because what’s more important is to get the following passages right. (Vera, HS, Excerpt 04)

Contrary to HS, the LS group would adhere to certain strategies even when these strategies did not help them complete the task successfully although they also monitored their reading process. For example, for short passages, Jane’s (LS) chosen strategy was aimed at helping her answer all the questions in a short time, but in doing so, she missed important details and thus could not answer questions smoothly. At this point, Jane then employed the self-monitoring strategy and realized that her original habit did not work well in a reading test. However, Jane chose to continue with the same strategy and ultimately became very anxious. The anxiety interfered in her answering procedure and it could be the anxiety level that hindered her comprehension, as Jane herself admitted.

From interviews with the LS, it was revealed that they seemed to be aware of the ineffectiveness of the strategy they had selected for the task. However, neither of the LS students made any change when facing difficulties. For example, when Allen was asked about his strategy for reading tests, he admitted that he kept using the same strategy by saying “…because I don’t know about other strategies”. This could indicate their low strategy competence, which means the “general knowledge about what strategies are, why they are useful, and specific knowledge about when and how to use them” ([41], p. 519). The unsatisfactory strategy competence may influence the learners’ strategy employment and keep them at an inflexible level. Flexibility in strategy selection has already been highlighted in early research as a feature of successful readers’ strategy use [13, 14, 31, 46]. Being able to employ strategies flexibly relies on the readers’ metacognitive knowledge about the task demands. As was indicated by Wenden [40], it is the knowledge of task classification that prompts the selection of appropriate strategies. In the face of challenging tasks (such as long passages in this study), the HS group was shown to demonstrate higher strategy competence in choosing suitable strategies for different types of reading texts and were able to use them flexibly. This finding corroborates with previous research [15] that showed successful language learners tend to choose strategies that work well for a certain task and orchestrate these strategies to help them achieve their learning goal. Anderson [2] also concluded that “strategic reading is not only a matter of knowing what strategy to use, but also the reader must know how to use a strategy successfully and orchestrate its use with other strategies” (p. 468).

Self-evaluation

The HS group demonstrated the use of the self-evaluation strategy to accurately assess their reading process and make modifications. May (HS), for example, admitted that she was not a detail-oriented reader, so she used to make mistakes on detail questions involving numbers and names. However, after self-evaluating past performances on the tests, she decided to adjust her reading speed in order to correctly answer detail questions. The following example showed her use of the self-evaluation strategy for a recent exam she attended:

I used to pay more attention to the big picture [rather than details], but [now] I will allocate more time [on detail questions] and work most slowly…Before, I would feel the pressure of time and strived to answer the questions, but this time I think I have adjusted myself to make sure that I don’t make careless mistakes. (May, HS, Excerpt 05)

Allen from the LS group, on the other hand, also evaluated his performance and weaknesses. This is in contrast to previous studies [3, 47] that reported the self-evaluation strategy was nonexistent for poor students in that they were found not to assess whether their strategy use was successful. Both Jane and Allen from the LS group evaluated their own use of strategy, and Jane even changed her strategy after taking the TOEIC test for the first time. However, limited vocabulary size appeared to be a factor that impacted Allen’s follow-up action after self-evaluation. For instance, Allen was aware that his vocabulary skill was not good enough to help him obtain a satisfactory test result and acknowledged that some changes to his English study habits were necessary. However, he claimed, “I just don’t know how because my vocabulary size is too small”. Even when his friends offered him new learning strategies, he could not persist in trying the new methods, claiming, “I don’t have much motivation. I feel really bored, and here it comes again. It’s time to check vocabulary.” His awareness of poor vocabulary capacity seemed to have stopped him from trying new learning strategies.

The finding that low-scorers’ limited vocabulary ability affected his use of different reading strategies was supported by previous studies [1, 8, 26]. Al-Nujaidi’s [1] research, for instance, reported that learners with better vocabulary proficiency used more reading strategies than their weaker counterparts. He concluded that “[e]xtensive vocabulary knowledge seems to trigger successful use of appropriate reading strategies, which in turn results in better reading comprehension” (p. 147).

Conclusion

This paper adopted O’Malley and Chamot’s [25] strategy taxonomy to examine metacognitive strategy use of high-scorers and low-scorers in EFL reading comprehension tests. Think-aloud and interview data did not show striking differences in terms of the variety of strategies; however, the HS and LS demonstrated the qualitative differences in strategy use, especially for strategies of selective attention, self-monitoring, and self-evaluation. The results lend support to previous research [2, 36, 38] that has indicated that the ability to orchestrate different strategies can be a crucial factor in distinguishing good from poor language learners.

The LS group was found to encounter two problems during the reading process. The first problem is related to their ability to make a correct judgment about the nature of the reading texts. That is, their limited vocabulary size made them only look for corresponding words that appeared in the question items, neglecting the coherence of the entire passage. Even though they attempted to use the selective attention strategy, their inability to see coherence within the texts could influence their efficacy in locating answers correctly. Another problem the LS faced comes from their lack of problem-solving capabilities when they found their original strategy use had been less than satisfactory. This could be due to their insufficient knowledge about various strategies, but also to limited vocabulary size. For one thing, without enough strategy competence, learners might not know alternative strategies when they encounter learning obstacles. For another, even if they are aware of the strategies, their poor vocabulary capability would limit their ability to make changes.

Based on the findings, metacognitive strategies are shown to be important in the successful completion of reading tests, especially with regard to learners’ monitoring and evaluation abilities. This research foregrounds the crucial role of metacognitive strategies. It is suggested that language teachers prioritize metacognitive strategies when conducting strategy instruction. As Chamot [12] asserted, EFL learners with higher metacognitive awareness will be equipped with the knowledge of strategies needed for successful learning because of knowing how to learn. This study provides researchers and teachers with a clear picture on the qualitative differences in strategy use of high- and low-scorers.

Apart from the strategy instruction suggested above, two pedagogical implications emanate from the findings regarding the self-monitoring and self-evaluating strategies. First, consistent with previous studies [7], it is found that less proficient learners did not know what actions should be taken after self-evaluation. This taps into two abilities these learners may be lacking: first, the learners’ strategic knowledge [41], that is, their knowledge of strategies suitable for a specific task; second, behavioral self-regulation [50], that is, learners engaging in making strategy adjustments to achieve better learning outcomes. It is therefore suggested that language instructors provide students with training in these two abilities alongside strategy instruction. Second, it is believed that attributions learners use to explain their learning outcomes can be changed [9]. By giving students feedback and helping them become aware of the link between strategy use and success or failure, learners will be motivated to try different strategies when facing setbacks.

Finally, this paper has a number of limitations with regard to research methodology. First, as Silva and Graham [34] cautioned, not all learners are able to think aloud, and this is especially so with less articulate learners. It is possible that participants in this study might not have verbalized their thoughts completely. Therefore, stimulated recall and retrospective interviews were relied on as a major tool for data collection with less articulate participants such as Allen (LS). However, data collected in this way is at risk of data repression—participants’ giving socially acceptable data [17]. It is therefore suggested that researchers provide participants with more training before the think-aloud activity takes place.