1 Introduction

Because classroom discourse reveals how the classroom context can facilitate the development of students’ knowledge or language ability, it has been the focus of pedagogical research (Cazden & Beck, 2003). Classroom discourse includes the interaction between teacher and students or among students. A typical teacher and student interactive pattern is a sequence of initiation, response, and feedback/evaluation (IRF/E) (Mehan, 1979; Wells, 1993). This is a pervasive discourse pattern in which a teacher proposes a question or nominates a student to share; the student then gives a response, and the teacher provides feedback or an evaluation of the student’s response.

In particular, studies have found that a teacher’s initiating question not only facilitates language development (Chen & Liang, 2017; Scull et al., 2013; Zucker et al., 2010), but also can impose greater cognitive demand on students (Massey et al., 2008; Zucker et al., 2010). While some studies have reported that teachers’ inferential questions (e.g., open-ended questions) tend to impose a greater cognitive demand, thereby requiring higher cognitive levels (e.g., Massey et al., 2008; Zucker et al., 2010), other studies have suggested that this cognitive demand can result from teachers’ use of authentic questions (Applebee et al., 2003; Li et al., 2016; Nystrand et al., 2003). Whether teachers’ questions are inferential or authentic, these questions are open-ended in nature and thus promote students’ higher-order thinking. In the present study, this higher level of thinking, or higher cognitive level, refers to the ability to analyze, evaluate, and make critiques (Anderson & Krathwohl, 2001; National Assessment Governing Board, 2013).

Nystrand et al. (2003) further exemplify how a teacher’s authentic questions can trigger high-level thinking in extended discussions. They observed the relationship between questions and the quality of discussions in 112 English language arts classes and 106 social studies classes for two grade levels (8th and 9th grade). When the teachers’ and students’ questions involved a higher cognitive value (e.g., making speculation) and evaluative value (e.g., reporting more than simply factual information), the students were likely to engage in extended discussions in which the students demonstrated high-level thinking. For example, a teacher’s question could be, “Well, Mr. ___, then what do you think Gandhi would have done if he had been in the cafeteria with us?” (p. 21). This is an authentic question that requires the students to consider possible reasons, state support for their ideas, and evaluate others’ opinions during the discussions. In contrast, a question that merely requires recitation or reporting of factual information does not trigger extended discussions or enhance students’ evaluative or cognitive ability. It should be noted that despite the importance of teachers’ questions, teachers’ question initiation and turn-taking are difficult to manage in a large class (Hardman, 2008), and this may discourage students from raising questions in discussions (Nystrand et al., 2003).

Because of the importance of question-raising, some researchers have tried to incorporate question-raising training for students in order to enhance their higher-order thinking development in text-based group discussions. “Quality Talk” (hereafter QT) is one of the discussion approaches in which questions-raising is adopted as an important discourse indicator in order to achieve interactions and higher-order thinking (Pennsylvania State University, 2016). The introduction and teaching procedures for QT are introduced in the Method section. QT has been found to be effective in assisting students’ basic-level comprehension and higher-order thinking (e.g., Davies & Meissel, 2015; Reninger, 2007). Basic-level comprehension means that students are able to remember and understand the meaning of the text, while higher-order thinking indicates that students are able to develop higher-order cognitive ability, such as analyzing, evaluating, and creating, based on their reading of the text (Anderson & Krathwohl, 2001; National Assessment Governing Board, 2013).

Studies have shown that QT is helpful for elementary school students (Davies & Meissel, 2015; Li et al., 2016; Reninger & Wilkinson, 2010), junior high school students (Nystrand et al., 2003), and even for low achievers (Reninger, 2007) in the development of higher-order thinking. For example, in order to help low achievers with their reading difficulties, Reninger (2007) adopted Quality Talk (QT) with the aim of developing students’ literal comprehension and higher-order thinking, including their ability to analyze, generalize, give personal responses, and elaborate on their explanations. Analysis of the researcher’s observation notes, transcriptions of student interviews and group discussions, and students’ writings revealed that students made improvements in their reading comprehension (e.g., remembering the facts in the text) and higher-order thinking by using authentic questions in their group discussions.

While Reninger (2007) observed the reading performance of individual students who received QT teaching, Davies and Meissel (2015) compared the effect of QT and a traditional type of discussion on students’ literal comprehension and higher-order thinking in one New Zealand elementary school. The students were randomly assigned to a control (i.e., regular group discussion) and an experimental (i.e., Quality Talk) group and their group discussions were observed. The students’ discussions were recorded, and an analysis has revealed that all the students interacted in a turn-taking style in the first time discussion (i.e., before QT intervention). After some practice with the QT approach, the students in that group became more familiar with QT and were more engaged in the discussions at the second discussion (i.e., after QT intervention). It was found that these students demonstrated higher-order thinking by using more authentic questions compared with the control group.

While the above-mentioned studies have indicated that QT is empirically effective in assisting students’ higher-order thinking, other studies have focused on issues that may influence the effect of QT, such as the genre of the text (Li et al., 2014) and the ability of the participants in the discussions (Murphy et al., 2017). It was found that narrative texts can trigger extended discussions compared with information texts (Li et al., 2014) and that heterogeneous grouping (i.e., students with different levels of ability) can sustain more extended discussions that induce higher-order thinking.

Based on the brief review above, it can be concluded that QT is beneficial for the development of students’ higher-order thinking and that additional factors (e.g., the genre of the text) should be taken into consideration. Therefore, the present study aims to investigate whether students’ academic backgrounds are likely to affect the development of higher-order thinking as indicated by the use of questions in group discussions. The research questions are listed below:

  1. 1.

    Does QT training help students attain higher-order thinking as indicated by the types of questions they ask?

  2. 2.

    Do academic disciplines (i.e., Social Science & Education, Humanities & Liberal Arts, and Science & Engineering) affect students’ higher-order thinking as indicated by the types of questions they use?

2 Methods

The present study adopted the Quality Talk teaching approach to help students develop higher-order thinking. This section is divided into three parts: (1) participants, (2) the design of the study, and (3) data collection and analysis. The design of the study includes instructional frame of QT, pedagogical principles of QT, discourse elements, and teaching procedures.

2.1 Participants

The participants in the present study were students in three freshman English classes; they were from different academic disciplines (i.e., Social Science & Education, Humanities & Liberal Arts, and Science & Engineering). Twenty-seven students (17 male and 10 female) were in Science & Engineering, 36 students (12 male and 24 female) in Humanities & Liberal Arts, and 31 students (6 male and 25 female) in Social Science & Education. The students’ English ability was at a high-intermediate level based on the college entrance examination, roughly comparable with the CEFR B2 level.

2.2 Design

2.2.1 Instructional Frame

Using the QT framework, the instructor built a friendly and student-centered learning environment. The instructor chose reading materials and discussion themes to help students avoid digressing. During the group discussions of the assigned readings, the students were in control of their group’s progress, giving their own ideas and interpreting the texts freely. The goal of these discussions was that students could understand and derive the information from the text (i.e., efferent stance), be able to express their personal idea (i.e., expressive stance), and be able to interpret beyond the text (i.e., critical-analytic stance).

2.2.2 Pedagogical Principles

The instructor incorporated the following three factors to encourage an interactive learning context, including interesting reading (e.g., superheroes), topics students were familiar with (e.g., role models), and discussion ground rules. In particular, a discussion-friendly context was built by following eight ground rules in the discussions:

  1. 1.

    Share your ideas (but nothing personal).

  2. 2.

    No need to raise your hand.

  3. 3.

    Interact with your group members instead of the teacher.

  4. 4.

    Respect each other.

  5. 5.

    If someone remains silent, ask him/her questions.

  6. 6.

    It is possible to have different ideas/opinions from your group members.

  7. 7.

    Build connections between your discussion and the article.

  8. 8.

    Give effective explanations.

2.2.3 Discourse Elements

In order to evaluate the students’ higher-order thinking, the questions they asked were used as indicators in the QT model. An authentic question is a primary question type, which does not have a correct answer and thus requires respondents to give open-ended comments. Authentic questions show a direct contrast to test questions, which have a single correct answer, that is, factual information in the text. Authentic questions can be further sub-divided into five secondary question types: uptake, speculation, high-level thinking, affective, and connection questions. Definitions and examples for these are displayed in Table 1.

Table 1 Types of question
Full size table

2.2.4 Teaching Procedures

These freshman English classes lasted 13 weeks, meeting two hours per week. The students used the book Q: Skills for Success: Reading and Writing 4 (Daise et al., 2011) because question-raising skills were stressed in this textbook. Five units in total were read and discussed in the present study. Unit 1 was about the characteristics of heroes by introducing familiar heroes such as police officers or the batman in the movie. Unit 2 introduced a researcher, Paco Underhill, whose research interest was customers’ shopping behaviors. In this unit, the students had a chance to discuss how to attract more consumers’ attention. Unit 3 was like the author’s autobiography in which she described her interactions with her friends and her father. Unit 4 discussed the climate change and Unit 5 discussed the characteristics of successful artists, such as persistence.

Before engaging in the QT discussions, the students received a training session in the first week, as indicated in the teaching schedules (Table 2). In the training session, PowerPoint slides prepared by Pennsylvania State University (2016) were used to teach participants about different types of questions. There was a total of six PowerPoint slides, and each one introduced a specific type of question. In particular, authentic and test questions were introduced in the same PowerPoint slide.

Table 2 Teaching schedule
Full size table

For each unit, the instructor used two weeks to complete a QT discussion procedure, as shown in Table 3. The instructor first conducted a whole-class warm-up discussion such as “Why are stories about superheroes so popular with people of all ages?” extracted from Unit 1 (Week 2). The warm-up discussion questions were general questions used to trigger the students’ interest and elicit their background knowledge before reading. The instructor then gave an introduction to vocabulary, including collocations, meanings, and sample sentences. In the second hour, the students were asked to read assigned texts by themselves. During their reading, they underlined the important key points or wrote down their ideas, such as comments and questions. After reading the texts, the students formed a group of four to five students and raised their own questions for practice. For example, in Week 2, each group was asked to think of two questions for each of two types of questions, namely, speculation questions and high-level thinking questions. This produced a total of four questions. Thus, twenty-eight questions were generated by the Science & Engineering students (seven groups) and Humanities & Liberal Arts students (seven groups), respectively. The Social Science & Education students produced 24 questions (six groups). The instructor reviewed the students’ proposed questions in order to correct language-level errors and confirm students’ understanding of the question types before the next class.

Table 3 Teaching procedures of a unit
Full size table

In the third week, the class together recited the eight ground rules listed in the above section, Pedagogical Principles before each QT discussion. Next, the students engaged in 20 minutes of discussion, using five discussion questions prepared by the instructor, such as “Among all of the superheroes, which character do you like the most?” (extracted from Unit 1). The teacher circulated around the groups to listen to their discussions. For example, for Unit 1, the teacher joined the discussions by groups 1 and 2; for Unit 2, the instructor joined groups 3 and 4. The students’ group discussions were recorded and then uploaded to a school platform where the instructor could keep track of each group and download their recordings for further analysis. Finally, the students received a comprehension check which included five multiple-choice questions and three short-answer questions. This comprehension check was used to evaluate whether the students understood the main idea of the texts.

2.3 Data Collection and Analysis

Data for this study included the students’ group discussions, which were recorded by the students and transcribed by a research assistant. The transcriptions were then analyzed by the research assistant and the researcher. The research assistant read through the transcripts and identified each type of question based on the definitions shown in Table 1. The researcher then reviewed the research assistant’s coding. If there were different interpretations of the students’ questions, the researcher and research assistant discussed these in order to reach a consensus. The coding reached a consistency of more than 80%.

In order to make comparisons across different units and academic disciplines, the numbers of types of questions were presented in terms of one minute. For example, when there were eight authentic questions in a twenty-eight-minute group discussion, there were 0.29 authentic questions per minute. A Kruskal–Wallis, nonparametric analysis, was adopted because the data were not normally distributed.

3 Findings and Discussion

3.1 Research Question 1

The students’ questions for the respective disciplines are displayed in Table 4. It is apparent that the students made gains in higher-order thinking, as indicated by the increased use of authentic questions (about 1 AQ per minute), uptake questions (about 1 UQ per minute), speculation questions (about 1 SQ per seven minutes), and affective questions (about 1 AfQ per eight minutes). The results suggest that QT was generally as effective as when it was applied in other studies to promote higher-order thinking (Davies & Meissel, 2015; Li et al., 2016).

Table 4 Types of questions used in the three academic disciplines
Full size table

AQ was the most frequently used question type by the college students in this study; this was also true for elementary (Li et al., 2016) and junior high school students (Davies & Meissel, 2015). The students can easily understand the concept of AQs and use them in their group discussions. One example from Unit 5 is given below to illustrate how AQs were employed (the grammatical errors in students’ output are retained throughout the examples in the present study):

Example 1

1

Student A:

I think Van Gogh …because he didn't get famous when he is alive but he still

2

 

works hard… He created many paintings in every two days. Work hard till he died

3

 

Now he is so awed around the world

4

Student B:

So… what has inspired you? [AQ]

5

Student A:

Even if his effort didn't solve by others… but now the whole world knows him

6

Student B:

I think …all the artist have their own experience …but we have different

7

 

environment … I don't I don’t have the specific artist that inspire me

8

Student C:

For me, I think I think no artist can inspire me either. Because their lives are

9

 

different from mine. I have no feeling… because painting will not be a job

Among the AQs, the students used more UQs, which possibly suggests that the students were engaged in more interactive dialogues. This result partly conforms to Bakhtin’s (2010) theory of how comprehension can be represented by one’s responses and how language is adopted as a tool of thinking in order to reach mutual understanding in dialogues. In Example 2 below, Student A’s UQ in line 6 indicates that they understand Student C’s statement in line 5. Only when mutual understanding is achieved among the students, is the dialogue then able to continue. This example is taken from the discussion of Unit 1.

Example 2

1

Student A:

If you have a superpower, which kind do you want? Why?

2

Student B:

I want energy…I want to spread love and happiness …because I want to see

3

 

everybody smile and happy so they can prevent them from melancholic… so

4

 

they can go out from the bad mood

5

Student C:

I want to stop the time

6

Student A:

What do you want to do? [UQ]

7

Student C:

If someone is dangerous, I can save him. So I want to stop the time…

8

Student A:

You stop a part of all the world? [UQ]

9

Student C:

Maybe like that…What do you want to do through this superpower? Like, go to

10

 

women's toilet? [UQ]

11

Student A:

Maybe. I just want to randomly go to another space like the past or future. Like

12

 

Doraemon (A character in Japanese comics) time machine. So …I can go to the

13

 

past to fix the mistake I have made it or I can just to future to …

14

Student B & C:

Change your life? [UQ]

15

Student B & C:

See your wife? [UQ]

In contrast to the increasing use of AQs, the fewer use of TQs may suggest that the students had learned to read beyond the lines, instead of reading only for factual information. The students in Social Science & Education appear to have successfully engaged in the QT discussions, in that they tended to ask more open-ended questions instead of test questions. No test question was used by students in this academic discipline. In contrast, the students in Humanities & Liberal Arts employed some test questions. In Example 3 below, Student C in line 4 sought to clarify Student B’s idea in lines 2–3 based on the text (Unit 4). Additionally, several examples from the text were provided by Student A in lines 5–8.

Example 3

1

Student A:

Okay, question one. Does anybody have any ideas?

2

Student B:

Hmm…I think climate change have impact on our health because every day we

3

 

breathe. If the air isn't clear, so my… we… our body may have a lot of problem

4

Student C:

Some problem such as? [TQ]

5

Student A:

According to the…article yeah article. Hmm…the author says that climate change

6

 

will cause a lot of healthy problems, like malaria and dengue fever. Yeah it's like

7

 

if the climate change to the hotter weather and mosquitoes will spread to other

8

 

place, the high…

9

Student B:

higher location

Although all of the students of the three academic disciplines showed, to some extent, higher-order thinking by using more open-ended questions, it appears that the students from certain disciplines tended to employ different types of questions, such as the use of TQs in Humanities & Liberal Arts. This varying use of question type is discussed in further detail in the next section.

3.2 Research Question 2

In order to examine the effect of the three academic disciplines on the students’ use of different question types, a Kruskal–Wallis test was adopted and the results have showed that there was a statistically significant difference in authentic questions according to students’ academic discipline, χ2(2) = 5.918, p = 0.049: a mean rank score of 40.43 for Science & Engineering; 56.28 for Humanities & Liberal Arts; and 48.55 for Social Science & Education.

A follow-up analysis has showed that the use of AQs was significantly different between Science & Engineering and Humanities & Liberal Arts (p < 0.05) and also between Humanities & Liberal Arts and Social Science & Education (p < 0.05). The students in the Science & Engineering programs used far fewer authentic questions than the students from the two other academic disciplines. The significantly fewer uses of AQs (about 1 AQ per five minutes) may be due to two reasons. First, the students in Science & Engineering may have yet to develop higher-order thinking or higher-order cognitive thinking ability through the QT discussion approach. Second, it is also possible that these students require additional training sessions or the teacher’s direct participation in their discussions because when the researcher listened to the students’ discussions or their recordings, it was found that they digressed off-topic more easily than students from the two other disciplines.

On the other hand, the students in Humanities & Liberal Arts used more authentic questions (about 1 AQ per 2 minutes) than the students in Social Science & Education (about 1 AQ per 4 minutes). It seems that the students from both disciplines show higher-order thinking, as indicated by their greater use of AQs. However, it was found that when discussion themes were closely related to students’ personal experience, they were more likely to engage in the group discussions. For example, although the students in both disciplines employed many AQs, they used more AQs—as shown in Example 4 (extracted from Unit 2)—when the theme dealt with personality or characters (e. g., Unit 1, 2, and 3), but not about weather (i. e., Unit 4). While Li et al. (2014) explained the impact of text genre on the effectiveness of QT, the present study finds that the themes of the text may also play a role.

Example 4

1.

Student A:

Ok, so we can.. the next question, what kind of personality did author have?

2.

Student B:

I think she is the shy but sensitive because she will observe events in her daily life

3.

 

even if a small thing, and she will…. to write down

4.

Student C:

How do you know? [AQ]

5.

Student B:

Because she observes that mother make-up by…and she observes that their record

As for uptake questions (UQs), their use was significantly different between Science & Engineering and Humanities & Liberal Arts (p < 0.05): one question per two minutes for students in the Humanities & Liberal Arts versus one question per five minutes for students in Science & Engineering. Although we cannot yet conclude that the uptake question is an indicator of students’ greater engagement in interactive dialogues, it indeed seems that the discussions among Humanities & Liberal Arts students were more interactive. In essence, the uptake question is a follow-up question. Thus, when the students pose more uptake questions, this suggests that they are listening closely to each other and want to know more from the interlocutors.

It can be concluded that the differences in students’ academic backgrounds do, to some extent, influence the effectiveness of QT. The students in Humanities & Liberal Arts benefitted from QT implementation through engaging in more interactive dialogues in which they raised more questions, while the students in Science & Engineering appeared to benefit less. The students in Social Science & Education, meanwhile, learned the key concept of QT (i.e., the differences between authentic and test questions) through avoiding test questions and using more AQs.

4 Summary and Conclusion

There are two major findings in the present study. First, all of the students made gains in higher-order thinking to some degree, as indicated by their use of more AQs and fewer TQs. More AQs suggest that the students paid attention to the discussion and engaged in cognitive thinking (e.g., to analyze or evaluate). Second, significant differences can be found among the three disciplines for two types of questions: AQs and UQs. The students in Science & Engineering used significantly fewer AQs, suggesting that they benefitted less from the QT. On the other hand, the students in the two other disciplines employed more AQs and UQs, suggesting that they had achieved higher-order thinking and benefitted from the QT approach. It should be noted again, however, that question-raising is only one of the discourse indicators of higher-order thinking.

Although AQs were found to be significantly different among the three academic disciplines, this does not suggest that the other question types are less important. It is possible to speculate that the students were not familiar with the questioning or interactive style in classes. They may need more time to familiarize themselves with the QT and may then possibly be able to use those questions’ types, such as connection questions. For further research in QT, additional factors should be taken into consideration as the focus of the present study, students’ academic background. For example, three interacting elements greatly influence the performance of reading comprehension: text, readers, and activity (RAND Reading Study Group (RRSG), 2002). Similarly, the three factors may also affect how higher-order thinking may be developed. Based on the present study, it has been discovered that the themes of the text change how they interact with each other. Therefore, more related factors may be expected in the future. What follows are the highlights of this chapter:

  • QT framework is a useful discussion approach. This approach generally enhances students’ higher-order thinking by raising more authentic questions.

  • The effect of QT framework does differ according to students’ academic backgrounds. To be more specific, Science & Engineering students benefitted less from QT framework compared with Humanity & Liberal Arts and Social Science & Education students.

  • It is speculated that more teacher’s directions are needed for Science & Engineering students. Humanity & Liberal Arts and Social Science & Education students enjoy the discussions and are able to be benefitted from QT discussions.

  • QT framework facilitates higher-level thinking because asking questions requires the participants to understand interlocutors’ meaning and think actively in response to others’ thoughts. In this process, language is used as a vehicle for co-reasoning among interlocutors.

  • In order to enhance students’ higher-order thinking through QT framework, students’ academic backgrounds should be taken into consideration. Their academic backgrounds greatly influence how they engage in the discussion process, which leads to the performance of higher-order thinking.

  • In the present study, it seems obvious that the students are able to enhance higher-order thinking through a student-centered classroom, which is one of the most important pedagogical principles in QT framework. When the students are responsible for their own learning, it seems that they learn better.