Introduction

Researchers define metacognition as awareness of and reflection upon one’s own cognitive process, which can induce self-regulation and conscious coordination of learning tasks (Brown 1987; Flavell 1976, 1981).

The research described in this paper is concerned with question posing, a higher order thinking skill, and its link to metacognitive knowledge. Therefore, the theoretical background relates to metacognition with emphasis on metacognitive knowledge, as well as question posing skill. A new learning unit in chemistry, Case-based computerized laboratories (CCL), developed at the Technion, Israel Institute of Technology (Dori et al. 2004), served as the source of the subject matter for the research. This learning unit is based, among other things, on learning through case studies, i.e., daily life stories with chemical orientation, presented as scientific articles. These case studies are characterized by a sense of reality, multidisciplinary nature, and dilemmas, often with no single, clear-cut solution. Each case study is accompanied by tasks and guiding questions at different thinking levels. It also includes a unique task, calling for posing questions related to the case study. Rather early during the instruction of the new learning unit, teachers exposed their students to a metacognitive strategy that enabled them to assess and analyze the quality of the questions they had formulated, and to characterize them according to a three-component question taxonomy. Assisted by the criteria included in the taxonomy, the students who studied the CCL unit were asked to undergo a metacognitive process of analyzing the complexity of the questions they had posed.

Our findings include qualitative analysis of students’ interviews and both qualitative and quantitative analysis of pre- and post-case-based questionnaires. Our qualitative analysis examined the thinking processes that students underwent while posing questions, assisted by the questions classification taxonomy. The quantitative analysis of the question posing skill from the case-based questionnaires served as a basis for two comparisons: one comparison was within the experimental group between pre- and post-questionnaires results, while the other was between the experimental group students and their counterparts in the comparison group.

Theoretical background

Metacognition and metacognitive knowledge

Flavell (1979, 1981) described metacognition as awareness of how one learns, knowledge of how to use information to achieve a goal, and ability to judge the cognitive demands of a particular assignment. Thus, metacognition refers to the awareness of one’s own cognitive processes and the self-regulation and management of those processes in relation to the learning task. This includes conscious selection of strategies and matching strategy to task demands. According to Koch (2001), metacognition is a hidden level of behavior that involves focusing on thinking about thinking and its relation to intellectual performance.

Metacognitive knowledge is often characterized by researchers as consisting of the following interrelated parts: (a) knowledge of one’s own cognition; (b) knowledge about the specific cognitive strategies that might be used for various learning tasks, and (c) procedural knowledge of when and where to use acquired strategies (Flavell 1976; 1979; 1987; Garner and Alexander 1989; Pintrich et al. 2000).

Schraw (1998) has made the important distinction between knowledge and regulation of cognition and argued that metacognitive knowledge is multidimensional, domain-general in nature, and reachable.

Strategic metacognition

Strategic knowledge is a component in metacognitive knowledge defined as knowledge of general strategies for learning, thinking, and problem solving. Students can have knowledge of various metacognitive strategies that might be useful to them in planning, monitoring and regulating their learning and thinking. These strategies include ways in which individuals plan, monitor, and regulate their cognition. For example: set sub-goals, ask themselves questions as they read a text, and re-read something they do not understand (Pintrich 2002).

Whereas cognitive strategies enable one to make progress in building knowledge, metacognitive strategies enable one to monitor and improve one’s progress by evaluation of understanding and application of knowledge to new situations (Flavell 1979). Pintrich (2002) argued that unlike discipline- or domain-specific strategies, metacognitive strategies are applicable across most academic disciplines or subject matter domains and can therefore be used across a large number of domains. Through metacognition, one can define the nature of a task or problem and select the most useful strategy for executing the task (Sternberg 1981).

Metacognitive strategies instruction

Increased learning is achieved when trainees are given the rationale for the strategy to be learned and are helped to see the direct relationship between strategy use and subsequent learning outcomes. These are significant advantages compared with blind training (Wong 1985). The more explicit teachers’ modeling of cognitive and metacognitive skills, the more likely it is that their students will develop cognitive and metacognitive skills (Butler and Winne 1995). Students aware of their teacher’s strategic preferences adapt better to the demands of this teacher’s classroom. Students who know about different strategies for learning, thinking and problem solving will be more likely to use them, since metacognitive knowledge of these different strategies enables students to perform better and learn more. There is a need to explicitly teach for metacognitive knowledge that is embedded within usual content-driven lessons in different subject areas (Pintrich 2002).

The methodology of repeating the same skill over and over again in different scientific contexts requires that teachers be able to plan their teaching with an eye to both content knowledge goals and thinking skill goals (Zohar 1999).

Promoting metacognition begins with building awareness among learners that metacognition exists, that it differs from cognition, and that it increases academic success. The teachers need to teach strategies, and help students construct explicit knowledge about when and where to use these strategies. A flexible strategy can be used to make careful regulatory decisions in order to plan, monitor, and evaluate learning (Schraw 1998).

Paris and Winograd (1990) have argued that students’ learning can be enhanced by becoming aware of their own thinking as they read, write, and solve problems, and that teachers should promote this awareness by informing their students about effective problem-solving strategies and discussing cognitive and motivational characteristics of thinking. Students who are not used to thinking in a metacognitive mode sometimes resist having to do so, especially if they have been passive learners for many years. Students need scaffolding instruction and strong support in the initial steps and later gradually to withdraw this support as they become more proficient at self-regulation (Hartman 1994). Simons and Klein (2007) examined how scaffolds influence inquiry and performance in a problem-based learning environment. They concluded that use of scaffolds has an important role in enhancing student performance within problem-based learning—PBL.

When investigating interventions that enhance students’ metacognition, researchers have found that if students’ metacognition was improved, then it was possible to improve their learning outcomes (Thomas and McRobbie 2001). An example of applying metacognitive declarative knowledge in a program designed to foster higher order thinking is found in CASE (Cognitive Acceleration through Science Education). In CASE, metacognition was applied in the sense of conscious summary of strategies successfully applied, and naming verbal tools used in every CASE lesson. Students learned to reflect on the thinking they were engaged in, to bringing it to the front of their consciousness, and to make an explicit tool that may then be transferred into a new context (Adey and Shayer 1993, 1994).

However, despite evidence that metacognition is important for high-quality learning in science classrooms (Tobin and Gallagher 1987), classrooms are often characterized by absence or lack of characteristics necessary for developing and enhancing students’ higher order thinking and metacognition, and by overemphasis on memorization and lower order thinking and learning. Therefore, means are necessary for informing educators of how they might enhance students’ metacognition while using interventions and changes in pedagogy (Thomas 2003).

Question generation

Young children are inherently curious, frequently asking a stream of questions. However, many elementary school students have stopped asking questions, and they do not articulate a desire to discover, debate, or challenge (Becker 2000).

Dillon (1988) found that when students did ask questions, the questions were seldom designed for increasing their personal knowledge or understanding. Rather, they were procedural, informational, and focused on the content covered in the next test.

By asking questions, students frequently reveal what they want to learn, what they know, and what they do not know. Questions are also part of social functioning when students seek their classmates’ views and communicate and negotiate during group activities. Students’ cognitive, social, and emotional growth is decreased when they do not ask questions (Becker 2000). The value of student questioning has been emphasized in the National Science Education Standards, which stated that “inquiry into authentic questions generated from student experiences is the central strategy for teaching science” (National Research Council 1996, p. 31). Emphasis on students’ questions conveys the message that in variety of science disciplines inquiry is a natural component and questions need to be constantly raised (Woodward 1992). Indeed, Dori and Herscovitz (1999; 2005) suggested that an effective strategy for improving problem solving ability is to foster students’ question posing skill.

Students’ questions can be valuable during discussions. They indicate that students are actively engaged in making sense of what they learn and may articulate issues that need to be addressed (Van Zee et al. 2001).

Students’ questions can be indicative of their “frame of mind” and the quality of their understandings (Watts et al. 1997). These are often not intended to be made formal, or even necessarily to be answered. Others have to do with exploring a situation rather than seeking a simple answer (Watts and Alsop 1995).

Categorization of student questions

The researchers, Arzi and White (1986) claimed that it is difficult to define the quality of student posed questions, but it is possible and desirable to provide teachers with research-based sets of working criteria for guiding their students. Watts et al. (1997) divided students’ questions into three categories: (a) Consolidation: Students may feel they have grasped an idea, and seek reassurance that this is in fact the case; (b) Exploration: Students’ questions seek to both expand knowledge and test constructs that they have formed; and (c) Elaboration: Students examine claims and counterclaims, elaborating on their previous knowledge and experience. Questions within this category are attempts to reconcile different understandings, resolve conflicts, test circumstances, force issues, and track in and around the ideas and their consequences. This categorization offers a starting point for analysis of the type of questions that occur during science classes.

Marbach-Ad and Sokolov (2000) explored the types of written questions students asked after reading one or more chapters from their textbook, and investigated the ability of students to improve their questions. Their semi-hierarchical taxonomy included eight categories of student questions. The lower-thinking level categories contained questions about definitions, concepts, or facts explained fully in the textbook. The higher-level questions were questions resulting from extended thought and synthesis of prior knowledge and information, questions that contain a research hypothesis, as well as ethical, moral, philosophical or sociopolitical questions, and questions for which the answer is a functional or evolutionary explanation. After the taxonomy was presented to an active learning class of undergraduate biology students, more students were able to pose better, written questions. Their questions became more insightful, thoughtful, and content-related, and were not easily answerable by consulting the textbook or another readily available source. The best questions could be recast as scientific research questions. These researchers (Marbach-Ad and Sokolov 2000) suggested that teachers present the student-question taxonomy to students at the beginning of the semester to let them know what is expected.

According to King and Rosenshine (1993) an important element in the success of guided cooperative questioning is the question structure. Particular structures are designed to promote learners’ cognitive and metacognitive activities that include critical thinking about the material presented, activation of relevant prior knowledge, and comprehension monitoring. Such questions induce students to engage in (a) thinking about applications; (b) developing examples; (c) analyzing relationships; (d) making predictions; (e) synthesizing ideas; (f) comparing and contrasting; and (g) evaluating.

The process of asking and answering those particular questions serves as the metacognitive strategy for helping students to monitor their understanding of the material (King and Rosenshine 1993).

Question generation as a metacognitive skill

Questioning directed toward higher order thinking plays a central role in comprehension, comprehension monitoring, self-testing and self-control (Davey and McBride 1986; Palinscar and Brown 1984). For students to be active learners and independent thinkers, they must generate questions that shape, focus, and guide their thinking (Singer 1978). Some studies have found that metacognitive activities that are externally imposed (by the teacher) generate questions that are less effective than those generated by the students themselves (Wagner and Sternberg 1984). From a metacognitive perspective, self-questioners know what they know and, as importantly, what they do not know (King 1989). In general, both cognitive and metacognitive strategies can be applied for question posing, but the degree to which these strategies can be applied vary and depend on the content area and prior knowledge of the students.

Good readers monitor their state of reading comprehension and engage in debugging strategies when they encounter comprehension failures, and they engage in review and self-questioning to verify that their reading and study goals have been met. Such conscious coordination indicates the existence of metacognitive processes (Brown 1980). Students’ self-questioning is a metacognitive or comprehension-monitoring activity because students trained in question generation may also acquire self-awareness of their comprehension adequacy (Palinscar and Brown 1984; Wong 1985).

Flavell (1976) suggested that by asking questions about the article one might improve his/her knowledge (a cognitive function) or monitor it (a metacognitive function). Metacognition is about self-regulation, not regulation by others. Consequently, Gourgey (1998) recommended that instruction must encourage students to generate and use their own strategies and self-questions. The effectiveness of question generation depends on the amount and type of training and practice that learners receive in question posing (Davey and McBride 1986; Dori and Herscovitz 1999, 2005; Palinscar and Brown 1984; Wong 1985). Generation of high level questions requires the adoption of questioning procedures with emphasis on thoughtfulness in questioning, extensive cognitive coaching, and practice with feedback (King 1989; Palinscar and Brown 1984). Learners need a specific strategy for questioning before they become proficient in asking thoughtful questions. A student-generated questioning strategy provides both freedom and structure, balancing learner autonomy and external control (King 1994).

Certain types of question generation training can have meaningful effects on students’ metacognitive reading strategies. Davey and McBride (1986) successfully trained sixth-grade students to generate, evaluate and answer questions about the meaning of a text passage. With practice, students improve their ability to ask clear questions, summarize main ideas, and take a more active role in leading group discussions (Hartman 1994; Palinscar and Brown 1984).

Research setting

Since the early 1950’s, chemistry teachers focused on students’ memorization of scientific facts and algorithms that could support them while solving textbook exercises and problems. Gabel and Bunce (1994) reported that the collection of exercises in textbooks made it possible for algorithmic thinking learners to come up with correct answers to a certain class of problems without creating the proper cognitive understanding related to those problems. As a result, Zoller (1993) argued that many students were not able to solve problems that require higher order thinking, which had no apparent resemblance to one of the patterns with which they had been familiar. Although it had been proven that lecturing and solving exercises with no relevant chemical concepts does not contribute to higher order cognitive skills acquisition, Zoller et al. (1995) claimed that lectures and algorithmic thinking continued to dominate the discourse in chemistry classes.

Until the beginning of this decade, the Israeli national chemistry matriculation examinations had also emphasized memorization of scientific facts and quantitative problems solving. Such examinations are foreseen, and the teachers and students work hard in order to pass them successfully (Dori 2003). In recent years, as alternative assessment approaches have begun attracting the attention of researchers and educators, the chemistry matriculation examination in Israel started to be supplemented with new modes of assessment (Barnea 2002). The reform in the Israeli chemistry curriculum included changes in the content of chemistry syllabus, such as reducing the number of mandatory topics, providing teachers with more flexibility, and the way students are assessed regarding their progress and achievements (Dori 2003).

The assumption underlying this reform was that by using a variety of forms of assessment, teachers would improve the monitoring of their students’ progress through assessment modes that encourage student thinking as well as application of skills (Gillespie et al. 1996).

At the Technion, we developed a Case-based Computerized Laboratory (CCL) learning unit along with embedded assessment. The unit was designed for 12th grade honors chemistry students (Dori et al. 2004). The honors curriculum in Israel consists of five study units, and the CCL curriculum is one elective unit of these. Developed within the framework of reforming the Israeli honors chemistry curriculum, the CCL unit integrates computerized hands-on experiments with emphasis on scientific inquiry and case studies. The CCL environment exposes the students to article reading and to metacognitive knowledge of question posing strategies, supported by a question classification taxonomy as we describe below. An important goal underlying the CCL learning unit was developing students’ higher order thinking skills. The unit includes reading case studies, posing question, computerized inquiry laboratories, and molecular modeling. One central component in the CCL environment was the case studies, followed by question posing tasks. Each of the five laboratory topics in the learning unit (e.g., energy, acid-base) began with a case study introducing chemical phenomena from daily life related to the inquiry laboratory that the students were about to experience. The last part of each topic included another case study which dealt with a different aspect of the subject matter under study.

All the chemistry teachers were exposed to the taxonomy, presented in Table 1, as part of a training program. The teachers of the experimental group students participated in a week-long CCL summer training program at the Technion. These teachers were directed to instruct the program with emphasis on the case-based method and the question posing metacognitive strategy.

Table 1 The classification taxonomy of chemical questions
Full size table

In their classrooms, after reading the first case study, the teachers worked on improving their students’ question posing skill, and they asked the students to pose as many questions as they could. These had to be questions related to the case study, to which the students could not find a direct answer from the text. After creating a list of 10–15 student-posed questions, the students’ next task was to sort the questions by categories, using only their own judgment, without any further explanation from the teacher. In each experimental class, the different questions were sorted by diverse parameters and categories which the students devised. These categories served as a platform for the teachers to expose the students to the metacognitive strategy of creating a question classification taxonomy. This taxonomy provided different aspects of examining questions posed in relation to a chemical text and defined what constitutes a “complex” question in this context. Having presented this taxonomy, the questions posed by the students were written on the board and sorted again by the students and the teacher together. Each question was analyzed for the different aspects of the taxonomy in a class discussion and a joint decision was made regarding whether one or more of the aspects was missing from it, and in what aspects the question could be considered as complex.

As the academic year progressed, while learning the CCL unit, students read more case studies (seven in total). Supported by the question classification taxonomy, they posed questions related to these case studies. As in the first time, the questions for each case study were written on the board and analyzed by the different taxonomy aspects. Repeating the same skill in different scientific contexts potentially helps the students in formulating better questions. A student who knows the teacher’s strategic preferences in question posing, is better able to adapt to the demands of this teacher’s classroom (Pintrich 2002).

Research objectives and questions

The objectives of the research were (a) to examine how an integrated metacognitive strategy affects students’ skill to pose complex questions and to analyze them according to a specially designed taxonomy and (b) to investigate the question posing—higher order thinking—skill of honors chemistry students.

The research questions were:

  1. 1.

    What are the characteristics of the metacognitive processes students undergo while developing their question posing skill assisted by the taxonomy?

  2. 2.

    What is the effect of a metacognitive approach based instruction on students’ question posing skill?

  3. 3.

    What differences, if any, in question posing skill exist between the experimental and the comparison groups?

Research participants

The experimental group of our three-year study included 793 honors 12th grade chemistry students taught by 28 teachers from Israeli high schools who studied the CCL unit. At the 2nd and the 3rd stages of the study, we expanded our research participants and added a comparison group. The comparison group consisted of 138 12th grade honors who did not study the CCL unit as part of their chemistry curriculum.

The research participants, 931 in total, were from schools in the north and center of Israel and included 45% male students and 55% females. Schools were located in cities as well as agricultural communities and their students came from a variety of socio-economical backgrounds. Most (91%) of the participants were from the Jewish sector and 9% were from the Arab sector. More Arab teachers joined this reform at a later stage and were investigated in a separate study (Abed and Dori 2007).

Table 2 presents the experimental group participants in the three stages of the study and the comparison group students in the last two years of the study (2nd and 3rd). We refer to our study as longitudinal based on the definition of White and Arzi (2005). They noted that a longitudinal study is a research in which two or more measures or observations of a comparable form are made of the same individuals or entities over a period of at least one year. We used our case-based questionnaires at the beginning of their 12th grade and toward the end of that year.

Table 2 Research participants
Full size table

Most of the comparison group students (70%) studied in inquiry- or industry-oriented laboratories. These programs integrated laboratory activities and modified science articles with emphasis on inquiry or industrial issues. The students of the inquiry- or industry-oriented laboratories programs experienced question posing tasks based on both adapted articles and laboratory activities. The rest of the comparison group students studied in a traditional teacher-centered style, which focused on theoretical studies with few laboratory activities. None of the students in the comparison sub-groups was equipped with the metacognitive knowledge concerning question posing skill.

Students in all the research groups were evaluated for their question posing skill by pre- and post-questionnaires, while only experimental group students were interviewed regarding the question posing skill in order to examine the metacognitive learning processes occurring in the CCL environment.

The teachers of the research groups participated in a summer training, and were familiar with the CCL unit and its characteristics. However, only the experimental group teachers participated also in an on-going training program throughout the academic year, received further help and solutions to problems that were raised while they instructed the new CCL unit. The experimental group teachers fully cooperated with the researchers, who, in turn, supported the teachers. Other teachers, who decided not to implement the CCL learning unit, were asked to be part of the comparison group. Most of these teachers, who taught the inquiry- or industry-oriented laboratories, also received support from another university and from local mentors throughout the academic year. Lacking the same level of commitment for the research as the experimental teachers, only few of them ended up participating in the research with their students serving as the comparison group.

To analyze the effect of students’ academic level on their thinking skills, we divided the experimental and the comparison groups’ students using Duncan’s Multiple Range Test into three academic levels—low, intermediate, and high-based on their total pre-questionnaire scores. The total score of the pre-questionnaire was calculated based on average scores of all the thinking skills examined in the CCL learning unit—question posing, inquiry, modeling,Footnote 1 chemical understanding–retention, graphing skills and transfer (Dori and Sasson 2008; Kaberman and Dori 2008; Sasson and Dori 2006)Footnote 2. According to the pre-questionnaire, we found a similar distribution of both the experimental and comparison students in low and high academic levels, with no regard to the teaching methods in their classes. No significant differences in students’ total scores were found in the pre-questionnaire between the experimental group participants and the different comparison sub-groups participants. Therefore, we elected to group the non-CCL methods as one comparison group.

Methodology

In order to present a broad view of the metacognitive knowledge of the students in our study, both qualitative and quantitative research tools were used (Denzin and Lincoln 2000; Johnstone and Onwuegbuzie 2004). We present the results of the interviews of six experimental students and the case-based questionnaires that 793 experimental and 138 comparison students responded to throughout the threeFootnote 3 year research. Each of the students and some of the teachers participated in only one of the three stages. The Mixed ProcedureFootnote 4 technique was used to analyze the data for all the stages of the research.

Students’ semi-structured interviews

We interviewed six students, three male and three female, who represented the experimental group students featuring high, intermediate, and low academic levels. The objective of the interviews was to understand the metacognitive processes these students underwent while developing their question posing skill and practicing it with the question taxonomy. At the beginning of the interview, each of the students read a case study, following which, one of the researcher asked him/her to pose questions about that case study.

Figure 1 described a case study which was presented to some of the students in the interview. Other interviewees read the patulin case study (see Fig. 3).

Fig. 1
figure 1

A case study used in the interviews

Full size image

During the interview, these students analyzed their questions using the think-aloud method, explaining why they had posed those particular questions and how they took the different aspects of the taxonomy into consideration.

The students were interviewed at an early stage, before completing the CCL learning unit, while they were still practicing the questions taxonomy. Because of the rather early stage of the interview, the interviewer intervened, clarified what she meant, and sometimes had to remind the interviewees, parts of the taxonomy. The interviewer’s guidance, which referred to the taxonomy as a questions’ metacognitive tool, encouraged the students to improve the questions they had posed in the beginning of the interview.

Case-based questionnaires

An important goal of the CCL learning unit was developing students’ higher order thinking skills, such as inquiry, graphing, and modeling (Dori et al. 2004; Dori and Sasson 2008; Kaberman and Dori 2008). To assess the question posing skill, we used pre- and post-questionnaires following the idea that the assessment tool should match the teaching and learning approach. Each of the questionnaires included a case study, related to a chemical story, and a variety of assignments for investigating various thinking skills, as presented in Fig. 2.

Fig. 2
figure 2

The examined skills in the case based pre- and post-questionnaires

Full size image

This research focuses on the investigation of question posing skill. However, in order to determine students’ academic levels, the assignments for all the thinking skills in the pre- case-based questionnaire were accounted for. The questionnaires were analyzed in two phases. In the first, qualitative phase, we applied content analysis of students’ responses to extract categories and used them to characterize students’ responses. In the second, quantitative phase, we scored each student’s response using rubrics and analyzed the results statistically.

For each of the three years of the research, different case studies were used, but the question posing assignment was the same for all three years. Students were asked to pose two questions to which they did not find a direct answer in the case study.

Figure 3 presents an example of a case study from the case-based questionnaire in the 3rd stage of the research.

Fig. 3
figure 3

An example of a case study—3rd stage

Full size image

The students’ questions were analyzed according to a rubric we had designed, based on the question taxonomy. This taxonomy helped us determine the complexity of each question a student posed based on the anticipated response to that question. Two aspects of this taxonomy—the question content and its required response’s thinking level—had been defined and evaluated in previous work (Dori and Herscovitz 1999; 2005). The third aspect—chemistry understanding levels required for responding—is presented and utilized in this study for the first time.

Table 3 presents the rubric we used to assess students’ ability to pose complex questions.

Table 3 Rubric for assessing students’ question posing skill
Full size table

Each question is scored separately for its content, thinking level, and chemistry understanding level. The total question score is the sum of these three aspect scores. Questions at different complexity levels that four students posed after reading the patulin case study are presented next. Their content analysis and score calculation, based on the rubric in Table 3, are provided in Table 4a–d, respectively.

Table 4 Analysis of the question in (a) Example 1; (b) Example 2; (c) Example 3; (d) Example 4
Full size table

Example 1:

A high complexity question, posed by student M.

Can the patulin production be prevented through genetic engineering even before the fruit is harvested?

Student M. gained 7 out of the 7 possible points.

Example 2:

A question of intermediate complexity posed by student D.

Why does patulin become biologically inactive in a basic solution?

Student D. gained 5 points out of 7.

Example 3:

An intermediate complexity question posed by student L.

Is there a health risk in using active carbon as an adsorbing substance?

Student L. also gained 5 points out of 7.

Example 4:

A low complexity question posed by student F.

What other fruit might the patulin be found in beside apples and pears?

Student F. gained only 3 points.

The examples above demonstrate that a question can score highly in one or two aspects and get a low score in the other aspect(s). For example, analyzing the question posed by Student L (Is there a health risk in using active carbon as an adsorbing substance?) according to our taxonomy, we find out that it features a high thinking level (critical thinking) while referring to a phenomenon that was not discussed in the case study. However, the question relates only to the macroscopic understanding level. Conversely, the question posed by Student D, which requires only the relatively low knowledge and understanding thinking level, calls for a complex explanation, requiring three chemical understanding levels.

When calculating students’ scores in question posing skill, we summed the scores for the two questions the student had posed, and normalized in to a 0–100 scale.

Findings

We first present the qualitative findings based on the questions students posed about a newly presented case study they read during their interviews. We start with demonstrating the improvement in the complexity level of the questions students posed as the interview progressed. We continue with analysis of the questions’ thinking levels and chemistry understanding levels along with a description of the metacognitive strategies that students applied while posing the questions.

The interviews findings are followed by the questionnaires’ statistical analysis which includes three parts. The first part is analysis of the pre- and the post-questionnaire scores in question posing skill for the experimental group students in each of the three research stages (years). The second part is the analysis of the questions that students posed with respect to the three aspects of the question classification taxonomy: content, thinking level, and chemistry understanding levels. Finally, we compare the question posing skill of the experimental students from the 2nd and 3rd stages combined to their comparison counterparts sorted by academic levels.

Qualitative analysis: experimental group students’ interviews

We demonstrate the improvement in the questions which students F. and A. posed by comparing the questions that they posed in the beginning of the interview, before discussing the taxonomy, to the questions they posed or improved during the dialogue between the researcher and the interviewee (see Table 5).

Table 5 Improvement in the complexity levels of questions students posed during the interview
Full size table

Analysis of the think-aloud scripts of the six students as they progressed in posing questions during their interviews revealed three main metacognitive strategies:

  1. a.

    Formulating a question

  2. b.

    Analyzing a self-posed question by thinking level

  3. c.

    Analyzing a self-posed question by chemistry understanding levels.

Tables 6 through 8 present the three main metacognitive strategies of the students based on their interviews conducted by one of the researchers.

Table 6 Cognitive and metacognitive processes while formulating a question
Full size table

Tables 6 and 7 present examples of the questions posed by the interviewees, demonstrating the cognitive processes they went through. For each question, a think-aloud quotation, representing the student’s corresponding metacognitive process, is provided. The metacognitive process in Table 6 pertains to the way the student formulated the question, whereas in Table 7, the metacognitive process relates to the way the student analyzed his/her self-posed question’s thinking level.

Table 7 Cognitive and metacognitive processes while analyzing a self-posed question by thinking level
Full size table

Interpreting the students’ quotations, we found that focusing on specific sentences or changing words order yielded low-level, knowledge type questions and the strategies students elicited characterize low level metacognitive process. Summary questions posed by the interviewees required both knowledge and understanding in order for them to be answered correctly. We classified the corresponding metacognitive strategy level as intermediate. Finally, students whose strategies were identifying the central topic of the case study or extracting the essence of each paragraph posed higher order thinking questions and their metacognitive level was classified as high.

The students were able to explain the kinds of questions that were considered as simple, with no complex characteristics, e.g., yes or no questions, questions that called for a one-word answer, or questions to which the answer could be found in the text. However, student A., for example, chose to pose types of questions which required answers with detailed explanations or critical thinking. Toward the end of the interview, most of the interviewed students formulated inquiry questions, which they (and we, the authors) considered as ones requiring higher order thinking. The metacognitive process the students expressed was in line with the amount of higher order thinking required to answer the posed question.

Table 8 demonstrates how students developed their questions as well as their metacognitive processes during the interview with respect to chemistry understanding levels.

Table 8 Development of the metacognitive processes while analyzing a self-posed question by chemistry understanding levels
Full size table

As Tables 6–8 show, a student who can ask a complex question and is able to analyze the question at an intermediate or high metacognitive level with respect to the thinking level aspect (see Table 7) is not necessarily able to analyze the question at the same metacognitive level with respect to the chemistry understanding levels (see Table 8).

Experimental group students—quantitative analysis

The students’ question posing skill was analyzed in both the pre- and the post-questionnaires. Figure 4 presents the average pre- and post-scores of the question posing skill for the experimental students in all three stages.

Fig. 4
figure 4

Students’ average scores in question posing skill—three stages of experiment

Full size image

The average post-scores of the question posing skill of students in the experimental group were higher in comparison to their pre-scores. The net gain (post-scores minus pre-scores) of the experimental students in the question posing skill was analyzed for each one of the three stages of the study. The effect sizes of the net gain scores of 1st and 2nd stages were 0.7, and that of the 3rd stage was 0.6. It was significant for all the three stages (p < 0.0001).

Analysis of the three aspects in the question classification taxonomy

To gain deeper understanding of the results, we analyzed the data according to the three aspects of the question classification taxonomy: the content aspect, the thinking level aspect, and the chemistry understanding levels aspect.

We chose to present the results of the different aspects as average scores of the three stages of the research. Looking at the results of each stage separately, the average scores tended to have the same pattern and could therefore be merged into one representation of the three stages together.

The content aspect

In order to explain the significant improvement of the students’ scores in the content aspect from the pre- to the post-questionnaire, we examined the percentage of students who posed questions related to a phenomenon, compared to the percentage of the students who posed questions which dealt with hazards and solutions, which were scored higher. Table 9 shows the distribution of the students who posed questions, sorted by content, as an average percentage of the three stages of the research. It also presents examples of posed questions—content-related and hazard- or solution-related.

Table 9 Distribution of students* who posed questions sorted by content (average of the three research stages)
Full size table

The results show that in the pre-questionnaire, more than one fifth of the students did not perform the task of question posing at all, while in the post-questionnaire only 1% of the students did not pose any question. Most students posed questions related to the text directly in the pre- as well as in the post-questionnaire. The increase from 59% of the students who asked questions related to a phenomenon from the text in the pre-questionnaire to 74% in the post-questionnaire can be explained by the fact that students who did not pose even one question in the pre-questionnaire, dealt with that task in post-questionnaire and posed phenomenon-related questions. In the post-questionnaire, one quarter of the students posed questions related to possible hazards or solutions to the problem in the case study, compared with only one fifth of the students in the pre-questionnaire.

The thinking level aspect

Questions to which the answers called for use of thinking level higher than knowledge or understanding were scored higher than knowledge or understanding responses. We therefore compared the percentage of students who posed higher-order thinking questions to the percentage of students whose questions required only low level thinking in the pre- and post-questionnaires. Figure 5 presents the distribution of the students who posed questions, sorted by thinking level, as an average percentage of the three stages.

Fig. 5
figure 5

Distribution of students who posed questions sorted by thinking level

Full size image

In the pre-questionnaire, fifth of the students did not pose any question, while in the post-questionnaire 99% of the students posed at least one question referring to the case study. Examining each stage separately, in the pre-questionnaire the percentage of students who did not respond to the task ranged from 15% to 30%, while in the post-questionnaire the range was between 0% and 2%.

In both the pre- and the post-questionnaire, most of the students posed questions that required a response at the knowledge and understanding level. Stage-wise, at this level, the range was between 60% and 70% in the pre-questionnaire and between 60% and 80% in the post-questionnaire.

The number of students who posed questions requiring higher order thinking answers in the post-questionnaire—29%—was double the number in the pre-questionnaire—14%. Stage-wise, at this level, the range was between 10% and 20% in the pre-questionnaire and between 20% and 40% in the post-questionnaire. As the results indicate, asking questions that require higher order thinking responses is a highly demanding task.

The chemistry understanding aspect

Our question taxonomy was partly based on previous question analysis tools that had been used in other studies (Dori and Herscovitz 1999; Dori et al. 2003; Marbach-Ad and Sokolov 2000) in environmental studies and biotechnology high-school lessons as well as in biology college classes. Our contribution is adding the chemistry understanding levels aspect. Researches in which four chemistry understanding levels were involved, dealt mostly with chemistry students solving algorithmic exercises (Dori, Barak and Adir 2003; Dori and Hameiri 1998, 2003; Gabel 1998; Johnstone 1991). Our question taxonomy analyzes the complexity of the posed questions according to those levels in chemistry high-school lessons, where students read case studies with chemical characteristics. The chemical understanding levels aspect is novel and unique to question posing in the chemistry domain.

When examining the questions students posed by the chemistry understanding levels that are required for answering them, we focused not only on the number of chemistry understanding levels being used, but also on the different and most common combinations of those levels. Figure 6 presents the analysis of all the questions that were posed in the three stages and their distribution according to the different combinations of chemistry understanding levels. The results in Fig. 6 are presented as the percentage of all the questions posed, while the results in Table 9 and in Fig. 5 presented the percentage of students who posed questions.

Fig. 6
figure 6

Distribution of questions posed by the main combinations of chemistry understanding levels for the three stages combined

Full size image

In all the three stages of the research, more questions were posed in the post-questionnaire than in the pre-questionnaire. Many questions posed in the pre- questionnaire called for a response that required the invocation of one chemistry understanding level only—the macroscopic or the process level. In the post-questionnaire in all three stages, less questions requiring response in only one chemistry understanding level were asked, and more of these questions called for invoking the microscopic level. There was an increase in the number of questions calling for responses that require the application of three chemistry understanding levels—macroscopic, microscopic, and process. Other questions required response that had to use different chemistry understanding level combinations, but since there were only a few questions dealing with symbols, we present only the main combinations that emerged from the questions students had posed.

The effect of academic level on question posing skill

To analyze the effect of the students’ academic level on their question posing skill, we divided the experimental group population, using Duncan’s Multiple Range Test, into three academic levels—low, intermediate, and high—based on their total pre-questionnaire scores. The total score of the pre-questionnaire was calculated based on average scores of all the thinking skills examined in the CCL learning unit—question posing, chemical understanding-retention, inquiry, graphing skills, modeling, and transfer. Table 10 presents students’ net gain scores in the question posing skill sorted by academic level for each one of the three research stages.

Table 10 Experimental students’ net gain scores in question posing skill sorted by academic levels—three stages
Full size table

In the 1st and the 2nd stages of the research, students’ net gain scores in the low academic level were the highest, meaning they improved the most from the pre- to the post-questionnaire in the question posing skill in comparison to intermediate and high academic level students.

Since the improvement of students’ net gain scores was very similar in the three stages, and the most obvious differences were between low and high academic level students (except for the 3rd stage), we decided to compare the population of the experimental group students to the population of the comparison group students based on these two academic levels only. Omitting the data regarding the intermediate group for some of the findings is the reason for the lower number of students for those issues, compared with the initial number of participants.

Experimental vs. comparison group students sorted by academic levels

Figure 7 presents the comparison between the question posing skill of the experimental group students from the 2nd and 3rd stage combined and their comparison group counterparts by academic level.

Fig. 7
figure 7

Students’ average scores in question posing skill—experimental vs. comparison

Full size image

We used the General Linear Model Procedure for comparing average net gain scores of experimental students vs. comparison students in low and high academic level in the 2nd stage and 3rd stage combined. A significant difference was found in favor of the experimental group in both academic levels (low academic level t = 3.71, p < 0.05; high academic level t = 3.96, p < 0.01).

Discussion

A central role of science education and science courses should be to develop in students an appreciation for posing questions. A student question-driven classroom may reinforce students’ creativity and higher order thinking skills (Shodell 1995). Question posing is a component of thinking skills for learning tasks and a stage in the problem-solving process (Dori and Herscovitz 1999). According to Flavell (1976), asking questions about an article might improve one’s knowledge—a cognitive function, or monitor the knowledge—a metacognitive function (Flavell 1976). Improvements in the comprehension, learning and memory of material can be achieved by training students to ask complex questions (Davey and McBride 1986; Dori and Herscovitz 1999, 2005; King 1989, 1994; Palincsar and Brown 1984).

The metacognitive strategy for posing complex questions which includes a taxonomy as a self assessment tool may help the learner gain deeper understanding of the subject matter to be studied.

This study described in this paper focused on question posing by 931 honors chemistry students and on ways by which an integrated metacognitive strategy affected students’ thinking skill to pose complex questions and to analyze them according to a specially-designed taxonomy.

One of the most demanding students’ tasks in this study called for posing a question to which the answer could not be found in the adapted scientific article. The assumption was that students who are trained in question posing may also acquire heightened self-awareness of their comprehension adequacy. Generation of complex questions is expected to foster subject matter comprehension, because creating such questions mandates deep analysis of what it is that a respondent must know in order to answer these questions correctly.

Interviews analysis

We interviewed six students who had studied the CCL unit in order to understand the metacognitive processes they underwent while developing their question posing skill and practicing it with the question taxonomy.

Interviews with students revealed that students were capable of analyzing the questions they generated based on the taxonomy they had been taught. The students related in their analysis to higher order thinking questions, including inquiry questions. They tried to raise questions which do not have a clear short answer, but require detailed explanations. During the interview, students improved the questions they had posed initially. Their questions became more complex, focused and well structured, indicating increased levels of critical thinking and question posing proficiency. Our qualitative findings revealed three main metacognitive strategies the students used: formulating a question, analyzing a self-posed question by thinking level, and analyzing a self-posed question by chemistry understanding levels.

The metacognitive strategies students used in order to formulate questions assisted them in reading the case study in a meaningful way. Before responding to the different assignments in the questionnaire, the self question-posing requirement made students who read the text comprehend it deeply and more successfully. Some of the interviewed students started to pose questions only after they had understood the central topic of the case study, divided it into paragraphs, and extracted the essence of each paragraph. Even students who demonstrated a low metacognitive level tried to pose summary questions or focused on specific sentences which seemed to them as important. Question posing as a first task after reading an article can significantly contribute to students’ text comprehension and to their ability to cope with subsequent tasks.

The thinking level of the questions was part of the question posing taxonomy that the students were introduced to. Indeed, most of the interviewees referred to the thinking level criterion as an important aspect that needed to be taken into consideration when formulating a “complex” question. The students developed metacognitive strategies that helped them formulating higher order thinking questions and explaining that inquiry questions can be defined as being at a higher level than knowledge and understanding.

Since the framework of the learning unit was computerized inquiry laboratories, students were exposed to formulating inquiry questions while planning and conducting inquiry experiments. In the process of posing questions about a case study, students transferred their skills from planning experiments by setting inquiry questions to the more general task of question posing after reading an adapted scientific article.

Comprehending the four levels of chemistry understanding is an important component of meaningful understanding of the chemistry domain in general. In this research, the chemistry understanding levels aspect was included for the first time as one of the three components in the question posing taxonomy. Initially, most of the interviewees did not mention that aspect as a vital criterion for question posing and did not base their questions on the four levels of chemistry understanding. When the interviewer tried to intervene and probed the students, encouraging them to talk about chemistry understanding levels, only the high academic level student who was interviewed could correctly analyze the questions she had posed according to chemistry understanding levels.

Since our learning unit was taught in chemistry classes and the case studies that students read were characterized by chemical orientation, we emphasized chemistry understanding in our taxonomy as one of the aspects that students had to analyze in their posed questions. However, the interviews were conducted at a rather early stage of the academic year, so students were not yet experienced enough to analyze their questions according to all the aspects of the taxonomy, and especially not according to chemistry understanding levels. Statistical analysis of the post-questionnaires indicated a significant improvement in students’ questions analyzed according to the chemistry understanding levels aspect. The post-questionnaire was administered at the end of the academic year and students became more competent in posing questions to which the answers contain two or more chemistry understanding levels. Based on the interview results, we recommend that teachers work with their students on this subject more intensively.

Case-based questionnaires analysis

The statistical findings show that students significantly improved their question posing skill. The number of questions students posed in the post-questionnaire and their complexity were both significantly higher than in the pre-questionnaire. The number of students who posed questions that required higher order thinking skills in the post-questionnaire was double that number in the pre-questionnaire (29% vs. 14%).

Our taxonomy exposed the students to higher order thinking aspects, and as a result, students posed questions at application, analysis and assessment levels. The number of inquiry questions increased and students even generated a significant number of judgmental questions. A similar process occurred in the classes investigated by Marbach-Ad and Claasen (2001) and by Marbach-Ad and Sokolove (2000), where the researchers exposed their students in the beginning of the semester to a question classification taxonomy, which emphasized what “good” questions were. In the end of the semester, students’ questions were characterized by deep insights and understanding, with the best questions being inquiry ones.

A significant increase was found in the number of students who asked questions that required answers in which two or three levels of chemistry understanding had to be invoked. In Dori and Hameiri (1998, 2003), a multi dimensional analysis tool for analyzing quantitative questions according to the four levels of chemistry was described. It was based on previous findings about difficulties students experience when asked to transfer from the macroscopic level in chemistry to the atomic or molecular level.

Since students do not deeply understand the different levels, they cannot transfer from one level to another (Ben Zvi et al. 1987; Gabel 1998; Gabel and Sherwood 1984; Johnstone 1991; Nakleh 1992). The researchers claimed that since students do not deeply understand the different levels, they cannot transfer from one level to another. These findings correspond to our findings that many of the questions to which the answer required reference to the macroscopic level only were posed in both the pre- and post-questionnaires. However, the literature does not refer to the four chemistry understanding levels as a tool for analyzing students’ generated questions. Moreover, researchers have not suggested using those understanding levels as a part of a metacognitive strategy for improving students’ question posing skill.

After experiencing the chemical understanding level aspect of our taxonomy, students posed questions also at the microscopic level, relating to the atomic and molecular structure of the substance. In all the three stages (years) of the research, we observed a significant increase in questions that required answers at two or three levels of chemistry understanding.

Experimental vs. comparison group: low and high academic level students

In all the three stages, the net gain of low academic level students in question posing skill was the highest, indicating gap narrowing. This finding is in line with Dori and Herscovitz (1999) and Dori et al. (2003), who showed that both high and low academic level students improved their question posing skill after a continuous instruction via case studies. In our research, the question classification taxonomy served as scaffolding for the students, explicating the expected criteria for posing complex questions and providing them with a valuable metacognitive tool. This scaffolding assisted the low academic level students the most, helping them to improve their scores and to narrow the gap that had existed in the pre-questionnaire scores between them and the high academic level students.

Comparing the experimental with the comparison group students’ question posing skill, a significant difference was found in the post-questionnaire in favor of the experimental group for both high and low academic levels.

Being taught in the inquiry- and industry-oriented mode since 11th grade, most of the high academic level students in the comparison group experienced question posing tasks while reading articles and laboratory activities. Those students did not improve their scores in the post-questionnaire even though they continued posing questions during their 12th grade. This finding can be explained by the realization that those students did not use metacognitive knowledge in order to perform the task. In contrast, the experimental group students were exposed to question posing only in their 12th grade, but they were equipped with an adequate metacognitive strategy that helped them to self regulate their learning and to gain the insights required to pose complex questions that require higher order thinking skills in order to answer them.

Low academic level students in the comparison group improve their scores in the post-questionnaire (49) compared to their scores in the pre-questionnaire (36). Low level academic students from both research groups narrowed the gap between them and their high academic level peers.

Our findings regarding the significant improvement of the experimental group in the question posing skill in all the three taxonomy aspects are in agreement with Gourgey (1998), who argued that metacognition enables one to use knowledge strategically to perform most efficiently. Students who use metacognitive strategies, self-monitoring, self-questioning, and self-assessment are more academically successful than students who do not use these strategies. Moreover, students can be taught to improve metacognitive proficiency through repeated guided practice (Gourgey 1998).

Research limitations and strengths

Our research has limitations and strong points as well as contributions to the knowledge base of students’ metacognitive strategies and question posing.

The limitations of this study are as follows.

  1. (a)

    The number of the comparison group students was small compared with the number of the experimental group students. According to White and Arzi (2005), while loss of subjects can affect any research, the length of longitudinal studies makes attrition particularly likely. Since our study was longitudinal, we indeed faced this problem. However, since the initial number of students in the comparison was relatively low compared with their experimental counterparts, the problem was more noticeable for the comparison group.

  2. (b)

    The comparison group consisted of three sub-groups, studying via diverse instructional non-CCL methods. A more homogenous comparison group would have helped us to further validate the significance of the results obtained.

  3. (c)

    For the experimental group students only, the post case-based questionnaire served as one component of the scoring of the advanced (five units) Israeli matriculation examination—the national assessment in chemistry. This might have motivated these experimental students to invest more effort in comparison to their peers in responding correctly to their post-questionnaires.

  4. (d)

    Another possible element, the portfolio, which has also become part of the Israeli matriculation examination in chemistry (Hofstein et al. 2004). Students who are examined oraliy in the matriculation examinations for inquiry-oriented laboratory are asked to pose inquiry/research questions (Hofstein et al. 2005). The question posing scores of this comparison group might have been higher, had we included their oral performance as well.

Beside these limitations, the research features the following strong points.

  1. (a)

    We defined a set of aspects for analyzing the students’ complex questions and for providing them with a metacognitive strategy. The case-based assignments and the criteria for content analysis of students’ self-posed questions were presented and discussed extensively in this paper.

  2. (b)

    Our research has proposed the four chemistry understanding levels as a tool for analyzing students’ generated questions and as a part of a metacognitive strategy for improving students’ question posing skill.

  3. (c)

    The fact that this research was longitudinal and lasted three years strengthens the generality of the results.

  4. (d)

    The research has impacted the policy of the Israeli Ministry of Education in the sense that our case-based assessment tool brought about changes in the national chemistry matriculation examination in Israel. Students are now tested for their higher order thinking skills in addition to their content knowledge. A case-based question is embedded nowadays in the matriculation examination and students are required to pose questions, analyze graphs, demonstrate inquiry skills, and transfer between molecular representations.

Recommendations

Cognitive skills tend to be encapsulated within domains or subject areas, whereas metacognitive skills span multiple domains, even when those domains have little in common. While high levels of domain-specific knowledge may facilitate the acquisition and use of metacognition, domain knowledge does not guarantee higher levels of metacognition (Schraw 1998). Especially young students should be instructed to acquire metacognitive skills in various domains and, subsequently, to apply those skills across the boundaries of tasks and domains. Such repertoire of general metacognitive skills may help them to manage new, unfamiliar tasks that are initially beyond their grasp (Veenman and Spaans 2005).

In agreement with Veenman and Spaans (2005), we believe that the question posing skill can cross domain boundaries, so the metacognitive strategy our chemistry students acquired can assist them in understanding other domains more deeply. Reading articles becomes a popular instruction method in several scientific domains, and posing questions about an article can help students understand the text and summarize its main ideas. The higher order thinking aspect in our taxonomy is a general aspect that concerns all subject matters, while the chemical understanding aspect is domain-specific and may assist chemistry students while reading a chemical article. Therefore, we recommend that science teachers use more authentic and up-to-date adapted scientific articles in their instruction and scaffold their students with the metacognitive strategy for question posing. Assisted by the taxonomy, students will be able to pose questions of higher complexity about the scientific article they read.

Questions generated by students promote active thinking and learning more than those created by teachers (Aldridge 1989; Hartman 1994; Paris and Myers 1981). It appears that self-questioning serves a form of self-testing that helps the learner to monitor understanding of the material presented. Learners who use self-questioning focus on the important aspects of the material they read. They analyze the content, relate it to prior knowledge, and evaluate it in a continuous questioning-answering-questioning cycle (Notle and Singer 1985; Palinscar and Brown 1984).

Marbach-Ad and Sokolove (2000) suggested that even if teachers feel uncomfortable about not covering enough content material in an active learning classroom, they should try as much as possible to encourage thoughtful student questions because they can be indicators of student thinking. To continue that line, we suggest that question posing becomes a part of the formative assessment of the students by their teachers in the classes. If a teacher invests a lot of precious time training his students to pose complex question, she or he needs also to include that skill in the assessment. As our findings indicate, low academic students are the ones who gain the most from such an assessment (See also Dori and Herscovitz 1999). The conjecture of this research is that high utilization of scaffolding that the metacognitive strategy provides, enables the low level academic students to narrow the gap between them and high academic peers.

We attribute the improvement in students’ questions posing skill to the metacognitive strategy to which they were introduced, giving them more control over their learning. Therefore, we recommend that science teachers and students will be exposed to our metacognitive strategy for generating complex scientific questions. This metacognitive strategy will enable them to be more aware of their own cognitive processes, thereby be able to better self-regulate their learning.