Introduction

Nature of science (NOS) has long been recognized as vital to the development of scientific literacy (Allchin, 2014; Kruse et al., 2017; Lederman et al., 2013). Clough (2007b) wrote:

The phrase ‘nature of science’ is often used in referring to issues such as what science is, how it works, the epistemological and ontological foundations of science, how scientists operate as a social group and how society itself both influences and reacts to scientific endeavors. (p. 31).

Understanding NOS has been argued to help students to value science (Tobias, 1990) and effectively evaluate scientific claims for the purpose of decision-making (Allchin, 2011; Karisan & Zeidler, 2017; Khishfe, 2012).

Unfortunately, despite the inclusion of NOS ideas in standards documents, teachers continue to struggle to teach NOS, even when they themselves have an adequate understanding of NOS (Bell et al., 2000; Pavez et al., 2016; Supprakob et al., 2016). While research has shown an explicit-reflective approach to NOS instruction to be effective in increasing student understanding of NOS (Akerson et al., 2007; Khishfe & Abd-El-Khalick, 2002; Schwartz et al., 2004; Yacoubian & BouJaoude, 2010), there is a need for more detailed and specific guidance on how to enact such instruction (Kruse et al., 2020). Because questioning is often touted as an effective way to guide student reflection (e.g., Chin, 2007; Oliveira, 2010; Van Zee & Minstrell, 1997), this paper first explores literature on questioning, explicit-reflective NOS instruction, and use of questions to enact explicit-reflective NOS instruction. We then report on an investigation to determine how students respond to different question types and discuss potential implications for NOS instruction.

Student-Centered Approach and Question Forms

Students have different expectations for classroom interactions based on the communicative approach of the teacher (Chin, 2007). Mortimer and Scott (2006) described communicative approaches as, “how the teacher works with students to develop ideas in the classroom” (p. 206). When a teacher takes an authoritative approach, they exercise control to ensure that a discussion follows a certain predetermined course toward a specific endpoint. The teacher takes responsibility for evaluating and elaborating on student responses, and the result is that student responses are shorter because there is little incentive for students to provide complex responses to teacher prompts (Pimentel & McNeill, 2013). In contrast, with a student-centered approach, a teacher gives students significant control over the direction of a discussion by using student ideas as prompts for further discussion and by increasing opportunities for students to talk. Student-centered questions, then, function to promote student voice.

Questions that promote student-centered classrooms are generally characterized as open-ended. Oliveira (2010) found that open-ended, student-centered questions yielded longer, better articulated student responses and showed more evidence of higher level thinking when compared to teacher-centered questions. Open-ended questions have also been associated with effective constructivist teaching (Erdogan & Campbell, 2008), increased student argumentation (Martin & Hand, 2009; McNeill & Pimentel, 2010), understanding of scientific explanations (Braaten & Windschitl, 2011; Windschitl et al., 2008), and conceptual change (Konfetta-Menicou & Scaife, 2000). Yet, even “why” or “how” questions can be closed if teachers are unwilling to accept multiple answers (Blosser, 1991).

While open-ended questions clearly have benefits for student thinking (Oliveira, 2010), we sought a more detailed system to classify questions beyond just open or closed. Therefore, for this study, we used the system developed by Gallagher and Aschner (1963) to classify questions according to the type of thinking expected to be elicited (see Table 1). In Gallagher and Aschner’s (1963) system, cognitive-memory questions entail recall of factual information, convergent questions are structured to lead students to respond with a particular viewpoint, divergent questions allow students freedom to respond with a variety of viewpoints, evaluative questions require students to judge an idea, and routine questions function as tools for classroom management.

Table 1 Gallagher and Aschner’s (1963) question types and definitions
Full size table

Regardless of the form of questions, science teachers often use discursive moves in an effort to shape student language to more closely resemble that of scientists (Chin, 2007; Soysal, 2020). Chin (2007) describes this collaborative process of knowledge construction as first eliciting what students think, encouraging them to elaborate, and then helping them construct new understandings (p. 808). While such approaches more accurately reflect the work and thinking of scientists, an implicit approach to NOS instruction is not enough.

Explicit-Reflective NOS Instruction

Instead of an implicit approach, an explicit-reflective approach, wherein NOS is treated as a learning outcome and planned for similar to other instructional objectives, has proved more successful in changing student conceptions of NOS (Abd-El-Khalick & Lederman, 2000; Khishfe & Abd-El-Khalick, 2002). Explicit-reflective instruction should not be thought of as direct or didactic instruction (Lederman & Lederman, 2019). Khishfe and Abd-El-Khalick (2002) pointedly explain that the “explicit” in explicit-reflective, “should not be confused with didactic teaching. An explicit and reflective approach does not entail drilling students to reiterate certain generalizations about the nature of scientific knowledge.” (p. 554). Rather, Lederman et al. (2019) emphasized the student-centered nature of the explicit-reflective framework:

we take for the term “reflective” to refer to the act of student contemplation, of mentally struggling with the meaning of their learning experiences. Thus, one might identify “reflective” teaching by asking if the instruction is student-centered and requires careful and extended thought (p. 201).

Questions to Enact Explicit-Reflective NOS Instruction

Despite the importance of explicit-reflective instruction, there remains some lack of clarity in regard to how such instruction can be implemented (Kruse et al., 2020). Questions are often used to generate the explicit-reflective discussions so crucial to NOS learning (Abd-El-Khalick, 2013; Kruse et al., 2020; Kruse & Borzo, 2010; Clough, 2007b, 2011, 2020a; Herman et al., 2013; Lederman et al., 2019). Yet, only recently has the formulation of questions received empirical investigation. Kruse et al. (2020) investigated the nature of student responses to different question types (i.e., general NOS questions and questions that target specific NOS ideas). Although their study investigated the role of specific and general NOS questions, all of the questions used by Kruse et al. (2020) were of a similar open-ended structure.

In reviewing the NOS literature, we noticed that the questions used as examples seemed to vary in terms of the constraints they placed on student thinking about NOS. Some questions simply pointed students’ thinking toward NOS in a general way. For example, the question “How were we being like scientists in what we just did?” (Fouad et al., 2015, p. 1118) prompts students to make connections between classroom experiences and NOS without specifying a particular NOS idea to be addressed. In contrast, a question like “How can scientists make their conclusions when they only have limited data?” (Khishfe, 2008, p. 477) specifically encourages students to consider the limitations of science. Other NOS questions required students to justify particular NOS ideas, for instance, the question “How does the notion of a universal step-by-step scientific method distort how scientists actually work?” (Clough, 2020a, p. 273) directs students to explain why scientists do not use a single scientific method. Lastly, some NOS questions prompted students to choose between two positions, as in “Do you think that Mendel’s personal bias played a role in the selection of the experimental plants and characters of peas? Why or why not?” (Kim & Irving, 2010, p. 196).

Given that NOS questions often seemed to differ in the degree to which they directed student thinking to a particular idea or perspective, Gallagher and Aschner’s (1963) question category system discussed previously can be used to classify example NOS questions found in the literature (see Table 2). After categorizing a sample of NOS questions from the literature according to Gallagher and Aschner’s system, the authors confirmed that there was significant variation in the types of NOS questions being offered as examples in the literature (see Table 2). The categories of “cognitive-memory” and “routine” were not observed in the NOS literature. Although Clough (2020a, 2020b) identifies vague open-ended, or divergent, questions as problematic and argues that more direct, educative questions provide the necessary guidance to draw students’ attention to more informed and nuanced NOS views, we found no clear attempts to investigate the effect of NOS question types other than Kruse et al. (2020). However, that study compared general and specific NOS questions rather than divergent, convergent, and evaluative questions.

Table 2 Example questions from NOS literature categorized by question type
Full size table

Purpose of Study

Although questions are often recommended to enact explicit-reflective NOS instruction, little guidance about questions is given in the literature. Indeed, several different forms of questions can be found, but are not differentiated in a meaningful way. Given the variable forms of NOS questions being asked in NOS literature, and the uncertainty as to how students respond to different question types (Dillon, 1982), the authors sought to examine how NOS question type (i.e., convergent, divergent, evaluative) might influence student responses. Specifically, we sought to answer: To what extent is there a statistically significant difference in the proportions of the kinds of student responses to various question types?

Methods

This study sought to use a quantitative experimental methodology to investigate student responses to NOS question types. Thus, participants were randomly assigned to various treatment conditions and data was coded into discrete categories so that non-parametric statistics could be used to compare across treatment groups.

Data Collection and Participants

For this study, the authors chose to target the NOS idea that not all science follows a linear scientific method. As McComas (2004, p. 25) noted, “Knowledge production in science includes many common features and shared habits of mind. However, in spite of such commonalities there is no single step-by-step scientific method by which all science is done.” A reading was created (see Appendix) to showcase a variety of approaches to scientific investigation. This reading consisted of short descriptions of the work of four scientists, each from a different field of study. These scientists were chosen because they pursued their ideas without the use of traditional experiments. The reading was at a 6.2 Flesch-Kincaid grade level (Kincaid et al., 1975; Stockmeyer, 2009), appropriate for sixth graders near the end of their school year.

Four different versions of the data collection tool were created, each with the same reading text, but a different question type (see Table 3) as determined using Gallagher and Aschner’s (1963) question category system. Because we found no literature recommending simple recall of NOS ideas, the cognitive-memory category of question was excluded and a general reflective question was added to serve as a non-NOS control. Below the question, the remainder of the page was left blank to provide space for students to respond.

Table 3 Questions used in data collection tool as categorized by question type
Full size table

To ensure that this study was ethically conducted, approval was sought from the university Institutional Review Board. The university Institutional Review Board (IRB) determined this project to be IRB exempt because the study involved data collected as part of normal educational activities. Furthermore, because only one piece of data was collected from each student, no student names or identifying information were used in the study.

The data collection tool was piloted at a private middle school located in the same metropolitan area as the main study. Forty-eight sixth, seventh, and eighth graders, all students of the same science teacher, participated in the pilot study. Students were randomly assigned a form of the tool and asked to do the reading and then respond to the prompt in writing. Pilot data was assessed to determine if students understood the reading and the questions. No analysis was done to compare student responses to the various questions types. Following the pilot study, prompts were adjusted to improve clarity.

The final study was conducted in a Midwestern United States of America suburb with a population of approximately 54,000. The district in which the study took place serves 11,000 students, 86% of students identify as White, 2% as Black, 4% as Hispanic or Latino, 6% as Asian, and 2% as two or more races. A total of 16% of students in the district qualify for free and reduced lunch. The study took place in the sixth grade science classrooms of one of the district’s two middle schools. A total of 285 students from the classes of three different sixth grade science teachers participated in the study. At the time of the study, the students were approximately 1 month away from completing their sixth grade school year.

On the day of the study, students were given a form of the data collection tool at random. Students were instructed to do the reading and individually respond to the corresponding prompt in writing. While teachers did read the story out loud, they did not read the question out loud as not all students had the same question. Students were informed that their writing would not be graded and were asked to do their best work. Teachers were instructed not to answer any additional student questions regarding the prompt or the content of the reading.

Data Analysis

Student responses were coded using the provisional codes (Saldaña, 2015) from Kruse et al. (2020): normative, descriptive, misconception, and off-topic. A second researcher coded 80% of the student responses. Intercoder agreement was 84%. Rather than resolving differences, the second coder was used to establish reliability of the first coder. Intercoder agreement above 80% is generally considered acceptable (O’Connor & Joffe, 2020). To more easily and accurately compare across treatment groups, qualitative codes were converted to quantitative numbers by simply counting the number of participants in each treatment group (i.e., general reflective, convergent, divergent, evaluative questions) that responded with each code (i.e., off-topic, misconception, normative, descriptive). Once data were quantified in this way, the percentages of response types for each question type were analyzed using a Chi-square test for independence to look for significant differences in the proportion of student response types across question types. More detail regarding response type coding appears below and in Table 4.

Table 4 Explanations and example quotations for student response codes
Full size table

Explanation of Provisional Codes

In terms of this study, “normative” refers to the tendency to describe things in terms of ideals or overgeneralizations. A normative response describes things as they should be rather than how they are. For this study, a response was coded “normative” if it expressed the view that scientists do not follow the scientific method without acknowledging that scientists are sometimes bound by rules, procedures, or conventions. For example, one student wrote, “Each scientist is different. Which means their ideas will be different.” Another student claimed, “They have to try lots of different methods before they are confident in their ideas.”

In contrast to a normative view, a descriptive view demonstrates greater complexity and nuance. Student responses identified as “descriptive” recognize that scientists do not necessarily follow the same method, but may sometimes follow prescribed procedures. These students did not completely dismiss the idea of scientific method and often noted that there are some things that scientists might do the same. However, the students seemed to recognize that not all scientists do the same things all the time. They often wrote that scientists “sometimes,” or “kind of,” use the scientific method. For example, one student responded, “they [scientists] can use the same step by step method if it works, but sometimes it’s better to use a variety of ideas/methods.”

For this study, “misconception” refers to an idea that is contrary to accepted NOS ideas. Responses were coded as misconceptions if students only noted scientists follow a procedural scientific method. For example, one student wrote, “Scientists do use step by step because if they don’t they will not go in that order and it will mess everything.”

Any student response that did not refer to scientific method was coded as off-topic. Responses in this category often referred to specific aspects of the reading or NOS ideas other than scientific method. For example, one student wrote, “shark teeth can look like stone.”

Results

A Chi-square test for independence indicated significant association between NOS question type and student response type, X2 (9, n = 285) = 141.36, p = < 0.001, Cramer’s V = .407. Given the p value and that Cramer’s V indicates a large effect size (Gravetter & Wallnau 2004), we conclude that question type does impact the proportion of response types observed in our study. Follow-up pairwise comparisons were conducted, and all comparisons were statistically significant using a Bonferroni adjusted p value of 0.008. Table 5 displays the percentages of each response type for each question type. Percentages were rounded to a whole percent.

Table 5 Count of student responses by code and question type
Full size table

For the general question (i.e., What do you notice or want to remember?), participants tended to respond off-topic with 60% of responses coded off-topic. Often, these off-topic responses focused on specific aspects of the reading. For example, one student wrote, “I remember Einstein investigating light and a guy studying fossils.” A relatively large percentage of students (36%) provided responses that were coded normative, such as “I think scientists don’t always use the same step-by-step method when doing experiments because everyone is different and you can’t use the same method on everything. Everyone thinks and acts different, so it just wouldn’t be realistic.” Few responses to general questions were coded misconception (2%) or descriptive (3%).

Responses to the evaluative question (i.e., Do you think scientists follow the same step-by-step scientific method and always use experiments?) were largely normative with over half (59%) of responses to this question type coded as such. An additional 24% of responses to the evaluative question were coded misconception. For instance, one student wrote, “They have to experiment it or no one will believe them.” An additional 18% of responses were coded descriptive, such as “I think scientists should follow certain steps to get their testing done efficiently, but the steps can be different. There is no certain way to do tests.” No responses to the evaluative question were coded off-topic.

Responses to the divergent question (i.e., What do you think about scientists’ following the same step-by-step scientific method and use of experiments?) were relatively more evenly distributed across response codes: 18% of responses were coded off-topic, 34% were coded misconception, 28% were coded normative, and 20% were coded descriptive. Out of all question types, the divergent question yielded the highest percentages of both descriptive responses and misconceptions. One misconception that seemed to appear repeatedly was the idea that a step-by-step method yields more accurate results. For instance, one student wrote, “I think that it’s a good way to investigate because it will give you the most accurate answers.”

Responses to the convergent question type (i.e., Why do you think scientists do not follow the same step-by-step scientific method and do not always use experiments?) were overwhelmingly normative. Three-quarters (75%) of student responses to the convergent question were coded normative. For instance, one student wrote, “Different scientists come up with different ideas all the time, sometimes even by accident. So scientists don’t always follow the same step-by-step methods or experiments.” Very few responses to the convergent question were coded off-topic (3%) or misconception (5%). An additional 18% of responses were coded descriptive, as in, “They use the same steps, but not the completely same process.”

Discussion and Implications

While it is not surprising that the form and wording of a question influences student responses (Chin, 2007; Clough, 2020a; Martin & Hand, 2009; Oliveira, 2010), the results of this study provide insight as to how questions can be used to enact explicit-reflective NOS instruction. Given our results, several recommendations can be made for enacting NOS instruction. Perhaps teachers can leverage different questions to achieve different pedagogical goals.

General Reflective

The majority of responses to the general question type did not address NOS. This trend is likely because the general prompt was the least topically constrained; students were invited to include anything that caught their attention while reading (e.g., What do you notice or want to remember?). Consequently, many students did not refer to scientific methodology in their response, often commenting on specific facts from the story or general impressions. This finding confirms the results of other studies that demonstrated students often do not attend to NOS ideas if they are not explicitly addressed (Akerson et al., 2000; Akerson et al., 2007; Khishfe & Abd-El-Khalick, 2002; Schwartz et al., 2004; Yacoubian & BouJaoude, 2010). When this study’s questions did not explicitly address a NOS concept, many students ignored NOS in the story. Therefore, when seeking to teach NOS, general reflective questions seem to be of little value.

Evaluative

Evaluative questions in the NOS literature were often observed to be dichotomous yes/no questions (e.g., Do scientists follow the same step-by-step scientific method?). Given this structure, one might expect responses to be split between normative and misconception codes. Observed percentages were 59% and 24%, respectively, perhaps because the reading was designed to illustrate how scientists utilize multiple methods. An additional 18% of responses were coded descriptive, and no responses were coded off-topic.

While in this study an evaluative yes/no question yielded a variety of useful responses (only 3 responses consisted of a simple “no”), a potential problem with yes/no evaluative questions is that students do not always choose to elaborate and may simply answer “yes” or “no.” Such responses are problematic because there is a range of possible reasonings for the responses (Wiśniewski, 2006), and a simple “yes” or “no” may mask the complexity of students’ thoughts and prevent teachers from assessing student thinking. Consequently, teachers may need to ask additional questions to probe student responses.

Divergent

Whereas the evaluative NOS question seemed to offer students only two opposing views, the divergent NOS question was open to responses addressing a range of views. While the divergent question in this study targeted one specific NOS idea (i.e., What do you think about scientists’ following the same step-by-step scientific method and use of experiments?), many of the divergent questions in the literature were more general; they often asked students to reflect on how some reading or classroom activity might generally relate to the discipline of science (Kruse et al., 2020).

Responses to the more specific divergent question in this study were relatively evenly distributed across response codes, possibly because the structure of the divergent question was somewhat less biased toward a normative view, and yet topically constrained enough to persuade students to discuss the intended NOS idea. Given the results of this study, a divergent question might be useful to teachers when they are looking to survey the landscape of student ideas about a specific NOS idea, possibly at the beginning of a unit or lesson. Based on the data from this study, students will likely respond to the question with a variety of positions, thus providing the teacher with information about how students are understanding the concept.

Convergent

Convergent questions in the NOS literature were generally biased toward NOS consensus views. Rather than allowing students to choose their position on an NOS idea, convergent questions often state a NOS view and ask students to justify. For example, the underlying assumption behind the question, “Why do you think scientists do not follow the same step-by-step scientific method?” is that scientists do utilize different methods. In responding, students need only explain why. Responding to this question with the misconception that scientists do follow the same step-by-step method would require a student to go against the structure of the question. Unsurprisingly, the majority of student responses to the convergent question were coded normative (75%).

As the convergent question in this study yielded largely normative responses, it seems the most appropriate choice for supporting and scaffolding students away from misconceptions and toward more accurate NOS ideas. Teachers might utilize divergent questioning to get a sense of student ideas about a NOS idea. Then, teachers might engage more convergent questions to lead students to desired NOS understandings.

Implementing NOS Questions

Although this study demonstrates that convergent and evaluative questions tend to yield normative responses from students while divergent questions yield a greater diversity of response types, additional factors ought to be considered for classroom instruction. That is, it is difficult to identify one “best” type of question (Ramsey et al., 1990) for NOS instruction. However, Scott et al. (2006) demonstrate how teachers’ communicative approach might vacillate between authoritative and dialogic throughout the course of a lesson:

Teacher encourages dialogic discourse to probe students’ everyday lives; later she adopts an authoritative approach to introduce the scientific point of view; then she prompts dialogic discourse as she encourages students to explore and apply the scientific view, and so the shifts in communicative approach continue throughout the lesson sequence (p. 623).

To enact such back-and-forth during NOS instruction, we often try to first make visible a variety of student views by asking divergent questions (e.g., What do you think about scientists’ following the same step-by-step scientific method?). Once we ascertain student thinking, and provide some experiences, implementing more convergent questions (e.g., Why do you think scientists do not follow the same step-by-step scientific method?) help us guide students to more accurate (often normative) understandings of NOS, particularly when misconceptions are present. However, we do not intend for students to stop thinking at normative views. Instead, we again use convergent questions to guide to more nuanced (descriptive) views of NOS by raising new questions and skepticisms about normative views (e.g., How might scientists conduct their work in similar ways?). Then, in later assessments, we might pose another divergent or maybe an evaluative question to assess student learning (e.g., Do you think scientists follow the same step-by-step scientific method? Support your answer with examples.).

Clough (2020a) would describe the example questions in the previous paragraph as educative. That is, the questions draw students’ attention to NOS ideas and push them toward nuanced understanding. Yet, based on the results of this study, the educative nature of the questions is enhanced by leveraging divergent, convergent, and evaluative questions in strategic ways. In addition, considering the level of specificity of NOS questions can increase their educative value (Kruse et al, 2020; Herman et al. 2013). General divergent NOS questions are sometimes used to begin NOS discussions (e.g., How has class today been similar to the work of real scientists?). Such questions can help use student thinking and student observations in productive ways. However, in a recently published study, Kruse et al. (2020) illustrate that such general questions tend to elicit relatively few NOS ideas. Instead, more specific questions that target particular NOS ideas help students address a wider range of NOS ideas (e.g., What does this story illustrate about how science knowledge is modified?). Combining specific NOS questioning strategies with divergent, convergent, or evaluative approaches as discussed above will further help expand teachers’ tools for probing student thinking about the NOS.

Given the results of this study, some of the question types examined may be more suited to certain NOS instructional purposes and circumstances than others. However, we recognize different question types do not result in only one kind of response type. Therefore, we seek to remain flexible in our responses to students and work to consistently understand and guide student thinking. Clough (2007a) explained, “Questioning is key for teasing out what students really think, helping them see the inadequacy of misconceptions, and piecing together a more accurate understanding” (pp. 3–4), and the results of this study provide additional insight with respect to how particular question types might be leveraged to enact effective NOS instruction.

Limitations

This study is limited because it examines the responses of a relatively homogenous group of students to individual questions about a particular NOS idea. Students of different ages and backgrounds may have responded to the prompts differently. Additionally, responses in this study may not be representative of responses in a whole-class verbal discussion or to questions asked about other NOS ideas. Prior knowledge, previous contributions, power dynamics, and reactions are just a few factors that contribute to how students respond to questions (Carlsen, 1991). Therefore, application of this study to the context of classroom discussion should be done with care.

Future Research

Although this current study and a recent study by Kruse et al. (2020) add nuanced evidence-based guidance for the enactment of explicit-reflective NOS instruction, continued investigation could illuminate ways to support NOS teaching and learning. Future research might utilize questions about different NOS ideas, explore the responses of students of different ages and backgrounds, or investigate the use of convergent, divergent, and evaluative questions within the context of verbal NOS discussions. This emphasis on teacher-student interactions and discourse may be a fruitful avenue for continuing to improve NOS instruction. For instance, Piliouras et al. (2018) found that in-service teachers were able to teach NOS more effectively after utilizing discourse analysis to reflect on their own classroom talk about NOS. While many scholars have investigated discourse in the science classroom (e.g., Chin, 2007; Oliveira, 2010), more focused work on NOS classroom discourse, including the role of question type, may provide additional guidance for science teachers to more effectively include NOS in their teaching.

Conclusion

NOS researchers have known for a long time that we can help teachers understand NOS. Yet, helping teachers enact NOS instruction has been challenging. We hope this study provides additional guidance to teachers and teacher educators about how to leverage different types of explicit-reflective questions to enact effective NOS instruction.