Student evaluations are the bane of many a faculty member’s existence and an identified source of real faculty anxiety (Ameen, Guffey, & Jackson, 2002; Gardner & Leak, 1994). As instructors we invest time, energy, and self in our teaching. We naturally expect our students to appreciate how we have taught them and reward our “performance” with good marks. When these expectations fail to materialize, we are disappointed, confused, and angry. Our reaction can lead us to discount these sources of information and may even contribute to our disillusionment with the value of our work. Many authors have written about the various factors that affect numerical ratings on student evaluations (for a number of references see Theall, Abrami, & Mets, 2001). Conversely, written student comments are often hard to interpret (Lewis, 2001) and appear idiosyncratic and anecdotal, lacking the heuristic power of numerical statistical data. They frequently reflect students’ affective issues, such as their perceptions of our interactions with and interest in them. Sometimes, however, we receive statements that give us valuable insight into intellectual challenges common to novice learners in our field, ones that are likely to recur from semester to semester. In this article we offer another lens through which to view students’ written comments on evaluations—one that brings the scholar’s eye to these “data” as a source of insight into student learning challenges. Using these comments as part of a scholarly approach to teaching can lessen our anxiety in reading them, form the basis for effective changes in our teaching approach, and may inform our thinking about curricular issues as well.

Recognizing the Importance of Personal Epistemology in Learning

As instructors, our idea of learning tacitly encompasses a complex array of cognitive processes: retaining and understanding content, analyzing information, synthesizing ideas, evaluating points of view, and creating new knowledge. For novice learners, however, these practices are difficult. Not only may they not understand the process of learning in the discipline, but also they may not see learning as a process at all. That is, students may have immature beliefs about how learning happens or how knowledge is created, not recognizing how tentative, iterative, and effortful a process it is. For instance, students may not realize that even experienced and competent readers struggle to understand difficult texts (Blau, 2003) or that expert problem-solvers may require a long time to work through a problem (Schoenfeld, 1985). These misperceptions may impede students’ intellectual development in our classes and may contribute in part to negative comments on our evaluations. Below we discuss some of the pertinent research on the development of students’ epistemological beliefs, beliefs that underlie and inform their intellectual development (see Bendixen & Rule, 2004, on the connection between epistemological beliefs and conceptual change). We then examine common student comments on evaluations from three specific situations: quantitative classes, writing-intensive courses, and classes taught with student-active formats. In each case, we maintain that the student comments on our performance—on the way we have designed exams, graded essays, and conducted class—reveal learning challenges that arise from their beliefs about learning itself.

Some of the early ground-breaking research in the area of students’ intellectual development is that of Perry (1999, originally published 1968). Almost 40 years ago he conducted in-depth interviews with Harvard men and identified nine stages of intellectual development that may be condensed into four general areas. In dualism students exhibit a right/wrong approach to knowledge. Students in this stage typically view instructors as all-knowing authorities and perceive their role as students to be receiving this knowledge from instructors and repeating it back at appropriate times. As students develop, they typically enter a stage he described as multiplism. They begin to recognize that some important questions do not have clear right or wrong answers. As a consequence they may think that since some knowledge is uncertain, all views or opinions are equally valid. They may be confused by instructors’ criticism of their work, assuming that it is based on personal whim.

In the stage known as relativism students begin to recognize how to use reliable information to make informed decisions. They perceive an instructor as an expert resource or consultant on disciplinary methods of analysis and their role as students as not just knowing facts but applying knowledge in different contexts and making conclusions based on evidence. Finally, students come to see knowledge as constructed and decisions as contextual; and they recognize the need to make choices based not only on informed judgment but also personal values—a stage known as commitment within relativism.

Perry’s work has been extended and modified in studies of both men and women learners and in diverse educational settings (for a comprehensive review see Hofer & Pintrich, 1997). His work contributed significantly to the field now known as personal epistemology, which concerns students’ beliefs about the nature of knowledge and how one comes to know. Models in this field now extend beyond Perry’s idea of discrete stages and include multidimensional, contextualized, and integrated theories of beliefs; but most models do acknowledge the sequence of absolutism-relativism-evaluativism in students’ epistemological development (Bendixen & Rule, 2004).

Most models of epistemological development are consistent with the constructivist theory of learning (Bendixen & Rule, 2004; Hofer & Pintrich, 1997; King & Kitchener, 1994). A common theme of constructivism is that meaning is actively constructed by the individual through his or her encounter with ideas and experiences (Piaget, 1970), and, according to social constructivists, through social interactions and influences (Vygotsky, 1978). In constructivist theory, metacognition, the ability to think about and monitor one’s own thinking, is essential to developing new conceptual understandings. Some versions of constructivist theory draw on the research in situated cognition (Brown, Collins, & Duguid, 1989; Lave, 1988), which emphasizes the importance of students working together with an expert or other learners on authentic tasks in relevant situations to maximize learning. Pedagogical approaches such as peer learning and collaborative or cooperative learning are based on constructivist theories of learning.

The integrated model of personal epistemology proposed by Bendixen and Rule (2004) illustrates a constructivist approach to development. They suggest that the evolution of epistemological beliefs over time has three interrelated elements: epistemic doubt, epistemic volition, and resolution strategies. We might describe these three facets as: questioning one’s beliefs and weighing evidence against them; confronting discrepancies and taking responsibility for one’s beliefs; and reconciling these discrepancies, often through interaction and reflection with others. Metacognition thus plays an important role in their model of epistemological development. They propose, in fact, that the more metacognitively aware a student is, the more lasting belief change may be (Bendixen & Rule, 2004, p. 74). In what follows, we discuss comments on student evaluations that reflect students’ beliefs about learning. In each case, we propose ways to encourage students to examine and reconceptualize their beliefs about how knowledge is made. Finally, we make recommendations for how to supplement or rewrite student evaluations to elicit more meaningful comments from students about our teaching and their learning.

A Common Issue in Quantitative Classes

Those of us who teach classes that include quantitative problem solving, such as economics, math, chemistry, physics, and other sciences, have probably had a remark on our evaluations similar to the following:

Problems on the exam were nothing like those in class/problem sets.

Although this comment may reflect some oversight on our part or our deliberate attempt to challenge students beyond class material, more likely it is a statement that we find puzzling and to some level irritating. We probably spent valuable time in class solving problems that we consider virtually identical to the ones on the test and asked students to practice such problems on homework. These remarks may lead us to assume that students are not paying attention, are not spending enough time on assignments, or simply are not studying hard enough. In some cases these assumptions may be true, but another interpretation is that these student comments reflect the differences in the way novices approach problem solving compared to experts.

Research on problem solving indicates that effective problem solving is a complex cognitive activity, one that requires disciplinary knowledge, a repertoire of strategies, a recognition of how one uses what one knows, and constructive beliefs about the process (Schoenfeld, 1985). Good problem-solvers typically think through the processes they are using, feeling free to explore, adapt, and reject various paths. Novice problem solvers often have simplistic beliefs about the nature of problem solving, e.g., that there is an equation for every problem and that all problems can be solved quickly (Schoenfeld, 1985). Novice learners often remember problems based on very specific learning prompts. For example, novice learners in physics often classify problems according to the kinds of objects involved, e.g., levers or pulleys, rather than underlying principles (Chi, Feltovich, & Glaser, 1981). College students typically rush to an answer, spending very little time thinking through various choices of procedure (Schoenfeld, 1985). They focus the importance of problem solving on the answer rather than the process.

As instructors we often consider quantitative problem-solving ability to be an indicator of student’s conceptual understanding. The approaches that novice learners frequently use in problem solving, however, do not support that understanding. In one recent study of university students in physics, conceptual understanding did not correlate well with problem-solving ability, even for students who had solved more than 1,000 problems (Kim & Pak, 2002). How do we help students focus on the mindful processes involved in productive problem solving, especially identifying and analyzing the principles that connect superficially different problems?

Our problem-solving exercises for students must explicitly require them to spend meaningful time analyzing principles involved and envisioning how these same principles might be “disguised” in other settings. For example, mixing up problems from different textbook chapters on assignments requires students to think through each problem rather than searching a given chapter for a pertinent equation. Helping students “conditionalize” their learning—that is, recognize when ideas are applicable—is an essential part of their developing meaningful problem-solving skills (Bransford, Brown, & Cocking, 2000, p. 19).

Another strategy that requires students to take a more thoughtful approach to their problem solving is to have them annotate their work on selected homework problems; this strategy, in essence, has them “talk” their way through the problem. This approach can be especially effective if students indeed talk aloud, explaining their reasoning to a partner (Schoenfeld, 1983) or to others in a group. The interaction with others helps students see each problem from different perspectives, developing their ability to recognize similarities among differences and common concepts within pertinent equations. These activities also make the decision-making process overt in ways that working alone does not. The metacognitive awareness that develops as students talk aloud about problem-solving may form part of the basis for the success of peer learning (Mazur, 1997) and collaborative learning (Springer, Stanne, & Donovan, 1999) in improving achievement in science courses. Providing time and incentives for students to look back at their problem solving reinforces their thinking about the principles they used and the choices they made, key elements that highlight problem solving as a process in and of itself.

A Common Issue in Writing-Intensive Classes

Those of us who assign writing tasks in our classes may establish and share with our students clear grading criteria that reward what we value in analytic writing: a thesis-driven argument. But despite this, we may still see the following complaints on our evaluations:

It might help if we knew what we will be graded off on.

I felt like the grading was unjustified.

Students’ intense focus on grades may be only one explanation for these kinds of statements. These comments no doubt allude in part to the difficulty inherent in explaining our grading choices to students in written or oral feedback on their writing. But the perception of being graded “off” or graded unfairly may also reveal a more serious misconception of what is being evaluated: not the ability to convey information (or the “right” information), but the ability to construct an interesting, persuasive argument. This misperception, in turn, may signal a failure to come to terms with the epistemological framework that college-level writing assumes. Analytic writing assignments ask our students to treat knowledge as produced rather than simply received and reproduced. More precisely, such assignments ask them to see knowledge “as dialogic, contingent, ambiguous, and tentative” (Bean, 1996, p. 18). John Bean explained that by requiring our students to support a thesis—which implies a counterthesis—we are asking them to take “a complex view of knowledge in which differing views about the nature of truth compete for allegiance” (Bean, 1996, p. 18). As Bean suggested, this demand may be especially difficult for novice learners, who are often what Perry called dualistic thinkers (Perry, 1999).

“To dualists, the only academic use of writing is to demonstrate one’s knowledge of the correct facts—a concept of writing as information rather than as argument” (Bean, 1996, p. 18). Students at the next stage of intellectual development, multiplism, “may accept the notion of opposing views, but they see these simply as ‘opinions’,” and thus “see little purpose in defending any particular view” (Bean, 1996, p. 18). Analytical writing assignments, which ask students to make and support an arguable claim, pose a challenge to how novice thinkers perceive knowing. This confrontation may be deeply uncomfortable. As Nelson (1999) noted, “The most fundamental lesson for teachers from Perry’s study is that critical thinking is acquired incrementally. The second key point is that learning critical thinking is existentially as well as intellectually challenging” (pp. 177–178). Following Perry, we should recognize that progressing to the understanding that knowledge is contextual and evidence-based may consume most of an undergraduate’s college career. How do we help students make this progress and develop as thinkers in our courses?

One way is to write a sequence of assignments that require students to identify and respond to a genuine interpretive problem and to openly discuss the challenges of such a task. These challenges include bearing what one scholar described as “the anxiety of incompletion that accompanies the act of writing” (Bhabha, 1994, p. xii). We can share with them our writing process or allow them to see our false starts and dead ends. We can make what Bean called “revision-oriented” rather than “editing-oriented” comments on drafts, responding to higher-order concerns (such as thesis and structure of argument) before commenting on lower-order concerns (such as sentence-level errors or stylistic awkwardness; Bean, 1996, p. 242). With guidance, we can ask our students to respond similarly to one another’s drafts in peer review. We can ask students to monitor the progress of their thinking, describing how their ideas have changed or their argument has deepened through the drafting and revision process. Helping students recognize that knowledge is made slowly and effortfully may help them construct and retain new beliefs about the nature of knowing.

A Common Issue in Student-Active Class Formats

So far we have discussed student comments that relate to learning challenges inherent in the type of disciplinary work we ask of our students. But learning challenges may also be pronounced if we transition from a lecture-based style of teaching to more student-active pedagogies, such as group work, case studies, or problem-based learning. Student response to this change is often variable, but it may be quite scathing as these common statements on evaluations attest:

I did not learn in this class because the teacher did not teach.

I didn’t come to college to teach myself.

Our earlier discussion of students’ simplistic beliefs about how and why one engages in the process of solving a problem or writing an essay highlighted the challenge of changing students’ prior conceptions about learning. Research in cognitive psychology confirms that one of the most important factors in human learning is prior knowledge (Halpern & Hakel, 2003). In fact, what students think they know is more of an impediment to learning than their ignorance. Many students’ expectations of or prior experience with college classes entail teachers standing in front of the room “telling.” Anything that deviates from this appears out of place and discomfits students for at least two reasons. First, all their experience says that our job is to do the talking, leaving them the choice of engaging with the subject or not. A second reason for their discomfort, however, is how they view the process of learning. Dualistic thinkers believe that gaining knowledge is as simple as listening to and repeating the views of an authority figure. Thus, when we employ group work in our classes, our involvement of other students as “teachers” appears to be a gross dereliction of our duty.

These student comments may especially trouble us because they may resonate with our own feeling of uncertainty in our roles as teachers. We, too, are frequently conditioned to think of teaching as telling, transferring our understanding and habits of mind by sheer force of will to our students. If we introduce student-active methods with the assumption that students will automatically like them and appreciate our attempt to support their learning in new ways, we may be particularly disappointed at student reactions. Even though the research on learning clearly shows that promoting students’ ability to retain and transfer ideas requires that they do more in class than just listen, neither we nor our students may really believe it.

Although lectures do enable us to transmit factual material, we also want students to develop deep conceptual understanding, integrate ideas across fields, and challenge prior knowledge. We as teachers often underestimate the power of students’ tacit beliefs about knowing in preventing them from integrating new concepts and ways of thinking into what they already know—or worse, think they know. Students’ beliefs about how one comes to “know” can act as a filter between our teaching and their learning, impeding such higher-order learning. As we discussed earlier, drawing on Bendixen and Rule’s (2004) model, in order to develop and change students’ beliefs, students must be willing to articulate these beliefs, weigh new ideas against that framework, and reflect on and confront these beliefs with others. Constructivist learning approaches such as peer and group learning involve students in metacognitive activities and provide a community of learners that can help students find new and deeper meaning in disciplinary knowledge. How can we help students understand these benefits of student-active pedagogical approaches to their development as thinkers?

It is always wise practice to share our reasons for how we structure the class. We may want to share with students some of the research on how people learn and how we humans develop intellectually. We can add any new elements to our teaching approach gradually, giving students a chance to adjust. We can invite them to be partners with us in the excitement of learning and mastering a new area of knowledge. Too often we provide students with the answers in our discipline before they even understand the questions. Focusing more of our classes around the questions in our discipline and how we strive to find some answers to them can help students see the processes involved in the human quest for knowledge.

Conclusion: Rethinking Student Evaluations

The comments section on student evaluations is often designed to ascertain student satisfaction with the class instruction. But is that what we really want to know? Edmundson (2004) suggested that we reconsider what we ask our students to evaluate in our end-of-semester questions. What if we asked them “to relate the quality of an encounter, not rate the action” (p. 9)? He proposed that we ask our students to consider how they have been changed by their encounter with course material, not how they have been entertained by our performance. In this article, we are suggesting that instructors read student evaluations for insight into learning challenges and stages of our students’ epistemological and intellectual development. We are also suggesting that instructors consider supplementing or rewriting final evaluations to emphasize the learning experience (Lewis, 2001). During the semester, instructors might ask students about their perception of the usefulness of problem sets, the quality of the feedback they are receiving, and the effectiveness of the class format. At the end of the semester, instructors might ask students how their thinking about the subject has changed, how the course has helped them develop as thinkers or as individuals. In doing so, we treat student learning—how it happens, how students have been changed by the experience—as a focus of inquiry. This scholarly approach not only helps us structure our own classes and assignments, but also can inform our thinking about department curricular planning and outcomes assessment.

Moving away from the questionable use of student evaluations to assess teaching effectiveness, we may instead view them as windows into the process of student learning and intellectual development. This perspective may lessen our stress as we read our evaluations and may encourage us to move more readily into the scholar’s question: “What is this student comment a case of?” As such, our evaluations may become a source of conversation with our peers, adding substance to department assessment endeavors and curricular planning. We may see our evaluations less as judgments of our performance and more as insight into our students’ intellectual growth—insight that may engage us in intellectual growth as teachers and scholars.