Introduction

Moving from initial ideas to innovation requires sustained creative work with ideas over an extended series of choice points, with decisions made under conditions of uncertainty (Brown 2009). To help build a citizenry attuned to the conditions of life in a knowledge society (OECD 2010), greater attention must be given to engaging students in extended idea development under realistic conditions of complexity and uncertainty. The Knowledge BuildingFootnote 1 approach represents an effort to refashion education as a knowledge-creating enterprise—to make it more attuned to the knowledge age (Bereiter and Scardamalia 2006). This requires that students function as epistemic agents—setting goals, monitoring progress, recognizing dead ends, rekindling interest, planning next steps, and so forth (Scardamalia 2002). To exercise such agency, students must continually make decisions under conditions of uncertainty about likely outcomes. Under similar conditions, mature knowledge creators will assess the promisingness of different topics, directions of inquiry, data sources, hypotheses, and so on. They will judge options not only on the basis of present value but also on the basis of their potential for further development—that is, judge the likelihood that an idea will be productive, decide on next steps, and analyze successes and failures following from their decisions. Through cumulative experience in making risky decisions, they develop promisingness knowledge (Bereiter and Scardamalia 1993)—domain-specific knowledge as well as general knowledge, both explicit and implicit, which can serve as a basis for future decisions and planning of knowledge building. In the context of Knowledge Building, ability to judge promisingness of community ideas can clearly have an important role in underpinning student agency and collective responsibility for knowledge advancement. The question arises, however, whether school age students have the necessary conceptual grasp and knowledge resources to make effective use of promisingness judgments.

The present study represents an early effort to explore promisingness judgments in an elementary school knowledge building context. In the following sections, we review the role of promisingness judgments in creative processes in various activities. We then situate promisingness judgments in the educational context, connecting it with relevant educational scenarios. Based on this review, we introduce a promisingness intervention based on a Promising Ideas Tool specifically designed to support the practice of identifying promising ideas to advance knowledge building. We then report results of this intervention, followed by discussion of results and implications for future work.

Promisingness judgments in creative processes

Promisingness is an everyday term that may be applied to actions, people, plans, tools—virtually anything considered from the standpoint of its future value. A quarterback earns money and fame based on ability to recognize promising passes; we all make daily judgments on the order of which route to drive home or what outfit to wear for a job interview. A normally active person probably makes dozens of promisingness judgments a day, but without consciously invoking promisingness as a concept.

Promisingness takes on additional meaning in the context of creative or “design thinking” (Martin 2009). Here it means “deserving of further investment in development.” Whether it is industrial designers working on a new product, scientists planning the next experiment in a research program, policy-makers planning social legislation, or graduate students choosing thesis topics, evaluation must be made about what is worth investment and likely to become fruitful in an uncertain future. Such evaluation is a significant challenge in scientific inquiry, as scientists are often confronted with “knowledge-poor” circumstances where principled knowledge about a problem space is scant (Bereiter 2009). To achieve a creative goal, they have to cope with many competing ideas that are usually in preliminary form and with uncertain prospects. The choices of which ideas to pursue are therefore of great consequence. In explaining creative processes, Gardner (1994) calls attention to the step of counting on intuition to detect “anomaly” or “discrepancy” when working in a domain; promisingness, as he explains, is what makes discrepant ideas stand out, encourages an individual to invest more effort, and eventually guides this person to breakthroughs. This claim about promisingness fits reported experience of creative individuals. For example, when discussing the development of the theory of relativity, Albert Einstein said, “During all those years there was the feeling of direction, of going straight toward something concrete … it was decidedly the case, and clearly to be distinguished from later considerations about the rational form of the solution” (quoted in Wertheimer and Wertheimer 1959, p. 228). In his case, the promising direction points toward breakthroughs, even though the richness of the promising idea will not be manifest until after the breakthroughs.

Promisingness judgments are also evident in other fields and professions requiring creative problem solving. In de Groot’s (1965) classic work on chess play, he refers to a feeling of promisingness that guides chess grand masters’ exploration of lines of play. Chess masters do not necessarily consider more options than experienced chess players; they simply think of better possible moves. Accordingly, what distinguishes masters from experienced players is the ability to recognize promising moves directly in their play. In engineering design, designers are often faced with “wicked problems” (Buchanan 1992), which require them to make design decisions that can account for a wide range of perspectives across disciplines (Pahl et al. 2007). While there are usually design axioms to follow, solving those problems requires recognition of more fruitful approaches directly from complex situations, rather than identifying and evaluating alternative courses of action; as expert-novice research indicates, experts do not necessarily employ more and different strategies than novices as they solve ill-defined problems, what they excel at is choosing strategies appropriate to the given circumstance (Schunn et al. 2005). In fine arts, painters make brushstrokes on the promise of advancing the artistic goal of the painting, with the painting as a whole conceived on the basis of an idea or image judged to be promising. As articulated by Fernando Botero when explaining his famous use of proportionally exaggerated or “fat” figures, “an artist is attracted to certain kinds of form without knowing why; you adopt a position intuitively and only later attempt to rationalize or even justify it.”Footnote 2 The same story can be told of the creative writer, chemist, or engineer. In summary, evaluation of promisingness, regardless of its rare appearance in literature, is integral to decision-making in creative processes of many kinds and plays a distinctive role in steering knowledge creation and innovation.

The feeling for promising directions comes with rich experience working in a domain (Bereiter and Scardamalia 1993). It relies on knowledge of promisingness—a type of impressionistic knowledge existing in forms of intuitions, hunches and feelings (Bereiter 2002a). This promisingness knowledge is acquired over time as people engage in creative practices, take risks, and learn from successes and failures. It is one type of “tacit knowledge” which is deeply rooted in action and one’s continual commitment to a problem (Nonaka 1991). Hence, promisingness judgments could be conceived of as educated guesses in that they draw on available knowledge from experience, even though the basis for judgment is often difficult to articulate or to defend rationally. To develop the capability of making promisingness judgments one needs to experience successes and failures from risk-taking in contexts of complex, ill-structured problems.

Situating promisingness judgments in educational contexts

If promisingness judgments play an important role in creative processes, they should legitimately have a place in education. Paavola et al. (2004) argue that education should go beyond pure propositional and conceptual knowledge and put more emphasis on hidden or tacit knowledge crucial for knowledge creation. Advocates of “teaching for wisdom” also attempt to find ways to nurture students’ capability in applying tacit knowledge toward the achievement of a common good (Sternberg 2001). Making promisingness judgments requires a form of tacit knowledge and may be treated as a type of wisdom.

Although promisingness judgment plays an essential role in creative expertise, education at all levels, by and large, ignores it or leaves such judgment to the teacher. In constructivist educational approaches, where students have a say in what questions they will investigate and how (Duffy and Jonassen 1992), judgments of promisingness become important so as to avoid going down blind alleys (Bickhard and Campbell 1996). However, it is not evident that promisingness is even considered in decisions about which questions to pursue, let alone reconsideration of work as it proceeds. Thus critical decisions to intensify or redirect work are seldom part of the inquiry process in typical constructivist classrooms. Common school practice usually helps students avoid unpromising directions by putting the highest-level executive processes for inquiry (e.g., issues to be investigated, evaluation of progress, time commitments, concluding activities) in the hands of teachers. Learning activities are usually structured (Kollar et al. 2007; Mäkitalo-Siegl et al. 2011) so that students do not spend a great deal of time with ideas considered unpromising by the teacher.

The alternative offered by Knowledge Building is to establish idea improvement as a norm and invite students to take collective responsibility for their work with knowledge and ideas (Scardamalia 2002; Scardamalia and Bereiter 2003). Responsibility extends to the most demanding aspects of their work such as setting goals, monitoring progress, and deciding next steps, with action taken collectively by community members. Making judgments of promisingness is essential for effective functioning of the community as these judgments drive design thinking. As in real-world knowledge-creating organizations, the challenge is to find a better way rather than focus so exclusively on uptake of true and warranted beliefs (Bereiter and Scardamalia 2003). Knowledge Building’s emphasis on promisingness raises the question, however, of whether young students are even capable of making useful judgments of promisingness. An assumption underlying the design of the current study is that this question cannot be answered except by design-based research aimed at facilitating the emergence of this ability. A program of design-based research was thus launched to devise supports for promisingness judgments in knowledge-building discourse.

A series of studies were initiated, starting with a pilot study with Grade 5/6 students (Chen et al. 2011). Results suggested that students intuitively held a “fact-oriented” rather than “knowledge-building potential” conception of promisingness; that is, without any explanation of the concept of promisingness, Grade 5/6 students tended to identify important-sounding facts as promising, rather than ideas having potential for deepening their understanding or leading to new directions in their work. This finding could suggest that promisingness, in the sense of knowledge building potential, is beyond the developmental capabilities of elementary school students. Alternatively, the dominance of successful school work equated with access to and use of true and warranted beliefs may mask hidden competencies (Scardamalia and Bereiter 2007). The hypothesis pursued in the research to be reported is that students are capable of promisingness judgments—judgments that reflect knowledge building potential. Toward this end we report on design-based research focused on the following central questions:

  1. 1.

    Do Grade 3 students have awareness of promisingness of their ideas? What is their intuitive understanding of promisingness?

  2. 2.

    Is it possible to raise students’ awareness of promisingness as potential for idea improvement, with this potential evident in selections of their ideas and those of their peers?

  3. 3.

    How might promisingness judgments influence knowledge-building discourse, at individual and community levels?

  4. 4.

    Can promisingness judgments facilitate knowledge advancement in a knowledge building community?

Methods

Experimental context and participants

The present study was conducted at a PreK-6 school affiliated with the University of Toronto. Principle-based Knowledge Building pedagogy and technology have been integral to the operation of the school, supporting core values and principle-based pedagogy focused on 12 principles used as design parameters rather than attempting to implement scripted procedures (Scardamalia 2002; Scardamalia and Bereiter 2006, 2014; Zhang et al. 2011). In the following description of practices, short-form descriptors of principles are italicized to convey their integration into design work and methods.

In the school, students are expected to take collective responsibility for community knowledge by contributing real ideas and authentic problems—ideas and problems they really care about rather than “authentic” problems designed for them by teachers. Authenticity is viewed from the students’ point of view. Collaborative knowledge-building discourse sustains knowledge advancement, making idea improvement a norm for discursive engagement. An online community space, Knowledge Forum (Scardamalia 2004), was used to support knowledge-building discourse. Of course knowledge-building discourse is not restricted to online environments, it is also supported in face-to-face conversations known as “KB talks” in this school. Efforts are made by both the teacher and students to establish respect for the ideas of others and a feeling of safety in sharing ideas. Idea diversity is respected, and democratization of knowledge a goal. Students’ ideas are center-front in conversations and students come to see the value in getting their ideas known by others in order to improve those ideas (Tarchi et al. 2013). In such endeavors, the community knowledge space often becomes the object of discourse in its own right: Ideas in the community space are projected on a whiteboard and class discussions focus on issues represented there, with emergent issues leading to suggestions for experiments, observations, and constructive uses of authoritative sources. When using authoritative sources, students are accustomed to searching for resources to advance their ideas rather than engaging primarily in text comprehension.

The laboratory school in which the present research was carried out has maintained a tradition of hosting bi-weekly Knowledge Building meetings attended by teachers, principal, vice principal, and researchers, to discuss Knowledge Building initiatives carried on in each class (Zhang et al. 2011). As a result the work at each grade level is informed by common principles reflected in similarities in design as elaborated above. The Grade 3 students taking part in this study typically entered the school in Pre-K, and so they had 3 or 4 years of prior experience in a Knowledge Building classroom.

Participants from the combined experimental and comparison classes were 40 students from two consecutive Grade 3 cohorts at the school. The earlier cohort (n = 20, 10 boys and 10 girls) was treated as a comparison group for the later experimental class (n = 20, 11 boys and 9 girls). The two classes were taught by two different teachers—the comparison group by a teacher with more than 3 years of experience with Knowledge Building and Knowledge Forum, while the teacher for the experimental class was at the school for a 1-year exchange program, thus was new to both Knowledge Building and Knowledge Forum. As a result, although two classes were led by two different teachers, to the extent that there may have been a “knowledge building” teacher advantage, the advantage would be in favor of the comparison group. Students in both classes, most of whom were at the school for at least the previous 2 years, had been taught by the same two teachers in their first and second grades. The dynamics in the comparison and experimental classrooms were quite similar. Not only did both teachers participate in weekly Knowledge Building meetings but the teacher who taught the comparison class provided considerable advice to the new teacher. Thus there were close parallels in terms of class design. The difference, as elaborated below, was the effort in the experimental class to integrate promisingness judgment into the classroom knowledge building culture.

Pedagogical approach

Previous exploratory research found that Grade 5/6 students tend to identify important-sounding facts as promising rather than ideas with greater knowledge building potential (Chen et al. 2011). This suggests that even for students engaged in knowledge building, when it comes to evaluation of promising school work, “true and warranted beliefs” are considered a better fit than ideas with “knowledge building potential.” Is this a simple accommodation to school life? In an effort to determine level of commitment to a fact-based versus knowledge-potential perspective, experimental work started with discussions of the concept of promisingness and work on the identification and further development of promising ideas, supported by an online tool for selection of promising ideas. The teacher’s role, as in Knowledge Building pedagogy generally, was to engage students in conversations regarding their work while providing support as needed to maximize opportunities for epistemic agency (Scardamalia 2002).

Technological supports for promisingness

To support this pedagogical approach, a “Promising Ideas Tool” was developed and integrated into Knowledge Forum. This tool included three components. The first was a highlighting feature to tag an idea within a Knowledge Forum noteFootnote 3 using a customizable categorization scheme (see Fig. 1, left side). By default, at the note level one student will not see other students’ highlights unless she clicks on a reveal button.

Fig. 1
figure 1

The Promising Ideas Tool has three components: a idea highlighting—on the left, a student can identify an idea with a customizable highlighting scheme; b idea aggregation—in the background window on the right side, all identified promising ideas from a view are listed, with identical or overlapping highlighted segments combined; and c idea export—a user can export selected ideas to a new view for further development (the foreground window on the right). Note: The color scheme could be customized to reflect different types of ideas, but in this study we let students choose whichever color they liked

Full size image

The second component was an idea aggregation window that collects all highlighted ideas within the current Knowledge Forum view,Footnote 4 merges overlapping ideas (based on text overlaps), and presents them in a list (see Fig. 1, right side). Ideas are ranked according to the number of “hits,” with most popular promising ideas at the top.

The third component, designed so students’ judgments would have real impact, was an exporting feature enabling export of select subsets of promising ideas to new workspaces for further knowledge work. This function was accessible to the teacher, to export notes to new views based on decisions taken collectively in class. In advance, the class reviewed promising ideas, and decided which ideas to export to a new Knowledge Forum view. They then created a new view, simply by clicking on the “Export Notes” button (see Fig. 1, right side) to populate it with select ideas. Students then refocused their work on the subset of ideas represented in the new view, a process designed to parallel that of scientists choosing to focus on “pregnant” ideas that they believe are promising to work on (Gardner 1994).

Procedures

Both the experimental and comparison classes studied a “Soil in the Environment” science unit for approximately 8 weeks. In studying this unit, both classes started with a “KB talk,” with students sharing their initial questions, ideas, and problems of understanding regarding soil. As is typical in knowledge building exchanges aimed at keeping ideas alive, students entered ideas from the KB talk into a Knowledge Forum database for further development. After the first few KB talks, interests in both classes focused on two central problems: “What is soil made of?” and “How to make soil?” Students kept recording ideas from their conversation in Knowledge Forum so others could advance ideas through online dialogue.

In the experimental class, the promisingness intervention included discussion of the promisingness concept, promisingness judgments, and collaborative idea refinement, as elaborated below.

  • Phase 1. For the first 2 weeks of the science unit, students proceeded as in the comparison class, participating in collaborative idea refinement through KB talks and working on a view titled “Grade 3 Soil 2010/11” in Knowledge Forum.

    The first element that distinguished the experimental from the comparison group occurred in week 3. The teacher first engaged students in a 30-min discussion of promisingness, eliciting students’ intuitive understanding of promisingness and advancing a “knowledge potential” account of promisingness through discussion. First, Grade 3 students were asked to consider the meaning of “a promising idea” and “a promising question.” They were then organized into eight small groups; each group discussed their ideas and recorded thoughts on a group worksheet. Ten minutes later, students came together for a whole-class discussion. They presented their different definitions and examples of promising ideas. The teacher helped distinguish different conceptions, especially important-sounding facts in contrast to ideas with high knowledge building potential. By the end of the presentation of examples and discussion the class elaborated their shared understanding of promising ideas as ideas that “they wish to spend time on,” “may change in further inquiry,” and “would deepen their shared understanding.”

    After the discussion of promisingness, students were introduced to the Promising Ideas Tool and then spent 30 min using this tool to conduct promisingness judgments in the “Grade 3 Soil 2010/11” view. First they worked individually using the highlighting function to tag promising ideas. Then they engaged in whole class discussion, aided by the idea aggregation function through which the whole class was able to review the “top hits” (the ideas most frequently selected as promising). Students then collectively identified three ideas to export to a new view. The focus of the exported ideas was “Where does soil come from?” and so that became the name given by students to the new view.

  • Phase 2. The second phase started with 3 weeks of collaborative idea refinement in the new view, followed by a second session of promisingness judgments on new ideas that emerged in this view. At the beginning of this new round of promisingness judgments the class discussed the concept of promisingness again, reflecting on their understanding of knowledge building potential. During the second round, students went through the same idea selection process as in Phase 1. They looked for promising ideas in the second view and exported three “most promising” ideas to a new view. This time their interests, reflected by selected ideas, shifted to earthworms. As a result, they named the third view “Worms and Soil.”

  • Phase 3. In Phase 3, students engaged in a new cycle of collaborative idea refinement in the new view for another 2 weeks. Because the goal of promisingness judgments was to initiate a new idea refinement cycle, no further promisingness intervention was conducted in the final phase.

In summary, the three elements—i.e., discussion of the promisingness concept, iterative cycles of promisingness judgments, and collaborative idea refinement—were implemented in the experimental class to provide a supportive socio-cultural context for students’ promisingness judgments. Class discussions of the promisingness concept were aimed in the earliest work to elicit students’ intuitive understanding of promisingness and later to advance their understanding of the “knowledge potential” account of promisingness. Promisingness judgments, facilitated by students’ use of the Promising Ideas Tool, directed the course of knowledge-building discourse through two rounds of “judgment–export” activities, with a goal of deepening collaborative idea refinement. The comparison class, in contrast, was only engaged in collaborative idea refinement—without use of the Promising Ideas Tool or explicit efforts to encourage promisingness judgments.

Data sources

To understand student engagement with promisingness, extensive quantitative and qualitative data analysis was conducted on Knowledge Forum databases and classroom observations. Knowledge Forum data consisted mainly of student notes and the “promising ideas” they identified. An overview of the Knowledge Forum dataset is provided in Table 1.

Table 1 An overview of data sources
Full size table

During the Soil unit, students from the experimental group worked in three Knowledge Forum views in three phases, as described in the Procedures subsection above. Students produced a total of 163 notes in these views. As for promising ideas, each highlight with the Promising Ideas Tool by any student was considered one promising idea. Students identified 57 and 94 promising ideas from the first two views in Phases 1 and 2. Since students could independently select the same idea, there were a number of repetitions. In Phase 3, students did not attempt to make promisingness judgments so no promising idea was highlighted.

In the comparison group, students produced 129 notes in the “Grade Three Soil” view and its four subordinate views—“Worm Anatomy,” “Worm Life Cycle,” “Worm Behaviour,” and “Worm Habitat.” The four sub-views were created by the teacher for ease of organization when the “Grade Three Soil” view became too large. Students in the comparison class did not use the Promising Ideas Tool for promisingness judgments, thus there were no promising ideas to evaluate.

In addition to data from Knowledge Forum, we collected students’ worksheets on which they recorded their initial thoughts of promisingness during the 30-min concept elicitation session. Eight small groups of students produced 35 notes conveying their early thoughts regarding promisingness. Additionally, video recordings of face-to-face interactions in two promisingness judgments sessions were collected to allow for triangulation of results.

Data analyses

Data analysis conducted in this study focused on research questions pertaining to three components in the promisingness intervention:

The concept of promisingness

Group worksheets containing group notes about students’ initial understanding of promisingness were collected from the initial session and qualitatively coded (Burnard 1991), with a goal of identifying types and variation in student conceptions. Through an iterative categorization process, key themes of student conception were identified and were then gradually refined through cycles of further analysis and verification. Video recordings of student discussion during the 30-min concept elicitation session were transcribed and analyzed to triangulate findings from coding of student notes and to help explain results.

Promisingness judgments

Students’ promisingness judgments were analyzed focusing on the following two aspects of their work.

Ways of contributing to knowledge-building discourse analysis. To assess progress toward a “knowledge building potential” conception of promisingness, students’ understanding of promisingness as reflected in their promisingness judgments was investigated. To this end, content analysis (Chi 1997) was conducted focusing on the epistemic nature of ideas selected by the students as promising. In this analysis, all identified ideas were coded according to the “Ways of Contributing to Knowledge-building Discourse” scheme developed by Chuy and colleagues (2011). This scheme provides an inventory of students’ types of contributions, with six major categories: (1) formulating questions (e.g., “Why do the plates have to move?”); (2) theorizing (e.g., “I think that the worms sense light through heat because the light has heat and dark doesn’t!”); (3) obtaining information (e.g., “Let’s make our own soil and compare it to real soil and see the difference”); (4) working with information (e.g., “Worms can tell when it’s night because it’s cooler. That’s why your mom and dad make you wear your hoodie when you go out for dinner.”); (5) synthesizing and comparing (e.g., “We have a sense of up and down worms have a sense of light and dark.”); and (6) supporting discussion (e.g., “Hey guys, let’s get down to business.”). The distribution of contribution types represented in identified ideas would reflect students’ approaches to promisingness: information-related contributions were considered a reflection of a “fact-oriented” conception of promisingness, whereas theorizing, formulating questions, or synthesizing were considered a reflection of a knowledge building potential conception. Despite the importance of facts and evidence in scientific inquiry, the goal underlying the current analysis was to determine the extent to which students had access to different conceptions of promisingness. Thus the analysis aimed to distinguish fact-based accounts as uncovered in earlier research (Chen et al. 2011) from the knowledge building potential account that is the focus of this study. Two independent raters conducted the analysis. The inter-rater reliability as measured by joint probability of agreement was 0.83. Discrepancies were discussed to reach agreement.

The effect of promisingness judgments on socio-cognitive dynamics of knowledge-building discourse. To determine if knowledge-building discourse was affected by promisingness judgments, discourse was analyzed at both the individual and community levels. First, individual Knowledge Forum activities were tracked to establish individual profiles, to reveal how students made use of promisingness judgments through their individual contributions. Specifically, note reading, note posting, and promising idea highlighting activities were identified for each student and arranged chronologically. Temporal relations among different types of activities were inspected for each student profile, to uncover the impact of promisingness judgments on knowledge building activities at the individual level.

Second, the impact of promisingness judgments on community cohesiveness was analyzed. The idea underlying the cohesiveness analysis was that promisingness judgments created a group profile of ideas; attending to agreed upon promising directions should create a more cohesive community context. To test this hypothesis, Social Network Analysis (SNA) (Scott 1988) was used to analyze students’ social interaction data recorded in Knowledge Forum. For this analysis three types of interactions involving note reading, building-on and promising idea highlighting, were used to construct social networks. To obtain a general understanding of each network, global level SNA measures focusing on network cohesion (Haythornthwaite 1996) were compared across different discourse phases in the experimental class.Footnote 5 In this context, network density reflects the extent to which students interact with each other and is closely related to the Knowledge Building principle collective responsibility for community knowledge. Related to density, the measure of average weighted degree provides another measure of how closely students are connected. The weight of a link between two students denotes the strength of their connection; for example, the weight of the link from Student A to B in a building-on network is determined by times Student A builds on B in Knowledge Forum. Average weighted degree denotes the average weight of connections among students and therefore additionally measures the strengths of connections in a community. Finally, average path length denotes the average number of steps along the shortest paths for all possible pairs of nodes in a network (Abraham et al. 2009). In the present study, this measure takes the network structure into consideration and provides perspective on how democratized or balanced a student network is. For example, a building-on network with a shorter average path length implies higher structural equivalence, implying more balanced and symmetric student discourse not dominated by a few prominent voices. Therefore, average path length is linked to the Knowledge Building principle of symmetric knowledge advancement (Scardamalia 2002).

Collaborative idea refinement

Promisingness judgments that foster social cohesion may promote idea improvement as well. To assess this, Knowledge Forum notes in experimental and comparison classes were first coded using the Ways of Contributing to Knowledge-building Discourse scheme (Chuy et al. 2011) to identify theorizing contributions. To evaluate knowledge advancement across discourse phases all theorizing notes were coded using a 4-point scale developed by Zhang and colleagues (2007):

  1. 1.

    Pre-scientific: containing a misconception while applying a naive conceptual framework

  2. 2.

    Hybrid: containing misconceptions that have incorporated scientific information

  3. 3.

    Basically scientific: containing ideas based on a scientific framework, but not precise

  4. 4.

    Scientific: containing explanations that are consistent with authoritative scientific knowledge

Two coders independently assessed the notes, and the inter-rater agreement measured by joint probability of agreement was 0.86. A two-way analysis of variance (ANOVA) was then performed to assess whether scientific sophistication scores of student notes could be predicted from class membership (experimental vs. comparison), phases of discourse (Phase 1, 2 and 3), and the interaction between these two factors. This analysis provided indication of the extent to which promisingness judgments facilitated knowledge advancement across discourse phases.

Results

Students’ intuitive understanding of promisingness

Previous research reported a tendency toward selecting fact-based ideas as promising (Chen et al. 2011). In the present study an effort was made to tap the full range of ideas that students might bring to the understanding of promisingness. This effort started with the 30-min discussion, including time for students to record their ideas. Thirty-five group notes written by students on their group worksheets during the first session were coded. This analysis identified three conceptions of promisingness:

  1. 1.

    Factual. Consistent with the results from previous research, “being true” or “truthfulness” represented a popular conception of what makes an idea promising. This conception was indicated in notes produced by all eight student groups. For example, some students thought a promising idea was “a true idea”, “an idea that is not incorrect”, “an idea you promise that it is right”, or “an idea that you are pretty sure is right”. These accounts explain why students from previous research identified “cool” facts such as “The universe is 13,000,000,000 years old!” and “the Grand Canyon could have 912,456 layers of rock” as promising.

  2. 2.

    High-probability of being right. Students recognized ideas with uncertain truth status as promising as long as there was high “likelihood” of being correct. This conception was evident in four groups. For example, “an idea that is very good and probably be right”, “an idea that might work”, “this is probably a right idea”, or “idea that is most likely or 90 % sure to be right”. This notion of “likelihood” went further than purely fact-oriented truthfulness. According to the Merriam-Webster dictionary, promising means “showing signs of future success or excellence,” “likely to turn out well,” or “likely to succeed or yield good results.” The notion of “likelihood” students captured is an essential component of promisingness, as judging promisingness is always a risky business. However, it should be noted that students expressing this view were treating likelihood in terms of an idea’s current but uncertain truth value. Plausibly, they were still not thinking of ideas on an improvement trajectory.

  3. 3.

    Knowledge building potential. Students also recognized ideas “leading to future actions” and having knowledge building potential as promising. This conception was displayed in three of eight groups. For example, “an idea you can spend time on”, “an idea/question you need to know”, and “an idea/question that can help you do something”. These beliefs are in line with the notion that a promising idea could be flawed but worth laboring on to be improved (Bereiter 2002b; Bereiter and Scardamalia 1993). This link to knowledge building potential provided a meaningful basis for engaging students in promisingness judgments to advance knowledge building.

To summarize, in early class discussions of the concept of promisingness, Grade 3 students in the experimental class demonstrated a full range of understandings of promisingness. During discussion, students explored these understandings with participation from the teacher. By the end of the discussion, the whole class arrived at a consensus that a promising idea is an idea “they wish to spend time on,” “may change in further inquiry,” and “would deepen their shared understanding,” reflecting the knowledge potential conception of promisingness that is the focus of the current research.

Students’ awareness of promisingness reflected in their promisingness judgments

After discussion and elaboration of the knowledge potential conception of promisingness, the stage was set to introduce students to the Promising Ideas Tool so that they could tag promising ideas in their discourse. The research question to be addressed: Would relatively brief discussions of promisingness along with use of the Promising Ideas Tool be sufficient to engage Grade 3 students in effective promisingness judgments? To answer this question, the Ways of Contributing scheme was applied to assess epistemic nature of selected promising ideas. The issue was: Would these young students adopt the knowledge building potential conceptualization and accordingly highlight more theorizing than obtaining information contributions?

As shown in Fig. 2, Grade 3 students in the experimental class identified a large portion of theorizing (68.9 %) and much less obtaining information contributions (6.7 %) as promising. In contrast, in an earlier study with Grade 5/6 students, 63.9 and 33.7 % of ideas highlighted fell into the theorizing and obtaining information categories respectively (Chen et al. 2011). So the proportion of fact/information with the third graders was significantly lower than reported in earlier research with older students.

Fig. 2
figure 2

The epistemic nature of promising ideas selected by the experimental class

Full size image

Items of information may be categorized as “clues,” meaning that they are judged promising with regard to solving a case. For example, in criminal detection there are promising facts in the sense that they are part of a problem solution or explanation. To see whether facts identified as promising by students in this study were promising in this sense, we further analyzed ideas surrounding the highlighted obtaining information contributions. This analysis revealed that these highlighted contributions in the Grade 3 discourse usually co-occurred with working with information contributions; information or facts in these contributions were originally introduced into the dialogue to support or refute a theory. Thus, factual information was incorporated in discourse constructively. For instance, in an idea identified by one Grade 3 student, the new information “when it’s night it’s cooler… in the day the sun is shining and it’s warmer” was introduced to support her hypothesis that “worms sense light by temperature.” In contrast, factual information contributions identified as promising by students in the earlier pilot research were typically standalone entries; that is, those students were more likely to identify isolated facts such as “the solar system formed 4570 million years ago” as promising (see Chen et al. 2011). Experimental group students in the present study continued to include factual statements among the ones they highlighted as promising (about 7 %), but it appears that they tended to select them on the basis of promise for future knowledge building.

Influence of promisingness judgments on discourse at the individual level

Knowing students in the experimental class were capable of making potential-for-knowledge-building promisingness judgments, the relationship between their selections and other knowledge building activities was pursued. Analysis at the individual level focused on qualitative data from students’ note writing and promising idea selections. As explained in the Methods section, each student notes and highlighted ideas were organized chronologically to examine the interplay between promisingness judgments and other knowledge building behaviors. Temporal qualitative analysis identified the following three themes in the third graders’ promisingness judgments:

  1. 1.

    Knowledge integration and revision: ideas highlighted as promising help advance previously posted ideas. Students participating in collaborative knowledge building typically bring their own ideas into the collaborative process. Thus it is reasonable that they would highlight ideas relevant to ideas they posted. Temporal relationship indicated that after highlighting an idea a student often reconsidered ideas posted previously and eventually revised their idea. For example, S9 posted her first note about how worms sense light: “My theory is because we have a sense of up and down worms have a sense of light and dark.”Footnote 6 A few days later, she identified a promising idea contributed by S6 that “the worms feel heat and the light has heat and dark doesn’t!” In the next note she wrote, “My theory is that it also has something to do with heat. Like when you walk into the dark it gets colder because there’s less sunlight… They don’t need eyes because they can feel heat like us.” Plausibly, she revised her original simple analogy to a more sophisticated idea with more coherent reasoning.

    In some other cases, highlighted ideas that complemented a student’s own ideas resulted in efforts to integrate them into stronger explanations. For example, S13 posted a note, “My theory is that soil is made out of rocks that get turned into sand. Then you maybe take a little bit of grounded up wood.” Then she tagged an idea from S8, “The soil is made from rocks it gets all broken up from the wind and getting rain or something watery on it.” In the next note she tried to integrate the highlighted idea, the “watery” part in particular, into her original one and created a more complex account: “My theory is that soil is dirt, rocks, little bits of water and life. The rocks get smashed up. Then mix it with the dirt. Then add water to make it moist. The worms help the soil and the poop and the worms make air holes.”

    As these examples suggest, by engaging students in the intentional effort of identifying promising ideas in the community, their attention was drawn to ideas adjacent to those generated by them that might otherwise be ignored. This process of considering multiple perspectives and formulating increasingly interconnected views of scientific concepts supports symmetric knowledge advancement (Scardamalia 2002), knowledge integration (Linn et al. 2006), and conceptual change in science learning (Vosniadou 1994).

  2. 2.

    Emergent topics and participation: ideas highlighted as promising help advance ideas beyond those previously found in the community workspace. After highlighting a promising idea new to the students’ conceptual space, students tended to write notes voicing agreement or reinforcing the idea, with subsequent participation leading to improvement of community knowledge. For example, S2 had not posted any note before highlighting an idea about worms: “the worms feel heat and the light has heat and dark doesn’t.” A few days later, he posted a note with an alternative explanation, “My theory is that worms don’t have eyes they can sense the difference between soil and the outside world. Because they can feel the difference in humidity.” One week later, he further developed his idea and came up with a more scientific explanation, “Worms don’t have eyes they have photoreceptors which catch the light and if they go outside too long the photoreceptors will ‘shoot off’ and the worms will get paralyzed…” New contributions extending the highlighted idea led several other students to grapple with the concept of “photoreceptor,” thus expanding the conceptual repertoire for the group as a whole.

  3. 3.

    Promising-ideas selected: some ideas highlighted have no discernible impact on subsequent knowledge building. Analysis also found students highlighting a number of ideas but not building on or making reference to them afterwards. For example, S20 was the most active student in identifying promising ideas in this intervention, making 35 highlights over two sessions of promisingness judgments. However, she highlighted many contradictory ideas and failed to make an effort to integrate them. Similarly, S8 highlighted a number of ideas about “where does soil come from,” but did not post any relevant note afterwards. In some other cases students identified facts or already heavily discussed questions as promising. For example, S7 and S8 picked the question posted early in the session by the teacher, which had been discussed by the whole class for an extended portion of class time. S5 and S11 identified a simple fact “some reptiles live in wet places,” with no evident follow-up in their discourse. Thus, while in many cases students could work productively with identified promising ideas, some of them may need additional support to incorporate promising ideas into their knowledge work.

In summary, promisingness judgments led to revision of earlier ideas, integration of multiple ideas, and emergent participation in new themes for discussion; however, promisingness judgments were also made with no evidence of pursuit of the promising line of investigation. Further research is needed to determine factors, domain knowledge and epistemic beliefs for instance, that may contribute to individual variations with the use of the Promising Ideas Tool and related knowledge building processes.

Social dynamics in the knowledge building community

Because knowledge building is a collective enterprise, in addition to analyzing individual profiles the impact of promisingness judgments on social interactions within the knowledge community was explored. Social Network Analysis (SNA) was conducted on three forms of interaction: note reading, building on, and promising idea highlighting. In each social network, nodes represent students and edges denote a specific type of interaction between them. To evaluate how social dynamics changed over time for the experimental group, Knowledge Forum discourse data was partitioned into three sections according to the three discourse phases.

Table 2 presents results of analysis related to network-level SNA measures. As explained in the Methods section, network density describes the percentage of connections out of all possible connections; average weighted degree denotes the average weight of connections among nodes in a network and is indicative of the cohesiveness of a network, whereas average path length means the average number of steps along the shortest paths for all possible pairs of nodes and implies how balanced a network is. It should be noted that because it was close to the end of semester students worked in Phase 3 for only 2 weeks while spending 3 weeks in the first two phases. Therefore, Phase 3 was left out of the comparisons.

Table 2 Measures of social networks in the experimental class
Full size table

Comparing networks of different types of interactions, the number of edges, density and average weighted degree in note-reading networks were much higher than building-on and idea-highlighting networks. This result was not surprising because reading activities are typically more frequent than note writing and idea highlighting.

Comparisons of social networks across different phases indicated increasing levels of connectedness and cohesion from Phase 1 to Phase 2 for both reading and building-on networks. For instance, the number of edges and the level of network density increased, showing higher level of connectivity. The average weighted degree in the reading network increased dramatically from 47.9 to 89.8, implying students were much more active in reading each other’s contributions. At the same time, the intensity of building-on activities, represented by the average weighted degree in the building-on network, also increased, indicating a higher level of collaboration. The average path length in both networks decreased, showing social networks were getting more symmetric and balanced from Phase 1 to Phase 2.

As for the idea-highlighting networks, it was interesting that while the networks had equivalent numbers of edges in Phase 1 and Phase 2, the average weighted degree increased. Although students were making more promisingness evaluation attempts, because the average path length did not necessarily decrease, the idea-highlighting network did not consequently get more balanced. These results fit with the finding of considerable variation in promisingness judgments uncovered in analysis of individual behaviors—some students made significantly more promisingness judgments than others. As a result, increased average intensity did not give birth to new edges between students. Taken together, SNA suggested increasingly intensive promisingness judgments in the community and more cohesive and balanced building-on networks (which are indicative of collaboration) across discourse phases.

Knowledge advancement in the knowledge building community

The ultimate goal of promisingness judgments is to boost community knowledge by refocusing community attention on promising directions. Through evaluating the promisingness of community ideas, students reflect on the cutting edge of their work and recognize ideas worth extended efforts. In this manner, students could devote their limited time and energy to more promising ideas, with better opportunities to grow individual and collective understanding.

In this study, our hypothesis was that the experimental class making promisingness judgments would achieve greater knowledge advancement than the comparison class. To test this hypothesis, student ideas in two classes were examined by rating students’ conceptual contributions according to a scientific sophistication scheme with four levels. In the experimental class, 91 theorizing notes were identified, with 26, 42, and 23 notes from respective Knowledge Forum views in three research phases. In the comparison class, a total of 68 theorizing notes were identified. Because the comparison group did not integrate promisingness judgments into their discourse, there was no natural divide of discourse phases; so student notes were sorted by time of creation and divided into three phases with equivalent number of notes.

A 2 (Group) × 3 (Discourse Phase) factorial ANOVA was performed to assess whether scientific sophistication scores of student ideas were associated with student group (experimental vs. comparison), discourse phases (Phase 1, 2 and 3), and the interaction between these two factors. Analysis of variance showed a significant main effect for discourse phases, F(2, 153) = 14.33, p < .001, η 2 = .16, indicating the mean scientific sophistication scores were different among three phases. The main effect for group difference was not significant, F(1, 153) = .03, p = .87. However, the analysis revealed a significant interaction between discourse phases and student group, F(2, 153) = 3.81, p < .05, η 2 = .05. This interaction is graphed in Fig. 3, showing a steeper gradient of improvement in the experimental group. This finding is especially important because the comparison favored the comparison group taught by a more experienced Knowledge Building teacher with her students from the beginning showing higher scores.

Fig. 3
figure 3

Change in rated scientific sophistication of ideas in the experimental and comparison classes

Full size image

Discussion

In the present study, design-based research was used to address four research questions. What do Grade 3 students understand by “promisingness of ideas”? (Question 1) and Could their understanding be moved to a higher level? (Question 2). Analysis of students’ written definitions of promisingness, produced in advance of a teacher-facilitated discussion of the concept, demonstrated that Grade 3 students brought with them a broad range of meanings of promisingness. Most students presented a “fact-oriented” interpretation but several presented definitions suggesting awareness of a “knowledge building potential” conception. Their different accounts in the earliest phase, and then later in continual class discussions, allowed them to consider a broad range of contrasting interpretations. The teacher reinforced the importance of the knowledge building potential conception and its relevance to upcoming work, using a Promising Ideas Tool to identify ideas in their Knowledge Forum work that they considered promising. Analysis of the epistemic nature of ideas that they selected as promising showed a significantly larger portion of theorizing ideas than obtaining information contributions compared with work in a previous pilot investigation with students several years older. This suggested that the Promising Ideas Tool and socio-cultural processes implemented in this study were effective in promoting students’ understanding of promisingness, extending that understanding beyond “true ideas” to ideas having a promising growth trajectory.

Do promisingness judgments influence knowledge-building discourse? (Question 3) Analysis of individual student profiles showed that ideas identified by a student as promising were often related to ideas they had posted previously, and led to subsequent knowledge revision or integration. By considering ideas from multiple perspectives, students were able to reformulate and explain ideas in multiple and more connected ways, leading to increasingly interconnected views about scientific concepts. Thus promising-idea selections served as building blocks for knowledge integration and conceptual change in science learning. The intentional effort of highlighting promising ideas also brought students’ attention to new ideas emerging in the community, leading students to work with ideas they identified as promising in a variety of ways. After highlighting an idea, students often committed themselves to the idea and made efforts to improve it. In other cases, however, students failed to act on an idea they had selected as promising.

Effects of promisingness judgments at the community level were explored using SNA. Network-level SNA measures showed rising intensity of connection in the reading and promisingness judgment networks and increasing cohesion and balance of the building-on network during the promisingness intervention. These results indicated rising awareness of community ideas and improved collaboration among students. These findings support those at the individual level: When promising ideas motivate individual knowledge building activities such as building on and revising ideas, this fosters collaboration, with new ideas emergents of promising idea selections and subsequent interactions that help create a more cohesive knowledge building community.

What are the facilitating effects of promisingness judgments on community knowledge advancement? (Question 4). Comparisons of scientific sophistication of student ideas across discourse phases and comparison-experimental classes showed the experimental class achieving significantly greater knowledge advancement in the soil unit, even though they started with slightly less scientific ideas.

Directions of future development to support promisingness

This was a small study and the basic design needs to be replicated at other educational levels. With older students there should be little question that they can grasp and apply the concept of promisingness. The question is what it will do for them. Can they genuinely adopt a “promisingness mindset” that will play a positive role in all their creative efforts? If so, we may have a powerful way of going beyond the brainstorming that sometimes passes for creative work with ideas and on to the sustained creative work with promising ideas that characterizes real world innovation and knowledge creation. To realize this potential, however, several design advances are called for:

  • Promising ideas from external sources. An obvious enhancement to the Promising Ideas Tool is to make it possible for users to select and import ideas from external sources. There are always concerns about authoritative sources pre-empting students’ own knowledge building efforts, but promisingness, with its emphasis on further idea development may offer a way of taking advantage of good ideas coming from outside without passively adopting them.

    • Promisingness integral to everyday work. Whereas in the present study work with promisingness was a special activity and a complement to regular knowledge building, our longer-range goal is to see this integrated into regular knowledge building, as something that goes on opportunistically as students engage in theory-building, historical interpretation, and other varieties of knowledge building. Toward this end, the technological supports will need to be redesigned so as to have an attractive presence in Knowledge Forum views and to be easily put to use there.

    • Justification of promisingness judgments. The tool had no provision for users to justify their choices of promising ideas. This was left for class-wide deliberation after judgments. However, by that time students sometimes forgot their reasons. We do not want to require students to have justifications. As modern dual-process theories of cognition make clear, instantaneous and rationally ungrounded choices can have great value (Gladwell 2005). But scaffold-type supports to help students when they do want to justify their choices (when defending them in a later discussion, for instance, or even when they are only concerned with rational justification for their own satisfaction) could be a valuable enhancement of the technology used in this study. We are experimenting with this in a more recent version of Knowledge Forum and users (in this case adults) have appreciated the opportunity to add short phrases to allow subsequent search and exploration of promising ideas.

    • Dealing with complexity. Knowledge Building requires that students thrive on complexity and, in turn, work through complexity to discover simplifications that get to the essence of a complex problem. The approach taken in the present study could be criticized on grounds that it encourages premature simplification by selecting only a few popular ideas for further development and that it fragments knowledge building by supporting a focus on individual ideas, whereas it should be supporting synthesis, the building of complex idea structures. These problems characterize many different facets of constructivist education, but it is possible that a more integrative way of dealing with promisingness could overcome these problems in a widely applicable way. Toward this end current experimentation is focused on goals underlying promising-idea selections, with ideas then discussed according to their merits in light of a specific shared goal.

    • Endless improvability. In Knowledge Building pedagogy, endless improvability of ideas is recommended as a working hypothesis although it is of course never fully realized in practice. Idea improvement is implicit in promisingness judgments; to say that an idea is promising is in effect to say that it or the situation it applies to, is improvable. Supports for assessing idea improvement could therefore play an important part in the further advancement of work with promisingness.

    • Portfolios of idea advancement. In the present study researchers conducted temporal analysis to determine effects of promisingness selections on knowledge building. Newer work is focused on storytelling regarding idea advancement, so that students are telling their own stories of knowledge advancement based on selections of promising ideas and subsequent work with those ideas in a community context.

Conclusions

This study has offered insights into the knowledge-building capabilities of elementary students. Students as young as 8 to 9 years of age have an intuitive grasp of a wide range of meanings of the promisingness concept and their understanding can be enhanced through making and discussing promisingness judgments. Results demonstrate the potential of promisingness judgments to improve individuals’ awareness of community knowledge and in doing so to improve collaboration aimed at advancing community knowledge. The reported research also opens up a broader space for research, including refined designs to support promisingness judgments, combined efforts to facilitate metadiscourse (Resendes et al. 2015) around promising ideas, research into cognitive processes behind promisingness judgments, and, central to all of them, further research aiming to expand our understanding of young children’s capability in making promisingness judgments.