Introduction

Reflection is an important prerequisite for students to achieve meaningful and deep learning (e.g. Dewey 1998). Yet, the complex cognitive and affective processes associated with reflection make it difficult for practitioners and researchers to inculcate reflective learning in students. Among various technology tools, the blog has been used to promote reflective thinking skills in educational settings, since it allows students to externalize their reasoning and reflection on experiences (Fiedler 2004). Although blogging can be effective for articulating ideas, students also show a tendency to merely record and forget blog entries over a short period of time (Sharma and Xie 2008). Reflective thinking requires students to examine, make sense of new knowledge, and make connections among parts of an experience—an integral activity of reflective thinking as defined by Dewey (1998). Davis and Linn (2000) called for a “mix of opportunities for reflection” and for knowledge integration. However, commonly seen blog tools do not provide any overt mechanisms for students to aggregate the information in blogs.

In this study, a new blogging-mapping tool was designed and developed to facilitate students’ reflective thinking and integration of learning. As a new and potentially powerful technology, understanding the parameters and consequences of this tool’s usage could have significant meaning for teachers, practitioners, and instructional designers who are interested in assisting reflective learning in students.

Diverse perspectives on reflective learning

Reflection is the mechanism by which learners form a “robust, coherent, conceptual understanding” for integrated knowledge (Davis 2003, p. 99). Reflective thinking has been described as a process of extracting concepts from learning content, and forming and reforming the propositions among concepts (e.g. Ausubel et al. 1978; Novak and Gowin 1984). Moon (1999) offers a more detailed depiction of how reflection functions in meaningful learning processes, suggesting that only through reflection can students move through each stage of a cycle that includes: taking notice of new information, making sense, making meaning, and working with meaning until transformative learning occurs. Moon suggests that each learning cycle results in an upgrading of representation of knowledge in the learner. Other research has emphasized the importance of systematizing domain knowledge for comprehensive understanding (e.g. Davis 1996, 2003; Linn 1995; Rosie 2000; Smith et al. 1993. For example, Linn (1995) identifies knowledge integration as the upper level of meaningful learning that involves reflective thinking. Integrating domain knowledge involves a process of linking ideas together to develop a “robust, coherent, conceptual understanding” (Davis 2003, p. 99). Davis (1996) specifically stated that “reflection provides one method for fostering conceptual change and knowledge integration by helping students to differentiate among ideas and make connections between them” (p. 5). (See Fig. 1 for a depiction of these different views.)

Fig. 1
figure 1

Different perspectives on meaningful learning through reflection

Full size image

Based on these various perspectives, in this study reflective learning was defined as a learner’s purposeful and conscious activity of manipulating ideas for meaningful learning and knowledge integration, including (a) revisiting and conceptualizing information, (b) reorganizing and differentiating concepts, (c) formulating connections between concepts, and (d) systematizing, integrating, and restructuring connections among concepts for comprehensive understanding.

Strategies for supporting reflective thinking

Various strategies have been used to promote reflective thinking, including blogging, use of self-monitoring techniques, and concept maps.

Blogging

Blogging or online journaling offers various benefits to instructors wishing to engage their students in reflective learning. A recent review of empirical research about the use of blogs in higher education settings (Sim and Hew 2010) indicates that blogging benefited students because it allows students to: (a) externalize their reasoning and reflective thinking through writing (Zeng and Harris 2005), (b) develop keener insights into assumptions and beliefs that interfere with their judgments (Sharma and Xie 2008), (c) prompts students to provide evidence, elaboration, justification and critically evaluate solutions (Loving et al. 2007), and, (d) engage in active transactions between the basic assumptions, motivations, and the more apparent descriptions and hence promote reflectivity (Stiler and Philleo 2003). Moreover, blogs, much like e-portfolios, allow students to see the progress of their thoughts over time (Ellison and Wu 2008). Baggetun and Wasson (2006) also suggested that blogs could represent an individual’s growing knowledge base.

Use of self-monitoring tags

Self-monitoring involves two processes: self-observation and self-recording (Cole et al. 2000). These processes tend to be facilitated by the use of explicit questions or tacit prompts. Various self-monitoring prompts have been shown to promote reflective thinking because the prompts required students to articulate domain-specific knowledge and their thinking processes (King 1990; van den Boom et al. 2004). For example, Schellens et al. (2009) prompted students to tag their posts in a discussion board by asking them to state specific types of expected cognitive processing during their collaborative problem-solving process. The tagging prompts obliged students to reflect upon the nature of their contribution and on its facilitation of the ongoing discussion. The results suggest that tagging thinking types significantly promotes critical thinking in terms of identifying a problem by going to the core of the matter and creatively generating new ideas. Similarly, prompting students to tag keywords to their blog entries will require them to read what they’ve written (the self-observing process) and write down the relevant core concepts in the entries (the self-recording process). For example, research on the Knowledge Forum (Scardamalia 2003) environment showed that prompting for keywords compelled students to make their thinking explicit and therefore fostered reflection and learning.

Concept mapping

A third effective instructional strategy for reflective learning is concept mapping. Among all kinds of cognitive benefits, a meta-analysis revealed that concept maps were effective for knowledge retention and transfer (Nesbit and Adesope 2006). Novak (1998) suggested that concept maps might be one method for capturing and utilizing knowledge, because concept maps allow students to analyze information and speculate on the relationships among pieces of information. For example, Scardamalia (2003) found that allowing students to map their bulletin postings helped social co-construction of knowledge. A technique similar to concept mapping, called “ordered tree” (where students sort concepts in a meaningful order), was also shown to help students to think about concepts, the connections between these concepts, and the structure of an entire course (Schraw and Moshman 1995). Another similar strategy, “concept chain” (where students list key ideas in an increasingly complex order), improved students’ learning because it required students to break down the progression of concepts and restructure them (Machin et al. 2004).

Based on these research findings, the researchers in this study designed a new tool that incorporated blogging, keyword-tagging, and concept mapping into an online environment. The following section describes the design of the tool, research questions and findings.

Design of a new tool

Blog environments could provide numerous advantages for a student’s reflection (e.g. Baggetun and Wasson 2006; Ellison and Wu 2008; Loving et al. 2007; Sharma and Xie 2008; Sim and Hew 2010; Stiler and Philleo 2003; Zeng and Harris 2005). However, the linear blog posts do not allow space for students to manipulate and rearrange concepts into a meaningful picture of their own knowledge or learning (Sharma and Xie 2008). The researchers in this study designed and developed a new blogging-mapping tool to combine the features of blogging with the ability to extract and manipulate concepts (see Figs. 2 and 3).

Fig. 2
figure 2

Attaching keywords

Full size image
Fig. 3
figure 3

Forming concept map

Full size image

As with existing blogging environments, the developed tool allows students to write posts on a website. In addition, students can also label each post with up to five keywords. Attaching appropriate keywords requires students to stop, think, and re-conceptualize and categorize their writing with keywords (Scardamalia 2003). Novak and Gowin (1984) have identified reflective thinking as “controlled doing, involving a pushing and pulling of concepts, putting them together and separating them again” (p. 19). To identify important concepts in their posts as “keywords,” students are literally “pulling” key concepts from their writing. Attaching keywords to posts could push students to reach beyond surface textual descriptions and perhaps promote thinking at a higher level.

Another reason for establishing keywords was to enable students to create a visual concept map where keywords acted as nodes in the maps. Connecting keywords would require students to form propositions among those keywords. While the selection of keywords might help students to identify the prominent ideas in a course (Cust 1996), the concept-mapping feature could encourage students to build a composite view about these core ideas rather than compartmentalizing them. As soon as a blog post was published, all keywords attached to that blog post were automatically fed into the concept mapping canvas. This feature allows accumulation of all keywords across blog posts and over time throughout the duration of the study. By manipulating the relationships of keywords in concept maps, students would further work with meaning by putting the concepts and key ideas together and separating them again (Novak and Gowin 1984). The following figure shows that in order to revise the concept map, each student will first choose the two keywords they wish to connect, and then add a linking word or phrase that indicates the relationship between the two concepts in the “attribute” box type. Students can repeat the steps until all keywords are connected with the rest keywords.

We also designed and embedded a “map archive” function in this online tool, which saved every version of a concept map whenever the student logged out of the system or closed the internet browser application. The map archive allowed each student to view a personal concept map history. The mapping activity was designed to support an iterative pattern of development and understanding (Land and Dornisch 2001) whereby students could revisit and revise their own work. While the design of the concept mapping tool allows manipulations similar to those seen with other concept mapping tools, i.e., the ability to generate and link nodes with semantic relationships, the tool developed for this study differs in that the nodes are based on keywords that are designated based on an a priori task—i.e., the selection of keywords for the blog post. Thus the tool requires the completion of three discrete tasks—generation of a blog post, reflection on blog post and identification of appropriate keywords, followed by manipulation and linking of keywords.

The concept mapping tool was programmed in PHP and MySql, which imposed some constraints on usage. (a) As mentioned earlier, the keywords attached to all blog posts were automatically fed into a concept map interface and accumulated throughout the duration of the study. With this feature, students were not required to brainstorm keywords at the time of concept map construction; (b) To revise a concept map, students’ only task in this activity was to link keywords with linking phrases (called “attributes” shown in Fig. 3). Unlike many other concept mapping activities, the tool also automatically generated the map structure. The technical constraints of PHP did not allow users to drag-and-drop objects on the concept-mapping canvas. Instead, the program automatically drew a graph as soon as an attribute was added to the map by using an anti-clustering algorithm. With this algorithm, the canvas was divided into a number of regions so that each keyword was placed into each region to avoid keyword overlapping while maintaining the shortest distances between keywords; (c) Because the map structure is automatically generated, the structure of the map could potentially change every time a new connection was added to the map.

Alignment of theoretical framework with design

This study used an approach similar to design-based research (Design-Based Research Collective 2003), thus requiring and addressing various validity issues in order for it to be rigorous. According to Hoadley (2004), the research validity of a design-based research study can be approached with the notion of alignment. He stated, “the validity of a study is the likelihood that our interpretation of the results accurately reflects the truth of the theory and hypotheses under examination” (p. 204). One type of validity that was carefully considered was treatment validity, i.e., that “the treatments we create accurately align with the theories they are representing” (p. 204). Figure 4 illustrates how the treatment/design is aligned with theory.

Fig. 4
figure 4

Alignment of design and theory

Full size image

Research questions

The research study focused on the concept map feature, which was focused on helping students to reflect on their knowledge and restructure their knowledge domain. The research question was thus: what evidence do concept maps (based on blogs) provide for student organization and connection of concepts over the semester? Since our goal in this study was to identify the extent to which concept mapping based on keywords attached to blog posts were able to support students in engaging in reflective learning (defined in this study as the process of identifying important concepts and generating connections between them), we sought to examine artifacts to identify and describe student organization of knowledge within the domain.

Method

Participants

The new blogging-mapping tool was implemented in an entry-level graduate course in instructional design. Apart from a 3-h face-to-face meeting every week, the class included hands-on projects, blog assignments, and a final reflection paper. The instructors required students to write posts at least once every week for 7 weeks to reflect on their learning in the course. Twelve students volunteered to participate in the study. However, three of them either entered the blogs at the very end of the semester or posted less than seven times. Therefore, their data were not included in this study.

Procedures

A blog writing aid (See Appendix A) describing the blog assignment was provided to students to guide their reflection during the first week of the class. The instructions, which offered a list of possible topics and keywords relating to the content of each week, acted as a supporting mechanism that “gave prominence to core ideas” to make such free writing assignment more successful (Cust 1996, p. 263). A 1-h class session was dedicated to demonstrating how to use the blog-map tool. During this session, the instructor and the researcher also explained the purpose of the features: by attaching keywords, students should aim to identify the prominent concepts in their blog posts that could represent their writing; the ultimate aim of attaching these keywords was to form a concept map continuously to represent their knowledge network throughout the semester. Students’ blog participation accounted for 10% of the course grade. Students were asked to write a blog post each week and revise their concept maps every 3 weeks. The researcher and the course instructor monitored the blog activities consistently.

Methodology

We employed a qualitative method to answer our research question, which focused on identifying patterns of students’ concept-mapping activity over time. Data were primarily collected from students’ artifacts created with this tool and included all their blog posts and all versions of the concept maps. Traditionally, qualitative study research can employ multiple sources of data such as interviews, observations, and documents. In this study, the research questions were focused on exploring and describing the evidence of organization and reorganization of student domain knowledge over time. Thus, data collection in this study primarily consisted of various documents. Interviews were not conducted because the focus of the study was not participants’ perspectives on, reactions to, or experiences of the blogging-mapping tool.

Data analysis

Analysis of concept maps

According to Hoadley (2004), measurement validity, a second aspect of research validity for design-based research, is “the ability to ensure that our measurements accurately reflect the constructs that we are trying to measure” (p. 204). The following section illustrates how we aligned data analysis with the treatment in this study.

Discussions about using concept maps as assessment tools abound (e.g. McClure et al. 1999; Rice et al. 1998; Ruiz-Primo 2004). Ruiz-Primo et al. (2001) compared two of the most commonly seen concept maps: a highly directed version called “fill-in-the-map” and a less-directed version called “construct-a-map-from-scratch.” Concept-mapping in this blogging-mapping tool closely resembled the “fill-in-the-map” technique, since students’ keywords were automatically accumulated and fed into a concept map interface throughout the semester so that they could not freely “brainstorm” concepts. The tool also automatically generated the map structure. The students’ only task in this activity was to link keywords with phrases. However, this tool was also significantly different from the concept maps referred to in other studies. This online blogging-mapping tool was available to students at any time (24 × 7) for a semester. The tool provided a platform where students could build and integrate knowledge over a relatively long period of time, unlike other “one-shot” concept-mapping activities. Hence, students’ performance over time was an important aspect to consider in understanding their learning.

In addition, learning content in this course was delivered in chunks of relatively self-sufficient packages of information each week. In other words, each week’s lesson focused on topics that were self-contained to some extent including topics such as needs assessment, task analysis, project management etc. It was relatively easy for students to connect concepts within one lesson because they naturally belonged to the same bundle of information. However, to connect concepts between lessons could be rather difficult since these concepts were probably assimilated into their schemata at different times without overt relationships. To associate such concepts, students needed to refer back to their knowledge network and themselves look for connections, thus making this task more effortful. With this blogging-mapping tool, students were encouraged to associate learning from different weeks and eventually systematize them into a unified whole. Thus, the degree to which students transcended the content of different lessons (by associating concepts from across different lessons’ posts) could serve as an important indicator of students’ learning. In summary, to study students’ concept mapping activity, the following factors were taken into consideration: (a) the existence of a relation between the concepts; (b) the accuracy of the label; (c) process of students’ construction of the concept map over time; (d) the degree of students’ knowledge integration indicated by within-post connections (i.e., propositional links between keywords generated for a single blog post) and cross-post keyword connections (i.e., propositional links between keywords generated for multiple blogs).

To measure the “accuracy” of the links, Ruiz-Primo et al. (2001) used a “proposition inventory” devised by subject matter experts within a specific content area. The inventory listed all qualitative variations of students’ propositions for all possible connections among concepts and assigned numerical scores to those propositions. In order to devise such an inventory, Ruiz-Primo and others engaged in a lengthy validation process with various subject matter experts. In addition, variations propositions were also derived from students’ previous work in this area. Given the constraints of the current research study, the researchers had no access to students’ previous concept maps (even if there were any) and were lacking additional subject matter expert support. Since there were few participants, analyses and rich descriptions of the links were provided instead, without assigning numeric scores to those links or referring to a pre-established inventory. Especially since our intent was to trace the degree of students’ concept mapping activity as illustrated through integration and cross-linkages of keywords, as well as the qualitative changes in concept maps over time, we refrained from imposing numeric assessments and instead focused on providing a snapshot of the rich activities of students over time by providing descriptions and graphic representations where appropriate.

Open coding

For concept map coding, as mentioned previously, participants could not arrange the appearance of their concept maps but only link concepts on the map. Thus, while analyzing the concept maps, the researchers first parsed the relationships from each version of the concept maps into the following format, “concept (linking word/phrase) another concept” as shown in the first column of the Table 1. To analyze the linking phrases on the concept maps, both researchers compared these propositions in relation to the blog posts with which they were associated. They were assigned to categories such as representative, distorted, partial etc. Table 1 shows examples of such categories and explanations how these categories emerged.

Table 1 Categories of relationships between concept map propositions and blog posts
Full size table

In addition, the researchers highlighted connections on the thumbnail versions of the concept maps to outline patterns of connections for all versions of the maps. These patterns provided the foundation for the researchers to study participants’ knowledge integration processes throughout the semester. After both researchers worked together on the first few posts and reached complete agreement on the analyses, one researcher processed the rest alone. Figure 5 shows an example of the open coding process for one participant’s concept maps.

Fig. 5
figure 5

Example of open coding concept maps

Full size image

Thematic coding

In the open coding stage, the researchers analyzed all of the student-generated concept maps, resulting in a synthesis of each student’s concept-mapping activity throughout the semester. Similarities and differences were extracted based on the synthesis of all student data, as suggested by Strauss and Corbin (1990). For cross-post connections of the final-version concept maps, a matrix (e.g. Miles and Huberman 1984) of all keywords and links (not shown in this presentation) was created for comparison and generation of themes.

Results

Changes of concept maps over time

Participants’ maps over time differed in two aspects: the percentage of connected keywords (e.g. number of connected keywords divided by the number of all keywords) and the components of the maps (the percentage of within- and cross-post connections). Within-post connections meant that propositions linked two keywords from one blog post whereas cross-post connections referred to those propositions that linked keywords from different blog posts. The researchers marked these patterns on all versions of participants’ maps. Tables 2, 3, and 4 depict participants’ connections on the concept maps by week and show the following themes about the revision patterns of their concept maps over time:

Table 2 Comparison of participants’ number of new connections by week
Full size table
Table 3 Comparison of participants’ percentage of connected keywords by week
Full size table
Table 4 Comparison of participants’ percentage of cross-post connections by week
Full size table
  1. (1)

    The number of new connections shown in Table 2. Except one participant (Joe), none of the other participants engaged in the mapping activities in a consistent manner. Most participants started making connections toward the end of the semester and revised their maps less than four times. Even though they started the mapping activity quite late into the semester, they did not make many connections on the map the first few times. The number of links dramatically increased the last time they revised their map for most participants,

  2. (2)

    The number of connected keywords as shown in Table 3. Similarly, except for Joe, the last few weeks of the semester saw a steady increase of the percentage of connected keywords. In addition, a surge of connected keywords occurred during the last week for most participants as they made dramatically more connections on the map.

  3. (3)

    The nature of cross-post connections as shown in Table 4. Because of the nature of the course, each lesson usually focused on one component of instructional design therefore, participants’ weekly post covered one content component. Although most of the participants started the concept mapping activities during the last few weeks of the semester, connections on the first few versions of their maps were mainly restricted to the keywords of their most recent blog posts. A high percentage of within-post connections signaled compartmentalized learning since it showed that student merely processed information within the boundary of the content component of that week without attempting to integrate knowledge from other areas. In contrast, if the map primarily consisted of cross-post connections, it could signal that the student attempted to integrate learning across different content components. As demonstrated by the percentage of cross-post connections across the participants, there was a steady increase of cross-post connections across participants over time except for Joe. In other words, most of the participants usually started the mapping activity by connecting the concepts within each content area and increasingly integrated concepts from different content components by the end of the semester.

The themes observed above seemed to indicate that students did not engage in the concept mapping activity in a consistent manner. The tool was provided for them to iteratively revise their knowledge network during the course of the semester. Yet, they refrained from doing so on the concept mapping tool and instead waited until the last week to reflect on the systemic relationship of the concepts of the course. According to Gestalt psychology (e.g. Perkins 1953), knowledge could involve leaps after flashes of insights that may follow a lot of step-by-step reflections and non-linear mosaic-like considerations, which could explain student actions. Also, knowledge integration did not appear to be easily facilitated. Students needed to allocate specific amounts of time for this effortful activity to occur. To proceed from “making sense” of knowledge components to “making meaning” and “linking ideas”, students needed to have accumulated a good amount of knowledge foundation of the content area.

Knowledge integration exhibited on their final maps

Since the final version of student concept maps indicated their ultimate level of systematized knowledge, the researchers also carefully examined the final versions of the maps to seek evidence of knowledge integration in addition to the analysis of week-by-week changes. Figure 6 captures the differences in final versions of the concept maps across participants. The X-axis represents the percentage of within-post and cross-post connections. Clusters refer to “within-post keyword connections” (that is, connecting keywords attached to a single post) whilst “All cross-post connects” indicate connected keywords attached to different blog posts. The Y-axis represents the percentage of a participant’s connected keywords. For example, George connected many of his keywords but not all. His position on the Y-axis should be close to the top but not exactly at the top. Many of his connections remained within clusters but some others were among keywords of different blog entries. Therefore, his position along the X-axis should be in the middle but closer to the all-cluster side.

Fig. 6
figure 6

Concept map connections of all participants

Full size image

Table 5 provides a numeric comparison of the two types of connections (within- and cross- post connections) across all participants.

Table 5 Comparison of within- and cross-post connections on the final map
Full size table

As indicated in the figure and table, three of the participants (Sue, Nancy and Emily) not only connected most of their concepts, but also presented an integrated picture of various content components by linking the concepts from all weeks. Their cross-post connections far outnumbered their within-post connections. Their final maps showed effortful attempts to integrate knowledge within this domain. On the other end of the spectrum, Mandy and Ross made few connections on the map and most of the connections were restricted to clusters of individual content components. Their maps showed little attempt at knowledge integration. Joe was an extreme case since his concept mapping behavior was consistently different from all other participants. The map archive showed that Joe constructed concept maps almost immediately after entering a blog post. The connections he made each time focused only on the newly attached keywords and he made no attempt to connect concept across content components. His final map demonstrated very compartmentalized connections. The following sections describe the themes found about the keywords, within- and cross-post connections on the final version of participant concept maps with details.

Unconnected keywords

What are the characteristics of the unconnected keywords? Examining their characteristics could possibly provide another avenue for elucidating participants’ learning and thinking. Analysis showed that most unconnected keywords had a marginal relationship with the blog texts to which the keywords were connected. For example, in one of Mandy’s blog posts, she described her previous experiences with instructional design when designing courseware. She realized that design needed application of theories to different situations. Besides concepts including “instructional design,” “learning theory”, and “courseware”, which were the key concepts in post, Mandy selected “computer assisted teaching” as a keyword. However, this keyword seemed to be a peripheral concept, because it merely provided a background for the design experience. When she connected her keywords in her concept map, she left such peripheral keywords out without connecting them to other concepts. Concept mapping required students to recall the course content and form relationships among them. As described above, these keywords seemed to be selected in a random manner or had a marginal relationship with the blog texts; the participants did not sufficiently process these concepts to understand their meaning. Hence, it could have proven difficult for participants to relate them with other concepts.

Within-post connections

Most within-post connections were restatements of the main ideas of the posts. For example, Nancy’s post detailed the iterative nature of instructional design. Nancy found from her experiences that instructional design did not follow a systematic but systemic pattern. The connection “models (types) systematic vs. systemic” reiterated her post. Although these connected keywords belonged to the same blog post, participants made the connections weeks after writing the blogs (as mentioned earlier, most participants waited until the end of the semester before revising their concept maps). The possibility was that these ideas were among the most engraved connections in their knowledge network. Two explanations could have accounted for the repetition of connections made in participant posts. First, some of the connections were among the main ideas of their post. When participants posted, they included many details before summarizing or offering conclusions. This post structure indicates that participants could have “worked with meaning” (Moon 1999) to an extent that the meanings became part of their knowledge network. Hence, even many weeks after they processed the information, they were able to remember those strong connections. Second, it is likely that common knowledge of the field could have been reinforced many times throughout the semester so that participants had internalized this knowledge by the end of the semester. It appears that this process of connecting keywords in the concept maps allowed participants to revisit their internalized knowledge, further refine and articulate the connections, and make the mental knowledge connections explicit.

Some connections made by participants indicated superficial processing of information, such as connections linked by phrases such as “inter-related,” “important in” and “of”, for instance, the relationship “objectives (important in) design” in Ross’s concept map. Ross only recognized the importance of objective to instructional design but he did not specifically pinpoint the exact nature of the relationship. He could have used propositions like “guide” to describe objectives guide design decisions. In addition, Beth and Ross connected some keywords without providing any linking phrase, for example “project management ()communication.” The relationships of these concepts might exist in participants’ knowledge network since they appeared in the same post. Yet, while making the links within the concept map, they refrained from defining the exact relationships. In other words, participants simply acknowledged the existence of the connections, but they might have been reluctant to make sense of those connections again.

Some other within-post connections reflected distorted (sometimes insensible) information of their blogs. For example, Mike characterized himself as a constructivist and reflected on how he learned in the past with different methods (behaviorist or constructivist approaches). Although he recognized these two approaches as different, he also admitted that they could supplement each other. In his post, he stated, “I think that both constructive and behaviorist are vital aspects of learning. It is my opinion that a well-rounded lifelong education will have elements of both.” Yet, in the concept map, he created “behaviorist (opposite) constructivist.” This word “opposite” could only describe one aspect of his characterization—different approaches, but not the other aspect—supplementation of each other. To account for the distorted or partial relationships, one explanation that fits the data is that as participants were engaged in identifying the relationships between two concepts, they might have focused on these concepts and other concepts in the vicinity within their knowledge network (Wang 2003), but they may have ignored or lost sight of the more holistic context. Table 6 compares the two types of connections of all participants.

Table 6 Comparison of two kinds of within-post connections on the final map
Full size table

Cross-post connections

The blogging-mapping tool was designed to allow students to form an integrated view about the core concepts in the field, instructional design in this case. The following section describes students’ knowledge integration as evidenced by their concept maps by analyzing connections to the central concept of instructional design, as well as the patterns of cross-post connections.

Connections to instructional design

The concept “instructional design” was the hub of five participants’ concept maps. The keywords connected to instructional design generally fell into the following three categories: phase or element of instructional design, underpinning base of instructional design, and nature of instructional design. Examples of the first category—phase or element of instructional design—might include, needs assessment, models, evaluation, ADDIE, project management, media selection, task analysis, implementation, instructional goal, learning objectives, and development etc. Examples of the second category–underpinning base of instructional design—might include constructivism, cognitivism, and behaviorism. Examples of the third category—nature of instructional design—might include, art, science, systematic, and interdisciplinary field. These different categories possibly meant that students’ understanding about the core concept of the course encompassed several dimensions. Since no single class presented a unitary view of the field, the final concept maps of many participants exhibited unified connections, especially many connections to instructional design. It probably indicated that students “worked with meaning” (Moon 1999) on a higher level than what was conveyed in a single lesson.

Indication of partial or sparse systematization of knowledge

The preceding section discussed the finding that five participants selected instructional design as the hub of their concept map and that the connected keywords covered three aspects of instructional design: component, nature, and fundamental philosophy. However, further analysis revealed that different students emphasized different dimensions but none of them covered all three aspects.

Sue, Beth and Emily connected keywords in the first dimension (phases and components), while Nancy’s connections leant more towards the second and third dimensions—nature and philosophical base of instructional design. Ross’s and Mandy’s connections to instructional design touched upon every dimension, but were not comprehensive in any aspect—these connections were rather chaotic without any conceivable pattern. The incomplete and chaotic nature of the connections might indicate that students failed to organize concepts in a meaningful order.

This partial systematization of concepts could have resulted from participants’ unwillingness to expend extra effort or from technological barriers presented by the blogging-mapping tool to their concept-organizing process. The technology constraint of the tool might have exerted some hardship for students to manipulate their concept maps. As previously mentioned, the concept mapping tool presented a different task demand and imposed a different constraint from the “construct-a-map-from-scratch” technique. If students were to build a map from scratch, students needed to identify concepts, clarify relationships among concepts, and organize relationships by rendering a structure to the relationships (Naveh-Benjamin et al. 1986). However, in this blogging-mapping environment, students could only focus on the task of clarifying relationships because the tool did not allow for a structure-rendering process. Also, the structure of the map automatically changed every time a new connection was added to the map. Therefore, it required much more cognitive processing for a participant to mirror every version of the map to their knowledge network structure. After a while, students may have become cognitively lost and concept mapping may therefore have become a relatively passive activity—grabbing two concepts randomly on the map and identifying a link, without coupling the map structure to their knowledge structure.

Conclusion of concept-mapping activity

Results showed that most of the within-post relationships in the concept map reflected the view of the participants’ blogs and were consistent with general knowledge in the field of instructional design. However, some connections still did not correspond to participants’ expressed views or generally accepted perceptions of the field. Some participants’ final concept maps showed possible attempts to integrate knowledge around instructional design. Yet, their knowledge integration was far from being completely unitary or systematic. As Fig. 6 showed, although most participants connected all of their selected keywords, their final versions of the concept maps were spread along the continuum of “clusters” and “cross-post connections.”

Discussions and future research

Writing blogs was a way to allow students to revisit the information they learned in class. In addition, the keyword-tagging feature was intended to allow students to stop and think about the content of their blog posts and further conceptualize the blog content for the subsequent concept-mapping activity. The concept-mapping feature aimed to encourage students to form a composite view about these core ideas instead of compartmentalizing them (Rosie 2000). Concept mapping each week aimed to offer students another opportunity to “work with meaning” by manipulating the relationships of keywords (Moon 1999). However, many users did not engage in consistent concept mapping activities. Most of the participants revised their concept maps less than four times. Moreover, most in-progress maps showed compartmentalized learning since the links were limited to keywords within the same posts. By doing so, students might have formulated connections between concepts but not systematized or integrated them until right before the semester ended. The final concept maps of Emily, Sue, and Nancy were well integrated, breaking through the boundaries of individual post or lesson content and demonstrated a relatively comprehensive understanding of the course content. Other students, however, made few attempts to integrate their concept maps. As Gestalt psychology (e.g. Perkins 1953) indicated that learning probably happens in big leaps and knowledge integration usually could not happen naturally. Students need to allocate specific amount of time for this effortful activity to occur. To proceed from a relatively shallow stage of learning to knowledge integration, students needed to have accumulated a good amount of knowledge foundation of the content area. This finding reiterates existing research that suggests that reflection is effortful and requires assistance (e.g., Harri-Augstein and Thomas 1991). However, the ability to distinguish within-post and cross-post connections from the concept maps suggests that one avenue for supporting reflective learning and systematization of knowledge is the identification of misconceptions or superficial connections, allowing facilitators to address and focus on important connections and understandings within the content area. The ability to examine and parse connections generated by the students, in conjunction with the narrative blog posts allows for a more comprehensive examination of student cognition and understanding (see example, Pellegrino et al. 2001).

Novak and Cañas (2008) stressed the role of “emotional commitment” in the process of students’ meaningful learning. They stated, “the learner must choose to learn meaningfully…The one condition over which the teacher or mentor has only indirect control is the motivation of students to choose to learn by attempting to incorporate new meanings into their prior knowledge” (p. 2). The researchers identified two kinds of strategies to promote students’ meaningful learning including instructional and evaluation strategies. This blogging-mapping tool only used instructional strategies because its features made it explicit to attach keywords and form connections among concepts. In addition, the Web-enabled concept mapping could enable students to save, revisit, reflect upon, share and explore complex problems flexibly from any Internet terminal (Luckie et al. 2004; Pea et al. 1999). However, the quality of participants’ activities on the site was not evaluated toward their final grades of the course. It is possible that students may have lost momentum without enough incentive or further encouragement. Although the blog writing aid might have imposed expectations from this assignment, without additional explicit evaluation rubrics for their concept mapping activities, students might have lacked emotional commitment to engage in meaning making through concept mapping activities. This finding also suggests that explicit reference to the tool and its utility, as well as to the importance of engaging in the process might help students engage more readily with such reflective tools and practices.

The study suggests that this blogging-mapping tool has some potential for being used as an effective tool to help students reorganize students’ learning and thinking in courses requiring integrated learning. The following issues can be reasonably implied from the result: (a) Integrating concepts gathered over the whole class period seemed to require more effort than just reorganizing and linking together the concepts in one post or lesson. Even though several students attempted to integrate concepts beyond lesson boundaries, their final concept maps still consisted of many clusters. In order to facilitate deeper learning with concept mapping activities, the instructor should provide definitive evaluation rubrics to sustain students’ emotional commitment. (b) The continuous availability of the tool (24 × 7) allowing students to revise the concept maps at any time did not necessarily encourage constant mapping activity. Most of the students constructed concept maps toward or at the end of the semester. Even though some students modified their maps during the semester, the maps did not show an iterative pattern of development or understanding, since the modification was limited to new learning. This lack of activity suggests that participants did not adopt the concept mapping strategy as a natural next step after keyword identification. Knowledge integration did not seem to be a progressive or gradual accumulation throughout the semester but rather an abrupt or instantaneous product at the end. Or at least it was less effortful to integrate information at the end of the study process. (c) The new tool allowed students to re-construct knowledge from concepts, unlike the Knowledge Forum (Scardamalia 2003) where students build concept maps with complete posts on a discussion board, wherein a post could encapsulate many concepts. In contrast, this tool allowed for knowledge construction from more fundamental units of information rather than packages of information as in the Knowledge Forum. Results showed that many students tried to break the boundary of individual posts and connect concepts from different entries. Many cross-post connections were meaningful and served as signs of integrated learning. However, some participants’ concept maps still had clusters. Because the clusters were built naturally on each topic during the semester, the breaking of these boundaries at the end of the semester and forming connections across different topics might require deeper processing of information. If students were asked to build connections among those stand-alone self-sufficient posts, students might be encouraged to compartmentalize information in each post. Besides, since each student wrote an average of seven posts, the connections among seven posts could have been much easier to build compared to those among 30–40 concepts—the latter situation would certainly require more cognitive processing.

The study was limited due to the scope of data collection, available methods of analysis, and the researchers’ identity (the researchers were the instructors and the designers of the tool). Data collected in this study were limited to the blogs, keywords and generated concept maps after the semester was over. Because of the non-intrusive nature of the research questions and limitations of researcher’s identity and involvement with the tool development, data collection precluded interviews or observations. Hence, it was not possible to identify what hindered students from using some features of the tool, and what students’ cognitive processes were when they attached keywords and revised their concept maps. The researchers’ own understanding of the field could have biased the processing of the blogs. These biases could have also affected the interpretations of the concept maps.

Further research should involve revision of the design including the improvement of the tool and flexible research procedures. With a new design of a tool allowing real manipulation of concepts by students, the instructor should provide more motivational strategies, such as explicit requirements or definitive evaluation rubrics of the blogging-mapping activities. Students may then feel strongly committed to the assignment and they would more likely engage consistently and meaningfully in the activity. The next phase of this research will look at “multiple aspects of the design and develop a profile that characterizes the design in practice” (Barab and Squire 2004, p. 4) by examining students’ experiences of using this tool in class with interviews or possibly observations about their usage.