Concept mapping is a method for representing knowledge graphically. Nodes represent concepts, and labeled links represent the relations between the concepts. The idea of concept mapping is based on Ausubel’s assimilation theory of cognitive learning (Ausubel et al. 1978). According to Ausubel, the mind organizes information in a hierarchical top-down fashion. Concept maps visualize how concepts are hierarchically related and how concepts on the same hierarchical level are interrelated. As several studies have shown, meaningful learning can be assisted through the construction of a concept map (Novak 1990). A variety of studies have demonstrated the effectiveness of concept mapping as a learning method, including a meta-analysis of 19 quantitative studies by Horton et al. (1993). The effectiveness of concept mapping has been compared to several other learning techniques. For example, learners who used concept mapping as a learning strategy performed better than learners who used underlining (Amer 1994), note-taking (Reader and Hammond 1994), discussing with co-learners (Chularut and DeBacker 2004), or outlining (Robinson and Kiewra 1995).

There has also been some research on worked-out concept maps. These are maps that have been constructed by a teacher or an expert and provided to students as a learning tool. In particular, students with low verbal abilities have been shown to benefit from studying such worked-out maps (O’Donnell et al. 2002; Rewey et al. 1989). In addition, learners with low prior knowledge of the content domain also profit in particular from learning from worked-out concept maps (Lambiotte and Dansereau 1992; O’Donnell and Dansereau 2000). The influence of prior knowledge and verbal abilities on learning by concept-map construction has not yet been tested.

In this article, we analyzed what characterizes good and poor mappers in terms of their cognitive processes during mapping and the type of map they construct. In addition, we tested whether general abilities facilitate effective mapping.

What makes concept mapping so beneficial?

Novak (1995) describes a variety of applications of concept mapping in learning. For example, concept maps can assist the preparation of lessons and the sequence of topics presented. They can also serve as a basis for discussions or as a tool for knowledge evaluation. Furthermore, concept maps can assist learning from texts. It is this latter application of mapping that is the focus of this article.

Concept mapping as a follow-up strategy in text learning can have several important functions. We differentiate four main functions:

  1. (a)

    Elaboration function. Due to the affordance of expressing notions in nodes and relations in links, concept maps foster elaboration processes (Weinstein and Mayer 1986). This means that learners have to relate new information to their prior knowledge in order to determine what concepts are important and whether and how they interrelate.

  2. (b)

    Reductionfunction. Weaver and Kintsch (1991) found that macropropositions which contain the top-level information of a text are recalled in more detail. Maps can enhance the acquisition and retention of macrolevel ideas (O’Donnell et al. 2002). Learners have to appraise the importance of concepts in order to decide whether they should integrate them in their concept map. Thus, learners concentrate on the most relevant macrostructure information of their learning topic.

  3. (c)

    Coherence function. Concept mapping requires the externalization of knowledge and its structure. Thereby working memory is offloaded and the construction of coherence is facilitated (Kintsch 1998). Labeling the links connecting nodes emphasizes the kind of relationship between concepts. Additionally, spatial arrangements or the use of similar colors can emphasize that certain concepts belong together. Thus, concept mapping fosters the building of a coherent structure of knowledge.

  4. (d)

    Metacognitive function. Metacognitive processes are supported through concept mapping. Knowledge and comprehension gaps can become obvious when constructing and explicating relations between concepts (e.g., Chi et al. 1989). At best, learners can overcome these gaps when they become aware of them.

When analyzing cognitive processes during concept mapping, activities that fulfill these functions should occur. We assume that successful learners perform such activities more often.

Beginners’ difficulties in concept mapping

Although concept mapping fosters learning and understanding, beginners are easily overwhelmed by the demands of mapping. Participants in a study by Reader and Hammond (1994) learned from a hypertext either by note-taking or concept mapping. Even though learners in the concept-mapping condition performed better in a post-test on the learning topic presented in the hypertext, qualitative analyses showed that they failed to structure and integrate the information provided by the hypertext in an appropriate way. The learners were not able to use the advantages of the method to the expected degree.

In order to cope with beginners’ difficulties in using concept maps for learning from texts, a few training studies have been conducted. For example, Chang et al. (2002) compared students’ learning outcomes after either correcting worked-out concept maps with errors included, constructing their own concept maps after a training session, or constructing concept maps without training. The students learning by correcting concept maps achieved the best results in a learning test on text comprehension. Students learning by constructing a map without training performed worst. It is, however, noteworthy that even after training students could still not perform as well as the students who learned by correcting concept maps. Similar advantages of studying worked-out maps in comparison to map construction were reported by Hauser et al. (2006).

Although the employment of worked-out concept maps is a promising method (e.g., Chang et al. 2001, 2002), the maps are often unavailable and laborious for instructors to construct. In the long term it is preferable that learners construct their own maps. In addition, instructional techniques such as using worked-out concept maps that are highly effective with inexperienced learners can lose their effectiveness with more experienced learners (cf. “expertise reversal effect”, Kalyuga et al. 2003). However, an ideal training method for students on how to use concept mapping that is effective, efficient, and directed to the typical needs of beginners in mapping, is missing at present. This is also due to the fact that there is little empirical evidence indicating which cognitive processes are actually crucial for successful mapping and which deficits beginners have in this respect. Most studies merely report some anecdotes of the learners’ difficulties during concept mapping (e.g., Jonassen et al. 1993). Knowing beginners’ specific deficits in detail is, however, necessary in order to develop effective training methods.

Research questions

In order to develop a strategy for teaching concept mapping that is adapted to the learners’ needs, it is important to first explore the processes that contribute to effective concept mapping. Thus, the aim of the present study was to analyze individual differences between learners with respect to those processes. Data on the cognitive processes elicited by concept mapping and concept-map characteristics were collected.

As several studies found that learners with low verbal knowledge profited more from learning with worked-out concept maps (O’Donnell et al. 2002; Rewey et al. 1989), we also measured the learners’ verbal abilities. In addition, we took the learners’ spatial abilities into account because those with greater spatial abilities could have advantages in mapping as it requires a spatial arrangement of concepts. Furthermore, it was questioned whether concept mapping as a follow-up activity to reading has a positive effect on learning in any way. The following specific research questions were addressed:

  1. 1.

    Does concept mapping—as expected—have a positive effect on learning? A variety of studies indeed found a beneficial effect of concept mapping on learning, including the meta-analysis by Horton et al. (1993).

  2. 2.

    Do the learners’ verbal and spatial abilities have an effect on learning outcomes? Prior studies such as Rewey et al. (1989) and O’Donnell et al. (2002) suggest a relevance of verbal abilities. The figural-spatial character of maps makes it sensible to also include spatial abilities.

  3. 3.

    To what extent are individual characteristics of the concept maps related to the learning outcomes? Are the learning outcomes associated with the quality of the cognitive processes during the concept mapping? To develop a training strategy for concept mapping we should take into account whether specific characteristics of concept maps are particularly beneficial. If this is actually the case, the training should instruct learners to realize these features. In addition, the cognitive processes that characterize effective concept mapping constitute a related issue that training of concept mapping should address. It is assumed that learning outcomes are fostered especially by those cognitive processes that refer to the elaboration function, the reduction function, the coherence function, and the metacognitive effect of mapping.

  4. 4.

    Can different mapping styles be identified with respect to the learning success? If there are different styles of beginners in mapping, different training variants may be necessary so that the intervention is tailored to the specific needs of the learners. It is also important so see whether specific styles are connected with some cognitive prerequisites, such as the learners’ verbal and spatial abilities.

Method

Participants

University students were invited to participate in the present investigation by posters in the university refectories and dormitories (University of Freiburg, Germany). They were informed that they would acquire a new learning strategy by participating. Thirty-eight students (17 male, 21 female, mean age 23.8 years) volunteered to participate in this experiment. Participation was rewarded with €8. All participants were native speakers of German.

Materials

Newspaper articles on stem cells

The learning contents were provided in six relatively short German newspaper articles on stem cells. The newspaper articles had a total of 2,116 words, the length of the shortest text was 80 words, and the longest text 1,029 words. The stem cell-topic was chosen because most students are relatively unfamiliar with the specifics related to this issue. Furthermore, it is a topic that is rather complex (i.e., requires integration of biological and ethical knowledge) and is widely discussed in Germany. Providing more than one information source increased the motivation (necessity) to engage in a follow-up activity after reading.

Mapping software

The participants worked with the Easy Mapping Tool, software especially developed for concept mapping (http://www.cognitive-tools.com). This software provided different forms and colors for nodes. Links to connect the nodes could be labeled by users.

Tests on verbal and spatial abilities

In order to test the learners’ verbal and spatial abilities we used two subtests of the I-S-T 2000 (Intelligence-Structure-Test 2000; Amthauer et al. 1999). The I-S-T 2000 is a widely used intelligence test in Germany. In this study, two subtests were used. Verbal abilities were measured by the subtest on complementing sentences. Each of the 20 tasks consisted of a sentence in which one word was missing. The correct word could be chosen out of a set of five words. Time for completing the subtest on verbal abilities was limited to 6 min. The subtest on spatial abilities consisted of cube problems. In each of the 20 problems the participants saw three sides of a cube with six sides. The participants’ task was to choose a rotated version of the cube out of five given cubes. The time limitation for the subtest on spatial abilities was 9 min.

Concept-map experiences questionnaire

To assess whether participants had experiences with concept mapping as a learning strategy prior to this study, they were first shown a picture of a concept map in a questionnaire. We asked: “Diese Abbildung zeigt eine so genannte Concept Map. Haben Sie eine solche Concept Map schon einmal gesehen?” [This is a picture of a so-called concept map. Have you ever seen such a concept map before?] This question could be answered with ‘yes’ or ‘no’. Afterwards participants were asked whether they had constructed concept maps on their own. This latter question was answered on a Likert scale ranging from 0 (never) to 5 (very often). Furthermore, some demographic questions had to be answered in this questionnaire.

Learning assessments

Two tasks measured learning outcomes. First, a post-test consisting of seven multiple-choice items on stem cells provided a measure of how much the learners integrated information of the different articles after concept mapping (integration test). To answer these questions, integration of the knowledge of at least two articles was required (e.g.: “Was erlaubt bzw. verbietet das deutsche Embryonenschutzgesetz?” [What is allowed and forbidden by the German stem cell law?] Two articles provided the information for the complete answer to this question). A still acceptable split-half reliability of .69 was determined.

Second, the students were asked to answer the question “Was sind Stammzellen und was kann man mit ihnen machen?” [What are stem cells and what can be done with them?] The ‘stem cell question’ had to be answered (1) in the pretest and had to be revised (2) after reading the articles and (3) after producing the concept map. Comparing the revisions of answers before and after concept mapping provided information on the changes in the participants’ declarative knowledge of stem cells. The participants answered the stem cell question on the computer using Microsoft Word software. This allowed them to revise their answers easily without having to re-write their previous answers or parts of it. This procedure assumes participants’ willingness to answer the same question three times.

Procedure

The participants first filled out a questionnaire regarding concept-map experiences and demographic data. Afterwards, participants completed the verbal and spatial abilities tests. Then they answered the stem cell question to assess their prior knowledge about the topic in question (i.e., stem cells). They were then given six newspaper articles concerning stem cells. Participants were provided with enough time to read each article once (15 min). After reading, they revised their answers to the stem cell question. Next, the participants produced a concept map about the topic of stem cells. To this end, they received a short introduction to concept mapping. The terms ‘node’ and ‘link’ were explained and an example of a concept map (topic: cows) was provided. Concept mapping was described as a task of selecting important concepts and linking them. No further tips for producing the concept map were given. The maps had to be produced with the mapping software Easy Mapping Tool and the participants received a brief instruction on how to use the software. Participants were instructed to think aloud during concept mapping. The thinking-aloud procedure was trained using a warm-up problem (Tower of Hanoi with three discs). For concept mapping 30 min were assigned.

The participants had access to the newspaper articles during concept mapping. Actually they were encouraged to rely on these newspaper articles for constructing their maps. They were asked to verbalize their thoughts during the construction of the concept map. The corresponding instruction was structured according to the guidelines of Ericsson and Simon (1993). Participants were only asked to talk aloud and verbalize anything that came to mind. They were not instructed to provide specific information. Thus, the participants’ spontaneous thoughts were assessed. After 15 seconds of not speaking, the experimenter asked the participants to “(Please) keep talking”. After concept mapping, participants had to revise their answers to the stem cell question again. Finally, they worked on the integration test.

Codings

Coding of the answers to the stem cell question

The stem cell question was answered by the participants before reading the newspaper articles on stem cells. After reading the newspaper articles the participants revised their answers a first time, and following concept mapping they revised their answers a second time. The answers and revisions of the answer to the stem cell question were segmented into propositions and classified into two distinct categories:

  1. 1.

    Correct: This category referred to extempore produced correct propositions (first answer to the stem cell question) or correctly reproduced propositions (revisions of the answer to the stem cell question) or propositions that contained a correct conclusion.

  2. 2.

    Incorrect: This category referred to information that the participants produced extempore (first answer to the stem cell question) or reproduced incorrectly (revisions of the answer to the stem cell question) or to an incorrect conclusion.

Interrater-reliability was determined by Cohen’s Kappa (Cohen 1960). It was good, κ = .78.

Knowledge increase

A measure for the change in students’ declarative knowledge on stem cells through concept mapping was provided by the changes in the number of correct and incorrect propositions from the first revision of the answer to the stem cell question (after reading the newspaper articles) to the second revision (after concept mapping). This measure was computed by summarizing the increase of correct propositions from the first to the second revision (correct propositions in the 2nd revision minus correct propositions in the 1st revision) and the decrease of incorrect propositions from the first to the second revision (incorrect propositions in the 1st revision minus incorrect propositions in the 2nd revision).

Concept maps

A coding system for the participants’ concept maps was developed, similar to that of Rafferty and Fleschner (1993). They presented a method to evaluate concept maps by scoring the number of valid relationships between concept-nodes, and the number of cross-links between distinct segments. In the present study, the scoring system was slightly modified. The following concept-map characteristics were analyzed (examples are provided in Fig. 1):

  1. 1.

    Number of concept nodes: All the nodes in the concept map were counted (see number 1 in Fig. 1).

  2. 2.

    Number of correctly labeled links: All the links that were correctly labeled to connect the concept-nodes were counted (see number 2 in Fig. 1).

  3. 3.

    Number of unlabeled links: All the links that were not labeled were counted (see number 3 in Fig. 1).

  4. 4.

    Number of cross-links: Cross-links are links that connect different sections of a map’s hierarchy, indicating knowledge about relations between different sub-areas of the topic. The number of cross-links was counted (see number 4 in Fig. 1 for an example).

Fig. 1
figure 1

Concept map on stem cells

Full size image

Thinking-aloud protocols

The thinking-aloud protocols were segmented into verbal units. A verbal unit comprised a segment of thoughts about one specific topic, for example, thoughts about a concept or a pair of concepts. Our coding system comprised ten distinct categories:

  1. 1.

    Elaboration of concepts. The learner connected his own knowledge with the learning content by generating examples or paraphrasing (Weinstein and Mayer 1986). This category refers to the elaboration function of concept maps (e.g., “If I took one of my skin cells and put the nucleus in an ovum, an embryo would grow that is just like me. Its stem cells would fit me perfectly.”).

  2. 2.

    Relevance. This category was coded when a statement referred to the importance of a concept. By including only the important concepts in the maps, learners can concentrate on the most relevant macrostructure information of the stem cell topic (O’Donnell et al. 2002) (reduction function; e.g., “Embryonic stem cells are an important sub-category of stem cells that is why I will include them in my map.”).

  3. 3.

    Relationships. The learner clarified the relation between concepts by thinking about how to label the links, by using colors, or structuring the concept map in a way that accentuates groups of concepts (coherence function; e.g., “There’s a connection between stem cells and pre-implantation diagnostics. You need stem cells for the PID. I’m going to label this link with ‘are examined’.”).

  4. 4.

    Negative monitoring. This category contained utterances about comprehension problems (Chi et al. 1989) (metacognitive function; e.g., “I just don’t get this thing with the embryonic germ cells. Are they the same as stem cells?”).

  5. 5.

    Positive monitoring. The learner stated his or her understanding of the contents (metacognitive function; e.g., “Yes, now I understand.”).

  6. 6.

    Planning and controlling. Verbal units were coded within this category that referred to the progress of the concept map such as planning the next steps or controlling whether contents are missing (e.g., “I didn’t explain the characteristics of stem cells. That’s what I have to do next.”).

  7. 7.

    False statements. Incorrect statements of the participants were coded in this category.

  8. 8.

    Activity report. This category was coded when the learner described what he was doing without referring to the contents (e.g., “Now I’m making a node here and connecting it to the other one.”).

  9. 9.

    Reading aloud. This category was scored for reading aloud the articles on the stem cell topic that the participants used for constructing their concept maps.

  10. 10.

    Non-content problems. The learner talked about general problems with the mapping method, the mapping software, or the demands of thinking aloud (e.g., “I’m not used to thinking in nodes. If I was learning this stuff on my own I would prefer making a table.”).

The thinking-aloud protocols were independently coded by both the first author and a student research assistant. The interrater-reliability expressed as Cohen’s Kappa was .73. This amount of interrater agreement can be regarded as satisfactory. In cases of divergence, the first author re-examined the protocols and made the final decision.

Results

The majority of participants (63%) did not know what concept maps were prior to the experiment. The question of how often the participants constructed concept maps prior to the experiment had to be answered on a Likert scale from 0 (never) to 5 (very often). Participants’ answers to this question ranged from 0 to 2 (M = .29, SD = .61). Hence, even though some learners had previous experience, they could not be considered “experts” in constructing or learning by concept mapping. There were no differences in the learning outcomes as measured by the integration test between learners who knew about concept maps and those who had not heard of this learning strategy before, t(36) = −1.07, ns. The frequency with which participants learned by concept mapping prior to the experiment did not correlate with knowledge integration either, r = .29, p = .074. However, there was a difference between learners who knew about concept maps and those who had not heard of this learning strategy before with respect to their knowledge increase as measured by the stem cell question, t(36) = −4.86, p < .001. With an average score for their knowledge increase of 5.58 (SD = 1.88), participants who knew about concept maps prior to this study outperformed participants who had not heard of this learning strategy before, M = 2.00, SD = 2.21. The frequency with which participants used concept mapping for learning prior to the experiment also correlated with knowledge increase, r = .37, p = .022.

Knowledge acquisition

In order to confirm the benefits of concept mapping for learning, we analyzed the answer to the question “What are stem cells and what can be done with them?” Table 1 displays the means of correct and incorrect propositions at the three different measuring times.

Table 1 Mean (standard deviations in parentheses) of correct and incorrect propositions at the three measurement points
Full size table

We performed an ANOVA with repeated measurements with Greenhouse–Geisser correction (Greenhouse and Geisser 1959) in order to test knowledge increase over the three measurement points (i.e., before the text, after the text, after mapping). We found that correct propositions increased significantly, F(1.7, 61.2) = 172.04, p < .001, η 2 = .82 (strong effect). The increase of incorrect propositions over the three measuring times was also significant, F(1.7, 61.9) = 54.13, p < .001, η 2 = .59 (strong effect).

The participants increased their knowledge on stem cells through reading. After concept mapping, a further significant increase of M = 3.6 (SD = 2.51) correct propositions was observed, F(1, 37) = 76.0, p < .001, η 2 = .67 (strong effect). The number of incorrect propositions after concept mapping did not significantly change, F(1, 37) = 3.70, ns. However, the change in incorrect propositions ranged from −3 to +3 for different individuals. Obviously, some learners corrected their wrong assumptions that they had before concept mapping, whereas other learners came to new wrong conclusions.

The changes in the number of correct and incorrect propositions from the first to the second revision of the answer to the stem cell question were used to compute knowledge increase. Participants’ value in this learning outcome score ranged from −2 to 9. The mean score for the knowledge increase was M = 3.13 (SD = 2.68).

Verbal and spatial abilities

In both the tests on verbal and spatial ability, a maximum of 20 points could be achieved. In the subtest of verbal abilities the participants reached M = 13.43 points on average (SD = 2.50). In the subtest of spatial abilities they reached M = 12.03 points on average (SD = 4.46).

Table 2 shows the correlations for the learning outcome measures with the participants’ verbal and spatial abilities. We found no significant correlation between verbal or spatial abilities and knowledge increase (Table 2). Thus, the learners’ cognitive prerequisites in terms of verbal and spatial abilities showed no relation to their changes in knowledge about stem cells as measured by the stem cell question. There was also no relation between knowledge integration and the learners’ verbal abilities. However, the higher the learners’ spatial abilities were, the higher was their achievement in the integration test, r = .33, p = .04 (Table 2). Thus, it is possible that higher spatial abilities helped learners to integrate the information that was provided in the six newspaper articles on stem cells to a higher degree when learning by concept mapping.

Table 2 Correlations between the verbal and spatial abilities and learning outcomes, concept-map characteristics
Full size table

As shown in Table 2, we did not find significant relations between the concept-map characteristics and the learners’ verbal abilities. The same was true for learners’ spatial abilities.

There were also no relations between learners’ verbal and spatial abilities and the cognitive processes during concept mapping, except for two variables (cf. Table 3). Firstly, participants with better verbal abilities showed less activity report, that is, they made few comments on the design of the concept map that did not refer to the stem cell topic, r = −.43, p = .008. Secondly, participants with higher spatial abilities planned and controlled the mapping process more, r = .44, p = .005 (i.e., ‘planning and controlling’ in Table 3).

Table 3 Correlations between verbal and spatial abilities and students’ cognitive processes during concept mapping
Full size table

On the whole, verbal and spatial abilities were not associated with most of the learning processes and outcome measures. However, learners with high spatial abilities planned and controlled the mapping process more and integrated information from different sources to a larger extent. Finally, learners with high verbal abilities provided less activity reports.

Concept-map characteristics

The concept maps that the learners produced were analyzed according to the number of concept nodes, the total number of links, the number of correctly labeled links, and the number of cross-links. The concept maps that learners produced contained, on average, 18.58 nodes (SD = 4.56). On average, 19.08 correctly labeled links connected the nodes (SD = 5.19). The number of unlabeled links in the participants’ concept maps was M = 11.55 (SD = 6.37). Cross-links appeared relatively seldom (M = 1.71; SD = 2.00). Table 4 provides an overview of the correlations of the concept-map characteristics with the learning outcome measures.

Table 4 Correlations between concept-map characteristics and learning outcomes
Full size table

Learners’ knowledge increase as measured by the stem cell question was positively correlated with the number of concept nodes in the participants’ concept maps, r = .33, p = .043 (Table 4). Thus, more concept nodes were related with a higher increase of knowledge about stem cells. No further correlations have been found for the learners’ knowledge increase (Table 4). The degree to which learners integrated the information from the six newspaper articles (integration test) showed a highly positive correlation to the number of correctly labeled links in the learners’ concept maps, r = .57, p < .001. Thus, the more learners labeled the links in their concept maps, the better were they able to integrate the knowledge about stem cells from the different newspaper articles. On the other hand, the more unlabeled links the learners included in their concept maps, the worse was their achievement in the integration test, r = −.41, p = .011. Neither the number of concept nodes nor the number of cross-links correlated with the learners’ scores for the knowledge integration (Table 4).

In summary, the concept-map characteristics that related most to the learning outcomes were the number of correctly labeled and unlabeled links. It seemed to be important for learning not only to recognize that certain nodes are related, but also to specify the type of relationship between the concepts. This interpretation was also supported by the results of the thinking-aloud protocols, as reported in the following.

Cognitive processes during concept mapping

The thinking-aloud protocols were segmented into verbal units and coded into ten distinct categories. Table 5 presents an overview of the relations between the cognitive processes during concept mapping and the learning outcomes.

Table 5 Correlations between students’ cognitive processes and their learning outcomes
Full size table

As Table 5 shows, the elaboration of concepts neither correlated significantly with the integration test nor with the increase of knowledge as measured by the stem cell question. Statements referring to the relevance of concepts were also not significantly related to learning outcomes.

Learners who reflected more often on relationships between concepts did not show a higher increase of knowledge about stem cells, however they performed better on the integration test, r = .40, p = .013 (Table 5). Thus, efforts to assay the precise relation between concepts have been found to be especially crucial. This is consistent with the finding that a greater amount of correctly labeled links in the participants’ concept maps was associated with better knowledge integration. A post hoc determined correlation of the amount of verbal units that deal with the relationship between concepts and the number of correctly labeled links in the participants’ concept maps shows that deep thinking about the kinds of relationships is indeed reflected in the concept maps, r = .36, p = .028. Learners who thought more frequently about the type of relationships between concepts labeled their links more often correctly.

Participants who were more likely to express comprehension difficulties (negative monitoring) achieved poorer results in the integration test, r = −.41, p = .012 (Table 5). Learners seemed to have recognized their knowledge gaps, but were not able to overcome them. Negative monitoring was negatively related to the explication of relationships (r = −.54, p = .001). Possibly, the comprehension problems hindered the learners from articulating the relationships between concepts, and this impaired the integration of knowledge. Negative monitoring did not correlate with the learners’ knowledge increase (Table 5).

Positive monitoring was related to a higher increase of knowledge as measured by the stem cell question, r = .43, p = .007 (Table 5). Apparently the learners’ self-estimation of their understanding was strongly associated with the actual increase of knowledge. However, positive monitoring was not related to the knowledge integration (Table 5).

Planning and controlling the mapping process was significantly correlated to the learning outcomes as measured by the integration test, r = .57, p < .001 (Table 5). Planning subsequent steps and checking whether there were contents missing seemed to be very important strategies for integrating information from multiple sources when learning by concept mapping. Nevertheless, the knowledge increase was not related to the amount of planning and controlling processes that were observed in the participants’ thinking-aloud protocols, r = −.01, ns.

Devoting much attention to what they were doing without referring to the learning contents impaired the performance in the integration test, r = −.47, p = .003 (cf. ‘activity report’ in Table 5). Preoccupation with the ‘design’ of the concept map alone should be avoided to ensure favorable learning outcomes in terms of the integration of information. The amount of learners’ statements in the thinking-aloud protocols that were coded as an ‘activity report’ on the other hand, was not related to the increase of knowledge about stem cells as measured by the stem cell question, r = −.16, ns.

No correlations were found between different learning outcome measures on the one hand and false statements and reading aloud on the other (Table 5). Finally, differences in learning gains were not found when considering participants’ problems in using the mapping software, the mapping process itself, or the demands of thinking aloud (see ‘non-content problems’ in Table 5).

Mapping styles

An explorative cluster analysis was performed in order to test whether there are different concept-mapping styles. The learners were grouped according to their similarity with respect to six variables that were specific for the mapping procedure and that proved to be related to the learning outcomes: learners’ prior experience in learning with concept maps, the number of correctly labeled links, number of unlabeled links, planning and controlling, relationships, and activity report. We used z-standardized variables to prevent certain variables from determining the cluster solution more than others due to larger variances. With respect to the relatively small sample of 38 participants, we decided not to include positive and negative monitoring in the cluster analysis although these variables also correlated with learning success. In this study, these metacognitive variables seemed to reflect primarily successful and unsuccessful learning, respectively.

The cluster analysis method employed was the Ward procedure with squared Euclidian distances. The resulting dendrogram (Fig. 2) favored a four or a six cluster solution. Aggregating the cases in five or three clusters strongly increased the residual variance (intracluster variance) respectively. As a six cluster solution resulted in very small cluster groups, we decided on a four cluster solution. Although the Ward procedure usually tends to result in equal size clusters, in this case relatively unequal group sizes were obtained (cluster 1: n = 6; cluster 2: n = 13 cluster 3: n = 9; cluster 4: n = 10).

Fig. 2
figure 2

Cluster analysis of mapping characteristics: Dendrogram using ward method

Full size image

We hypothesized that different mapping styles led to differences in measures for learning outcomes. To test this hypothesis, a MANOVA for the knowledge integration and the knowledge increase as a dependency was performed. A significant difference between groups for these two dependent variable was found, F(6, 66) = 6.99, p < .001, η 2 = .39 (strong effect). Looking at the learning outcome variables separately revealed that there was no significant difference between the mapping style clusters with respect to the knowledge increase, F(3, 34) = 1.21, ns. However, the mapping style clusters differed with respect to knowledge integration, F(3, 34) = 12.72, p < .001, η 2 = .53 (strong effect). In order to detect which clusters differ from each other, post-hoc comparisons were conducted using Tukey’s Honestly Significant Differences (HSD). The post-hoc comparisons revealed that learners assigned to cluster 1 outperformed learners in cluster 3 (p = .001) and in cluster 4 (p < .001) in the integration test. Learners that were assigned to cluster 2 also performed better in the integration test than learners in cluster 3 (p = .001) and learners in cluster 4 (p < .001). Thus, participants in clusters 1 and 2 had good learning results in terms of the integration of information from the six newspaper articles (cf. Fig. 3), whereas participants in clusters 3 and 4 did not perform very well in the integration test.

Fig. 3
figure 3

Cluster analysis of mapping characteristics: Z-standardized means of the learning outcomes and the cluster variables in the resulting groups

Full size image

In the following, the different clusters are characterized by their profiles in the learning process and outcome variables. Participants in cluster 1 performed above-average in the integration test and exhibited an above-average knowledge increase as measured by the stem cell question. They had prior experiences with learning by concept mapping. They produced concept maps with an above-average number of correctly labeled links and did not include many unlabeled links in their concept maps. As their over-average amount of thoughts about relationships in the thinking-aloud protocols indicated, they reflected considerably on how to label the links. Also, their amount of planning and controlling processes as determined by the thinking-aloud protocols was very high (Fig. 3). Members of this group could be labeled ‘advanced beginners’.

The concept maps, as well as the thinking-aloud protocols of participants in cluster 2, showed that they took effort in labeling their links as well. Their amount of planning and controlling processes were also above-average. However, in contrast to cluster 1, these participants did not have much experiences with concept mapping prior to this study, thus they were true beginners in this learning technique (Fig. 3). Their knowledge increase as measured by the stem cell question was about average. However, their performance in the integration test was above average. Thus, this group can be referred to as ‘successful beginners’.

Clusters 3 and 4 could be described as unsuccessful—at least in terms of the integration of information from the six newspaper articles on stem cells. The knowledge increase of participants in clusters 3 and 4 was about average. The individuals in cluster 3 did not care much about labeling links and they did not give much thought to the relationships between the concepts. Their amount of planning and controlling processes was slightly below average (Fig. 3). We called this group the ‘non-labeling mappers’.

The concept maps of individuals in cluster 4 contained a below-average number of correctly labeled links. They showed a medium amount of unlabeled links, and their thinking-aloud protocols showed a medium amount of thoughts about relationships. This group was characterized by very little planning and controlling processes and a high amount of activity report (Fig. 3). That means that these individuals very often only reported what they were doing without referring to the contents (i.e., stem cells). This group can be referred to as ‘non-planning mappers’.

In order to determine whether the resulting cluster solution yielded a grouping of learners that was related to verbal and spatial abilities, the extent to which these scores varied between clusters was examined. In respect of the learners’ verbal and spatial abilities, a MANOVA was performed. A significant difference between groups for these two dependent variables was found, F(6, 66) = 3.10, p < .010, η 2 = .22 (strong effect). Looking at the dependent variables separately revealed that the participants in the cluster groups did not differ with respect to their spatial abilities, F(3, 34) = 2.23, ns. However, the mapping styles differed concerning the verbal ability, F(3, 34) = 4.54, p = .009; η 2 = .29 (strong effect). Table 6 shows the means of verbal and spatial abilities for the cluster groups.

Table 6 Means (standard deviations in parentheses) of participants’ verbal and spatial abilities for the different mapping styles
Full size table

Post-hoc comparisons using Tukey’s HSD method revealed that learners classified as non-planning mappers performed worse on the test of verbal abilities than learners classified as advanced beginners (p = .010). No further differences were found (all p > .05). Thus, non-planning mappers had lower verbal abilities than advanced beginners.

Discussion

The aim of this study was to examine interindividual differences between learners constructing concept maps as a strategy to learn from texts. To this purpose, cognitive processes elicited by concept mapping, concept-map characteristics, and the learners’ verbal and spatial abilities were analyzed.

The effectiveness of concept mapping for learning was once more confirmed by the results of this study. As in a variety of other studies (e.g., Horton et al. 1993), we also found a substantial increase of knowledge after the construction of a concept map. This is particularly interesting as the participants learned from six separate texts that only partially overlapped in their information. As Britt et al. (1999) showed, often learners cannot integrate information from multiple texts to a substantial degree. Concept mapping could be a useful tool to foster learning that particularly involves multiple sources.

In contrast to Rewey et al. (1989) and to O’Donnell et al. (2002), we did not find an advantage for learners with low verbal abilities. On the contrary, we found that the cluster group with the lowest learning outcome had significantly worse verbal abilities than learners in the cluster with the best learning outcome. However, participants in the experiments of these authors did not construct concept maps on their own but learned from worked-out concept maps. For spatial abilities we found that higher spatial abilities were related to a higher learning outcome as measured by the integration test. Future studies need to clarify the differences in the influence of learners’ verbal and spatial abilities when learning either from worked-out maps or from the creation of concept maps on their own.

Characteristics of concept maps and of cognitive processes during concept-map construction that were related with a good learning outcome concerned two processes in particular—the labeling of the links that connect the concept nodes and the meta-processes of planning the map construction and of controlling the progress of the concept map. As the analysis of the thinking-aloud protocols showed, learners who cared more about the labeling of the links (i.e., what kind of relationship the concepts share) were better able to integrate the information of the multiple text sources. The importance of carefully thinking about the interrelations between the concepts is backed up by the results of the analysis of the concept maps the participants produced: We observed a better integration of information in learners who conscientiously labeled the links in their concept map. In addition, the analysis of the learners’ cognitive processes during concept mapping indicates that processes of content-related planning and controlling are associated with preferable learning outcomes. In contrast, describing the pure technical design of the concept map without referring to the contents that were represented therein was disadvantageous to learning. Paying attention to the content information displayed in the concept map is the crucial factor.

Different mapping styles could be identified that varied in their success in learning. We found two distinct successful styles (i.e., the ‘advanced beginners’ and the ‘successful beginners’). Both groups of successful learners carefully labeled their links and planned and controlled their mapping processes. Thus, these two successful groups of learners put effort in those processes that we identified to be connected with better learning outcomes. However, the advanced beginners group engaged more intensively in these processes than the successful beginners group. Also, the advanced beginners did not only reach a very good learning outcome as measured by the integration test, they also had a high increase in knowledge. Hence, for learning successfully by concept mapping, learners need to plan their mapping process and during mapping they have to concentrate on the connections between the nodes and how to label them. They also have to monitor their progress. Learners with more experience in concept mapping seem to conduct these processes more easily and with better results.

Maybe the most interesting finding is that there are two types of unsuccessful mappers (i.e., the ‘non-labeling mappers’ and the ‘non-planning mappers’). Non-labeling mappers constructed maps with many unlabeled links and they did not think very much about how the concepts are interrelated. However, they at least planned and controlled their mapping process to a medium degree. On the other hand, the non-planning mappers thought about the relationships between the concepts to a medium degree. However, they did not plan and control their mapping process. Instead, they engaged a lot in activity reports. Consequently, carrying out one activity alone—planning and controlling or carefully labeling the links—seems to be insufficient.

A correlational study is appropriate as a type of needs assessment that analyzes the restrictions of impromptu mapping that is not instructionally enhanced. Nevertheless, some limitations are connected with such a study. It could be argued that the differences in the learning outcomes are not a result of the different concept-mapping strategies but of ability in general. However, the cluster groups did not differ with respect to their spatial abilities. Although learners in the group of non-planning mappers have been shown to have lower verbal abilities, the processes that were related to good learning outcomes were not correlated with the learners’ verbal abilities. On the whole, the present findings do not indicate that general abilities are the primary determinants of learning by concept mapping. This conjecture must, however, be regarded as preliminary. For example, it is undetermined whether our measure of spatial ability that included 3-D mental rotation problems was optimal with respect to the 2-D mapping task. Additional studies employing 2-D spatial ability problems (e.g., embedded figures) would be necessary to further analyze the influence of spatial abilities.

Another restriction of this study is related to the sample size. Although this study included a relatively large sample for a thinking-aloud study, it included a relatively small sample for a comprehensive analysis of the deficits that typical concept-mapping beginners have. A replication study would help to evaluate the reliability of the present results.

A final restriction refers to the fact that we used a single-item test for knowledge growth. We employed a very general question (“What are stem cells and what could be done with them?”) in order to prevent the learners’ from devoting attention during reading or mapping to specific facts, concepts, arguments. An increase of knowledge with respect to issues that were raised in questions during the whole learning process (reading plus mapping) would have been rather trivial. In other words, our very general question did not “tell” the learner what to learn. Nevertheless, a further study including a control group that does not map after reading is necessary to determine the precise benefit of mapping after reading. In such a study a more comprehensive test that assesses knowledge growth could be used.

Consequently, what implications for a concept-mapping training can be drawn? Fifty percent of the participants belonged to the two unsuccessful groups. The fact that only half of the learners could be labeled as successful, underlines the need for effective training to enable more learners to achieve preferable learning outcomes. The results of this study lead to the following recommendations for concept-mapping training:

  • Instruct learners to pay careful attention to labeling links

  • Emphasize the relevance of planning the mapping process and controlling the improvement of the concept map

  • Guide learners to shift their attention from the pure technical design to the learning contents as much as possible.

Two different approaches are conceivable in order to help learners engage more in successful activities when learning by concept mapping. Direct training would teach learners to primarily use the successful activities during concept mapping. As this study indicates, the planning of the concept map is the first step in learning the process of concept mapping successfully. While actually constructing the concept maps, learners definitely have to pay attention to the relationships between the concept nodes. The progress of the map has to be steadily controlled and—if necessary—learners have to engage in a new planning activity to revise their concept map and thus begin the process over again. This process could be conveyed to the learners. One result of this study was that learners with more experience in concept mapping were better able to integrate the information that was provided by the learning texts. Consequently, it seems reasonable that subsequent to the presentation of the successful concept-mapping processes learners should be given the opportunity to practice them. However, especially in initial skill acquisition, learners are often overwhelmed by the opportunity to solve such a task on their own (Atkinson et al. 2000; Renkl 2005). Often, the primary goal of novices (i.e., learners) is to find a solution which leads to performance orientation (Dweck and Leggett 1998). In concept mapping, this primary goal would be to construct a concept map for one specific topic. To learn how to construct concept maps, it would be preferable that learners concentrate on the process of mapping. One possibility would be to confront learners with examples that model the process of concept mapping. Example-based learning has been successfully employed in the initial skill acquisition of cognitive skills in well-structured domains (Atkinson et al. 2000; Renkl 2005) as well as in more complex domains (Hilbert et al. 2006; Rummel and Spada 2005). We are presently investigating whether examples that model the concept-mapping process could also be a helpful method to teach learners how to construct concept maps.

An indirect training approach (i.e., no explicit instruction of the favorable processes) would be to design the mapping task in such a way that the successful activities are induced. This could possibly be accomplished by prompting the learners to plan their mapping process, to carefully label the links and to control their progress while constructing their concept map. Prompting procedures have been successfully employed in several learning scenarios. For example, Atkinson et al. (2003) showed that prompting learners to self-explain the underlying principles while learning from examples fostered their learning outcome. The effectiveness of prompting to induce productive learning activities while using a new learning method has been shown in the writing of learning protocols (Berthold et al. 2004; Hübner et al. 2006; Schwonke et al. 2006). The positive effect of prompting learners to be engaged in successful activities even remained stable in a follow-up test where learners wrote a learning protocol without being prompted (Berthold et al. 2004; Hübner et al. 2006). Further studies should test whether a similar prompting procedure focusing on the mapping activities that were related to learning outcomes in this study helps learners to profit more from concept mapping.