Introduction

One of the most basic cognitive process in human learning is causal reasoning (Rehder 2003) when concepts are combined into propositions (e.g., heat increases pressure) to determine the functional properties of concepts and to understand the essential parts and causal-and-effect relationships that exist within a system (Guenther 1998). Given that causality is the core property of all science (Keil 1989), the ability to reason causally is an essential cognitive skill that is central to understanding and predicting the behavior of complex systems (Brewer et al. 2000; Carey 1995; Corrigan and Denton 1996; Schlottmann 2001; Thagard 2000; Wellman and Gelman 1998). The National Science Education Standards emphasizes that students reflect on observations in ways that indicate that he/she is attempting to find patterns and causal relationships (National Research Council 1996). The Standards place more emphasis on science as argument and explanation and on activities where students analyze science questions using evidence and strategies for developing explanations. All of these standards focus on understanding causal relationships.

Causality can be understood in terms of the priority principle, covariation (co-occurrence) principle, and mechanism principle (Bullock et al. 1982; Kelley 1973). The priority principle refers to the temporal relationship between cause and effect. A cause must precede the effect for a cause to be valid (e.g., “the boy kicked the stationary ball” and “the ball rolled”). The covariation principle describes a causal law (Kelley 1973) which predicts that repeated occurrences of the association between cause and effect over time is a necessary condition for a causal relationship to be legitimate. In this case, the strength of the correlational relationship indicates the probability of the cause producing the effect (Hung and Jonassen 2006). The mechanism principle describes the beliefs that people construct to explain relationships between cause and effect. The causal mechanism is the causal chain of intermediary events that connect a cause and an effect (e.g., factories increased in number → jobs increased in number → city population increased). Temporal, covariational and mechanistic understanding may all be necessary to achieve full understanding of causal relationships in a complex system. For example, Rapus (2004) found that information about how covariation strength is used depends on the detailedness of mechanistic information and the scope over which covariation information is defined.

Causal reasoning with causal mapping tools

One fundamental assumption of this article is that humans understand the world by constructing mental models (or internal symbols and representations) of the world in their minds that serve as structural analogs of real-world or imaginary situations, events, and processes (Johnson-Laird 1983). Science and mathematics educators recognize the importance of modeling in understanding scientific and mathematical phenomena (Confrey and Doerr 1994; Frederiksen and White 1998; Hestenes 1992; Lehrer and Schauble 2000, 2003). Extensive research has been conducted on the effects of using visual diagrams like concept maps and/or knowledge maps to support learning in the classroom (Nesbit and Adesope (2006). Constructing causal maps (one variant use of concept maps) to examine causal relationships underlying complex phenomena is an important and fundamental process in scientific inquiry. Modeling helps learners to: (a) express, externalize, and share their thinking; (b) visualize, discuss, and test components of their theories; and (c) make materials more interesting. These processes of modeling can, for example, support conceptual development, and conceptual change (Nersessian 1999; Vosniadou 2002). When solving problems, learners construct models in memory and apply those models to solving the problem rather than applying logical rules (Vandierendonck and de Vooght 1996).

However, the models that learners construct are often analogical, incomplete, and fragmentary representations of a given system (Farooq and Dominick 1988). Students often possess overly simplistic models of complex systems which fail to recognize the interconnectedness of variables within a system, and ignore indirect effects, and/or view all variables as direct causes (Barman et al. 1995; Griffiths and Grant 1985). To construct more complete and accurate models, computer-based tools can be used to produce both computational and visual representations of their models (Richardson 1999). Although causal models can be constructed both quantitatively and qualitatively, most of the research on modeling has focused more on the use of quantitative and computational tools like Stella to facilitate the modeling process. However, qualitative representations may just be as important as quantitative representations of student’s model/understanding (Ploetzner and Spada 1998). When students try to understand a problem using quantitative approaches, students often do not conceptually understand the underlying systems and computational formulas. As a result, it may be necessary to help learners construct qualitative representations of a problem to facilitate the construction of quantitative representations—especially for novice problem solvers (Chi et al. 1981; Larkin 1983).

Causal maps can be used by learners to explicate causal relationships using a more qualitative approach. A causal map is a visual-graphical network of nodes (e.g., graphical squares, rectangles, or circles) and unidirectional links (e.g., arrows that point in only one direction) used to represent variables and the causal relationships between variables. Causal maps and concept maps in general have been used in science education as a tool to teach and assess learners’ systemic understanding of complex problems and phenomena (Leelawong and Biswas 2008; Owen 2002; Ruiz-Primo and Shavelson 1996). Specifically, maps have been used to elicit, articulate, share, identify similarities/differences, trigger and support discussions, refine, assess, and improve understanding, analysis, and the identification of causes and effects, their temporal relationships, and causal mechanism underlying a complex problem or system (Jeong 2009, 2010a).

A growing number of studies on causal maps and/or concept maps in general have formulated various metrics to measure the accuracy and structural attributes of students’ maps (parsimony, temporal flow, total links, connectedness)—particularly attributes believed to be correlated to map accuracy and attributes that can be potentially used as guidelines to help students create more accurate maps (Nicoll 2001; Scavarda et al. 2004; Ifenthaler et al. 2011; Jeong 2009; Plate 2010). Studies have been conducted to determine how different constraints imposed on the map construction process affect student’s maps and learning—constraints like imposing hierarchical order by allowing students to move and re-position nodes (Ruiz-Primo et al. 1997; Wilson 1994), providing terms for nodes (Barenholz and Tamir 1992), providing labels for links (McClure and Bell 1990), and allowing more than one link between nodes (Fisher 1990).

In addition, studies have been conducted to develop computer-supported tools to automate and reliably measure the accuracy and structural attributes of maps. Computer software like the Highly Integrated Model Assessment Technology and Tools or HIMATT (Ifenthaler 2010, 2011) and jMAP (Jeong 2010b) are being used to not only to address issues of rater reliability and validity, but also to test the correlation between different structural attributes and accuracy of students’ maps (Ifenthaler et al. 2011). In addition, these tools are being used to measure how maps change over time and how observed changes over time contribute to convergence in shared understanding between learners (Jeong 2010a). HIMATT is a web-based tool that enables learners to construct causal maps and teachers or researchers to quantitatively measure how an individual learner’s causal map compares to an expert’s map. jMAP is an Excel-based program that also enables learners to construct causal maps using Excel’s autoshape tools. Most of all, jMAP also enables users to “join” (or aggregate) and graphically “juxtapose” maps to make visual and quantitative comparisons between maps produced by an individual, the collective group of learners, and/or the expert (in any paired combination).

Limitations of causal mapping tools

However, students’ maps can vary widely in both accuracy and form when a student’s map is compared to another student’s map or to an expert’s map (Ruiz-Primo and Shavelson 1996; Scavarda et al. 2004). Based on a review of prior research, Ruiz-Primo and Shavelson (1996) concluded that maps should not be used in the classroom for large-scale assessments until students’ facility, prior knowledge/skills in using maps, and associated training techniques are thoroughly examined. Furthermore, variations in the processes used by students to create their maps (and how these processes affect map quality and accuracy) need to be identified and thoroughly examined in order to ensure that observed variances in the accuracy of students’ maps are the result of the differences in students’ causal understanding and not the result of individual differences in the processes students use when constructing their maps. Most of all, the processes that help students create more accurate causal maps must be determined and thoroughly tested so that mapping processes can be standardized and implemented to eliminate variance in map accuracy attributed to individual differences in the causal mapping processes used by students. At this time, the research that has been conducted to examine how various attributes of students maps correlate to learning outcome have not specifically examined the attributes (temporal, covariation, mechanisms) that can directly impact causal understanding.

Given the issues described above, new research is needed to: (a) identify the processes students use when constructing causal maps prior to receiving instruction and training on causal mapping; and (b) determine to what extent particular processes contribute to causal understanding measured in terms of the accuracy of students’ maps (the match between the student’s and expert’s map). A clear understanding of the processes and their effects on map accuracy will provide the foundation on which to identify the most appropriate interventions for improving the map construction process, the accuracy of students’ causal maps, and students’ causal understanding of complex systems (e.g., causal mechanisms, temporal relationships). This correlational study examined the accuracy of students’ maps in terms of the ratio of correctly/incorrectly identified root causes and in terms of total number of correctly identified root-cause links (links stemming from root causes) to gauge how well students understand the causal chain-mechanisms and mediating factors underlying cause-effect relationships between root causes and outcome node. Each of these two measures of causal understanding were correlated with three attributes observed in students’ causal maps: total number of causal and unidirectional links present in the student’s map (total links), ratio of links that point from left-to-right versus from right-to-left (temporal flow), and the distance measured by the number of pixels between the position of the outcome node and the left-most edge of screen (position of the node representing the final effect/outcome).

The purpose of this study was to determine which attributes were correlated with (and possibly contributes to) high scores in map accuracy and causal understanding. The findings can be used to identify which attributes can be implemented in student training and instructions and/or integrated into the causal mapping software interface to enhance and/or scaffold specific mapping processes (e.g., limiting number of links, creating default links pointing from left to right, positioning by default final outcome nodes at right portion of screen). To address these issues, this case study examined two research questions:

  1. 1.

    Which attributes (total links, temporal flow, lateral position of final outcomes) are correlated with causal understanding?

  2. 2.

    What is the relative magnitude of each attribute’s impact on causal understanding?

In this study, the total number of causal links observed in student’s maps was hypothesized to be negatively correlated with causal understanding. In the case where event A causes B and B causes C (a total of two causal links), students with little understanding of direct and indirect causes may insert causal links between A–B, B–C, and A–C (a total of three causal links). As a result, students’ inability to distinguish direct causes (B) from indirect causes (A) and inability to construct parsimonious models could in theory inflate the number of causal links in their causal maps and therefore reduce the accuracy of their causal understanding.

The degree to which students’ causal maps exhibit temporal flow (the percentage of links pointing from left to right) was hypothesized to be positively correlated with causal understanding because the conventional practice of sequencing temporal events from left to right (at least in the Western culture) may lead students to engage in more reflection on the temporal nature of causality. This in turn may enable students to apply the priority principle—one of the three underlying principles of causality noted above—as they attempt to identify possible cause-effect relationships.

The distance (lateral position) between the outcome node and the left-most edge of the mapping screen was hypothesized to be positively correlated with causal understanding. Positioning the outcome node to the far right of the mapping area can help to create adequate space for students to (either intentionally or unintentionally) re-position and move direct causes to the middle portion of the map (closer to the outcome node) and move indirect causes to the left of the mediating causes (farther away from the outcome node). As a result, the relative positions of the nodes may assist students in recognizing and distinguishing which factors might be direct and indirect causes and successfully identify the causal mechanisms to explain how indirect causes affect the final outcome.

Given the hypotheses presented above, the total number of causal links, temporal flow, and lateral position of the final outcome node was hypothesized to be significant predictors of causal understanding measured in terms of the ratio of correct/incorrect root causes and the number of correct root links. Given the absence of prior research on the relative impact of these three variables on causal understanding, no hypotheses were made on the relative magnitude of each attribute’s impact on causal understanding.

Method

Participants

The participants in this study were 19 graduate-level students enrolled in an online course on the topic of computer-supported collaborative learning at a large university in the southeast region of the U.S. in the summer of 2008. Eight participants were male, and eleven were female ranging from 22 to 55 years in age.

Procedures

The students were given two class activities to examine the cause-effect relationships between factors that influence learning in collaborative learning groups and to create a personal theory that explains how student learn successfully in collaborative groups. In week 2 of the course, students used a Wiki web page to share and construct a running list of factors that they believed to influence the level of learning in collaborative groups. Students classified and merged the proposed factors, discussed the merits of each factor, and submitted votes on the factors believed to exert the largest influence on the outcomes of a group assignment. The votes were used to select a final list of 14 factors (see Fig. 1).

Fig. 1
figure 1

Instructor map superimposed over a student’s map to reveal matching and missing causal links. Note: Dark gray link is present in student’s map; light gray link missing in student’s map

Full size image

In week 3, students were presented six example diagrams to illustrate the characteristics (temporal flow and parsimony) and functions of causal maps. Students were then provided a MS Excel-based software program called jMAP to construct their first causal map. The map template was pre-loaded with the final list of14 factors. The nodes were randomly placed along the left and bottom edge of the screen with the outcome node placed on the right portion of the screen. The purpose of each student’s map was to graphically explain their understanding of how the selected factors influence learning in collaborative groups. Using the functions in jMAP, students connected the factors with causal links by creating each link with: (a) varying densities to reflect the perceived strength of the link (1 = weak, 2 = moderate, 3 = strong); and (b) varying link styles (continuous, dotted, sparsely dotted) to convey the amount of evidence (from past personal experiences or from empirical research) that is available to validate/verify the presence of a causal relationship. The course instructor also used jMAP to construct an expert map (see Fig. 1) that was used in this study to assess the accuracy of students’ maps. While constructing the expert map, the instructor applied the same guidelines that were shared with students before they constructed their causal maps. In following the guidelines, the instructor: (a) immediately positioned the outcome node to the far right of the causal map; (b) purposefully sought to establish temporal flow while examining and identifying direct from indirect causes; and (c) purposefully removed causal links inserted between indirect causes and final outcome node in order to achieve a parsimonious model. As students constructed their causal maps, students were permitted to omit any factors that he/she did not believe to directly or indirectly influence the learning outcome. Personal causal maps were completed and electronically uploaded within a 1-week period. Any student that submitted a map received 10 class participation points, and as a result, the causal maps were not graded. Class participation points earned by students (out of 275 total possible points) accounted for 25% of the course grade. Students used and made references to their causal maps in a written assignment due in week 4 that described their personal theory of collaborative learning (accounting for 10% of the course grade).

Once all the students submitted their first causal map, the instructor used jMAP to download and aggregate all diagrams (n = 19) to produce and share with students a matrix conveying the percentage of diagrams that possessed each causal link. For example, the matrix in Fig. 2 shows that the causal link between ‘Individual Accountability’ and ‘Learner Motivation’ was observed in 47% of students’ diagrams. The highlighted cells in the right matrix in Fig. 2 identify the common links observed in 20% or more of the students’ diagrams (note: this criterion was specified by the instructor when aggregating diagrams in jMAP). Presented in the left matrix are the mean strength values of only those links observed in 20% or more of the diagrams. The highlighted values reveal links that are present or absent in the expert’s map (i.e., dark = links and strength values match, medium dark = links match, but strength values do not, light gray = missing target links).

Fig. 2
figure 2

Matrix displaying percentage of students’ maps with each causal link and mean impact values assigned to causal links observed in 20% or more of students’ maps. Note: Dark gray links in instructor’s map with matching strength values; medium gray links in instructor’s map with different strength value; light gray instructor’s link not present in less than 20% of students’ maps

Full size image

In week 9, students were presented the matrix that revealed what percent of students’ maps (Fig. 2) possessed each link. In an online discussion forum hosted in the BlackboardTM course management system, the instructor created an individual discussion thread for each factor pairing. Within each discussion thread, students posted messages to explain, defend, and challenge the rationale behind each proposed causal relationship. Each posted explanation was labeled by students with the tag ‘EXPL’ in message subject headings. Postings that questioned or challenged explanations were tagged with ‘BUT.’ Postings that provided additional support were tagged with ‘SUPPORT.’ In weeks 9, students searched and reported quantitative findings from empirical research in a Wiki to determine the relative impact (strength values) of one factor on another factor.

Finally, in week 10, students reviewed the discussions produced in week 9. Within each discussion thread for each proposed causal link, students posted messages to report whether they rejected or accepted the link (along with explanations). At the end of week 10, students revised and submitted their causal maps (map 2) based on their analysis of the arguments presented in class discussions. Similar to the first map produced in week 4, students received 15 class participation points for submitting their final maps. All students received the 15 participation points for simply creating and submitting a final map. With three students that did not submit their final maps, a total of 16 final maps were collected.

Data analysis

This study measured students’ causal understanding in terms of (a) the ratio of correctly/incorrectly identified root causes (nodes with only out-going causal links) and (b) total number of correctly identified root-cause links (links stemming from root causes) used to gauge how well students understand the causal chain-mechanisms and mediating factors underlying cause-effect relationships between root causes and outcomes. The students’ scores on each of these measures of causal understanding were based on a direct comparison of each student’s causal map and the instructor’s causal map. Matrices like the one in Fig. 3 were automatically produced by jMAP in MS Excel worksheets for each student’s map to facilitate the process of comparing and scoring each student’s map in relation to the instructor’s map. Each matrix represents one student’s causal map with causes listed by row, effects listed by column, dark and medium-dark colored cells identifying correctly identified links, light gray colored cells identifying missing links, factors with empty columns identifying root causes (factors with no incoming links), and red triangles denoting the presence of a student’s personal explanation for inserting the causal link stored in the cell’s comment box. The values entered in each cell that denote the perceived strength of impact each cause has on each effect (first value in cell) and the amount of existing evidence to validate/verify the cause effect relationship (second value in cell) were not examined in this study.

Fig. 3
figure 3

Matrix representing a student’s causal map with causes listed by row and effects listed by column. Note: Dark gray links in expert map with matching strength values; medium gray links in expert map with different strength value; light gray expert link not present in the student’s maps

Full size image

Each of these two measures of causal understanding were correlated with three attributes observed in students’ causal maps: total number of causal links (total links), ratio of right/left pointing links (temporal flow), and distance of final outcome node from left edge of screen (outcome position). Total links was measured by counting all links in each student’s causal map using the matrices like the one illustrated in Fig. 3. Temporal flow was computed by visually examining the actual maps produced by each student to count the number of right and left pointing links. The temporal flow of each map was then scored by dividing the number of right pointing links by the number of left pointing links. Links that pointed straight up or straight down were not included in the computation. Outcome position was based on the number of pixels separating the left edge of the screen and the middle position of the outcome node titled “Level of learning achieved”. The descriptive statistics for each of the measures are presented in Table 1.

Table 1 Descriptive statistics on each of the five measures
Full size table

To determine how the attributes might potentially affect causal understanding (research question 1), correlations were computed between all the examined attributes (total links, temporal flow, lateral position of final outcomes) and the two measures of causal understanding (ratio of correctly/incorrectly identified root causes, number of correctly identified root cause links). One correlation matrix was produced for all causal maps (map 1) students produced before participating in the online discussions/debate over their causal relationships between the factors and outcome. A second correlation matrix was produced for all maps (map 2) students produced after they presented, discussed, and examined the causal relationships in the online discussions/debate in week 9.

To determine the relative impact of each attribute on causal understanding (research question 2), two regression models were tested on the maps produced before discussion and on the maps produced after discussion:

Model 1

Ratio of correct root causesi = β0 + β1(number of total linki) + β2(ratio of temp flowi) + β3(outcome node position).

Model 2

Number of correct root linksi = β0 + β1(number of total linki) + β2 (ratio of temp flowi) + β3 (outcome node position).

Results

Correlations between attributes and causal understanding

The correlations between the attributes and measures of causal understanding are presented in Table 2. In the student’s initial maps (map 1) produced prior to discussion, temporal flow was negatively correlated to the number of correct root links (r = −0.461, P = 0.047), outcome position was negatively correlated to the number of correct root links (r = −0.465, P = 0.045). In the maps produced following discussion (map 2), temporal flow was positively correlated with ratio of correct/incorrectly root causes (r = 0.688, P = 0.003), while total causal links was negatively correlated with number of correct causal root links (r = −0.523, P = 0.037).

Table 2 Correlations between two measures of causal understanding and the three causal map attributes
Full size table

These findings suggest that: (a) understanding of the underlying causal mechanisms during early map construction can be adversely affected if students are instructed to temporally sequence nodes to flow from left to right and position the outcome node farther away from the left edge of the map relative to other nodes in the map; and (b) the ability of identify root causes during the later process of map construction following class debate over the causal links can be increased by instructing students to minimize the number of causal links and create a map that temporally flows from left to right.

Relative impact of attributes

The regression analysis produced one model of statistical significance (see Table 3). The regression model for the ratio of correct/incorrect root causes identified in students’ maps produced after the online discussions (F(3,12) = 5.025, p = 0.017) explained 44.6% of the variance (Adjusted R 2 = 0.446) and power was 0.73. In this model, the variables that were the most to least predictive were temporal flow (β = 0.772, P = 0.004), total links (β = −0.263), and outcome position (β = −0.092). These results suggest that during the later causal mapping process, temporal flow can make the greatest impact on students’ ability to identify the root causes. An increase in temporal flow by one standard deviation (while holding total causal links and outcome node position constant) can potentially increase the ratio of correct/incorrect root causes by 0.722 standard deviations.

Table 3 The unstandardized and standardized regression coefficients for the variables
Full size table

Although the model was not statistical significant (P = 0.076), the regression model for the number of correctly identified root links following online discussion explained 28.1% of the variance (Adjusted R 2 = 0.281). In this model, the variables that were most to least predictive were total links (β = −0.554, P = 0.028), temporal flow (β = 0.321), and outcome position (β = 0.143). This finding suggests that instructing students to minimize the number of links in their causal map during the later process of map construction (following discussions) can make the greatest impact on explaining how root causes directly/indirectly impact outcomes.

Although the models tested to determine the impact of the attributes early in the map construction process (prior to discussions) were not statistically significant, what is worth noting is that the impact of total links was very low or non-existent while temporal flow and node position appear to have an adverse impact on causal understanding. On the other hand, the impact of these attributes during the later processes of map construction are quite different in that total links appears to make a small impact (as opposed to no impact) while temporal flow makes a positive (not negative impact) and node position makes little or no impact.

Discussion

Overall, the findings in this correlational study (though not conclusive) suggest when and what methodological approaches students should use to construct causal maps in ways that help them to achieve a better understanding of underlying causes and causal mechanisms in complex systems. One of the findings suggests that students’ understanding of the causal mechanisms underlying the cause effect relationships between root causes and outcomes achieved during early processes of map construction can be adversely affected when (a) students are instructed to temporally sequence nodes to flow from left to right and (b) position the outcome node farther away from the left edge of the map relative to other nodes in the map. On the other hand, temporal flow and outcome node position seem to make a positive impact when implemented after class discussions. Students’ ability to identify the root causes later in the map construction process can be increased by instructing students to create maps that flow temporally from left to right. In addition, students’ ability to identify root links and underlying causal mechanisms can be increased by instructing students to minimize the number of causal links in their causal maps.

One plausible explanation as to why these three attributes appear to have an adverse and/or little or no impact early in the map construction process is that the process of implementing these attributes may be imposing too many constraints on students as they brainstorm and explore the relationships between all possible factor pairings. For example, a student who re-positions a given node closer to one node to convey their temporal relationship is at the same time positioning the given node farther away from other nodes. The increased distance between the given node from other nodes (and thereby reducing their visual proximity) may lead students to skip and omit from consideration the relationship that the given node might have with other nodes. As a result, these actions may pre-empt students from conducting a more thorough exploration of all possible cause effect relationships and hence lead students to produce less accurate maps and poorer causal understanding.

On the other hand, the findings in this study suggest that once students participate in deliberate discussions and debates over the validity of the causal links in their maps (and have mentally winnowed down the number of possible cause-effect relationships), establishing temporal flow during the map construction process can help students substantially increase the their ability to identify root causes. One possible explanation for this finding is that the process of positioning nodes in temporal sequence creates a task-demand characteristic that encourages students to identify mediating causes and thereby distinguish root causes from mediating causes. This finding contradcts the previous finding where hierarchical structure was found to have no effect on accuracy (Ruiz-Primo et al. 1997). However, computing and scoring temporal flow in each student’s map in this study did not pose any methodological problems like those Ruiz-Primo et al. (1997) reported in their study in their attempt to formulate an operational measure of hierarchical structure. The differences in the measures and the instructional tasks (e.g., create initial map individually, structured online debates on causal links, etc.) used in this study versus Ruiz-Primo et al’s (1997) study may have contributed to the different findings.

The findings in this study also suggests that if students are encouraged to keep the number of causal links to a minimum (and to achieve parsimony), students are better able to correctly identify root-cause links given the negative correlation found between total links and ratio of correct/incorrect root links. Some plausible explanations for this finding is that: (a) the students that inserted an excessive number of links did not mentally recognize potential redundancies between direct links (A-outcome) and mediated links (A–B-outcome) and as a result, did not remove redundant links (A-outcome) from their causal maps to minimize the number of links and to increase the accuracy of their maps; (b) the excessive number of links was a reflection of student’s inability to break down and understand the causal mechanisms and relationships and as a result, these students may have intentionally created as many links as possible in order to maximize the chances of identifying the correct links in their maps. To prevent students from making large numbers of wild guesses, it may be necessary in the future to inform students that their maps will be assessed not on the number of correctly identified links but on the ratio of correctly/incorrectly identified causal links. However, assessing and assigning grades to students’ maps cannot be legitimized until thorough research is conducted to achieve a complete understanding of how individual differences in causal mapping knowledge, skills, and processes affect the quality of students’ maps (while controlling for individual differences in the knowledge and understanding of the domain/content under study).

Given the limitations of this study, future research will need to: (1) replicate this study using a substantially larger sample (and with different student populations) while controlling for individual differences in students’ knowledge of the domain and concepts that students are mapping and prior experience with causal and/or concept mapping; (2) set the default position of the outcome node at the center of the screen rather than to the right portion of the screen in order to fully assess the effects of outcome node position; (3) analyze videos of each student’s desktop as they are constructing their maps to identify and determine how other real-time processes impact student’s causal maps and causal understanding (work in progress); (4) compare the effects of constructing causal maps that flow temporally from left-to-right versus right-to-left to examine which reasoning process (forward vs. backward or predicting vs. explaining) helps students better identify root and mediating causes; (5) validate the quality and accuracy of the instructor’s causal map by using outside criterion and/or by generating the target map by aggregating maps produced by multiple experts using the same mapping guidelines examined in this study; (6) integrate the desired attributes into the causal mapping software and conduct a controlled experiments to determine the effects of limiting number of links, manipulating the option to create links that can point in any or in only one direction, and intentionally varying the default position of outcome nodes; (7) determine the extent to which students are able to correctly interpret their own maps to correctly and verbally report the root causes and verbally explain how root causes directly/indirectly impact outcomes; and (8) consider how the effects of each constraint vary when examining causal maps across different domains or topics that are or are not naturally temporal in nature; (9) include other measures of map accuracy and causal understanding such as the number of correctly identified causal chains and students’ rank ordering of causes on overall impact based on number of out-going links stemming from each cause); and (10) examine how specific attributes and processes also affect causal understanding in terms of covariation (or strength of effect) in addition to temporal and mechanistic nature of cause-effect relationships.

Although the findings in this study are not conclusive given the limitations noted above, some of the preliminary findings were nevertheless consistent with our predictions—predictions that were based on established principles and procedures used to construct causal maps. For the findings that were not anticipated, logical and yet plausible explanations were presented to help understand possible reasons behind the findings. Furthermore, this study fills an important research gap by providing a framework and bringing to our attention the types of mapping processes and measures of map accuracy that need to be thoroughly investigated and understood before we can justify the large-scale use of causal maps (and/or concept maps in general) as an instructional and assessment tool. This framework will help set the necessary groundwork to developing an empirically based methodology on how to use causal maps in teaching and learning. The findings reported in this study provide useful and preliminary insights into the processes of constructing causal maps and insights on how these processes can positively as well as adversely impact level of causal understanding, problem analysis, and problem solving. These and future findings will provide useful data for developing and testing more rigorous and empirically based procedural models on how to construct causal maps in ways that maximizes the affordances gained from using causal mapping as tools for assessing and facilitating learning and problem solving.