Keywords

1 Introduction

1.1 Background

Diagrams are essential tools for representation, communication, and reasoning. In education, diagrams have been used widely and play an important role in STEM learning and problem solving [e.g., 1–5], learning and comprehension of complex systems [2], judgment [6, 7], reasoning [8], analogical transfer [9, 10], planning [11], and data representation and interpretation [12, 13].

But like any tool, a diagram must be well chosen for the task at hand, and its use affects both the process and the product of the activity. First, as an external representation of a cognitive or educational problem, salient aspects of the diagram must map to relevant aspects of the problem [14, 15]. Second, structural, visuospatial, and implicit aspects of the chosen diagram can influence and alter people’s inferences, judgments, and the perceived relations and structures of the represented information [e.g., 6, 8, 13]. How a diagram steers people to making certain inferences and judgments concerning the represented concepts and relations is not arbitrary. Rather, it stems from cognitively natural ways of mapping visuospatial elements and relations to conceptual content and relations, externally or internally, based on shared metaphorical (or analogous) similarity of abstract relational structures [e.g., 6, 15–17].

A rich body of research has explored how specific types of diagrams affect inferences in reasoning and judgment tasks. When asked to describe the relation of individual data points shown in statistical graphs, people given a bar graph tended to describe the relation as comparisons of discrete entities, but as trends of continuous change when given the same information depicted as a line graph [13, 18]. To describe complex mechanical systems depicted by diagrams, people reading mechanical diagrams with arrows described the functions of the systems, whereas those reading the same diagrams without arrows gave structural descriptions [2]. Other evidence [8] has demonstrated that when people use diagrams to keep track of individuals’ locations over different time points, presentation of different graph formats led to different inferences by participants.

In general, people’s inferences using diagrams are systematically related to the schemas that different diagram formats convey. For example, lines connect and associate entities, indicating paths, relations, and movement [8, 13, 19]; bars and boxes suggest enclosures and separate categories [8, 13]; and arrows show asymmetric directions and sequences from actions to goals and causes to effects [2, 19].

In this paper, we investigate the role of diagrams in probability problem solving. One general question we address is: can diagrams affect the interpretation or processing of probability word problems? Another is: do diagrams facilitate solution of probability problems? These questions can be answered in different ways. First, authors of probability textbooks seem to believe that diagrams aid in the understanding of probability principles, since such textbooks sometimes feature outcome trees and tables in chapters on probability [20]. Second, there is evidence that spontaneous use of diagrams by probability problems solvers is associated with higher solution success, but only when the diagram type used is appropriate to the problem type [4, 21]. Finally, there is suggestive evidence that providing a generic diagram with a problem text may in some cases facilitate solution of the problem, especially when the problem is somewhat difficult for the would-be solver [22].

Providing a generic diagram might be beneficial to students solving a probability word problem simply because it offers another representation of the problem, and multiple representations have been argued to lead to a deeper understanding of the problem [23]. But if the structure of the diagram is indeed cognitively mapped to the problem structure and the diagram is then used to reason about the problem and its solution, using different types of diagrams to represent a given problem might steer the problem solver in different directions in terms of solution strategies or processes. Such process evidence would provide more diagnostic evidence than mere solution success for how diagrams steer, and ideally facilitate, problem solving processes.

1.2 The Present Study

In the present study, we explore how using different types of diagrams can affect both process (strategy choice) and product (solution success) in probability problem solving. Specifically, we focus on two types of schematic diagrams that are commonly used as visual aids in probability and statistics domains, trees and tables.

Trees and tables differ in diagrammatic structures, and thus may lead to inductions of different schematic information and applications. Novick and Hurley [15] analyzed the basic structures and schematic components of these two general types of diagrams. As they observed, the global structure of a table features the cross-classification of two variables or sets. The rows and columns each represent a variable or set, and the intersection cells represent the combinations of items from the two sets. The global structure of a tree diagram (or hierarchy) features hierarchical levels of events, generated by a branching process. Different levels of events are often dependent so that the identities of one level depend on the identities of the preceding level. Items listed at the same level are mutually exclusive and identical in status, whereas items listed at different levels differ in status or sequence.

Research suggests that these two types of diagrams are chosen for representing different situations and schemas [e.g., 24]. In probability problem solving, tables are used to represent factorial combinations [e.g., 10], and trees may be used to keep track of sequential selections [e.g., 25]. Zahner and Corter [4] found that using tree diagrams was particularly useful for solving conditional probability problems, a type of probability problems that involve sequential and dependent occurrence of events. These empirical findings suggest that trees and tables may evoke different schemas of mathematical relations and choices of different solutions.

Thus, we sought here to investigate whether and how trees and tables lead to different results in interpreting mathematical structures and selecting solutions. To do this, we used problems representing two topics in combinatorics: combinations and independent events. These specific types of problems were chosen because they admit of multiple types of solution strategies (described and illustrated in the Method section), and each can be represented by both trees and tables. We compared the effects of providing three different types of generic diagrams for these combinatorics problems: N-by-N tables, binary trees, and N-ary trees. The diagrams were designed to manipulate two factors: the type of diagram structure (tree versus table) and the abstraction level of the represented outcome space (either a large space of equally-likely outcomes, or a smaller space of unequally-likely outcomes).

We hypothesize that the choice of solution strategies can be influenced by the diagrammatic representation of a problem. The idea is that different types of diagrams should bias people to formulate different mathematical solutions, due to the similarity correspondences between a diagram’s visuospatial structures and the selected solution’s mathematical structure. That is, the problem solver tends to choose a solution with procedural or mathematical structure that can be easily aligned with the visuospatial relational structure of the diagram.

2 Method

2.1 Participants

The participants were 48 students (39 or 81.3 % female) recruited from a university in New York City. Most were paid $8 for their participation; the others participated for course credit. Their average age was 25.6 years. To be qualified for the experiment, a participant had to have taken at least one undergraduate- or graduate-level statistics course prior to participation. On average, participants reported to have taken 2.33 statistics courses. However, their level of training varied: 19 (or 38.6 %) participants reported having taken one statistics course, 11 (or 22.9 %) participants reported two, 10 (or 20.8 %) reported three, and 8 (or 16.7 %) participants reported four or more such courses.

2.2 Materials

Each participant solved three elementary probability word problems, representing three different probability topics: combinations, independent events, and conditional probability. The first two problems/topics were the target materials for this study, because they admit of two distinct salient solution strategies, which we refer to as outcome-search and sequential-sampling strategies (described below). The third problem, involving conditional probabilities (and referred to below as the Weather problem), does not admit of salient alternative solution strategies, and was treated merely as a filler problem for purposes of this investigation.

The problem text for the independent events problem (also called the Spinner problem) was:

Two spinners are constructed. Each spinner has 3 color sections of equal size: red, white, and blue. The two spinners are spun at the same time, and the result of each spinner is recorded. What is the probability of getting the same color on both spinners?

In the diagram conditions, either a tree (Fig. 1a) or a table (Fig. 2a) was also provided to the problem solver. These diagrams were unlabeled, but the number of branches (or rows and columns) was appropriate to the problem.

Fig. 1.
figure 1

Panel 1a (on left) shows the tree diagram provided with the Spinner problem for some participants. Panel 1b (on right) shows how the tree diagram was annotated by one participant.

Full size image
Fig. 2.
figure 2

Panel 2a (on left) shows the table diagram provided with the Spinner problem for some participants. Panel 2b (on right) shows how the table diagram was annotated by one participant.

Full size image

The problem text for the combinations problem (also called the Work-Group problem) was as follows. The generic diagrams for this problem are presented in Figs. 3 and 4.

Five students are in a work group. The teacher randomly selects two of them to present the group work. If there are 2 boys and 3 girls in this group, what is the probability that the teacher selects 2 girls?

Fig. 3.
figure 3

Panel 3a (on left) shows the tree diagram provided to some participants for the Work-Group problem. Panel 3b (on right) shows how the tree diagram was annotated by one participant.

Full size image
Fig. 4.
figure 4

Panel 4a (on left) shows the table diagram provided to some participants for the Work-Group problem. Panel 4b (on right) shows how the table diagram was annotated by one participant.

Full size image

Although these two problems represent two different probability topics, the same two broad strategies can be used to solve each one. We refer to these two approaches as “outcome-search” and “sequential-sampling”. The outcome-search strategy finds the probability by first counting the total number of equally-likely outcomes in the outcome space. For the combinations (Work-Group) problem, this involves computing the number of all possible combinations of 2 girls out of 3 girls (=3), and dividing into this the number of all possible combinations of 2 students out of 5 students (=10). Thus, the probability of selecting two girls is 3/10. The sequential-sampling strategy views the problem as sequential sampling without replacement, so the solver first defines the simple or conditional probability at each stage in a sequential order. For example, for the combinations (Work-Group) problem, this strategy first defines P(G1) = 3/5, the probability of selecting a girl on the first draw, and then P(G2|G1) = 2/4, the probability of selecting a girl again on the second draw, conditional on P(G1). Finally, P(G1∩G2), the probability of selecting two girls, is equal to P(G1)P(G2|G1) = (3/5)(2/4) = 3/10.

2.3 Design and Procedure

Three test forms were used, each presenting the problems in a different order (Table 1). The first problem for each test form was given in text only, with no provided diagram. For the second and the third problems, one was given with a generic tree diagram, and the other one was given with a generic table diagram. Problems were presented in different orders, counterbalanced to equate possible carry-over effects. Thus, each problem was attempted by three independent groups of participants, with one third of them solving it with no provided diagram, one-third with a tree diagram, and one-third with a table diagram. As Table 1 shows, three types of provided diagram were used: an N-by-N table and a binary tree for the Work-Group (combinations) problem; and an N-by-N table and an N-ary tree for the Spinner (independent events) problem.

Table 1. The three test forms and the design for the study.
Full size table

Participants (N = 48) were randomly assigned to the three test forms, with 16 participants in each test form. In the task, each participant was given a booklet in which each problem was presented on a separate page. No explicit training was provided; they were asked to solve all three problems, showing their step-by-step solution procedure on the worksheet. To prevent participants from seeing more than one diagram type at a time, they were instructed not to look at any other problems when they were working on a problem. Participants were explicitly instructed to use the provided diagram for problem solving when one was provided. A probability formula sheet was also provided, although participants were told that the formula sheet was optional for them to use. Participants also filled out a brief survey about their basic demographic information and statistics training experiences.

3 Results

3.1 Coding

Two outcome variables were of focal interest: choice of a solution strategy, and problem solving correctness. Solution types were coded based on whether an outcome-search strategy or a sequential-sampling strategy was used for problem solving. Use of each strategy was coded independently and dichotomously, with a value of 1 if the strategy was used, and 0 otherwise. This strategy determination was made without reference to correctness, which was coded independently. Examples of participants’ work using each of the two solution strategies for the two problems are shown in Figs. 5 (the Work-Group problem) and 6 (the Spinner problem). To check reliability, a second coder independently coded 24 of the problem solutions. For each strategy, percent agreement was .96, with r = .92. The sole discrepancy occurred for an incorrect, somewhat disorganized response.

Fig. 5.
figure 5

Panel 5a (on left) shows an example of a solver using the outcome-search strategy for the Work-Group problem. Panel 5b (on right) shows an example of a solver using the sequential-sampling strategy for the Work-Group problem.

Full size image
Fig. 6.
figure 6

Panel 6a (on left) shows an example of a solver using the outcome-search strategy for the Spinner problem. Panel 6b (on right) shows an example of a solver using the sequential-sampling strategy for the Spinner problem.

Full size image

Solution success was assessed with two measures. Answer correctness was coded dichotomously, based on whether the final answer has the correct value, regardless of whether all the solution steps were correct. Procedural correctness was coded to indicate if the solution steps a solver followed were appropriate, regardless of final answer correctness. For example, if a solver’s solution procedure was correct, but a computational error led to an incorrect final answer, procedure would be coded as correct (=1), but answer correctness would be coded 0. On the other hand, one participant obtained a correct final answer by an incorrect procedure. For these reasons, procedural correctness was analyzed as our main measure of solution success [cf. 26]. Inter-rater reliability for procedural correctness was perfect (=1.0) for the 24-solution set.

3.2 Analysis of the Work-Group (Combinations) Problem: Effects of the N-by-N Table Versus the Binary Tree

Strategy Choice.

As hypothesized, solvers’ choices of the solution strategies were strongly biased by diagram types (Table 2). Because use of each strategy was coded independently, neither or both might both be employed on a given problem; however use of both strategies was relatively rare. Participants given the N-by-N table more frequently used the outcome-search strategy (93.8 %) compared to participants given no diagram (62.5 %), χ 2(1, 32) = 4.571, p = 0.083 (by Fisher’s Exact Test); and less frequently used the sequential-sampling strategy compared to no diagram (6.3 % vs. 43.8 %), χ 2(1, 32) = 6.000, p = 0.037 (by Fisher’s Exact Test). On the other hand, participants given the binary tree showed significantly more use of the sequential-sampling strategy than those in the no-diagram condition (81.3 % vs. 43.8 %), χ 2(1, 32) = 4.800, p = 0.028; but significantly less use of the outcome-search strategy than those in the no-diagram condition (25 % vs. 62.5 %), χ 2(1, 32) = 4.571, p = 0.033.

Table 2. Raw and proportional frequencies of solvers’ solution choices for the Work-Group problem as a function of diagram type.
Full size table

Solution Success.

Table 2 also shows participants’ rates of procedural correctness and answer correctness. The procedural correctness rates for the no-diagram condition (81.3 %) and the binary tree condition (81.3 %) were high and identical. However, procedural correctness for the N-by-N table condition was significantly lower (31.3 %) than for the no-diagram condition (81.3 %), χ 2(1, 32) = 8.127, p = 0.004.

Error Analysis.

We analyzed unsuccessful solvers’ error patterns to better understand why the table decreased problem solving success for this problem. When participants were given the N-by-N table for the Work-Group problem, 6 of the 11 erroneous solutions resulted from incorrectly defining the total number of equally likely outcomes, and 5 of these cases resulted directly from counting repeated selection of a single student (represented by the diagonal cells) as valid combinations. Thus, it was a common error to count all 25 cells as valid outcomes by using the 5-by-5 table (Fig. 7). This corresponds to treating the selection of two (distinct) students as sampling with replacement. Another 3 errors involved incorrect use of the combinations formula, or failure to convert the count of outcomes to probability.

Fig. 7.
figure 7

Counting diagonal cells (=self-repeated combinations) as valid outcomes, a common misuse of the N-by-N table that led to erroneous solutions.

Full size image

Thus, more than 80 % of the procedural errors made with the N-by-N table involved the solver attempting to use the outcome-search strategy but being led astray by the structure of the N-by-N table. Specifically, in order to use the table correctly, problem solvers must recognize that all the diagonal cells should not be used, because the self-repeated combinations that these cells represent are impossible outcomes when sampling without replacement. Put another way, the structure of the table does not map in a one-to-one way with the structure of the problem.

The binary tree led to distinctively different error types, so that three out of five incorrect answers involved incorrectly defining stage-wise probabilities. For example, the correct probabilities for the two sequential selections should be P(G1) = 3/5 and P(G2|G1) = 2/4. However, the erroneous solutions involved incorrect probability values for these two events.

Discussion. For the Work-Group problem, the tree and the table altered the frequency of using particular strategies, with the N-by-N table biasing solvers to select an outcome-based strategy (the “outcome-search” strategy), and the binary tree leading them to select a sequential strategy based on an event-level representation (the “sequential-sampling” strategy). These differences in turn led to differential error rates for the provided-diagram conditions, and also to characteristic error patterns that are highly distinctive. In particular, the N-by-N table led to more use of an outcome-based strategy, and more errors in identifying the correct equally-likely outcome space, while the binary tree led to more use of a sequential strategy, with typical errors in identifying the correct stage-wise probabilities for unequally-likely outcomes.

We explain these solution-strategy effects in terms of the compatibility between the diagram and the relevant problem characteristics. Combinations problems (e.g., “How many ways can N objects be selected k at a time?”) are typically interpreted as involving the simultaneous sampling of k entities from a larger set of N entities (e.g., by application of the formula for the number of combinations of n objects selected k at a time), but can also be formulated and solved as involving sequential sampling (k draws, a single object at a time, without replacement). However, in the latter case, order of selection is implied to be relevant, so answers may require appropriate adjustment.

The N-by-N table displays the outcomes in an outcome space simultaneously, by integrating all outcome cells into a single table. Therefore, it cues solvers to search for all possible outcomes in the whole outcome space and for the target event as a subset embedded in the whole outcome space on the table. Note specifically that an unlabeled N-by-N table implicitly cues solvers to consider all outcome cells as relevant to the problem, including those diagonal cells that represent self-repeated selections, although these are impossible outcomes for the combinations problem.

Thus, the table diagram led to more erroneous identifications of the correct outcome space, because its representational format violates the Principle of Congruence articulated by Tversky, Morrison and Betrancourt [27], in that there is poor fit between the structure of the diagram and the structure of the problem.

In contrast, the binary tree steers problem solvers towards use of an outcome space with only four unequally-likely outcomes: S = {BB, BG, GB, GG}. Further, we believe that the tree diagram’s left-to-right hierarchical structure cues viewing the problem as involving sequential sampling without replacement. Therefore, the binary tree steers problem solvers towards a sequential-sampling strategy. Correct execution of this strategy requires correctly specifying conditional probabilities for the second student selected, as in Fig. 5b. Here, errors made by the binary tree condition tended to involve incorrect specification of these probabilities.

3.3 Analysis of the Spinner (Independent Events) Problem: Effects of the N-by-N Table Versus the N-ary Tree

The two generic diagrams offered as aids for the Work-Group problem actually vary two aspects of the diagram at once: the diagram structure (trees versus tables), and the abstraction level of outcome space (an equally-likely outcome space based on the specific students selected versus an unequally-likely outcome space based only on sex of the two selected students). For the Spinner problem, exemplifying the use of the fundamental principle of combinatorics, we chose to control for the abstraction level of the outcome space and vary only the diagram type: trees versus tables. We also refer to this problem as the “independent events” problem, because the outcome of the first spinner is independent of the outcome of the second spinner.

Strategy Choice.

As seen in Table 3, for the Spinner problem, the N-by-N table diagram led to more frequent use of the outcome-search strategy (68.8 %), compared to 31.3 % when no diagram was provided, χ 2(1, 32) = 4.500, p = 0.034. However, the N-ary tree, which cues both use of the equally-likely outcome space and a sequential strategy involving the definition of stage-wise probabilities, led to mixed choices of solutions: 50 % of participants in the N-ary tree condition used the outcome-search strategy, while 50 % of them used the sequential-sampling approach.

Table 3. Raw and proportional frequencies of solvers’ solution choices for the Spinner problem as a function of diagram type.
Full size table

Solution Success.

Table 3 also shows the frequency and percentage of participants in each condition who successfully solved the Spinner problem. Compared to the no-diagram condition (50 % procedural correctness rate), both provided diagrams increased the percentage of procedurally correct solutions. The increase for the N-ary tree, to 62.5 %, was not significant, χ 2(1, 32) = 0.508, p = 0.476. However, the increase to 87.5 % for the N-by-N table was significant, χ 2(1, 32) = 5.236, p = 0.022.

Error Analysis.

For the Spinner problem, the most common error was to calculate the probability of obtaining one specific color twice, instead of any color twice. Specifically, this error involved incomplete solution of the problem: finding the probability for the two spinners to both land on a particular color to be (1/3)·(1/3) = 1/9, and stopping there. However, the correct solution should be three times this probability (=1/3), because there are three colors on each spinner, and hence three same-color outcomes. In other words, solvers making this error failed to solve the problem completely due to failure to integrate intermediate results and all possible outcomes. This type of error and other types of errors were most likely to occur with no diagram, or with the tree. On the other hand, the N-by-N table, by displaying all possible outcomes for the independent events problem in a visually efficient way, facilitated coordinating the multiple substages of the problem solution, and thus improved the success rates.

Discussion. For the Spinner problem, the N-by-N table was the most effective representation, improving procedural correctness over the no-diagram condition. This is not surprising, since the table represents the N·N = 3·3 = 9 equally likely outcomes in a simple and direct way, even allowing space for labeling the 9 outcomes. Furthermore, the table representation naturally suggests the semantic aspects of the problem: that there are two different spinners that are of equal status or priority (corresponding to the rows and columns of the table) [cf. 15]. These findings support an explanation in terms of diagram congruency, that the more compatible the content and the structure of a diagram to that of the represented problem, the more likely it is to be facilitative for solving the problem.

The N-ary tree would also seem to offer advantages for this Spinner problem: it too displays the nine equally-likely outcomes with roughly equal salience, allowing space for convenient labeling. However, the tree’s hierarchical structure suggests a sequential process, and here it is not explicitly stated whether the spinners are spun simultaneously or sequentially. Also, the tree does not emphasize the multiple target same-color outcomes for this problem to the same degree as the table; the table places these same-color outcomes on the main diagonal, where they are particularly prominent, and grouped after a fashion. In this way, the N-by-N table was able to provide extra external visual support for solvers to manage all the possible outcomes by chunking the essential information for computational efficiency [6, 28].

4 General Discussion

The present study explored whether and how different types of provided diagrams (trees or tables) influence problem solvers’ choice of mathematical structures and solution methods for probability word problems. It was hypothesized that a diagram will cue use of a particular solution strategy to the extent that structural and connotative properties of the diagram match to structural properties of the mathematical solution procedure or the formal representation of the solution space [6, 13, 14, 29, 30].

Based on the structural differences of trees and tables, we specifically predicted that N-by-N tables would lead to more use of an outcome-search strategy, and trees to more use of a sequential-sampling strategy. This prediction was confirmed for both the problems studied: the tree diagram tended to elicit a sequential-sampling strategy, while the table increased use of the outcome-search strategy.

However, for the Work-Group problem there were confounds in our diagram manipulation: the diagram conditions not only differed in basic diagram structure, trees versus tables, they also differed in the abstraction levels that the diagram suggests for the outcome space. The diagrams contrasted for the Spinner problem (the N-ary tree and the N-by-N table) removed this confound. The results showed that the N-by-N table strongly cues the use of an outcome-based strategy. Note the N-ary tree has features that can cue either solution. The N-ary tree conveys a sequential schema, which maps to the mathematical relation of sequential definitions of simple or conditional probabilities; and it also offers the cue to use low-level and visualizable equally-likely outcomes. Indeed, solutions were evenly split between outcome-based and sequential solution strategies for this condition.

The present results support and extend previous findings on the cognitive effects that visual representations have on reasoning, inferences, and problem solving [e.g., 7, 13, 31]. In particular, this study builds on and extends previous research on the structural properties of tree and table diagrams [e.g., 15, 24], by assessing the effects of using trees and tables on probability problem solving behavior. In particular, our results suggest that performance is affected not only by diagram type (tree versus table), but also by the abstraction levels of the outcome space that they suggest. Most prior diagram studies have focused on problem solving success as the criterion variable. Importantly, the present study demonstrates that diagrams can also affect strategy choice and the types of characteristic errors made in mathematics problem solving. This finding has instructional implications, because it implies that while diagrams can be an important and powerful form of scaffolding, they should be used in a carefully targeted fashion, taking into account their specific effects on the solution process. Also, it suggests that student-generated diagrams might be useful diagnostic evidence for inferring student problem-solving strategies and solutions.

The present results also show that the degree of the structural compatibility between a diagram and the particular type of problem to be represented affects problem solving success. For a combinations problem, which can be thought of as sampling without replacement, the use of an N-by-N table actually decreased the proportion of correct solutions. Because the diagonal cells are depicted with equal status as all the other cells, the table structure can mislead solvers to perceive all types of combination outcomes as valid, including those self-repeated combinations that do not arise in sampling without replacement. On the other hand, for an independent events problem, which involves sampling with replacement, the N-by-N table significantly improved participants’ solution correctness. For combinations of independent events, the table structure better conforms to the probability problem structure, cueing an appropriate solution.

Thus, the findings of the present research offer support for the Congruence Principle in diagram design, the idea that external visual representations must have analogous relational structures with the conceptual schemas and relations they are designed to represent in order to facilitate construction of appropriate mental representations and correct inferences [e.g., 6, 11, 27, 32–34]. Such congruence of structure also ensures that inferences or insights spurred by the diagrammatic representation can be accurately transferred back to the problem domain.