Epistemic trajectories: mentoring in a game design practicum

Abstract

Innovative professionals rely on a specific ways of thinking to solve the nonstandard problems that come up in practice (Goodwin, Am Anthropol 96(3):606–633, 1994; Schön, The reflective practitioner: how professionals think in action, 1983; Educating the reflective practitioner: toward a new design for teaching and learning in the professions, 1987; Sullivan, Work and integrity: the crisis and promise of professionalism in America, 1995). The professions have reproductive practices for transmitting these ways of thinking, such as practica (Schön, Educating the reflective practitioner: toward a new design for teaching and learning in the professions, 1987). In this paper, we examine the learning relationship between a mentor and team of college students through an ethnographic study of a game design practicum at a European arts school. To examine the role that the mentor played in this practicum, we use two theoretical constructs. Epistemic frames—the configurations of the skills, knowledge, identities, values, and epistemologies that professionals use to think in innovative ways—provide a model for examining professional expertise (Shaffer, Comput Educ, 46(3):223–234, 2006a). Epistemic network analysis (ENA) (Shaffer et al., Int J Learn Media, 1(2):33–53, 2009) is a method for quantifying changes in epistemic frames (Shaffer, The bicycle helmets of “Amsterdam”: computer games and the problem of transfer, 2010). Our results here suggest that the mentor leads the team on a path that illuminates the nature of learning to think professionally, as well the function of a mentor in that process. We argue that the mentor, rather than providing a direct map to a professional vantage point, scaffolds aspects of the epistemic frame of game design that, in turn, aid in the development of a more professional frame. Using ENA to understand the way that mentors help coach learners to develop epistemic frames should be useful for further studies of professional education, as well as for studies of apprenticeship-based programs for youth.

Introduction

There is growing concern that the 20th century mode of education, with its focus on basic facts, skills and problems with standardized answers, is leaving increasing numbers of young people unprepared for the future (Collins and Halverson 2009). The professions—and more specifically, the processes of creating new professionals—offer an alternative model for educational activities better aligned with the abilities required to navigate a complex and changing future (Gee and Shaffer 2010; Shaffer 2005b).

Innovative professionals rely on specific ways of thinking to solve the nonstandard problems that come up in practice (Goodwin 1994; Schön 1983, 1987; Sullivan 1995). Furthermore, the professions have their own institutions of learning and their own reproductive practices (Schön 1987). In practicum experiences, for example, novices “take on real-world projects under close supervision” (Schön 1987, p. 37). In this paper we consider in more detail the nature of the “close supervision”—also known as coaching, or more generally mentoring—in a professional practicum. We use data collected during a professional practicum in educational computer game design to assess the extent to which Schön’s “Follow me!” coaching model (1987, p. 212) offers insight into the nature of the “close supervision” that mentors use to help novices develop professional ways of thinking.

The importance of such a study is timely and necessary: recent work has described the pedagogical and developmental benefits of apprenticeship-based learning models in which young people are guided by mentors (Halpern 2010; Rose 2004). An interest in how the training practices of professionals can serve as models for the development of technology-supported learning environments for K-12 students (Shaffer 2006b) has informed other work looking at the processes by which mentors instill professional ways of thinking (Nash and Shaffer 2010).

In this paper, we examine the learning relationship between a mentor and team of college students through an ethnographic study of a game design practicum at a European arts school. In the practicum, seven advanced undergraduate students were assigned to design and build a computer game by an external client from industry. Their mentor, a professional game designer and game design consultant, guided them in this endeavor.

To examine the learning relationship between the mentor and learners in this practicum, we use two theoretical constructs. Epistemic frames—the configurations of the skills, knowledge, identities, values, and epistemologies that professionals use to think in innovative ways—provide a model for looking at professional expertise (Shaffer 2006a). Building on epistemic frames is epistemic network analysis (ENA) (Shaffer et al. 2009), a method for quantifying changes in epistemic frames. ENA is a tool for comparing epistemic frames held by different parties or by the same party at different times (Shaffer 2010).

Our claim here is that the mentor, using Schön’s “Follow me!” coaching model (Schön 1987), leads the team on a path that illuminates the nature of learning to think professionally and on the function of a mentor in that process. In this practicum, the mentor does not model the full professional way of thinking—or even the same parts of a professional way of thinking—throughout the practicum. He scaffolds different aspects of the epistemic frame of game design at different times in the practicum, and the cumulative effect of such differential scaffolding, rather than continuous modeling of the full range of professional thinking skills, aid in students’ development of a more professional frame.

Theory

Learning to think like a professional

The issue with learning to be a professional is that professional problem-solving is itself problematic. As Donald Schön argues, “the problems of real-world practice do not present themselves to practitioners as well formed structures” but as “messy, indeterminate situations” (1987, p. 4) in which the problem itself is not yet known. These situations, as Dreyfus and Dreyfus point out in their discussion of developing expertise, have “a potentially unlimited number of possibly relevant facts and features” (1986, p. 20) that could inform a course of action.

Different professions approach ill-defined situations differently. Charles Goodwin calls the shared way that professionals see and categorize their domain “professional vision” (1994). Professional vision, according to Goodwin, is employed by a community of practitioners who expect from each other a common way of organizing the world that is consistent with the values and methods of the profession. In other words, each profession is a “community of practice” (Lave and Wenger 1991): a group of people who share similar ways of seeing and solving problems. Goodwin (1994), for example, notes that archeologists will see the same patch of dirt differently than farmers would. Professional communities of practice rely on a “set of values, preferences, and norms in terms of which they make sense of practice situations, formulate goals and directions for action, and determine what constitutes acceptable professional conduct” (Schön 1987, p. 33). Thus, to learn to be a professional, one must be initiated into a professional community of practice.

All professional communities of practice face the challenge of “how to teach the complex ensemble of analytic thinking, skillful practice, and wise judgment upon which each profession rests” (Sullivan 1995, p. 195). Schön (1987) argues that for most professions, this complex ensemble is developed in practicum experiences. Although literature on the purpose, value, and shortcomings of the practicum in professional education exists (for example, Ryan et al. 1996), here we focus on the mechanisms of learning within the practicum. In the professional practicum, novice professionals do the work of a practice in “simulated, partial, or protected form” (Schön 1987, p. 38) while guided by mentors who engage them in reflective conversations.

Mentors’ roles within the practicum is to facilitate novices’ work and reflection on that work by scaffolding what they are not yet ready to do.¹ Mentors scaffold within the discourse of the practicum, demonstrating and describing the profession’s way of seeing and solving problems. At the beginning of a practicum, learners lack both the vocabulary to talk about the work and the experience that would give that language any meaning. Yet as the practicum proceeds, and novices begin to do the work and talk about it with their mentors, Schön (1987) notes that they seemingly become more fluent in the language the mentors use. This convergence of meaning occurs as the learners imitate mentors’ speech and actions until it is internalized. Characterized by Schön as a “Follow me!” model of coaching, this way of mentoring is fundamental to the practicum experience.

Implicit in the “Follow Me!” coaching model is that the learner ultimately goes to where the mentor is: to the professional vantage where competently expert decisions may be made. Less clear, however, is where the mentor needs to take them within the process of learning in the practicum. If it is the mentor’s job to arrange, as Schön puts it, the “right” kinds of experiences for the learners, by what logic is an experience “right”? The answer depends in part on the specific professional practice that the practicum simulates and the particular phase of activity in which the learner is presently occupied. Yet Schön is clear that professional expertise must be grasped as a whole in order to be grasped at all. It cannot be learned in a molecular way, for different activities and understandings “tend to interact with one another and to derive their meanings and characters from the whole process in which they are embedded” (1987, p. 158). If the path to professional expertise is not linear, what is the path that is taken? What would be the logic of such a path?

In order to answer such questions, it is necessary to carefully choose a way to examine the complex ensemble of professional vision and the process of developing it. As Pickering (1995) describes in his analysis of scientific practice, there are an indefinite variety of tools, material and conceptual, for capturing and framing aspects of the “mangle of practice.” To capture and frame practice with “machinic grips,” as Pickering (1995, p. 188) calls such tools, is to provide a situated framework for productively understanding and describing the complexity of real-world phenomena. The machinic grips to be used in this study are theoretical constructs for examining and measuring professional thinking.

Epistemic frame theory

The professional way of seeing the world that Schön describes as “the competence by which practitioners actually handle indeterminate zones of practice” (1987, p. 13) can be opaque to those outside the community of practice in question. Some specification is provided by Goodwin (1994), who, in his description of archeological field work and courtroom practices, describes the concrete ways practitioners highlight and elide things that are important or not according to their professional perspective. Likewise, Sullivan (1995) offers a sense of how professionals employ an ensemble that includes intellectual, practical, and ethical components. Shaffer (2006b, p. 160) extends these insights by emphasizing the relationships between “the combination—linked and interrelated—of values, knowledge, skills, epistemology, and identity” that characterize the professional ensemble—what he calls the “epistemic frame”. Shaffer’s theoretical construct of epistemic frames is faithful to prior conceptions of professional expertise in that frames are both social and individual, and constituted with elements that inform particular ways of thinking, seeing, talking, doing, and decision-making.

Epistemic frame theory emphasizes how aspects of professional thinking are linked in practice (Shaffer 2010). For example, reporters may write a certain way because they view their job as serving the important societal function of being a community watchdog. In this epistemic frame, a particular journalistic skill is informed by a specific journalistic value and a specific sense of professional identity. Although it could be argued that working a certain professional way, possessing certain professional knowledge, and having certain professional values are always unavoidably linked, particular frame elements are not always linked to all of the others in practice. As Shaffer (2010, p. 17) puts it, “the purpose in marking certain frame elements as significant to a community of practice is to understand how they become linked in the ways characteristic of expertise within the community.” That is, by not assuming a priori that all epistemic frame elements are always linked in practice, one can examine when they are linked, and thus gain a valuable perspective on how professionals see and solve problems in practice. An assessment method aligned with this learning theory must therefore be able to chart and compare epistemic frames comprised of distinct patterns of frame links.

Epistemic network analysis

If epistemic frames are appropriate models for looking at professional expertise, a method for observing change in epistemic frames is required to chart their development. Epistemic network analysis (ENA) is a technique for quantifying and analyzing an epistemic frame (Shaffer et al. 2009). ENA adapts a social network analysis framework—typically used for mapping the relationships between social actors—to instead map the constituent elements of complex thinking.

In social network analysis, the objects of interest are not the individual actors themselves; rather the focus is on the structure of the relationships among those actors. Similarly, epistemic frame theory suggests that complex thinking is not characterized merely by a collection of values, knowledge, skills, epistemology, and identity, but rather by a particular structure of relationships or connections between these components of expert practice. Thus, analyzing thinking in terms of simple counts of frame elements in an (aspiring) professional’s frame is not sufficient to account for the development of expertise. Instead, ENA uses co-occurrence of frame elements in discourse to model the pattern (or patterns) of association characteristic of a particular professional community. By quantifying the relationships between epistemic frame elements, ENA describes and quantifies the structure of an epistemic frame. Where one frame may have one set of important relationships between its constituent elements, another frame may emphasize different relationships. ENA provides a method for examining when and how often frame elements are linked, and can show trends in how epistemic frames change over time, between individuals, or across different interactional contexts (Bagley and Shaffer 2009; Nash and Shaffer 2010). ENA is thus a potentially useful tool for comparing the epistemic frames of a team of learners and a mentor in a practicum.

Many of the studies that examine the learning that takes place in a practica rely on surveys (Ryan et al. 1996), which do not measure learning in situ and rely on self-report data, or are qualitative, such as ethnographies (see, for example, Hutchins 1995). While both of these methods shed light on the processes of learning and of mentoring in a practicum, ENA is a potentially useful addition to the scholarly toolkit because it can quantify thinking in the context of action. In the case of a practicum, the conversations between mentor and learner provide the occasion for the mentor to model professional thinking and for the learner to imitate that modeling (Schön 1987). Using ENA to examine the reflective conversations in the practicum, we can further examine whether (and, more important, how) the team of learners imitates or “follows” the mentor, as suggested by Schön’s model.

We operationalize such a question by comparing the “distance” between a mentor’s frame and the learners’ collective frame when they have meetings together across the time of the practicum. By distance, we mean the projection of the structure of the epistemic frame, as quantified by ENA, into a high dimensional space. Using multi-dimensional scaling, we can visualize epistemic trajectories: models that show the distances between team and mentor’s frames throughout the practicum change over time. Finally, constructs borrowed from social network analysis can help us interpret these trajectories. For example, the relative centrality of a frame element quantifies the extent to which it is connected to other elements in the professional discourse, and therefore can help us interpret the mathematical concept of “distance” by suggesting in what ways two different frames are similar or different.

This study: epistemography of a game design practicum

The starting point for this study, then, is the idea that in a practicum, a mentor leads learners through the authentic work of a professional practice and that it is through the mentor’s coaching that the learners develop a particular epistemic frame. We examine the epistemic frames of a mentor and a team of game design college students in a practicum.

As in all practica, the students in a game design practicum do the work that professional game designers do. They seek to create games that feature meaningful play, which game design experts describe as occurring “when the relationships between actions and outcomes in a game are both discernable and integrated into the larger context of the game” (Salen and Zimmerman 2004, p. 34). In other words, a critical job for game designers is to make a game in which players can perceive the immediate outcome of the actions they take—the relationship between action and outcome in games is often called game-play (or game mechanic)—and that those outcomes are consistent with the game as a whole. The game as a whole, also known as the game concept, can be simple (e.g. Angry Birds) or complex (e.g. World of Warcraft), and encompasses the vision of the overall player experience of the game. It often includes narrative elements, and in the case of educational or “serious” games, the learning goals (Abt 1970; Gee 2003; Squire and Jenkins 2004). In addition, both the game-play and the game concept simulate some phenomena that are usually, but not exclusively, real-world (Salen and Zimmerman 2004). Thus, one important activity in game design is researching aspects of the phenomena to be simulated; these aspects are referred to as the content domain. Learning to link these three elements—game-play, game concept, and concept domain—is a fundamental task of game design. This study looks at the role of mentoring on the learning trajectory of the team of students as they learn to link these elements of game design’s epistemic frame.

To do this, we conduct an epistemography: an analysis of the structure of a professional practicum through the lens of epistemic frames (Shaffer 2005a). Epistemographies focus on the participant structures—“conventional configurations of communicative activities and roles” (Shaffer 2005a, p. 4)—of a practicum. Some of these structures are activities that correspond to the practices of a specific domain and serve a particular pedagogical purpose in terms of epistemic frame development. For example, Hatfield and Shaffer (2010) showed how different participant structures in a journalism capstone course, which included activities such as news meetings and copy editing, integrated particular groupings of epistemic frame elements in a way that helped students bind together a journalist frame. Other participant structures, however, are particular to practicum experiences but are not necessarily domain-specific: Schön’s coaching model is employed by mentors in the practice of a variety of different professions. The participant structure of interest here is this coaching model. Although the study is situated in an examination of one particular practicum, we wish to use this analysis to capture how novices in practice come to think like professionals under the guidance of a mentor.

We ask three questions:

• 1. Does the team “follow” the mentor in the sense that the team and mentor’s frames during meetings become more similar over time?

• 2. Where does the team follow the mentor? In other words, what is the relationship of the mentor and team’s frame trajectories in terms of distance?

• 3. What is the explanation for how the mentor and team’s frames change in distance across their trajectories?

To answer these questions, we conduct a qualitative (ethnographic) analysis of the practicum and then triangulate this analysis in quantitative terms using ENA.

Methods

Setting: school and practicum

The game design practicum took place in a northwestern European country at an undergraduate level arts school. The practicum course occurs in a Game Design and Development program that terminates with a Bachelor of Art and Technology degree. The semester-long practicum experience is organized around the production of a single game. At the beginning of the semester, students are assigned to teams, and teams to clients.

The team in this study was assigned to a client who works with the government’s Ministry of Agriculture. The client assigned the team the task of creating a game “in which the complex chains and reality of the different fish-production is made clear to consumers” with the goal of educating them “about the problems with fish food.” The stated goal was to provide at least a working prototype of the game by the end of the semester. The game was expected to be faithful to real-world phenomena related to the aim, and the team was provided with research questions related to the content area of interest.

Other than three mandatory lectures and a mid-semester review, the teams of students were free to schedule their work and meetings as they saw fit.

Participants

The team was comprised of seven students, ranging from 18 to 29 years old. Two were international exchange students who did not speak the language of the host country. Since every student spoke English as a second language, all conversations were in English.² The mentor assigned to work with the team is currently a professional designer and developer of educational software.

Data collection

Data were collected though observations of every team meeting during the first 2 months of the practicum, until the midterm review. Meeting data were collected in digital audio recordings and supplemented with field notes. Recordings were transcribed to provide a detailed record of interactions.³ No other demographic information was collected about the students.

We interviewed the team’s mentor twice during the 2 months, asking questions designed to elicit the mentor’s opinions about the epistemic frame of game design, as well as questions about the progress of the team and his role as a mentor. We conducted the same type of interview with two other mentors who were assigned to different teams. These interviews were audio-recorded and transcribed. The full set of interview questions is available in Appendix 1.

Data analysis: coding scheme

From interviews conducted with three mentors at the school, we used a grounded theory approach (Strauss and Corbin 1998) to generate a set of qualitative codes representing the epistemic frame of game design. The list of the game designer frame elements are outlined with definitions and examples in Appendix 2.

The codes for the key game design frame elements described in the previous section are included in Table 1.

Table 1 Codes for three key game design epistemic frame elements

Data analysis: segmentation and coding

The focus of this study was to examine the learning relationship between mentor and team. There were four team meetings at which the mentor was present: three of the four meetings were comparable in the sense that each lasted more than 3 h and contained sustained substantive discussion of the design project in progress. The fourth mentor-team meeting recorded was a 15-min check-in, and was excluded from this analysis because of its structural differences from the other three.

We segmented the three meetings into interactive units (“stanzas”) which were defined as sequences of utterances with a consistent topical focus. For example, if the team started discussing their strategy for an upcoming meeting with a client and then switched to discussing the profile of their target users, the switch in discourse topic would indicate two separate interactive units. There were 14 stanzas in the first meeting, 17 in the second, and 24 in the third. The length of these stanzas varied.

For the three team meetings, we coded each stanza for the contributions of both the mentor and the team for articulations of the elements of the game design frame (as generated from the interviews with the three mentors.) If discourse in the stanza was determined to meet the criteria of a code’s definition, that stanza was coded with a ‘1’ for that code. If none of the discourse in that stanza met the criteria of the code’s definition, that stanza was coded for a ‘0’ for that code. In order to compare the team’s frame to the mentor’s frame, the team members’ comments were coded collectively instead of individually within each stanza. In other words, if no team member discussed a given frame element, that stanza would be coded with a 0 for that element, but if at least one team member talked about a given frame element, that stanza would be coded with a 1 for that element.

Data analysis: measuring frames

Adjacency matrices

For this study, the epistemic frame of game design is assumed to be composed of the frame elements described in Appendix 2. For the team and mentor, we looked at each data segment (each stanza of each meeting) for evidence that the participant is using one or more of the elements of the game design epistemic frame. We then created an adjacency matrix for that segment. Adjacency matrices record the links between individual frame elements. When a pair of elements co-occur within a stanza, they were considered conceptually linked (Shaffer 2010).

By representing the epistemic frame in use during a segment as an adjacency matrix, we can use the tools of epistemic network analysis to examine the cumulative impact of strips of activity on a developing epistemic frame. For both the team and mentor, we constructed final adjacency matrices for each meeting by summing each meeting’s constituent stanzas’ adjacency matrices. For an example of the data structure and an adjacency matrix, see Appendix 3, which shows the raw codes for meeting 1, and the final adjacency matrix for the team in meeting 1.

Relative centrality

For both the mentor and the team, we calculated the relative centrality of each of the frame elements that constitute their epistemic frames in each meeting. The relative centralities of the component frame elements show the shape of the frame at a given time or context. Relative centrality is a measure of how often each element is connected to all of the other elements in discourse; in other words, it is a measure of the relative weight of an epistemic frame’s constituent elements (Shaffer et al. 2009).

To compute the weight of a frame element, the square root of the sum of squares of its associations with its neighbors is calculated. Measuring the strength of association for a given frame element f _i emphasizes those elements with tighter linkages to individual neighbors f _j . We compute the sum-of-squares-centrality C(f _i ) or weight of an individual node f _i in matrix M as:

$$ C(f_{i} ) = \sqrt {\sum\limits_{j} {(M_{ij} )^{2} } } $$

(4)

To compute the relative centrality of an individual frame element, its weight is divided by the frame element with the greatest weight in the network (Shaffer et al. 2009). We compute the relative centrality R(f _i ) of an individual node f _i in matrix M as:

$$ R(f_{i} ) = \frac{{C(f_{i} )}}{{C_{\hbox{Max} } (M)}} $$

(5)

where C _max(M) is the maximum node weight of any node in M. For an illustration, see Appendix 3, which shows the relative centralities of the team’s frame in the first meeting, by stanza.

Data analysis: comparing frames

Frame similarity index (FSI)

The frame similarity index provides a testable measurement of the similarity between two epistemic frames (see, for example: Bagley and Shaffer 2010). The Euclidean distance (D) between two frames (f _r , f _t ) is calculated by finding the root mean square of the differences of the relative centrality of each individual ( _i ) frame element (C) between the two frames.

$$ D\left( {f_{r} ,f_{t} } \right) = \sqrt {\sum\nolimits_{i} {\left( {C_{{f_{{r_{i} }} }} - C_{{f_{{r_{i} }} }} } \right)^{2} } } $$

The maximum theoretical distance between two frames (MaxD) is the Euclidean distance between a frame where every element has a relative centrality of 100 (all possible connections) and a frame where every frame element has a relative centrality of 0 (no connections).

$$ {\text{Max}}\,D\left( {f_{r} ,f_{t} } \right) = \sqrt {\sum\limits_{i} {(100 - 0)^{2} } } $$

The maximum theoretical distance between two frames with 32 constituent frame elements would be 565.68. Dividing the distance between the two frames by the maximum possible distance between those two frames provides the distance expressed as a percentage of the maximum possible distance.⁴

To estimate confidence intervals for the difference between two frames, a jackknifing method can be used. The relative centralities of the frames to be compared are systematically recomputed leaving out one stanza at a time. The distances between the subsample of resulting frames (one frame per stanza, per meeting) are then calculated. The standard deviation and confidence intervals of the FSI statistic are then estimated from the variability within the calculated subsamples.

Data analysis: epistemic frame trajectories

While useful, the FSI statistic does have limitations. The most significant limitation is that it can only consistently compare two frames at a time.⁵ To see all of the team and mentor’s meeting frames in relation to each other, and to more accurately compare those frames, we require a more sophisticated model: epistemic frame trajectories.

First, the distances between the mentor and team’s frames in and across each of the three meetings are calculated using FSI. In the case of this study, those points of interest are the mentor and team’s frames in each of the three meetings. These distances are organized in a symmetric distance matrix, made up of 6 distance vectors, as seen in Fig. 1.

Fig. 1

A 6-dimensional symmetric distance matrix

A classical multidimensional scaling (MDS) algorithm is applied to the distance matrix in order to identify the dimensions that capture the most variance in the data. Two dimensions that account for a high amount of variance are then chosen to project the points representing these six points into a two-dimensional space. Because MDS does not preserve directionality, specific dimensions in the dimensional projection are not interpretable (Bartholomew et al. 2008). However, relative position in space is still meaningful in that points closer together in the space have more similar patterns of co-occurrence than points farther apart.

Data analysis: highlighting specific relationships

When a frame is composed of many elements, the relationships between frame elements of particular interest can be difficult to discern. To examine the specific relationship between the elements of interest, unaffected by the rest of the frame, we can create subset adjacency matrices for just those elements. These matrices are constructed as described above, but only include links between frame elements of interest in each stanza. The stanza subset adjacency matrices are then summed to create a final subset adjacency matrix.

We use the values from the final subset adjacency matrices to graphically represent the relationships. To control for variable meeting lengths, we normalize the values in the final subset adjacency matrices by dividing by the number of constituent stanzas in each meeting. This normalization produces comparable matrices across meetings of different lengths, in the sense that the relationship between two frame elements in a meeting in which they co-occur in every stanza will always have a value of 1, and all other weights are scaled proportionally.

Using these tools, as well as a qualitative examination of the team and mentor’s discourse in each of the three meetings, we operationalized our research questions in the following ways:

Research question 1

Does the team follow the mentor? In other words, do the team and mentor’s frames during meetings become more similar over time? To answer this question, we first qualitatively examined the discourse of the mentor and team in each of the three meetings. We then used FSI to measure the distance between the team and mentor’s frames in each of the three meetings. We looked to see whether the distance between the team’s frame and the mentor’s frame was reduced with each successive meeting.

Research question 2

Where does the team follow the mentor? In other words, what is the relationship of the mentor and team’s frame trajectories in terms of distance? To answer this question, we first qualitatively examined the discourse of the mentor and team in each of the three meetings. We then constructed epistemic frame trajectories to show the distances between the team and mentor’s frames across the meetings in a two dimensional projection of a multidimensional space. We used multi-dimensional scaling (specifically principal coordinates analysis) to create two dimensional trajectories from the six dimensional space of the team and mentor’s meeting frames. We used the second and third of the resulting six dimensions to display the two dimensional trajectories. Although the first dimension captured the most variance, and thus could be argued to be the most important, it appeared to be highly correlated with time. Since we already knew that the meetings were separated by time, we considered the second and third dimensions more revealing.

Research question 3

What is the explanation for how the mentor and team’s frames change in distance across their trajectories? To explain the trajectories, we examined the change in relative centralities of the frame elements across the three meetings for both the team and mentor. Since the positions in the epistemic trajectories are determined by FSI scores, which in turn are calculated by the similarity of the relative centrality of the frame’s constituent elements, changes in relative centrality would influence the position of the team and mentor’s epistemic frames with each meeting. In order to further understand how particular epistemic frame elements—game-play, game concept, and concept domain—were linked by the mentor and team in each of the meetings.

Results

We describe our observations of the game design practicum in four parts below. The first part is a qualitative look at the three meetings, and the final three parts quantify what was observed in that qualitative investigation.

Qualitative results

The team had met a handful of times before their first meeting with the mentor and brought with them a list of brainstormed game concepts. The meeting took place in the mentor’s apartment and was close to 4 h in duration. The mentor ran the meeting, and was firm, honest and critical of the work the team had done thus far. He began by setting an agenda for the meeting that included items such as: working as a team, tools for planning and meeting and setting deadlines, the requirements of the assignment, the goals of the client, and the research that the students had already done and the game concepts they had brainstormed. The meeting had a contentious mood, and the team bristled at many of the mentor’s suggestions about how they should work together.

Near the start of the meeting, one student observed that, “We’ve mostly just talked a lot.” The mentor agreed, and told the team that they needed to “quit the endless brainstorm and start working.” He felt that not enough work had been done, that the team was not adequately organized, and that they were prioritizing the wrong things. The team had brainstormed game concepts, but as one student admitted, “There isn’t any gameplay yet.” None of the team’s game concepts were grounded in either a vision of gameplay or rigorous research. The mentor, recognizing the flimsy nature of the team’s concepts, explicitly reinforced the link between gameplay and game concept, telling the team that “the challenge of your game has to be in line with your educational challenge.” In other words, the mentor was telling the team that the challenges a player faces in the game need to reflect the things the player is supposed to learn by playing the game.

The mentor was similarly explicit about the link between game concept and the content domain. Noting that the team’s research into the content domain is only “okay”, the mentor described the relationship between the two:

…we’ll also do the next step of research. Like if you want to do the fish tycoon concept, that means you have to do a lot of extra research on fish management and business. And if you take the fisherman concept, you have to do another type of research.

Finally, the mentor linked all three elements:

I push you to come up with the game mechanics. What does the player do? Make a list of the most important objects of the game. If you have shrimps what do they do? [Get] born, die, ill, bigger, get sold. Think about events. What happens in your game? Gear up your research. Be careful that it is correct.

The main problem with the team’s work in this stage was that their game concepts were not connected to game play or a content domain. The mentor sent the team away with instructions “to fill it in. A lot of concepts seem nice, but then you [need to] fill it in.”

Whereas the first meeting was run by the mentor, the second meeting was run by the team. They used planning tools such as the action list that the mentor had suggested they use in the first meeting, and they had, as the mentor had instructed in their first meeting, done “the next step of research.” They spent most of the second meeting reporting their research into the content domain, answering questions about fish farming, transportation, pollution, piracy, and fishing methods. They had filled the gaps that the mentor was concerned about in the first meeting.

The mentor actively prompted the team to be explicit about the utility of their new information, at one point asking, “So, what is the main question you need to know for your game with respect to the storing of the fish?” It seemed that the team was on the same page. Much of the activity in the meeting was precisely this kind of decision making. For example, a student remarked how “it’s very important to be more specific about what fish we’re going to use.” When the mentor asked whether the team was deciding to abandon fish farms as part of their game concept to focus more specifically on fish marts, one student responded:

Yah, this is crucial now. I mean, this week, at the end of the week, we are supposed to have a document or at least an idea of what will happen, what the game will be like from beginning to end. The whole game. We have to map the game completely, and that’s the deadline. That’s the most important thing for this.

Mapping the game completely required the team to describe the concept and game play in terms of the content that they have researched. As one player said, “There’s a lot we don’t know yet and have to come up with.”

Although the topic of the second meeting was on content, the mentor did not allow the importance of the link between the game mechanic and the game concept to slip away. Toward the end of the meeting, as the team was figuring out what to do next, the mentor reinforced again the relationship between the three elements, explaining,

In the process, I think it’s important to first, to get your idea of gameplay clear. And to rev up the research, to see if it will go. The other thing is, did you elaborate on the design idea…?

Although all three are important, he suggested that the team now prioritize the gameplay. Finishing the research on the content, he said, “is also important, but it also can be done today or tomorrow or next week.” The time for prioritizing the content domain was done.

As in the second meeting, the third meeting was run by the team rather than the mentor. At this meeting, the team was preparing for two impending deadlines. They were planning on conducting play-tests of some of their game prototypes, in order to get feedback on the game mechanics. They were also preparing for a mid-term presentation, at which they needed to present their game concept, complete with working prototypes, and reports of the research they had done to support their design decisions. At the meeting, the team shared and discussed a number of prototypes for mini-games within their game concept, as well as game art that had been created for those prototypes.

The mentor continued to give explicit advice about the three key frame elements. In a discussion about one of the mini-games the team was creating for their game, the mentor talked about the “story” of the game. The mentor said:

The story has to be matching the reality. If you say there’s always bad-catch, you can only say that if that’s the reality. Or you could say, this is just ordinary fishing…. Then later in the story maybe you say something about environmental, dolphin-friendly tuna. Or you play this game again and the score is really [about how you] find and get no dolphins or throw the dolphins out again.

The game story is a way of talking about the game concept: it is the sequence of challenges the player must face in order to complete the game and, since this is a game with a pedagogical purpose, to learn the intended lesson. Both the sequence and the individual challenges must make sense internally (in terms of the game mechanic) and externally (in terms of the content domain).

The team, having done enough research to start building their game, saw these connections too. Listing a series of mini-games that made up their game concept, one student used content domain language to describe how the gameplay must be persistent in their concept:

One game is like fishing in the Baltic Sea and avoiding environmentalists. The other is scaring off locals with your bulldozer. The other one we haven’t defined yet. That means that, in the end, all those things have to be present again.

Game mechanic and game concept needed to be inextricable at this point. As one student, thinking of how to divide the work in the coming days, put it:

Games, mini-games and story are so dangerously linked with each other, so that we cannot divide it in the groups. The ones who are working on the mini-games should also be already working on the story.

Qualitative results summary

The qualitative observation of the mentor’s role in the team’s development of a game designer epistemic frame shows the mentor reinforcing relationships between key aspects of the frame, and the team following the mentor’s lead by focusing on those same relationships in subsequent meetings. In the first meeting, the mentor was most concerned that the team’s game concept ideas were not conceptually linked to either a specific vision of game mechanics or rigorous research into the content domain from which the game is supposed to be inspired. He implored the team to do this necessary work. The team did, and they spent most of the next meeting reporting on their research. The mentor continued to talk about the importance of treating the content together with the game-play and game concepts, and the team responded by also considering them together. In the final meeting, the team and mentor focused on game prototypes, and thus mainly discussed their game concept. The difference from the first meeting was that the team now talked about their concepts in terms of both the content domain and the game play. To further examine the extent to which and how the team followed the mentor in terms of their professional thinking, we turn to the quantitative analysis.

Result 1: following the mentor?

The FSI was calculated to compare the team’s epistemic frame and the mentor’s epistemic frame in each of the three meetings. As seen in Fig. 2, in the first meeting the distance between the team and mentor’s frame is .208, in the second meeting the distance is .166, and in the third meeting is .128.

Fig. 2

Distance between Mentor and team’s frames in 3 meetings (The error bars represent two standard deviations from the mean of jackknifed subset: .023 for the first meeting, .015 for the second meeting, and .004 for the third meeting)

With each successive meeting, the team’s game design frame was closer to the mentor’s in the same meeting. This result suggests that the team increasingly mirrored the mentor’s game design discourse in the meetings.

This convergence occurred even though the relative amount of talk contributed by the team or mentor varies by meeting. In the first meeting, during which the mentor was almost lecturing the team on what they needed to do, the mentor-team ratio of words spoken is 3.46. The team led the second meeting, talking about twice as much as the mentor, with a mentor-team ratio of .524. Finally, in the third meeting, the mentor and team contributed much more equally to the conversation; here the ratio is .951.

Result 2: the nature of the team and mentor’s path

To examine the team and mentor’s paths, we created epistemic trajectories of the mentor and team’s frames across the three meetings. The trajectories show the convergence reported above, with the team and mentor frames becoming successively similar with each meeting (Fig. 3).⁶

Fig. 3

Team and Mentor epistemic frame trajectories. In MDS, the transformation from a high dimensional space to a low dimensional projection optimizes the preservation of distance but does not guarantee the preservation of direction. For this reason, the dimensions in MDS projections cannot be interpreted as having semantic significance (Bartholomew et al. 2008)

Although the distance between the team and mentor’s frame was reduced with each meeting, both the mentor and team’s frames in the second meeting were much further from the frames in either the first or third meetings. In other words, the team’s third meeting and first meeting frames were more similar than their frame in the second, and the same is true for the mentor. These trajectories suggest that the team’s epistemic frame development did not proceed uniformly from the first to the final meeting. Furthermore, the mentor and team’s frames similarly transformed along the trajectories.

Result 3: why that path?

To see what might have caused the nonlinear trajectories, we looked at the change in the relative centralities of three key frame elements across the three meetings. Around the time of the third meeting, the team was supposed to have a working prototype of their game, and therefore needed a game concept with a consistent game mechanic, both of which needed to simulate some real-world phenomena. Therefore, we examined the change in relative centrality of the following frame elements: the skill of concept development, the knowledge of game mechanic, and the knowledge of the content domain (Fig. 4).

Fig. 4

Change in relative centralities of three frame elements across three meetings

The change in the relative centrality of these frame elements shows how the second meeting might not have fit neatly between the first and third meetings. The development of a concept was important for both the team and mentor in all three meetings. Only in the second meeting did content research become the most central concern. Finally, the game mechanic became increasingly important across the three meetings. The spike in the relative centrality of K/Content Domain would appear to explain the large distance between the frames in the second meeting and the other meetings.

To examine the specific relationships between the three elements, unaffected by the rest of the frame, we created final subset adjacency matrices for just those three elements for the team and mentor for each meeting. To control for the variable meeting length, we normalized the values in these matrices by dividing them by the number of constituent stanzas in each meeting (there were 14 in the first meeting, 17 in the second, and 24 in the third). We then created network graphs for the mentor and team in each meeting, with line weights corresponding to the rate of co-occurrence.

Figure 5 shows comparatively how often each of the three elements was linked to the others in the team and mentor’s discourse in the three meetings.

Fig. 5

Strength of links between 3 frame elements in 3 meetings for mentor and team (No link means those two frame elements were linked in no stanzas in the meeting, while a value of ‘1’ would mean they were linked in every stanza of the meeting; line weights in the graphs are proportional to the strength of association)

In the first meeting, the team linked their S/Concept Development to the other frame elements very loosely. At that point, they had not connected K/Game Mechanic to K/Content Domain at all. By contrast, the mentor linked S/Concept Development to the other two frame elements quite closely.

In the second meeting, the team followed the mentor’s lead. They now linked their concept to the other frame elements much more closely. In particular, they connected the S/Concept Development to K/Content Domain more tightly than anything they had connected thus far, and as tightly as the mentor did in the same meeting. The mentor connected all three frame elements equally. Notably, his connection between S/Concept Development and K/Game Mechanic became looser.

In the third meeting, both the team and mentor emphasized the link between S/Concept Development and K/Game Mechanic. Links to K/Content Domain diminished, though were still present and important.

Quantitative results summary

The quantitative analysis both echoed and extended the qualitative findings. As seen in the FSI statistic, the team and mentor’s professional discourse grew more similar with each meeting. The epistemic trajectories, however, revealed that this convergence did not show the full picture. When viewing the meeting discourse in more than one dimension, we discovered that neither the team nor mentor proceeded directly to the third meeting. An examination of the change in relative centralities of the three key frame elements suggested that their indirect path was due in part to the focus of their second meeting: the content domain that their game was meant to simulate. The rates of co-occurrence between the three key frame elements in the team and mentor’s discourse across the three meetings revealed that the mentor consistently linked the three key elements. More specifically, he pushed the team to first focus on researching the content domain so that they ultimately could make better design decisions that connected their game concept to their game mechanics. In order to bring the team to a target way of thinking, the mentor seemed to have strategically over-emphasized some connections in the epistemic frame as part of the process of ultimately emphasizing more significant linkages later.

Discussion

The results presented here describe how a team of novices in a practicum come to think like professionals under the guidance of a mentor. By the time the practicum was half over, which was the end point of this study, the team needed to have a game concept and game mechanic that were tightly bound together and that simulated a well-researched content domain. The story presented here is about the path and mechanism by which the team arrived there.

First, the team imitated the discourse of the mentor. The team’s epistemic frame grew more similar to the mentor’s frame with each meeting. A feature of the “Follow Me!” coaching style is the mentor’s demand that the learners follow them through imitation (Schön 1987). It is, of course, possible that the increasing similarity of discourse in the later meetings is due to differences in the degree of freedom in the conversational topics: perhaps earlier in the practicum the conversations were simply more “open-ended” and thus the participants talked in different ways. We argue, however, that the convergence of professional discourse is by design, not happenstance. The conversations the team had in their meetings were informed by the particular intentions and direction of their mentor as well as the general construction of the practicum.⁷ Regardless of whether any particular topic of conversation was due to one or the other, the conceptual development of the game designer frame requires this convergence. Where the mentor and learner go together depends in part on the specific professional practice and the particular phase of activity within that practice. Indeed, the value of practicum participant structures that map to the activities of professional practice is that they provide learners with experiences that mobilize the connections between important aspects of the professional epistemic frame (Shaffer 2005a). A participant structure such as the coaching model, however, does not have an obvious analogue in actual practice.⁸ Mature professionals do not need the same type of guidance that learners in a practicum need because they have presumably already internalized the epistemic frame of their profession. The “Follow me!” coaching model serves to guide learners who are doing work without necessarily knowing how to do it or why they are doing it.

The epistemic frame trajectories show the path down which the mentor led the team by offering an additional perspective on how the team and mentor’s frame’s converged. Mentors and students did not take a “direct route” from where they started to where they each ended. Instead, their second meeting, distant from the first and third meetings, indicated a developmental “detour.” The change in the importance of three key frame elements—similar for the mentor and team across the three meetings—helps explain indirect path. The development of a concept was important in all three meetings. In the second meeting, the content domain became the most central concern. The game mechanic became increasingly important across the three meetings.

Although the game ultimately must have had a mutually reinforcing concept and mechanic, the mentor first led the team to other connections: to the connections between the content domain that the game is simulating and both the game concept and mechanic. That is, the development of a concept and game mechanic rely on the development of an understanding of the content domain. We see the students progressed from where they have “mostly just talked” without “any gameplay yet” to where they used specific content domain language to link their game mechanics to their game concept.⁹ In other words, the mentor concentrated on one part of the frame in order to scaffold another part of it, which suggests that the apparent detour in the epistemic trajectories was not a detour at all. Rather, the shortest distance from novice to professional thinking may not be to simply model best expert practice.

That the team has this indirect trajectory is perhaps unremarkable. After all, sometimes learners take steps backward on their trajectory forward. That both the team and the mentor share this type of trajectory, on the other hand, implies a learning experience quite different from the way traditional curricula, instruction, and assessment are organized. Most school subjects, for example, are organized to be taught in a strictly atomized and sequential manner. By contrast, a practicum is composed of very different participant structures. The trajectories demonstrate how mentors’ coaching in practicum is a participant structure that scaffolds learners’ engagement with other activities, both literal and conceptual. If learning to think like a professional requires rather than just accommodates indirect learning trajectories, as these results suggest, then the type of coaching by which mentors scaffold different connections within an epistemic frame is a type of learning relationship that deserves more attention.

Another valuable finding of this study is that epistemic network analysis was shown to be a useful way to quantify the development of epistemic frames, as well as the relationship between the students’ and mentor’s frames. Other methods of discourse analysis may well offer similar or additional results to those found in this study. However, the promise of ENA is that it is not driven by frequencies of qualitative codes, but rather frequencies of the co-occurrence of qualitative codes, and thus captures how practitioners connect the aspects of professional vision. In particular, projecting the distances between interactive units—whether they be meetings, activities in within meetings, turns of discourse within activities, or who the unit is associated with—by creating epistemic trajectories is a promising way to explore the nature of developing epistemic frames and complex ways of thinking in general.

The results presented here have several limitations. The ethnographic nature of this study necessarily means that any conclusions are limited to what one particular group of students and experts did in the context of one particular capstone course. Therefore, as Shaffer and Serlin (2004) argue, the purpose of significance tests under such circumstances is to show that additional observations made under the same conditions would show similar results. Further, this study focuses on a limited amount of data.¹⁰ Analyzing team meetings at which the mentor was not present would add essential information about what aspects of the epistemic frame the team internalized. Similarly, examining the epistemic trajectories within meetings would allow us to better map how frames are developing in relation to the activity and discourse in the meetings. In addition, when looking at the relationships between the mentor and student frames, this study treated the students collectively as a team, and so does not show individual development. The “Follow me!” coaching strategy might tend to work differently for different students, or differently during different activities, or at different stages of the practicum. Finally, epistemic network analysis is a new method for understanding the development of an epistemic frame. As such, we expect it to develop in ways that allow us to better test significant events in frame development.

Despite these limitations, the results here suggest that focusing on how mentors coach learners to develop epistemic frames should be useful for further studies of professional education, as well as for studies of apprenticeship-based programs for youth, and that epistemic network analysis is a useful tool for uncovering these learning processes.

Appendices

Appendix 1: Mentor interview questions

• 1. Tell me about what’s going on in this process right now. What’s supposed to be happening? What is happening?

• 2. What should the mentor’s role be in this situation? What has your role been? How have you been able to play that role?

• 3. How are the students doing? Are they learning what they are supposed to be learning? What are they getting out of it? What are they missing or struggling with?

• 4. Are there particular skills they should be developing? What are they developing?

• 5. Is there specific knowledge about the field and process they should be learning? Are they?

• 6. Are there particular values they should be developing? Are they?

• 7. Is there a particular process of decision making that they should be learning? Are they learning how to make appropriate decisions?

• 8. Do you think they see themselves more as students or professionals? Which should they see themselves as?

• 9. How is what they are doing the same or different than what a professional design team would be doing?

Appendix 2: The designation, definition, and an example of the game designer frame elements, as generated in a grounded analysis of the interviews with three mentors at the school

Code	Definition	Example from Mentor interview
Epistemology/contract	Justify decisions by referring to previously agreed-upon decisions	The game designer is the one who really has to make sure that any detour they make…doesn’t hurt the initial power of the concept, the essence
Epistemology/argumentation	Justify decisions by argumentation based on evidence	What they are confronted by or confronted with now is that decisions are made on arguments that are really slowly built….. Every decision is an add-on to your heuristics, because you’re further on in the process and it’s a logical one following from what you had when you started out
Value/good team	Importance of team synergy/good team	Everything, the quality comes from the group dynamic, not so much the individual skills of the people
Value/getting it done	Importance of working hard to accomplish tasks and to produce results	… you have a deliverable very day. You get so used to finishing stuff that you hardly have any deadlines left to celebrate. It’s all in a day’s work
Value/originality	Importance of making something new	I’d rather have them and then come, try something new. Maybe it’s a failure, but at last we tried some new game play than we play it on the safe side…
Value/concrete	Importance of having concrete goals	It’s a goal…for them to work concrete and specific, as soon as possible and that is speeding up. I really, especially, in the first couple weeks, I want to have very clear and short deadlines
Value/client	The importance of serving the client (including informing, clarifying, educating client)	What’s happening here is that the client says, ‘I don’t know anything about games. I’m not sure that I need a game, but I want a game.’ ….They have to feed the client with every possible answer that they will have to do to find arguments for their assignment
Value/control	Importance of distributed expertise and assigned roles/jobs	The problem is every example in game design is that there’s really only one person who really knows what the game should be like. But to get it exactly like that, others have to make decisions that this designer cannot decide on. He doesn’t have the specialty, the specialized knowledge
Knowledge/design document-term of art	The design document is the record of the research and design decisions that team will use to develop game	A design document is really kind of an “I told you so” book. If it’s well done, the game designer goes to page 72 and says, ‘It’s right there, you should have read it.’ It’s his responsibility to keep it alive. If it’s alive, then it can change. If it’s alive you always see the whole thing. You can identify your part, but you can always see the whole thing
Knowledge/market	Understanding of context in which game will be played (where, by whom)	In terms of research into the particular subject it’s getting knowledge of their target audience…knowledge of the context in which the game is going to be running… knowledge of technical specifications and requirements
Knowledge/content domain	Understanding of world game will simulate	Well, they should get a lot of knowledge about the content. So, they should know all about fish and sustainability and climate change and all this
Knowledge/prototype-term of art	Prototypes are models built to test feasibility and usability of a design idea	I am happy that they are already producing prototypes that are, more or less, working. Because I know that often game development projects don’t have enough time in the end to do that
Knowledge/target user group-term of art	The target group or user is the intended type of person who will play the game	…if we want to reach to this particular target group. That means it must be an internet game…
Knowledge/client-toa	The client is the person for whom the game is being built	Did you ask the client?
Knowledge/patterns	Understanding of logic in the world that can be translated into a game simulation	You should kind of teach them to see what’s happening around them so that they can really have an insight into their own design process. And that their own design process is, includes the people they work with. It’s not only their head and their scrapbook…. it’s also about learning… to see things more and look at patterns more
Knowledge/game mechanic	Understanding of game mechanics, interactions,	They should be learning to… see what makes… quality in a game…. it’s not about… the most beautiful pictures, but it’s probably more about interaction
Knowledge/action list-toa	The action list is a document the team uses to organize their activity	… they have to also have a sort of longer-term planning…That’s actually what’s missing… the action-decision list on a higher level
Skill/translate	Ability to transfer language across audiences, ideas across media (transfer)	…. they will have to be able to translate this concept to a data set that they start programming with… you have to translate them in numbers if you talk to a programmer and measurable values. You have to translate them into situations if you talk to an artist. You have to translate them into monetized values if you talk to most clients
Skill/testing	Ability to perform usability tests	Testing skills. Like being able to formulate good research questions… It can be, you know, interaction mechanics, user group, psychological impact and so on. Being able to set up a testing lab, testing situation
Skill/research	Ability to research (content, market, psychology)	Students here, they always tend to think that any good idea is always coming from inside out. But I believe it’s a mix of things coming from your brain and also input from whatever inspiration you have. In this case, books …and reports, and interviews can help the concept development very much
Skill/manage social	Ability to manage social relationships	If you have no one who is taking care of the personal relationships, you get a really cold team that is not nice to be in
Skill/observations	Ability to pay attention to world	It’s not only their head and their scrapbook…. it’s also about learning to look around you, to see things more and look at patterns more…
Skill/strategy	Ability to create a work plan (task division, prioritization, identify tasks)	See, they have six weeks, and they have to come up with five mini-games and a good story. It all has to look nice and work good. So that means that some things are more important than other things. So they have to be able to prioritize these things
Skill/assessing resources	Ability to assess resources (time, person, constraints)	You have to fit…the people’s resources you have on your team to the concept you thought of
Skill/concept development	Ability to develop a concept (idea, direction, metaphor)	Often I see with these groups that they float around for a month or so, or even longer, in the sort of concept phase. They go from brainstorm to brainstorm. I hope that they see that if you go to more specific, more concrete actions quickly, that the actual creativity is much higher
Skill/prototype	Ability to turn concept into prototype	…if you only have one day to make a prototype and to make a concept, then you have to come up with shortcuts and fast ideas. Not only work until you have the best idea, but first idea and build it
Skill/group communication	Ability to communicate with group	Of course, the social skills, they learn about…. talking about worries that you have on a project as a student, or defending your opinion, or listening to other people, or negotiating stuff
Identity/programmer	Seeing self as a programmer	I would say the programmer’s role is very focused on getting it done…
Identity/artist	Seeing self as an artist/animator	They think they understand each other, because they all fought the bear in level 6. But the artist is thinking about the hairs in the neck of the bear
Identity/team- mate	Seeing self as part of a team	What I’m aiming at is that every team member in the end has so many strategies… [and] can face different people and different kinds of problems being constructive and creative and still being themselves
Identity/professional	Seeing self as a professional	I hope they seem themselves more as professionals, because it also helps them…coming over conflicts. It helps them to say, ‘…we’ve got a job to do’
Identity/game designer	Seeing self as a game designer	The game designer is the one who really has to make sure that any detour they make or everything they skip on the project, that it doesn’t hurt the initial power of the concept…

Appendix 3: Data analysis

Coded data is a series of excerpts of meeting discourse (stanzas) tagged with identifiers that associate those codes with the participant (mentor or team) and the meeting (1, 2, or 3). The following table shows the codes for the key frame elements for the mentor and team in the first meeting.

Each stanza is transformed into an adjacency matrix by counting the co-occurrence of codes within it. The final adjacency matrix of a meeting is the sum of all of the meeting’s stanza’s adjacency matrices. The following table is the final adjacency matrix for the team’s first meeting.

A subset adjacency matrix looks the same as the one above, but with only the frame elements of interest included.

Full adjacency matrices are used to calculate the relative centralities of the codes. The following table shows the relative centralities of each frame element for the team in the first meeting by stanza.

Since relative centrality is calculated cumulatively by stanza, the relative centrality of the last stanza for each meeting is used as the state of the frame for that meeting. The final stanza of each meeting is used to calculate the distance between the team and mentor’s meeting frames.

Equations for generating these matrices and measures, as well the equations for calculating the FSI statistic and an explanation of the process of creating epistemic frame trajectories, can be found in the “Methods” section. The 2-dimensional projection of the epistemic frame trajectories can be found in the “Results” section.