Keywords

Introduction

Wiki adoption has grown incredibly quickly over the last decade in primary, secondary, and tertiary institutions throughout the world. In a recent National Center for Education Statistics survey, 38% of public school teachers reported using blogs or wikis for class preparation and administration, and 21% of teachers reported requiring students to contribute to these online environments (Gray et al. 2010). As these platforms have grown in popularity, educational researchers have contested the utility and promise of wikis as collaborative learning environments, where students are immersed in communities of practice (Wenger 1998) engaged in knowledge-building practices (Scardamalia and Bereiter 2006). Much of this literature—design research, case studies, and theoretical development—has been derived from the investigation of individual classroom environments, even though one of the signature characteristics of wikis is that they can be published openly to the world.

To contribute to the debate over the viability of wikis in typical educational settings, I examined a large, representative, and diverse sample of wikis created in US K-12 classrooms. From analyzing these wikis, I developed a taxonomy of collaborative behaviors found on classroom wikis, and then measured the distribution of these collaborative behaviors in a sample of 406 wikis drawn from a population of nearly 200,000 wikis. I situate these findings in the context of the lack of collaboration in other peer-production environments used within and beyond education. One of the signature challenges for advocates of technology-mediated collaborative learning must be to confront the paucity of collaborative behaviors found in peer-production platforms used in typical educational settings.

Contested Views of the Possibilities of Wikis

Glassman and Kang (2011) argue for an optimistic view of wikis in the classroom, where wikis provide affordances that allow for the fulfillment of a progressive educational vision devised by John Dewey nearly a century ago. They argue that Dewey, Charles Peirce, and other pragmatists developed a system of logics known as abduction, as an alternative to the well-known logics of deduction and induction. Rather than following first principles or inferring directly from data, abduction calls for developing a series of hypotheses to be tested through scientific experiment. Glassman and Kang argue that Dewey’s pedagogical efforts were an attempt to infuse this “logic of discovery” into educational settings, and Dewey failed in his time, at least in part, because he lacked the technology to implement a “democratic classroom” where students have a meaningful voice in hypothesis generation. For Glassman and Kang, wikis provide a technological foundation that allows education to shift from the practice of transmitting known facts to the practice of generating and testing hypotheses. They argue that wikis allow Dewey’s vision to finally be realized: “Wiki technology may fit the promise of Web 2.0 in education more than any other technology. It fosters integrated problem solving, and advanced understanding of the fungible nature of information and cooperation” (lines 703–5). They describe this moment in history as a “cusp of a revolution in education” where new technologies allow a fundamental shift in learning practices.

In contrast, Dohn (2009) argues that the attractive affordances of wikis are systematically undermined by the educational imperative to grade and rank students individually. Dohn further argues that Web 2.0 activities—which involve sharing, co-constructing, and publishing for a wider audience—operate under a participationist metaphor that appears to be fundamentally at odds with the accumulationist metaphor of schooling, where individual students are responsible for their own preparation and teachers assess and sort students as individuals. Dohn shares the view with Glassman and Kang that a pedagogy capitalizing on the affordances of wikis would be novel, powerful, and well-suited to an increasingly networked and interconnected world; however, Dohn argues that such a pedagogy is nearly impossible to implement in schools. Institutional imperatives for measuring individual achievement are one major hurdle, and even where these can be institutionally modified or neutralized, students still come to school acculturated to the norms of an environment that only measures individual competencies.

Glassman and Kang; Dohn can be positioned as two poles of thought on the pedagogical possibilities of wiki-integration. One emphasizes the transformative potential of wikis and the other emphasizes how institutional constraints of formal schooling restrict (perhaps necessarily) the capacity to use wikis in transformative, or even meaningfully collaborative, ways. Other case studies can then be located on this spectrum.

For instance, in the article “I DON’T CARE DO UR OWN PAGE,” Grant (2009) examined the use of wikis in a grade 9 classroom in the UK. The class used a wiki platform to create collaborative presentations about the history of modern technologies. Grant found that classroom norms around the individual ownership of text powerfully constrained the collaborative potential of wikis. Since students were used to controlling their individual contributions and to being evaluated on these contributions, very few students approached the wiki project collaboratively and those that did were rebuffed, sometimes harshly. Even efforts at commenting on other students work tended to be no more than perfunctory. Ultimately, students used a divide and conquer approach, where each student individually produced discrete content which was then assembled into a “shared” product.

Forte and Bruckman (2009) offered a somewhat more promising case study from an AP science classroom. They also found that “institutional assessment regimes for both students and teachers inhibited collaboration” (p. 23) and that students used wikis primarily as sites of individual production. In the course of struggling to produce ScienceOnline, a wiki-based website for publishing science reports and articles, students developed a richer understanding of the “genre” of wiki writing. Participants became more aware of the norms of standard, individually produced school writing, and the norms of collaborative writing within peer-production systems. In other words, even if students could not overcome hurdles to collaboration or successfully produce work within the genre of collaboratively produced writing, they learned from the process of becoming aware of these norms and barriers.

In a third case, Pifaare and Staarman (2011) described a highly structured wiki project in an elementary classroom, where students wrote an opinion piece about the feasibility of developing a colony on Mars. Students first worked in pairs to develop initial positions. They then formed groups of six students and they used a wiki platform to negotiate the synthesis of all three positions and develop a final product. Pifaare and Staarman argued that students in the project actively discussed each other’s positions and then jointly created a final product, and in doing so the students developed digital competencies needed for collaborative knowledge creation. As presented, the case appears to warrant some of Glassman and Kang’s pedagogical optimism.

In all five wiki case studies cited above, broad agreement exists over the theoretical potential of wikis to support collaborative learning. All five articles argue that wikis provide a technologically robust platform for co-constructing knowledge, for developing collaboration skills, and for nurturing dialogue about the contested nature of knowledge. All five articles discuss the theory of knowledge-building from various papers by Scardamalia and Bereiter (2006), as a useful framework for examining the social constructivist practices enabled by wikis and attempted in their own case studies. Four of the five articles cite theories of situated learning (Lave and Wenger 1991) or communities of practice (Wenger 1998) as additional lenses through which wikis can be seen as technological enablers of established pedagogical frameworks of social learning. Broadly speaking, these articles share a vision of how wikis might help prepare students for a globally networked world, and the locus of debate—as highlighted by Dohn versus Glassman and Kang—is whether this vision can be realized in the structures of actual classrooms around the world. The three subsequent case studies represent various points along a spectrum of success in nurturing collaborative learning with wikis, and thus they provide evidence that might support both the optimist’s and the pessimist’s position on wikis. (I have chosen three case studies drawn from primary and secondary educational settings, and similar examples from higher education supporting the optimists’ (Wichman and Rummel 2013) or pessimists’ (Cole 2009) view on the potential of wikis can be found as well.)

One approach to testing the merits of the optimists’ and pessimists’ perspective toward wikis would be to continue to accumulate these kinds of case studies and attempt a meta-analysis or synthesis of these cases. Such an approach, however, has several important limitations. In the five cases cited above, the classrooms being studied were taught or assisted by a wiki researcher, and/or used a special researcher-designed curriculum, and/or used a special build of wiki software, such as a MediaWiki installation with a series of modifications. It is not clear how well findings from these “hothouses” generalize to the experiences of wiki-using classroom teachers without such supports. In addition, the cases are from classrooms in different countries, in different subject areas, serving different grades of students, from different cultural backgrounds. Such diversity provides a richness of examples to spark provocative thinking, but it presents challenges for systematic comparison. Thus, this debate can benefit from research employing methods that allow for evaluating patterns of activity across large, diverse, and representative samples of typical learning environments. Close examination of particular learning environments should be complemented by studies that can provide a wider view of the educational landscape.

Examining Wiki Practices at Scale

Part of the Glassman and Kang, and Dohn debate hinges upon the impact of the hundreds of thousands, probably millions, of wikis that have been created for use in classroom settings. Is the wiki-inspired “revolution in education” underway or do these thousands of new learning environments show little sign of nurturing Dewey-inspired forms of collaborative learning? One way to contribute to this debate is to address research questions about how wikis are used in typical settings. What kinds of collaborative behaviors do students perform when using wiki learning environments? To what extent do students perform these different collaborative behaviors when using wiki learning environments?

In a sense, the Glassman and Kang, and Dohn debate is about what we should expect to happen when wikis are used in typical classroom settings. Their theories postulate competing “expected distributions” of collaborative activity in classroom wikis. The data warehouses of public online learning environments create tremendous new opportunities to give researchers insight into those settings. They allow researchers to estimate actual distributions of collaborative behavior in the population of classroom wikis, which can be used to test competing theories.

In this study, I examined wikis produced on the PBworks platform, one of the three largest hosts for free wikis. Hundreds of thousands of PBworks education-related wikis are publicly accessible on the Web. For each of these wikis, researchers are able to access the complete edit history, maintained continuously to the second, of every word and tag added, changed or deleted by teachers and students in these environments. These data provide both tremendous scale, with the many thousands of cases, and rich historical depth, with their real-time edit histories.

I drew a random sample of wikis from a population of nearly 180,000 publicly viewable, education-related, and then research assistants under my direction used the Wiki Quality Instrument (Reich 2012a, b) to measure opportunities that each wiki provided to develop twenty-first-century skills such as collaboration, expert thinking, or new media literacy. While coding wikis, research assistants used a special browser interface—called the Wiki Coding Tool—that computationally generates basic information about each wiki, facilitates the rapid viewing of a sequence of edits to a single page, and allows researchers to restrict their viewing to only certain time periods (such as the first seven days of a wiki’s lifecycle). Using these strategies, I identified a taxonomy of defined collaborative behaviors, measured the presence of those behaviors on a sample of wikis, and estimated the distribution of those behaviors in the population of US K-12 wikis. In this paper, I then use those empirical findings to contribute to the Glassman and Kang, and Dohn debate about the potential of wikis in educational settings.

This approach complements case studies and design research in several important ways. First, by randomly sampling from a large population of wikis, I examine typical settings rather than “hothouses” with special instructors, curriculum, or resources. While special settings can be very helpful in mapping out the possibility space for a learning theory or a technology, exclusively studying special environments can result in the literature providing a biased impression of the implications or implementation of tools, practices, or theories. Relatedly, random sampling allows for the study of wikis that are rich, complex, sustained, and successful, as well as those that are incomplete, useless, or a failure. In previous studies, I showed that nearly one-third of wikis do not persist past 24 h (Reich et al. 2012). Because of their short lifetime, these wikis are very hard to investigate through qualitative methods (since they exist too briefly to be observed). Nonetheless, studying these short-lived experiments is very important to developing a complete understanding of wiki usage in schools. Finally, random sampling from a large population of wikis allows me to make claims about the degree of collaboration in wikis across the US K-12 system.

Research Questions

To summarize my argument to this point: Glassman and Kang, and Dohn provide two competing perspectives about the potential of wikis to support rich learning experiences. Glassman and Kang argue that wikis allow the instantiation of Dewey’s pedagogical vision, and Dohn argues that the individual logics of assessment in school prevents the theoretical affordances of wikis from actually being realized in classroom settings. The scholarly literature on classroom application of wikis provides a variety of case studies that can contribute to this debate and to theory-building around peer-production tools in education, but researchers need to be careful about drawing conclusions exclusively from research conducted in special “hot-house” environments.

This study attempts to advance our understanding of classroom wikis specifically, and peer-production environments in education more broadly, by conducting a large-scale content analysis of wikis used in typical settings. In this study, I posed two research questions: (1) What kinds of collaborative behaviors do students perform when using wiki learning environments? (2) To what extent do students, in US K-12 schools, perform these collaborative behaviors when participating in wiki learning environment?

Research Design

In this section, I describe the dataset, sample, instrument, and data-analytic strategies used to evaluate the distribution of collaborative behaviors in US K-12 wikis. This study was part of a larger research program, the Distributed Collaborative Learning Communities project, to comprehensively examine the use of wikis in US K-12 settings, and more detailed descriptions of these methods have been published (Reich 2012a, b). Below, I describe the relevant research design information for this study of student collaborative behavior.

Dataset

PBworks.com is a wiki-hosting service that allows educators and students to set up free wiki workspaces and it ranks among the top four most-visited sites providing free wikis (Alexa 2010). From this company, I obtained longitudinal usage data on all 179,581 publicly viewable, education-related wikis that had been created between the founding of the company in June 2005 through August 2008. These were wikis whose creators designated them as for “Education” during the creation process, as opposed to “Business” or “Personal.”

Each of these 179,581 wikis represented a discrete subdomain on PBworks. The unit of analysis in my study was the wiki subdomain. Hereafter, when I refer to a “wiki” in my dataset, I refer to a publicly viewable, education-related wiki subdomain hosted by PBworks. I had both a set of usage statistics on each of these 179,581 wikis and the capacity to examine closely the content of each wiki. I could examine the present state of any wiki, and I could also access every version of every page ever saved during the lifetime of the wiki. In this study, I worked with the entire population of publicly viewable, education-related PBworks wikis, without restricting my population based on the number of edits, the number of days of activity, or other similar criteria. This preserved my capacity to compare the full range of wiki learning communities.

Sample

For the analyses presented here, I drew a 1% random sample of 1799 wikis from the population of 179,581 education-related wikis made available to me by PBworks. From the 1% sample of 1799 wikis, I identified 406 wikis created in US K-12 schools for further study. I disqualified 18 wikis that were set to be privately viewable (removed from public view) during the observational period, 502 wikis that were either deleted or never changed (which unfortunately are collapsed in one category—I believe that the vast majority of wikis in this category were never changed), 448 wikis that were used exclusively outside the United States, and 425 US wikis that were not identifiable from K-12 settings (most of which were from higher education).

These 406 US K-12 wikis were used in different institutional contexts. Within the sample, 322 wikis were created within the US public school system. Of the rest, 43 were created in independent schools or home-schooling organizations serving K-12 students, two were created in public libraries, three in university settings serving K-12 students (e.g. a summer school) and 36 were from sources serving K-12 students but with unclear institutional affiliations. Wikis were used throughout the grades levels, with 27% serving elementary school students, 29% serving middle school students, and 44% serving high school students. We found wikis used throughout subject areas including language arts, science, mathematics, computer science, social studies, and other subjects. These wikis were used for a very diverse range of pedagogical purposes: teachers posted syllabi, assignments, class newsletters, and course content; students posted papers, introduced themselves, described hobbies, planned presentations, curated portfolios, engaged in academic discussions, and wrote stories. As noted in previous research, teacher activity dominates most wikis. Only 26% of wikis have any student involvement at all; 38% of US, K-12 wikis are “trial balloons” and cease development almost immediately and 34% are platforms for teacher-centered content delivery (Reich et al. 2012).

Sample Limitations

There are two important limitations of this sample. First, we have access only to PBworks wikis, raising questions as to whether PBworks wikis are representative of freely available wikis hosted by other providers. The only major comparable alternative host of free online wikis for education is Wikispaces.com. PBworks and Wikispaces trade places from week to week as ranked 3 and 4 on the Alexa rankings site for wiki-hosting services (Wikia and WetPaint, which do not have a significant share of educational wikis, ranks 1 and 2 typically).

There is one structural feature of PBworks that substantially differs from Wikispaces, MediaWiki, and most other wiki-hosting solutions. At the bottom of each PBworks content page, there is a space for comments and discussion. This is instead of the “Discussion” pages that are paired with content pages in Wikispaces or MediaWiki. This may influence the distribution of commenting and discussion behaviors on PBworks wikis, though a systematic comparison with Wikispaces or MediaWiki wikis would be necessary to determine whether or not this is the case.

The second limitation of my sample is that it does not include privately viewable wikis. Publicly viewable wikis represent 70% of the wikis created on PBworks from 2005 to 2008, so even if my findings are only generalizable to the population of public wikis, they are generalizable to the majority of wikis. It is not clear whether one should expect privately viewable wikis to be used differently. While many might assume that most wikis with student activity would be kept private, there was extensive evidence of publicly viewable student activity in my analytic sample.

Instrument

To measure the collaborative behaviors of students in wiki learning environments, I used the Complex Communication subscale of the Wiki Quality Instrument (WQI). The WQI is an instrument designed to assess the opportunities that wikis provide for the development of twenty-first-century skills such as expert thinking, complex communication, and new media literacy (Reich 2012b). The complex communication subscale operationalizes a taxonomy of collaborative discursive moves made by students participating in wiki learning environments.

With a team of colleagues, I developed and piloted the WQI over an 18-month period using a rigorous design process. A full description of the development of the WQI and associated protocols is available online (Reich 2012a, b), and below I describe key features of the development of the instrument.

Instrument Development

I used three research strategies to develop the taxonomy of collaborative behaviors in the Wiki Quality Instrument scale. First, I conducted a thorough literature review to assess whether existing measures of online collaborative behavior might be available. Second, I conducted multiple rounds of preliminary content analysis on our wiki samples to identify patterns of collaborative practice on wikis, then to test a series of preliminary measures of collaboration, and then to iteratively refine these measures. Third, I triangulated this content analysis with descriptions of student collaborative behaviors from wiki-using students and teachers in interviews, focus groups, and classroom observations.

In developing the taxonomy of collaborative behaviors on wikis, I conducted an extensive literature review to investigate how other researchers and scholars had approached the evaluation of Web 2.0 learning environments. Research into Web 2.0 learning environments—wikis, blogs, discussion forums, proprietary environments, and other platforms—has primarily been conducted through design research experiments and qualitative case studies. Most studies examine one or a few classes of students, often in courses taught by the researchers. Often these studies were conducted within a single subject domain, such as algebra (Chiu 2008), business ethics (Jeong 2003), or American history (Lawrence 2009). One result of this pattern is that existing measures of collaboration developed in the literature tend to be particular to a certain subject, classroom, or even a specific assignment. For instance, Trentin (2009) developed measures of individual contributions to a wiki project by looking at four sites of participation in particular wiki: (1) in the forum used for the planning stage, (2) in the peer review, (3) in the development of the wiki’s reticularity; (4) and in the development of its contents. These kinds of specific indicators provide examples of venues for collaboration in a particular wiki learning environment, but they would not be adequate for mapping the full range of collaborative activity that we found in my diverse sample of K-12 wikis. Since the measurement strategies that I found in the literature had similar levels of specificity, I developed a taxonomy of collaborative behaviors that could be applied to a much wider array of wiki learning environments.

I developed this taxonomy of collaboration through a series of content analysis exercises guided by a grounded theory (Charmaz 2006) approach. Research assistants under my direction coded wikis in multiple iterations guided by a series of increasingly refined questions. In the first round of coding (used to separate the US K-12 wikis out of the full sample of 1799 publicly viewable, education-related wikis), researchers evaluated each wiki and identified its purpose and the kinds of activities found on the wikis. Coders were given no instructions or definitions for “purpose” or “activity”, although they were asked to attempt to use consistent language in describing similar wikis. From these codes, I developed a sense of how wikis were used, by whom, and to what ends. In a second round of coding on the set of 406 US K-12 wikis, I directed research assistants to evaluate “patterns of practice,” routine moves made by teachers and students, that seemed likely to promote twenty-first-century skill development, including collaboration and complex communication. I then attempted to winnow down these descriptions of patterns of practice into a formal taxonomy of collaborative behaviors.

In parallel with these wiki coding activities, I also conducted an extensive program of qualitative research. My team conducted 68 teacher interviews, observed 19 classrooms in 6 states, and held 40 focus group sessions with students from these 19 classrooms. In these qualitative research activities, I sought to evaluate how teachers and students used wikis, how they defined high-quality work with wikis, and how they assessed quality in wiki learning environments. The interview transcripts and field notes were coded multiple times, first through a round of open coding, and then through a more focused examination looking for teacher and student descriptions of students’ collaboration and communication practices. I triangulated these data sources with data from our content analysis to develop a taxonomy of collaborative behaviors in US K-12 wikis.

Taxonomy of Collaborative Behaviors: The Complex Communication Subscale of the Wiki Quality Instrument

At the completion of this design process, I developed a taxonomy of seven common collaborative student behaviors in wiki learning environments, which I summarize in Table 11.1, and I describe in greater detail in section “Findings”. The Complex Communication subscale of the Wiki Quality instrument, measures the presence or absence of these seven moves on a wiki at a particular occasion of measurement, as described below in section “Procedures”.

Table 11.1 Taxonomy of collaborative student behaviors that comprises the coding categories in the complex communication subscale of the Wiki quality instrument
Full size table

Procedure

The wikis in my sample are extremely diverse. They are used with elementary schools through high schools, in nearly every subject area imaginable, and for a wide variety of educational purposes. They range in size and complexity from a single page with no revisions to wikis with hundreds of pages revised thousands of times. Accurately characterizing the activity on wikis is very challenging work. In this section, I present strategies to meet these challenges.

To measure collaborative activities on my representative sample of US K-12 wikis, research assistants used the Wiki Coding Tool (Reich 2012b). This tool is a Web interface that draws on the PBworks’ data warehouse and permits a coder to examine the appearance of a PBworks wiki at any particular day in the wiki’s development. Because the entire historical record of every edit to every page of every wiki is stored by PBworks, our Wiki Coding Tool is a “time machine” for assessing wiki usage. The wiki coding tool also allows raters to rapidly scroll between page revisions and examine differences between two revisions, allowing much more efficient evaluation of collaboration moves than allowed by the native PBworks interface. The wiki coding tool provides the “distillation for human judgment” in Baker’s (2011) taxonomy of educational data mining methods.

Each wiki was coded by two raters at six occasions of measurement, on 7, 14, 30, 60, 100, and 400 days after the wiki’s creation. (These occasions were determined by quantitative analysis of wiki edit histories see (Reich 2012a).) Each wiki was coded as long as the wiki continued to change. Thus if a wiki’s final change was on day 25, it was coded on days 7, 14, and 30, but not further. On each occasion of measurement, the two raters evaluated every revision to every page, all page comments, and all documents uploaded to the wiki up to that time period. On small wikis with only one page, this might take only a few minutes. On the largest, most complex wikis on their 400th day, this process can take several hours. For every item on which the two raters disagreed, a third rater reconciled the disagreement. As I explain later, in this paper I do not present my findings in a longitudinal framework, but understanding these multiple measures is important for evaluating our inter-rater reliability.

One limitation of our procedure is worth highlighting. In evaluating students’ collaborative behaviors, research assistants were dependent upon students logging in with their own user ID or leaving bylines associated with their contributions (e.g. “Irish History, by Jane McDonnell”). In many cases, students adhered to these norms, and researchers were able to measure collaborative activity with precision. I know from classroom observations, however, that sometimes students do not log in under a unique ID and sometimes multiple students work on a page while logged in under one ID, while sitting next to each other and sharing a computer in a school lab. Research assistants could not credit collaborative activity that they could not identify affirmatively. Therefore, it is possible that I have underestimated collaborative activity within my sample of wikis. This was a topic of discussion on several occasions in our weekly team meetings, and the consensus of my team was that over the hundreds of wiki we evaluated, raters felt that there were few occasions where they believed they might be under-representing collaboration because of ambiguities in user identity (usually because wikis appeared to be unambiguously created by one person). In one sense, this issue is resolved by clearly defining the collaborative behaviors in my taxonomy as those that observably occur within the wiki environment. The instrument and my procedures did not measure dimensions of collaboration happening within classrooms and computer labs, dimensions which are certainly important and worthy of anthropological study.

In terms of quality-code agreement, our research team coded 406 US K-12 wikis at 1219 time points, an average of 3 occasions of measurement for each wiki. Average inter-rater agreement across all five subscales of the Wiki Quality Instrument at all occasions of measurement was 0.92 and inter-rater agreement on the Complex Communication subscale was 0.96 (Reich 2012a).

Measures

Though my data gathering procedures were sophisticated, the presentation of my measures in this paper is simple. Although my team measured wiki quality at multiple time points across each wiki’s lifecycle, two features of our data rendered it unnecessary to present my findings with such precision. First, collaborative activities occurred very infrequently in our sample of US K-12 wikis. Second, in most wikis, if collaborative behaviors occurred they were evident within the first few weeks of the wiki lifetime, so longitudinal representations of collaborative activity were not substantially more informative that simpler representations. As a result, I simply chose to present measures of student collaborative behavior as we found them on the last date a wiki was changed or on our final occasion of measurement at 400 days after wiki creation.

Thus, in this chapter, I report the proportion of wikis in our samples containing each of the seven discursive moves included in the taxonomy of collaborative behaviors, as measured after 400 days of observation. I also report the Complex Communication subscale scores for each wiki—a scale ranging from 0 to 7 representing the number of collaborative student behaviors identified on the wiki—as measured after 400 days of observation.

Data-Analytic Strategy

To address my first research question—How do students collaborate in wiki learning environments?—I present a detailed description of the seven discursive moves identified in my taxonomy of student collaborative behaviors.

To address my second research question—To what extent do students collaborate in wiki learning environments?—I present frequency counts of each of the seven collaborative moves in my representative sample of 406 US K-12 wikis as measured after 400 days of observation. I also present the distribution of complex communication subscale scores—the sum of these seven collaborative moves, in my wiki sample. Together, these descriptive statistics provide a landscape view of how students in US K-12 settings collaborate, or not, in wiki learning environments.

Findings

How Do Students Collaborate in Wiki Learning Environments? A Taxonomy of Collaborative Behaviors on Wikis

From the instrument development procedures described above, I identified seven different types of collaborative moves that students performed within my sample of US K-12 wikis (summarized in Table 11.1). In this section, I define each of these seven behaviors and present the proportion of wikis exhibiting each behavior.

Concatenation occurs when students post discrete content to a single page. For instance, if a team of four students was assigned to create a wiki page about trees, one student would individually write a paragraph on leaves, another on branches, another on the trunk, and the fourth on roots. They would then each add their discrete paragraphs to a common page. Concatenation might be thought of as the lowest possible level of collaborative page construction: it represents merely posting in the same place. It is the second most common form of collaborative behavior, occurring on 5.91% of US K-12 wikis in my sample.

I identified two other forms of co-writing. Copyediting is when students edit the grammar, spelling, or syntax of another student’s contribution to a wiki page but do not make substantive changes. Copyediting occurs in 1.72% of wikis in my sample. Co-construction is where students substantively edit another student’s contribution to a wiki page. Figure 11.1 shows a screenshot from a wiki where students co-construct a paragraph, and, conveniently for researchers, highlight each individual contribution in a different color. This behavior occurred on 4.43% of sampled wikis. It is important to note that thresholds for identifying the presence of these behaviors were quite low. While Fig. 11.1 shows an unusually extensive example of co-construction between multiple students, if one student contributed one phrase in the middle of another student’s paragraph, that would also be considered evidence of co-construction. Even though our findings show very low levels of collaborative behaviors, our measures were very inclusive of even trivial forms of collaboration.

Fig. 11.1
A screenshot has the title guided research questions slash prompts. It presents a question and responses in a color-coded format.

Screenshot from wiki demonstrating co-construction. Contributions from different students are rendered in different colors

Full size image

I identified two forms of student dialogue on wikis. Commenting is when students comment upon the work of another wiki user, student or teacher. The two most common forms of commenting behavior involved students responding to a prompt posted by a teacher and students leaving a comment evaluating the posted work of another student. In the PBworks environment, there is a defined space for comments at the bottom of every wiki page, although we also measured comments that were left in the body of a wiki page. Commenting is the most common form of collaboration, and it occurs on 6.4% of wikis in my sample. Discussion occurs when comments have at least four conversational turns among a group of commenters. This occurs on 2.46% of sampled wikis. To clarify the distinction between commenting and discussion: if a teacher poses a question on a wiki page and students respond directly to the question on the same page, this would be an example of commenting. If within those responses, students respond to each other’s contributions, with at least four conversational turns in the process, then this would be discussion. If each student simply responds to the original prompt, then this would be commenting.

Finally, I identified two additional collaborative behaviors related to using the wiki as part of a work process. Scheduling occurs when students populate a list or calendar. For instance, a student might post a list of times that group members can meet, and the group members put their names next to potential times. Or a teacher might create a list of roles for a group project (such as editor, leader, proofreader, and so forth), and students sign up for those responsibilities. Scheduling is present in 1.72% of wikis in my sample.

Planning is the final collaborative move. The vast majority of student discursive moves on wikis involve creating some piece of content meant for publication and presentation. It made sense, therefore, to create a category in the taxonomy to identify moves where students were not creating content but instead planning to do something. Planning occurs when students use the wiki to develop plans with other students for creating work products (on the wiki or elsewhere). Planning occurs on 2.46% of wikis in my sample.

I considered including two additional categories related to collaborating across institutional boundaries: beyond school collaboration and international collaboration. Beyond school collaboration would be when students from two or more schools use the same wiki, and international collaboration would be when those schools are in different countries. We did not find evidence of these behaviors in our sample, so we did not include them in our taxonomy, although several well-known exemplary wiki projects, such as the Flat Classroom Project (http://flatclassroomproject.org), exhibit these behaviors (Lindsay and Davis 2012).

This taxonomy of collaborative behaviors defines and categorizes the most common discursive moves found in US K-12 wikis. This taxonomy provides a structure for systematically analyzing the distribution of collaborative behaviors in these wikis.

To What Extent Do Students Collaborate While Using US K-12 Wikis?

Student collaboration is rare in US K-12 wikis. In Table 11.2, I show the distribution of student collaborative behaviors in our sample of 406, US K-12 wikis, as measured after 400 days of observation. Notice that the most common forms of collaboration—concatenation and commenting—occur in only 5.9% and 6.4% of wikis, respectively. These most common collaborative moves are also the simplest. Indeed, these forms of “collaboration” do not require any real interaction between students at all, but instead represent individual contributions that co-occur in the same virtual space.

Table 11.2 Distribution of collaborative behaviors in 406 US K-12 wikis, as measured after 400 days of observations
Full size table

In analyzing these percentages, it is also important to remember that the decision rules defining each of these seven behaviors were designed to be broadly inclusive of a wide variety of behaviors. If a student posts “This is stupid” or “Good job” at the bottom of a page, even these trivial moves would be counted as comments. If a student corrects a single misspelling or adds a single comma, that wiki would be designated as including “Copyediting” behavior. So even with our broadly inclusive measures, student collaboration on wikis is infrequent.

Another way to evaluate the distribution of collaborative behaviors is to count the number of identified behaviors occurring on each wiki. In Table 11.3, I show distribution of the number of collaborative behaviors found in my sample of 406, US K-12 wikis as measured after 400 days of observation. The striking finding from this table is that 89% of wikis have no identifiable student collaboration at all. Another 6% of wikis have only one identifiable form of collaboration, and 5% of wikis have between two and seven collaborative behaviors. If Table 11.2 shows that each individual collaborative behavior is rare, then Table 11.3 demonstrates that wikis that include multiple forms of collaboration are also quite rare, even given our low standards for what can be considered collaborative behaviors.

Table 11.3 Distribution of complex communication subdomain scores in 406 US K-12 wikis, as measured after 400 days of observation
Full size table

Discussion

With these empirical findings concerning the distribution of collaborative behaviors in US K-12 wikis, I can now return to a discussion of the Glassman and Kang (2011) and Dohn (2009) debate about the promise and potential of wikis. The evidence presented in this paper is much stronger warrant for Dohn’s pessimistic position than Glassman and Kang’s more hopeful view of the potential of wikis. On the whole, most wikis are individual constructions, and when students do engage in collaborative behaviors, students tend to be just posting in the same place rather than participating in intensively collaborative knowledge-building.

The patterns of collaborative behaviors presented above also cohere with Dohn’s arguments about the power of institutional conditions and conditioning to foster a strong sense of individual ownership of text. Consider the three forms of cooperative writing that I define: concatenation, copyediting, and co-construction. On first glance, it appears that co-construction is a more sophisticated form of collaboration than copyediting; fixing commas is less challenging than fixing meaning. One might hypothesize, therefore, that copyediting would be more frequent than co-construction since modest changes to grammar, syntax, and spelling are simpler than more substantive co-writing.

It turns out, however, that copyediting is one of the least frequent collaborative practices. My hypothesis is that while copyediting is technically simpler, it involves the deletion of someone else’s text, and therefore is actually perceived by students as more invasive than co-construction, which can be accomplished by threading additional phrases and sentences onto the writings of others without ever deleting anything. My quantitative findings align with the anthropological perspective on student’s strong feelings about individual ownership of text presented in Grant’s (2009) “I DON’T CARE DO UR OWN PAGE.” Both of these studies lend different forms of evidence to support Dohn’s position that the structural and cultural features of schooling are inhospitable to peer-production practices in institutionalized school settings.

My findings do not necessarily contradict Glassman and Kang’s position on the theoretical possibilities of wikis; I found a small handful of highly collaborative wikis within my representative sample. To some extent, Glassman and Kang are making an argument about the possibility space for Dewey-inspired education enabled by wikis. It could be argued, therefore, that this study shows that collaborative wiki learning communities on the progressive edges of that possibility space can be found in representative samples of typical wikis drawn from huge populations.

My reading of Glassman and Kang, however, is that they argue that knowledge-building, content co-creation, and communities of investigation are not merely made theoretically possible by wikis, but that educators should understand wikis as places where these advanced learning behaviors can emerge with some regularity, indeed, enough regularity to inspire a revolution. The evidence presented here suggests that these particular arguments should be tempered with the caveat that, in practice, most wikis are individually produced platforms for content delivery, more often created by teachers than by students.

As with all social science debates, this one study certainly does not “settle” any issue, and of course the revolution predicted by Glassman and Kang could indeed be right around the corner. I am currently conducting several follow up studies on samples of wikis collected in 2010 to evaluate whether collaborative patterns have changed over time. If Dohn is correct, then the fact that teachers will have several more years of experience with wikis and peer-production tools will prove to be, in the main, irrelevant for the advancement of richer collaborative practices, since the institutional and culture barriers to social practice and knowledge-building remain unchanged. If I find growth in the distribution of collaborative practices, especially the more intensive forms of collaboration, then the Glassman and Kang, and Dohn debate will need to be reexamined through these new data.

Situating Case Studies in the Landscape of Wiki Practices

In the absence of broad, generalized, contextual data about practices with emerging technology and pedagogy, it can be very difficult to ascertain how a particular learning environment compares to other learning environments employing similar tools or design principles. For instance, the signature event in Grant’s (2009) “I DON’T CARE DO UR OWN PAGE” is when a student logs in to edit another student’s wiki page and makes a few modest changes, and she is strongly rebuked by the original creator of the page for attempting to make edits. Grant presents the case study as an example of the failure of a new technology to foster the development of new collaborative practices. In reference to the standards for collaborative behaviors explicitly set by the teacher and implied by the researcher, the project fails.

A somewhat different picture emerges by comparing the practices identified in Grant’s (2009) case study to the taxonomy of collaborative behaviors presented in this study. In the article, Grant shows clear evidence of at least three of the seven collaborative moves from the taxonomy—concatenation, copyediting, and commenting—occurring on the wiki being studied. The article text suggests there may have been co-construction as well. If this wiki does in fact have evidence of four types of collaborative moves by students, then it has more collaborative moves than 96% of US K-12 wikis. Seen from this perspective, it is perhaps more remarkable that students were even willing to attempt invasive collaborative moves than the fact that those moves were so sharply rebuked. From its own internal frame of reference, Grant’s case study appears to be a story of failing to meet expectations. With a larger perspective, it might be possible to revisit Grant’s case to identify what factors enabled the relatively high (if ultimately disappointing) levels of collaboration that the learning environment did manage to foster.

A similar kind of reframing might occur in reference to the two other case studies mentioned early, by Forte and Bruckman (2009) and Pifarre and Staarman (2011). Set against the backdrop of the distribution of student collaborative behaviors in US K-12 schools, it becomes clearer that these particular cases are examples involving very high degrees of student collaboration and they are examples of very atypical situations. Within this frame of reference, Forte and Bruckman’s case might be positioned, like Grant’s case, as less a story about the barriers to collaborative practice that the students and educators failed to overcome, and more about the remarkable degree to which they fostered any collaborative activity at all. Pifarre and Staarman’s case is one of the few design research studies presented in the literature as a more or less unqualified success, and the efficacy of their example appears much more remarkable in comparison to the very low levels of collaboration found in US K-12 wikis broadly.

The data presented in this study allow readers and authors of case studies to situate these individual classroom cases within a larger landscape of wiki usage. In particular, these findings help highlight the atypical nature of those wiki learning environments that foster any degree of student collaboration. From this perspective, it becomes apparent that the research literature on classroom wikis consists almost exclusively of cases at the tail of the distribution of collaborative behaviors, potentially biasing conclusions about classroom wikis drawn from this literature. Case studies that explore the reasons why teachers use wikis as sites for individual production may be as useful for understanding how to advance teacher practice as those that examine more collaborative approaches.

Beyond Wikis, the Challenges of Peer Production in Education

Wikis provide an example of a platform designed for peer production, though in K-12 education settings they are primarily used for individual production. An emerging set of studies of other peer-production platforms within and beyond education suggests that this pattern is typical of peer-production platforms. Acknowledging this widespread pattern of individual production in collaborative environments requires raising difficult questions about designing learning experiences using these platforms.

Scratch (Resnick et al. 2009) is an online platform (http://scratch.mit.edu) where students can program games, simulations, and animations using an open-source visual programming language. Any participant can build their own project, but they can also edit and remix any project shared within the Scratch community. Hill and Monroy-Hernandez (2013) studied all 536,245 projects created in a one year period in 2010 on the Scratch platform. They found that only 7% of these projects were remixed within one year of their creation, and only a tiny percentage of Scratch projects were ever remixed after one year. The vast majority of Scratch projects are individual productions. As with wikis, Scratch makes collaboration possible, but most Scratchers choose to work alone.

Hunt (2011) analyzed the activity from a district installation of a blogging platform, and he found similar patterns of production. First, as with wikis, blogs were primarily tools for teacher communication. Even though students outnumber staff five to one in the district, teacher posts on blogs outnumber student posts two to one. Moreover, blogs were primarily individual productions: 80% of posts had no comments at all. Among the 20% of posts with comments, most comments were mandated responses to teacher questions without any real discussion (“comments” instead of “discussion” in the parlance of my taxonomy of collaborative behaviors), evaluation of student work by teachers, or trivial responses by students to other students (e.g. “good job.”). As Hunt writes, “If the purpose of the blogging engine was and is to provide students and staff access to each other for the purposes of being in conversation about teaching and learning, then the blog engine is a gross failure” (p. 48).

As an example from outside the education space, Sourceforge is an extensively researched platform for the development of open-source software projects with a robust infrastructure for collaboration. As with educational wikis, most Sourceforge projects have been created and developed by individuals. Data show that 70% of Sourceforge projects have only one developer, 14% of projects have two developers, and only 2% of projects have more than ten developers. (Hill 2011; Schweik and English 2007). Leveraging collaborative platforms for individual production, therefore, does not seem to be limited to educational environments.

K-12 wikis, blogs, Scratch projects, and Sourceforge projects all display a Pareto distribution of collaborative behavior, where most projects on peer-production platforms are individually produced and a small fraction are actually collaborative. One simple design principle that I adduce from analyzing these patterns is that the introduction of a new peer-production platform into a particular learning environment or into the ecology of schools more broadly is very unlikely to, in and of itself, bring about more collaborative behaviors. That point is perhaps obvious, but it is still worth emphasizing.

However, the claim made by Dohn (2009), and to some extent Grant (2009) and Forte and Bruckman (2009), is that what curtails that development of collaborative practices in educational settings is the logics and culture of individual assessment in schools. One possible line of argument following from that premise is that educators interested in supporting collaborative learning need to change the organization and cultures of schools and classrooms to make collaborative learning viable. The comparison with Sourceforge, however, suggests that barriers to collaboration in peer-production environments may not be particular to educational institutions. If this is the case, mitigating barriers to collaboration inherent in the design of schools may not be sufficient to ameliorate the barriers to collaboration that appear to exist more broadly in wider society. Getting people to collaborate online may be hard even without including the special challenges faced in educational environments.

If it is indeed the case that most peer-production platforms primarily support individual rather than collaborative activity, then this raises serious questions about the ethical and pedagogical responsibilities of educators who might introduce peer-production tools. Probabilistically, it is unlikely that an average educator’s typical effort will yield high-quality collaborations. Should educators avoid introducing peer-production tools until the organizational structures of school change sufficiently to make them more viable pedagogical options? Should educators produce learning designs that anticipate failed efforts at collaboration, such that individual productions can still yield meaningful learning opportunities? Should educators produce learning designs that anticipate failed efforts at collaboration for the specific purpose of analyzing why they fail, as Grant (2009) and Forte and Bruckman (2009) suggest? Is it reasonable to have students engaged in a learning activity where the desired results are unlikely to be achieved? These are vital questions to address in learning designs involving peer-production platforms, and theories of technology-mediated collaborative learning need to pay closer attention to the difficulties implied by these questions.

The data presented in this study, and the studies of blogs, Scratch, and Sourceforge, suggest that these questions are not particular to certain kinds of classroom or educational settings, but universal challenges across diverse environments for collaborative learning. Even when teachers put collaborative online environments in the hands of “digital natives” (Palfrey and Gasser 2008; Prensky 2010) who are members of a “participatory culture,” (Jenkins 2009) most work is completed individually or assembled from the discrete contributions of individuals: students just posting in the same place. One of the signature challenges of the technology-mediated learning in the decades ahead will be to reshape these power curves and develop the design principles that enable peer-production tools to support meaningful collaborative learning at scale in online settings. Confronting the paucity of collaborative behaviors currently found in these peer-production environments will be an essential part of that challenge.