Introduction

Community computing refers to socio-technical interventions and infrastructures that support community interactions and civic activities among people sharing common resources (Carroll, 2001). For example, the Blacksburg Electronic Village (BEV) community computing infrastructure supports a web-based network that hosts local, online community information and activity (Carroll, 2005). Community computing, in general, supports human computer interaction (HCI) and computer supported cooperative work (CSCW) among community members, both local and distributed, working toward shared goals.

A core issue in developing and maintaining community computing infrastructures is sustainability. Many community computing projects fail because the underlying infrastructure does not meet end user requirements; the community is unable to maintain a critical mass of users consistently over time; there is insufficient social capital to support significant contributions by community members; or, as it typically happens with funded community computing initiatives, financial and human resources become constrained or even unavailable to adequately maintain the infrastructure. When community activities and practices are supplied hierarchically, such as by formal institutions, instead of developing organically and being maintained by the community, they are often construed as belonging to others and are typically underutilized (Rheingold, 1993; Schuler, 1996). As a result, the community fades away and its infrastructure fails.

In this paper, we present a case study of successfully and iteratively designing and sustaining a community computing infrastructure. Our case study is an online environment called Tapped In® (http://tappedin.org/) that supports activities of a large and diverse community of distributed education professionals. Drawing on more than nine years of participatory design experience with Tapped In users, we present design interventions that were introduced in the Tapped In community to sustain its computing infrastructure. Our participatory design interactions with the Tapped In community have enabled end users to articulate problems and propose high-value improvements to the infrastructure. These recommendations, in turn, have enabled Tapped In designers to continually improve the infrastructure over time.

Our contribution in this paper is a case study analysis of developing Tapped In. Specifically, we present four design interventions that have helped sustain the Tapped In community computing infrastructure. These interventions represent broader design strategies for developing online environments to support professional communities of practice. These design interventions and strategies would be of interest to software designers and community computing developers.

In the following section, we present our conceptual framework with respect to prior literature. We explain our participatory design methodology and research methods in Section 3. Section 4 provides background of Tapped In that sets a historical context for our case study. In Section 5, we present four design interventions that have been successful in sustaining the Tapped In infrastructure and its community of users. Section 6 discusses three broader strategies for technology design to support online communities of practice.

Related work

Over the past several years, we have been developing and refining the Tapped In infrastructure, which is intended to support the online activities of a large and diverse community of education professionals. The community of practice framework (Lave and Wenger, 1991; Brown and Duguid, 1991; Wenger, 1998; Orr, 1996; Cothrel and Williams, 1999) has guided us over the years to develop online teacher support activities and the community computing infrastructure to support such activities. Communities of practice are described as emergent, self-reproducing, and evolving entities that are distinct from, and frequently extend beyond, formal organizational structures, with their own organizing structures, norms of behavior, communication channels, and histories. Members often come from larger professional networks spanning multiple organizations, drawn to one another for both social and professional reasons.

The community of practice framework suggests that a teaching professional’s community of practice can have a direct (positive or negative) impact on professional growth through various forms of informal collegial interactions (Barab and Duffy, 2000; Brown and Duguid, 2000). The recognition that communities of practice can play an important role in professional learning has spurred a great deal of interest in how to harness the power of such communities in the context of systemic school reform and professional development projects. Researchers (Loucks-Horsley et al., 1998; Stein et al., 1998; Little, 1990; Garet et al., 2001; Smylie et al., 2001), practitioners (Rényi, 1996; Wilson and Berne, 1999), and policy-makers (President’s Committee of Advisors on Science and Technology, 1997; National Commission on Mathematics and Science Teaching for the 21st Century, 2000) are converging on a shared vision of effective teacher professional development as more than a series of training workshops, institutes, meetings, and in-service days. It is a process of learning how to put knowledge into practice through engagement within a community of practitioners.

A major part of the challenge in designing and developing community computing infrastructures is sustaining the infrastructure and its critical mass of users over time. The theme of sustainability has long been addressed from different perspectives in the community informatics literature. Merkel and colleagues (2005) provide an overview of what sustainability means in community computing settings. Broadly, sustainability is centered on how people in community computing settings can best achieve their goals consistently over time. According to Merkel and colleagues (2005), this question has been asked in different ways, with researchers and practitioners focusing on: (a) the feasibility of various models and the physical, social, and technical requirements that must be in place to ensure technology access to citizens (Clement and Shade, 2000; Benassi et al., 2004); (b) the role of the government in addressing issues that affect the public good, such as providing access to government information through web portals and to the Internet itself, especially for marginalized members of society who may lack the resources or training necessary to access such services (Doody, 2004; Musgrave, 2004; Malina and Ball, 2004; Rideout and Reddick, 2004; Schauder et al., 2004); (c) outcome-based approaches that study factors needed to encourage long-term changes in the lives of users (Gordon and Gordon, 2004); and (d) socio-technical investigations of information technology adoption and features of one’s social network (e.g., social capital) that tend to support or inhibit technology adoption (Day and Cupidi, 2004; Prell et al., 2004).

For Tapped In, our concern with sustainability is related to developing design interventions to keep up with the changing needs of our community of users. Our mantra has been that design interventions that enhance end user participation and interaction with the designers of the community infrastructure can lead to sustainability. When end users have a greater stake in the community computing infrastructure, they feel empowered as they have the ability to guide design, and they are actively engaged in discussions with peers and designers, motivating subsequent community interaction and contribution.

Previous literature outside the domain of teacher professional development has looked at factors that motivate participation (Lampe and Johnston, 2005; Lakhani and Hippel, 2003; Ling et al., 2005) and enhance socio-technical capital (Resnick, 2002) in online communities. Our interest lies in enhancing participation and social capital between end users and designers of the community in order to foster collective initiatives to improve the underlying community computing infrastructure.

The work most closely related to our investigation is the practitioner-oriented set of design lessons by Amy Jo Kim (2000). Kim proposes specific design principles that characterize successful, sustainable online communities. We reiterate five of Kim’s design principles that we have used directly to design the four interventions for Tapped In that we present in this paper:

  • Build flexible, extensible gathering places. Online gathering places provide a flexible medium for end users and designers to work together to evolve and continually define and articulate the purpose of the community.

  • Design for a range of roles. As the community grows, it will become increasingly important to provide guidance to newcomers while offering leadership and ownership opportunities to more experienced members.

  • Develop a strong leadership program. Develop a leadership program because community leaders are the fuel in the community: they greet visitors, encourage newbies, teach classes, and answer questions.

  • Facilitate member-run subgroups. To grow a large-scale community, provide technologies to help community members create and run subgroups to drive member loyalty.

  • Create and maintain feedback loops. Successful community building is a constant balancing act between the efforts of management (designers) to plan, organize, and run the space and the ideas, suggestions, and needs of community members.

Our integrated conceptual framework draws inspiration from the various pieces of literature mentioned above, especially Kim’s design principles. Our framework has been developed over several years to support professional collaboration and peer support among teachers (Schlager and Schank, 1997; Schlager et al., 1998, 2002; Fusco et al., 2000; Derry et al., 2000; Tatar et al., 2002). The major conceptual components of our framework that we directly leverage in this paper are:

  • Multiple interaction formats and technologies. Our framework calls for a range of tools and workspaces that (a) support work practices of large numbers of different groups; (b) enable users to know with whom they are interacting and what is going on around them; (c) allow users to create, store, and share discourse objects (e.g., notes, overhead slides); (d) communicate in real time or asynchronously, as the need arises; and (e) engage in group activities hosted by designers as well as their own circle of colleagues.

  • Identity and trust. User profiles and induction activities are aimed at building trust in the system and developing a strong sense of community and group identity.

  • Ownership and empowerment. The framework facilitates a sense of ownership and empowerment in the community by encouraging members to contribute to community activities and resources, assist other members, and use the online environment to support their own collaboration with others.

  • Heterogeneity. A key indicator of community health is the participation of a population with diverse interests and a range of expertise. The framework encourages the participation of teachers at all levels and from all disciplines, as well as district staff, researchers, university faculty and students, staff developers, and administrators.

  • Community management, leadership, and sustainability. No professional community can be sustained without management and committed community leadership. Recognizing and rewarding informal leadership and centralizing community management can help coordinate activities across projects, increase efficiency, and create economies of scale.

One facet of our research deals with understanding the kinds of online activities and content that teacher professional development organizations can develop to achieve their goals and support teachers more effectively (Schlager et al., 1999). A second facet of our research, which is the focus of this paper, addresses the issue of sustainability with respect to the long-term and evolving design interventions that allow the online teacher professional development community to engage in participatory design with the designers of the infrastructure.

Methodological approach and research methods

Our methodological approach most closely resembles participatory design. Participatory design (PD) is a practice among design professionals that explores conditions for user participation in the design of technology (for detailed discussions, see Clement and van den Besselaar, 1993; Greenbaum and Kyng, 1991; Kensing and Blomberg, 1998; Schuler and Namioka, 1993). Participatory design, as it is referred to in HCI and CSCW, has its roots in socio-technical systems theory (Mumford, 1983). Historically, Emery and Trist (1960) (Emery, 1993) were pioneering thinkers in understanding the importance of including the membership of a community in the design process.

Our approach to participatory design brings end users and designers together in mutual commitment, where users learn about what computer technology can do for them and designers learn about the application domain in order to build a flexible and efficient system to fit the users’ needs (Bjerknes, 1993). Most of our participatory design interactions occur online in an asynchronous manner.

Our participatory design approach also has a flavor of action research. We assume that the end users who are scrutinized in our research and are potentially affected by our research can be, or can be qualified to become, co-researchers. Overall, our methodological approach can be described as a design experiment, in which our investigation includes research to design professional activities and technical capabilities that we conjecture will help establish and sustain online teacher professional development communities.

Since the Tapped In project started in 1996, it has spawned many different smaller projects (e.g., theses and dissertations). It is difficult to briefly describe how data were collected, analyzed, and evaluated over such a long period of investigation. For the purposes of this paper, we refer to snapshots of our efforts with data collection, analysis, and evaluation that have occurred during multiple instances in the nine years of our research with Tapped In. These snapshots establish that we followed a rigorous and systematic research investigation.

Data collection

Field research began in 1997 when Tapped In went online. We have used a multitude of quantitative and qualitative instruments in collecting data. Because our methodological approach was guided by participatory design mostly through online interactions, the primary methods of data collection were online observations recorded through field notes, surveys, activity logs, and interviews. Secondary sources of data included documentation (e.g., newsletters), archival records (e.g., e-mails), and physical artifacts (e.g., design mockups and scenarios).

As reported in Schlager et al. (1999), we have collected data for online member activities (e.g., objects they access, rooms they visit, when they log in and out). All Tapped In members are informed of such research data collection efforts when they apply for membership. Strict confidentiality is maintained, and the content of conversations is never recorded without additional explicit permission from the participants. The only exception is our After School Online (ASO) sessions, which are recorded and posted in Tapped In’s transcript archive for Tapped In members and guests to access in the future. Participants are informed that by participating in an ASO session, they agree to the publication of their transcript.

Our membership and list of partners have grown steadily over time. As of January 1999, we had more than 2,500 members and an average of more than 60 logins a day. In July 2006, we had about 20,000 members and approximately 1,200–1,500 member logins a day. As membership has grown, the monthly login rate has remained steady at approximately 10–20% of the membership. For example, in July 1998, 378 different members (out of 1,700+) logged in. In July 2006, 2,100 different members (out of about 20,000) logged in. Members log in every day of the week and almost around the clock. Logins are relatively equally distributed from Monday through Friday and shrink by about two-thirds on weekends. Currently, about 40% of our members describe themselves as K-12 teachers and 25% are composed of researchers, university faculty and graduate students, staff developers, school support and administration staff, and pre-service teachers. The remaining 35% describe themselves as “other”.

Over the years, we have also held summer institutes, workshops, training sessions, and online seminars to expand our data collection efforts. One of our largest efforts started in summer 1997, when we were part of two 2-week summer institutes in July and August. We followed seven teams of two to four high school and/or community college teachers who attended each institute to gain hands-on experience with software and techniques used in earth and space science. Periodic follow-up online meetings with the 14 teams also occurred over the course of the school year. At least one representative from each of the 14 teams was asked to log in and report on their progress, obstacles, and lessons that they wanted to share with other teams. Transcripts of all the meetings were collected via Tapped In’s automated transcript logging mechanism. We analyzed these meetings and learned a great deal in this research about how people learned to communicate, share information, and collaborate online. In the course of three meetings, we observed people being able to work and accomplish things online (Schlager et al., 2002).

We have also administered surveys with Tapped In users. For example, one such survey was developed to help us learn who our members are and how their experiences in Tapped In have affected their professional lives. We collected data on standard demographics and professional development activities, technology use, and Tapped In use, affordances, and barriers (Fusco et al., 2000).

Data analysis

The data collected were analyzed by using the general analytic strategy of developing case descriptions (Yin, 2003). A descriptive approach was followed to help identify the complex stages of designing and sustaining a community computing infrastructure for Tapped In. Our perspective on participatory design guided our analysis of the data, reflecting important socio-technical elements of designing the Tapped In infrastructure. However, the data were also used to inform the participatory design approach itself, in that the design emerged as an iterative process taking place throughout the data collection and analysis phases. For example, the designers and researchers of Tapped In addressed many features and bugs in the order end users prioritized them.

We have used discourse analysis on meeting transcripts to interpret our data (Schlager et al., 2002). As an example, one of our transcript analyses shows that even with a group that uses technology minimally over a period of several months, the structure of their meetings shifts from a focus on technology and group norms to a predominantly task-oriented focus, similar to dialog captured in face-to-face meetings (Olson et al., 1992).

We have also coded our data to address specific research questions (Schlager et al., 2002). For example, with the data we collected from the summer institutes, a coding scheme was developed to quantify the structure and flow of the online meetings, based in part on studies of face-to-face dialog in collaborative design group meetings (Olson et al., 1992). We coded each utterance and nonverbal action as an instance of one of several categories of discourse. As an example, the four most common categories of discourse that emerged were business focused, meeting management, technology related, and social.

Data evaluation

To achieve rigor in our data analysis and interpretation, we triangulated the multiple sources of data collection. To ensure reliability and plausibility of our results, the Tapped In research group attended biweekly meetings to discuss their field observations. The research group included developers and designers with considerable experience in online communication technology. All members of the group reflected on the collected data to generate collaborative interpretations. Discussions related to design and improvements of the Tapped In infrastructure were the primary focus of these meetings. This process of collectively reflecting on data interpretations helped to remove individual researchers’ subjective biases, thus increasing the reliability of data analysis. During our coding efforts, the transcripts were read by two researchers, who coded independently and then came together to calibrate their findings. Differences between the two coders’ ratings were resolved by a third reviewer.

Because many of our research group members were geographically dispersed, we often used Tapped In ourselves as a communication and collaboration mechanism for our research meetings. We analyzed our meeting transcripts as well. These analyses revealed many episodes of knowledge building, mentoring, argumentation, and resolution, all key characteristics of productive group work. A research issue we encountered in our multidisciplinary research group was learning each other’s jargon and interpersonal styles (Schlager et al., 2002). We also had to develop our own norms for interacting as a dispersed group. We were all used to the social constructs of face-to-face meetings – rapid-fire dialog, long monologs, whispered side comments, topic shifts – and the skills needed to break into the dialog at just the right moment or guide a meeting through the items on an agenda. Online collaboration requires adjustments to these constructs and skills.

The data we present in the forthcoming sections has been anonymized. For screenshots, we have blocked out the last names of the participants. In some cases, staff members have given permission to leave their names unaltered.

Background of Tapped In

From 1996 to 2002, we developed and hosted the Tapped In Testbed, a MOO-based platform (Curtis, 1992) in which we cultivated a diverse education community of more than 20,000 members with the aim of understanding the nature and affordances of online communities of practice in the service of teacher professional development. Two critical pillars of the infrastructure were the establishment of a live Help Desk and a discussion series called After School Online (ASO). We felt that greeting new members was an important first step in welcoming them into the Tapped In community. We established the Help Desk in the reception room for this purpose. Although the Help Desk was originally staffed by members of the research group, community members acknowledged its importance by adopting it to such a degree that it was eventually staffed primarily by volunteers. ASO was originally conceived of as a venue for our partner organizations and community leaders to reach out to teachers, but it grew over time into a way for members to meet others with similar interests, to gain comfort with the technology, and to develop online discourse and leadership skills in a low-pressure, motivating context.

To further support the key activities of an online community of practice and move forward with our research, we decided to abandon the MOO platform for a more modern, flexible, and extensible architecture. The MOO used an unsupported language, was single-threaded and hence scaled poorly, and was a text-based system at odds with the increasing use of multimedia to support online learning and collaboration.

Starting in mid-2001, we began working with our partners and community to incorporate new features and capabilities, including groups, discussion boards, and search, among others. We used a scenario-based, participatory design approach (Rosson and Carroll, 2001), bringing together a design team representing researchers, teacher educators, technology developers, regional education support providers, national teacher professional development organizations, and our core constituency, the Tapped In members. After a rigorous needs assessment process and multiple design iterations, mockups, and user tests, the resulting feature set and interface design were reified in a web-based demonstration and a set of functional specification documents, including feature prioritization.

The development team chose open-source, scalable, Java-based solutions in which to implement a redesigned system that would be robust, versatile, and scalable. By building on open-source foundations, we benefited from and contributed back to existing development communities. We released a basic system for alpha testing ahead of schedule in September 2002. As a result of a second formal round of user testing, we made a major conceptual design change to the “place” metaphor employed in the user interface (Schank et al., 2002). We continue to develop features in the system based on suggestions by community members. Screenshots of the current system are shown in Figure 1.

Figure 1
figure 1

Redesigned Tapped In user interface, with the addition of many new tools and services. Information tabs and navigation support are at the top, content and room tools are in the center; and awareness, chat, and instant messaging tools are at the bottom. Clicking a tab at the top overlays the tab content over the room until you close the tab overlay. The screenshot on the left show the reception room and variety of tools available in the room (notes, files, links, etc). The screenshot on the right shows the campus map overlaying the reception room view.

Full size image

Case description and analysis

We presented a brief history of Tapped In’s evolution. Many design interventions have been introduced in Tapped In’s infrastructure to facilitate the participatory design process. In this section, we discuss four such design interventions that we believe have been instrumental in successfully enhancing and sustaining the Tapped In community and its infrastructure. We believe these design interventions represent useful design strategies, particularly for readers of this special issue on “Infrastructures,” and in general for designers of online communities of practice.

Table I summarizes our contribution in this paper in terms of the four design interventions. We chose these specific interventions because they are significantly distinct from each other on at least four dimensions: goal of the design intervention, primary mode of communication, core participants (primary users of the design intervention), and implications for use. These interventions have allowed Tapped In community members to weigh in on and influence the design in the spirit of our participatory design methodology.

Table I Design interventions summary
Full size table

Contact and bug forms

An early design intervention through which Tapped In community members and guests (who did not create a Tapped In user name and password but could still log in) contributed to the design process was a submission form available on Tapped In’s web site. One part of the submission form, known as the contact form, was dedicated to contacting the staff for technical help, suggestions, and general comments. The second part of the submission form, known as the bug form, was specifically used for reporting bugs. End user input from these two form submissions were e-mailed to a mailing list that several Tapped In team members monitored daily. Using contact and bug forms, Tapped In community members could establish direct communication with the designers and developers. Although users sometimes reported suggestions for new features or enhancements of existing ones that spawned e-mail discussions, the vast majority of submissions related to technical difficulties with lost passwords, chat configuration, and firewalls. However, community members did often engage the Tapped In design team in clarifying various features of the infrastructure. For example, the following was a request by one Tapped In community member:

Date: July 3

Subject: Saving info in my Music History Room

Comment: I’d like to save ALL the information from my Music History Room, including discussions, postings, and mail to me. If I can’t do this directly, how can I access this information in the future? I may have students come back at a later date for grade information....

The first responder to this request was Kari from the Tapped In design team. She did not know the exact answer to the query, so she engaged the rest of the team within a couple of days. She sent the following message to the Tapped In team mailing list:

Date: July 5

Would archiving save this info for him? Or should he just keep renewing the group?

One of the other members of the Tapped In team, Patti, was also unsure how to answer the original query. To achieve consensus, she drafted a reply and asked for confirmation:

Date: July 5

Archiving will record the discussion messages and notes, I believe (right Zaz)? He’ll lose any files that were attached, though. When he says “mail to me” that’s totally separate from the group (I assume he means Saved Messages?)

In this message, Patti tries to engage another team member, Zaz, a programmer who developed the group archiving feature. Zaz has the final response to this internal discussion thread:

Date: July 5

That’s right, everything is recorded (except perhaps folders? not sure about that) but binary files are deleted. So he’d have any metadata (title, description) but not the file itself.

Saved messages have no relation to groups, and we don’t keep records of any messages sent to the group (from About Us, e.g.) not via the discussion board.

The first respondent, Kari, finally replies to the Tapped In community member:

Date: July 6

You could either just keep renewing your group so that it does not expire or you could archive the group. Archiving will record the discussion messages and notes, but you’ll lose any files that were attached.

Let me know if this helps or if you have any further questions.

Such a discussion thread is a frequent occurrence within the Tapped In team. All design team members are active in promptly replying to end user queries. Such prompt action reinforces that Tapped In community members are important and that their queries are being addressed.

One of the most important ways to sustain a community infrastructure is to retain and increase user critical mass by constantly engaging them to contribute to the infrastructure. Because the contact and bug forms are external to the Tapped In infrastructure – they are simply web forms on the Tapped In web site – the Tapped In design team also encourages end users to share their contribution and learning experiences with the larger Tapped In community. For example, in the following e-mail, an end user contacted the Tapped In design team to share educational web resources:

Thought your users/members would appreciate this information on our social studies curriculum involving the use of popular song lyrics to engage students and raise their awareness of important environmental, historic, social, and political issues as a prelude to action and activism.

Our web resource for teachers and students has recently moved to... http://www.learningfromlyrics.org/

Be sure to check out numerous examples of student works including photos of the 2006 Student Memorial Projects in the Gallery Section... http://www.learningfromlyrics.org/gallery.htm

and here’s a recent web posting about the student Memorial Projects... http://www.eltonjohnworld.com/coranto/news/2006/April/SittinginTheClassroom.html

The lead Help Desk member (part of the Tapped In design team) urged this end user to be a guest speaker in one of the Tapped In community sessions:

What a wonderful resource! Why don’t you present this information yourself as a guest speaker during one of the Tapped In After School Online discussions? I didn’t see your name in the member directory, so that would be my first recommendation...

I lead a monthly Arts and Literacy discussion (one is scheduled for July 3 at 7 pm EDT/4 pm PDT) and a monthly ArtsSites discussion that covers a variety of arts topics (your resources would fit nicely in either discussion). Please get back to me if you’re interested in more information or would like to schedule a date and time to be a guest speaker.

The contact and bug forms have been instrumental over the years to serve as a medium for the Tapped In design team to interact with end user community members and achieve participant buy-in. Many characteristics of these forms have led to the sustainability of the Tapped In infrastructure. Prompt replies by the design team have reinforced the importance of the presence of community members. These replies have not just been one-way dialog. They have led to constructive discussion threads and have engaged end users in exploring further features of Tapped In, a mechanism that encourages the community to keep coming back. Involving end users in community-wide sessions has also helped to ensure that, ultimately, Tapped In is co-directed by the larger end user community and the design team.

Needed Features group

As Tapped In has evolved, community members and guests have also taken advantage of the infrastructure’s new features to create additional avenues by which to inform the designers of their suggestions, frustrations, and experiences interacting with the online environment and its denizens. For example, a prominent Tapped In community member used groups and discussion boards to foster a lively sub-community expressly to provide a place for other community members to contribute their thoughts on the design of Tapped In. This group, “TI2 Needed Features,” evolved into a valuable resource for the Tapped In team (TI2 refers to “Tapped In, version 2.0”). An important characteristic of this group is that it was not created by the Tapped In design team but rather by one of Tapped In’s community members. “Kathleen,” who created this group in 2002, was motivated to incorporate features from Tapped In’s predecessor MOO-based system.

Because the Needed Features group is public and open, any member in the community can join and post to the group’s discussion. These discussions include musings on the differences between the old and new systems, bug notifications, wish lists for new features, and feedback to the staff on recently introduced features. The simple “me too” recurrence of the same request or the enthusiasm expressed by users helps the Tapped In team decide how to prioritize the implementation of new features and assess which features from the old system are most sorely missed. The group also provides a means for staff to announce when new features are made available, allowing for an informal beta test before the feature is mentioned in the monthly newsletter or advertised by Help Desk volunteers. This open line of communication also engenders a positive feedback cycle in which users feel that the Tapped In team is attuned to their needs and informs them as progress is made and features are prioritized. The Tapped In team has directly experienced the gratitude of community members when they post their thanks after the introduction of a new feature.

The following are two typical examples of discussion threads in the Needed Features group that document the interplay between the community and the design team. The first example (see Figure 2) shows an unsolicited feature request from a user for foldering and hyperlinks within note text and the responses from a member of the design team letting the group know the priority of the features requested and when they were completed.

Figure 2
figure 2

A discussion thread in the TI Needed Features group room, showing a request for foldering and hyperlinks.

Full size image

The second example (see Figure 3) comes from several prominent Tapped In community members, emphasizing the need for being able to create K-12 student groups. This example demonstrates the open lines of communication between users and the design team and illustrates that the urgency of the needs of these users is taken into account when prioritizing and implementing features and enhancements to the system.

Figure 3
figure 3

A discussion thread in the TI Needed Features group room, showing a request for K-12 student groups.

Full size image

In 2003, we created a technological mechanism to allow K-12 teachers to bring their students online. Many teachers had asked whether they could bring their students online, and then a discussion about the wish for this feature came up (in the TI2 Needed Features room). Tapped In staff wanted to create a way that would make it safe for the students and also keep them from entering and disrupting the Tapped In community. (We imagined that some teachers might view Tapped In as a sanctuary from students.)

While we were creating the technology that would give K-12 students their own special accounts and a place of their own to interact in the system, we also created a group room for the teachers who would be bringing their students online so that they could easily find others who were doing so. We imagined they would want to discuss how the technology could be used in new, interesting, and effective ways. This was a deliberate effort to increase the social capital among a subset of members and the first time we had invited people to join a group because of an action (getting a K-12 student group room) they took in the system. The invitation to join was automatic during the K-12 group creation process.

Currently, there are 310 K-12 group rooms in Tapped In and 446 members in the group to support teachers interested in bringing their students online. In early 2004, a member learned that it was possible to bring K-12 students online, and that there was a group to support teachers doing this, and asked whether she could work with and support the teachers in the group room. We were delighted to have a member take charge in this way. The member was a doctoral student and an education consultant. Previously, she was a middle school teacher, a curriculum developer, a professional development facilitator, and an education researcher.

Task list

As development of the redesigned Tapped In began to ramp up in 2001, the number of features to implement and bugs to fix rapidly became too high to manage without some formal process. The Tapped In design team created a list of these items, the initial purpose of which was simply to record the features and bugs. However, the list soon also came to function as a mechanism for designating a priority category (1 = high to 4 = low) to each item, assigning items to individual developers, establishing the status of each item (e.g., in progress), and organizing items into types (e.g., enhancement versus bug). As development continued, this priority list was continuously updated and added to as new features were dreamed up, bugs were reported, and the design of the new system was refined. Eventually, the list was also used to estimate development time and record how long items took to be completed. With the small Tapped In design team and the varied nature of the items (e.g. features versus bugs), a simple Excel document with a single individual as the primary gatekeeper was easier to maintain than a more sophisticated bug tracking system such as Bugzilla (http://www.bugzilla.org/).

Table II shows a part of the task list. Darker colored rows indicate high-priority items. The tasks in the list are also organized according to three categories (not shown in the table): ongoing tasks, possible vendor bugs, and completed tasks/unverifiable bugs.

Table II Sample rows from the task list
Full size table

The task list is shared among the Tapped In design team. One person maintains the list for synchronization purposes. The team members typically e-mail about new tasks and/or talk about them face-to-face to clarify the scope and identify the best person to do the task. The developers estimate the time and management sets the priority. During periods of intense development, the Tapped In team meets at least weekly to discuss the task list and prioritize tasks. In times of sparse development (such as when funding is low), the team focuses on tasks requiring the least time and cost.

The following is an exchange of e-mails within the Tapped In team to discuss addition of new features. The original e-mail thread started off with the following:

Here are some top features Zaz and I came up with for TI2...any you want to add? Do the estimates look right?

  • For private messages, make it easy to delete multiple messages, e.g., add select all and delete selected buttons (1 day?)

  • Buddies Tab working (5 days?)

  • Login Aliases (2 days?)

  • Calendar Search (2 days?)

  • Implementing Resources jsp for TI2 (3 days?)

  • Implementing Newsletter jsps for TI2 (3 days?)

  • Preference for timestamps (hide/show) in transcripts (1 day?)

  • User Reaper (3 days?)

  • Allow custom header/footer for login and registration pages (2 days?)

  • Integrate search indexing into results for discussion, other (5 days?)

  • Make “new” marker smart (new to you) (5 days?)

The following reply from one of the developers reassessed one of the features and estimated times for all the tasks:

I would double all those estimates:-)

Is ‘transcript access via web’ worthy of this list?

Once Tapped In users began to migrate from the MOO-based system to the new one and as they became a source of input – particularly via the Needed Features group – the developers found themselves frequently referring to the task list when communicating with the community. As more active community members became aware of the existence of the priority list, it became obvious that the next logical step was to periodically publish the list to the community in the interest of strengthening and making more transparent the feature development process. In the following example, one of the Tapped In community members encourages the developer team to consider his improvements:

Spent 2 plus hours on Tuesday participating and observing two of the sessions in the TI2 Launch Festival. Exciting to see the level of interest and to see the activity. The two sessions I participated in were well attended – 40 plus and 50 plus people in attendance.

I hope you don’t mind, but I noted some issues that I ran into while using the system and, where appropriate, took the liberty of offering my ideas for addressing them. I know that given resource constraints, you are very limited in terms of what you can afford to do, but I wanted to capture my observations so that you have a record. Please accept as food for thought as you move ahead with the system.

…[suggestions]

Fred

The Tapped In team was prompt in replying to this e-mail and explicitly referred to the task list. For instance, in reference to the private message window icon in Tapped In, one of the developers replied with the following:

each time the private message icon is clicked whether previously opened or not, it should be given focus and moved to the foreground. [Fred’s suggestion]

Hmm, this may be on our list, I’ll check. [Developer response]

For one of the other features (a help button in Tapped In), the developers realized that it should be put on their list:

Explore the possibility of creating a separate help button outside of the “Actions” menu that spawns a small window in which the help can be viewed statically and concurrently with the Chat window. [Fred’s suggestion]

Good idea; we can put that on our list. [Developer response]

The list is an Excel file, so publishing it as an uneditable PDF file in the Needed Features group room is a straightforward task. To enhance the usefulness of the task list, we recently began recording for each item the number of times it is mentioned or requested by Tapped In users. Keeping in tune with the needs of the community and prioritizing the items on the list are particularly critical, given the small development team and the minimal funding for Tapped In. As such, sometimes larger, more complicated items that may not be as frequently used drift down the list in favor of more pressing needs, which often are “low-hanging fruit” – that is, smaller features or fixes that make a large difference to users.

Overall, we consider the use of the task list a success for systematically monitoring feature updates to the infrastructure. The task list is also helpful as a tangible resource when communicating with the Tapped In community. Most importantly, we feel that the task list facilitates the management of tacit knowledge within the Tapped In developer and design team. Although documentation plays an important role in managing knowledge within an organization, it is often considered a “side task”. The Tapped In team’s view on the task list – a lightweight version of formal documentation – is that it promotes sustainability because it helps to legitimize less formal mechanisms of documentation.

Help Desk volunteers and long-standing members

Originally, in 1996–1999, Tapped In staff members were the leaders and managers of the community and were online continuously. However, as the community grew, the leadership and governance transferred (as is necessary for a successful community) to the community members. Tapped In staff members still take part in the community and know many community members well, especially the most dedicated and long-standing members. The community volunteers and leaders are highly dedicated and provide assistance and a warm welcome to new members for as many as 12 hours a day, sometimes more.

While they are online, these members help guests and new members get oriented, lead tours, provide technical assistance, and grease the wheels of the community in many other subtle ways. They are often present as members or guests arrive in Tapped In Reception for the first time, try out new features, or exclaim as they discover a useful aspect of the system. By helping scores of new members and guests every day, Help Desk volunteers become aware of misconceptions or problems users may have, perhaps indicating design choices of the system that require clarification or modification.

Problems and successes are communicated to the Tapped In design team through various means. There is no single formal process for Help Desk volunteers to report experiences and impressions, but the Tapped In team and the volunteer community are close-knit, so they see each other regularly online and make themselves available to each other. Positive feedback and ideas for improvements are often communicated informally online or through e-mail. There is also a group room, Helpdesk Central, where some of the more formal communication occurs. Figure 4 illustrates information flow from a Tapped In staff member (Mark) to a core Help Desk volunteer (BJ), who in turn posts it to the Helpdesk Central group room discussion board, where it is disseminated to the 54 volunteers who are members of the group.

Figure 4
figure 4

An example of positive feedback flowing through the close-knit Tapped In team and volunteer Help Desk community.

Full size image

In addition, there is a link between the Help Desk and the Needed Features group, since most of the Help Desk volunteers are members of the Needed Features group and add requests and suggestions there. In this way, Help Desk volunteers act as conduits between newer community members and the design team. Because the Help Desk volunteers have a great deal of experience with using the system, their comments are often given more weight as they can usually better articulate the feature request or frame an idea for a new feature and suggest how it might be integrated into the current design. Perhaps more importantly, the experienced volunteers can usually provide a concrete design rationale for a new feature and express why it supports the overarching goal of the Tapped In community. All of these “extras” that an experienced member provides helps the Tapped In design team to make better decisions about where to invest their limited resources.

The Help Desk volunteers are so much a part of the design cycle and so in tune with the community that they recently (October 2005) noticed a slight decrease in ASO session attendance and alerted Tapped In staff. Tapped In staff called a meeting of all interested members and held a brainstorming session about what could be done. Some members hypothesized that new technologies (e.g., audio or video capabilities, blogging tools, or a new look and feel) might make Tapped In more useful and interesting to members. Others thought that Tapped In was just fine as it was, but that more outreach needed to be done by staff and volunteers. The solution will most likely require social infrastructure improvement, but new technologies also might help. The important thing is that through the mechanisms we have set up, developed, and allowed to grow organically, the community brings these issues to the attention of the Tapped In staff as they arise.

Summary

We have presented four design interventions (Contact and bug forms, Needed Features group, Task list, and Help Desk) that have contributed to the sustainability of the Tapped In infrastructure. We perceive these interventions as a measure of our success in iteratively designing the Tapped In infrastructure and keeping the community members interested in using the infrastructure for their online teacher professional development and social networking. Although the four interventions we presented may be considered as standard design practice, the value lies in their integrated use as participatory design mechanisms to enhance end user participation and interaction with designers.

The community design processes we established to collect feedback and suggestions from users have led to many improvements to the community infrastructure that supports Tapped In. These processes grew organically from minimal structures we had in place initially – such as simple contact and bug report forms – into a more complete feedback system of task lists, discussion boards, and interactions with Help Desk staff and long-standing members. These processes also help members feel ownership in the community: members feel that they are recognized, helpful, and contributing to a larger cause.

The design interventions introduced through the process of participatory design were instrumental in empowering the Tapped In community members. Data in the preceding sections indicated that end users had a stake in the design process and on many occasions drove this process. We think achieving such buy-in through participatory design is essential for maintaining and increasing the critical mass of users and improving the community computing infrastructure.

Discussion

Although the social sciences provide a rich body of theoretical and empirical literature about human behavior from a socio-psychological perspective, CSCW has rarely taken advantage of such research (Kraut, 2003). This is due partially to the fact that CSCW is primarily a design-oriented domain (Farooq et al., 2006). Though our contribution in this paper is targeted to advancing the design science of developing sustainable community computing infrastructures, the design interventions we presented are interwoven with theoretical and empirical foundations acknowledged in prior literature.

In our previous work (Schlager and Fusco, 2004), we proposed several guideposts for technology design to support online communities of practice for teacher professional development. Here, we expand on these guideposts by reflecting on three design strategies that tie back to theoretical and empirical literature in community computing. These higher-level design strategies, as defined by Dourish (2001), comment upon the general characteristics of design interventions articulated in the previous section.

First, investing in bonding social capital is critical for maintaining feedback loops between community end users and designers. Putnam (1993) uses the phrase bonding social capital to refer to the relationships that are developed within a homogeneous community. For online communities of practice, such as Tapped In, one important aspect of bonding social capital between the end users and developers of the community computing infrastructure is the feedback that end users provide. This type of social capital grounded in participatory design (between end users and designers) is not typically discussed when people think about designing an online community and its potential social support or resources. However, we would argue that it is necessary to keep the community moving forward, improving its offerings and growing at the same time. Through our design interventions, we have facilitated the creation of social capital in the Tapped In community, evidenced by the steady growth of Tapped In users and their contributions in the online community. The Needed Features group is a good example of bonding social capital with the goal of getting feedback by the community; the group is led by Tapped In community members and serves as a primary channel for requesting new infrastructure features from the designers.

A second design strategy is to provide multiple online gathering places for engagement with a range of community end users. The need for communities in general to have a gathering place of some sort is well acknowledged. For face-to-face geographical communities, these gathering places might take the shape of coffee shops, bars, and bookstores. Oldenburg (1989) refers to these as third places, the first being home and the second being work. For distributed online communities, providing effective gathering places is a design challenge. On the Internet, a gathering place can be a mailing list, a chat room, a virtual world, a blog, or some combination of these spaces (Kim, 2000). Online gathering places, just like their geographical counterparts, nourish relationships, develop a sense of community, and promote social interactions (Kim, 2000). For online communities of practice, where participants can range from being legitimate peripheral participants to core members (Wenger, 1998), an important aspect is to design multiple gathering places for the different types of community end users. Tapped In provides such multiple gathering places. The contact and bug forms provide an asynchronous gathering place that caters to peripheral participants who may not have transitioned into core participants of Tapped In but still want to explore Tapped In features and weigh in on design. The Needed Features group and the Help Desk provide interactive gathering places for more committed and experienced Tapped In community members.

A third design strategy is to reinforce leadership roles organically from within the community. Community leaders perform organizing, governance, networking, brokering, and other social support services for the community (Lieberman, 1996; Spillane et al., 1999). They empower and sustain the community and maintain order and etiquette (Kim, 2000). Community leaders are instrumental in developing, managing, and participating in multiple overlapping social networks within and across community of practice boundaries (Wenger, 1998; Cothrel and Williams, 1999). A major difference from other online communities is that Tapped In facilitates organic growth of leadership rather than well-defined and highly structured leadership roles that are put in place by designers or administrators. Indeed, it is plausible that leadership by community members, who are intrinsically motivated to give back to the community, entails longer-term sustainable consequences than designing contrived and possibly constraining leadership roles. In Tapped In, for example, the Help Desk continues to be a viable option for developing organic leadership; it leverages experienced volunteers, who are self-motivated, as conduits between other community members and the design team.

Although community computing research in general has been discussed in CSCW, design work on community computing infrastructures has been less well documented. This special issue seeks to fill this gap. In this spirit, we have presented a rich history of a successful community computing infrastructure, Tapped In, to support an online community of practice for education professionals. We used a case study analysis to reflect on four design interventions that were instrumental in sustaining the Tapped In infrastructure on a community-wide scale for more than nine years. The case study was not just technical in presentation but was based on our integrated conceptual framework, which emanated from broad and interdisciplinary, theoretical and empirical literature. The follow-up discussion points abstracted from our case study analysis represent broader design strategies that serve as a source for constructive debate and future investigation in the CSCW design community.

We are currently exploring the viability and application of our design interventions and strategies for other community computing infrastructures. Based on Tapped In technology, we started CLTNet (pronounced as C-L-T-Net; http://cltnet.org/) in 2003 as an online network to support the US NSF-funded (National Science Foundation) Centers for Learning and Teaching (CLTs). This network helps the CLTs share knowledge and resources, disseminate findings and expertise to the education community at large, and provide information on training, professional development, and job opportunities to teacher education researchers and practitioners. In contrast to the Tapped In community, the CLTNet community is quite small (approximately 800 members) because of the limited number of people participating in CLTs. The meeting space is used infrequently and was intended primarily to support scheduled online events. Because CLTNet members rarely use the meeting space facilities (such as chat), there are fewer opportunities for informal interactions between CLTNet members and staff, less need for Help Desk staff (mainly to answer questions sent by e-mail), and many fewer feature requests received from the community. As such, we have noted that CLTNet has not grown into a community of practice to the same extent as Tapped In.

It is important to acknowledge that not all online communities become communities of practice. CLTNet has similar technology support as Tapped In. Many members certainly make use of CLTNet resources for various activities, such as organizing courses, working groups, and events, but no community of practice has specifically moved into CLTNet to do its work. Why? It may be that the main users of CLTNet (university faculty and graduate students participating in a Center for Learning and Teaching) have existing communities of practice that are not fully defined or are not catered to by CLTNet. For example, many university professors in education belong to communities of practice through the networks of professionals that they or their universities have developed. Given the technologies available to them, they may not see a need to move their existing communities of practice to a specific online environment such as CLTNet. It also may be that their visions for their communities of practice are still being defined; certainly, a vision for a community of practice is key to success. We are continuing to leverage our Tapped In experience to investigate the successes and failures of our design interventions and strategies in other case studies such as CLTNet.

It was our intent in this paper to serve a broad audience of scholars interested in socio-technical interventions that lead to the design of successful community computing infrastructures. We believe our contribution is of value to researchers and practitioners interested in designing online communities and using information technology to build community capacity, enhance social capital, and achieve sustainability. In general, scholars in community computing can reuse our design interventions and strategies in their own research investigations to engage their communities of users and apply our design knowledge in their own contexts. We also recognize that, as architect Ludwig Mies van der Rohe is quoted as saying, “God is in the details”. We conclude this paper with a few pragmatic details of our four interventions that, although perhaps not standard design practices, may benefit community computing practitioners in particular.

The first pragmatic detail is our practice of always closing the communication loop in order to establish and gain trust and confidence of end users in our leadership. We achieve this by respectfully responding to users who submit a bug, complaint, or suggestion, and as appropriate, informing the community concerning what is being done. If we cannot address the problem (e.g., solution is too costly) or choose not to address the issue at that particular time, we explain why. Explaining our rationale and theoretical underpinnings simply and succinctly typically quelled any anger or frustration on part of the end users. In some cases, the self-reflection we engaged in as a result of eliciting our rationale and theoretical underpinnings helped reinforce existing or establish new community policies.

Regarding our Help Desk, an important pragmatic detail is that although we envisioned it as helping users learn to use the system, in practice, it quickly became a set of services more aptly described as those of a doorman and concierge. The Tapped In team members shed the researcher and developer persona to put a friendly face on the community from the first moment a new member logged in (note that a vast majority of our users had never experienced a “chat room”). We tried to help with all questions people asked, regardless of whether the question or problem was professional, academic, or personal. This stance built tremendous good will and set the tone for all interactions among Tapped In members (we like to say that new members “imprinted” on the Help Desk staff). We believe this is the chief reason Tapped In has had so few problems with inappropriate behavior.

Regarding prioritization of our task list, design teams must recognize that there is an inherent tension (especially with limited funding) between what the users want and what we ultimately prioritize on the list. It is easy to implicitly discount the priorities expressed by the community (after all, we are the experts!). We were able to recognize and overcome this bias by having a user ambassador on the core team who sincerely and consistently represented the will of the community in all design and development discussions.

Finally, we cannot emphasize enough the value of applying the adage “eat your own dog food” to every member of the design team. We were most effectual in identifying and fixing problems when we got frustrated trying to participate in an online group or activity. Moreover, we learned to understand what was truly important to the community by interacting with the members on their terms and on their virtual turf.