Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

The incidence of humans collaborating via computer mediation is rising over time; groups collaborate online to author encyclopedias, to write software, to optimize search engines and to solve a whole range of problems from uncovering the structure of an enzyme to documenting blotches on the surface of Mars.

One main way in which combined human and computer groups manifest in the daily lives of individuals and organizations is in the form of online or computer-mediated teams. Online teams have become so common in organizations today that surveys estimate that as many as 78 % of professional workers and executives have at some point worked in online teams (Martins et al. 2004; The Economist Intelligence Unit 2009). Online teams are used in almost all industries and in a variety of areas, such as scientific innovation (Fiore 2008), software development, customer service, sales and R&D (Carmel and Agarwal 2001; Hertel et al. 2005; McDonough et al. 2001).

This chapter examines human computation through the lens of online collaboration. It begins by considering the challenges associated with assessing group performance. This leads to a discussion of collective intelligence as a measure of group effectiveness, and considers the factors that influence this measure of group performance. These factors then serve as a focal point for developing techniques that foster the emergence of greater collective intelligence in human computation systems that manifest as collaborative groups. Next, consideration is given to how these techniques can help overcome limitations of the virtual communication medium and ultimately give rise to unprecedented degrees of collaboration efficacy. Finally, new research directions are identified.

Historical Challenges of Performance Measurement in Online Teams

Studies have compared the performance of traditional teams and online teams with mixed and sometimes conflicting results. While some studies report greater effectiveness for online teams (i.e., Sharda et al. 1988), others found that online teams could not outperform traditional teams (McDonough et al. 2001; Warkentin et al. 1997). Still others detected no difference between the two types of teams (Burke and Aytes 1998; Burke and Chidambaram 1996; Galegher and Kraut 1994).

The disparate conclusions regarding the performance of online teams reflect the difficulties in assessing team performance more generally. Considerable work in fields such as social psychology, organizational behavior, and industrial psychology has been conducted to characterize factors that predict group performance on individual tasks. Traditionally, performance has been examined in terms of an “input-process-output” model, where researchers observe or manipulate inputs to the teams (such as individual differences, task definition, and resources), then measure process variables, and finally observe the effects on performance (Ilgen et al. 2005). Much of this research explores why groups so often appear to under-perform, given the potential of the individuals in the group (Steiner 1972; Tziner and Eden 1985). While some of the tasks that have been examined in teams are complex and multifaceted, such as tasks performed by top management teams (e.g., Kilduff et al. 2000; Wiersema 1992) or product development teams (Katz 1984), even in these domains performance has been examined as the outcome of a particular task at a particular point in time, despite the wide array of subtasks necessary for a team’s success. Thus, conclusions about the performance of computer-mediated teams can vary as a function of the group’s task, the technology available, or both, making generalization of conclusions across studies quite difficult.

Collective Intelligence in Human Groups

Research on collective intelligence in groups was motivated initially by a desire to measure group performance in a manner that would generalize across tasks and settings (both face-to-face and online) and predict a group’s performance on future tasks. In exploring alternate ways of conceptualizing and measuring group performance, initial studies of collective intelligence in human groups built upon work in individual psychology and concepts for understanding and predicting individual performance. Psychologists have repeatedly shown that a single statistical factor—often called “general intelligence” or “g”—emerges from the correlations among individual people’s performance on a wide variety of cognitive tasks (Deary 2000; Spearman 1904). But, perhaps surprisingly, until recently none of the research on team performance has systematically examined whether a similar kind of “collective intelligence” exists for groups of people.

Recent research has sought to explore the degree to which the concept of intelligence generalizes to groups. In two studies with 699 individuals, working in 192 groups of size two to five, researchers found converging evidence of a general collective intelligence factor that predicts a group’s performance on a wide variety of tasks (Woolley et al. 2010).

The groups in this study spent approximately 5 h together in the laboratory, working on a series of tasks that required a range of qualitatively different collaboration processes (McGrath 1984). Example tasks included brainstorming uses for a brick, creating a logistical plan for a shopping trip, accurately typing a large amount of text into a computer, discussing a moral reasoning problem, and answering questions from an individual intelligence test.

In a factor analysis of all the groups’ scores, the first factor accounted for 43 % of the variance in performance on all the different tasks. This is consistent with the 30 %–50 % of variance typically explained by the first factor in a battery of individual cognitive tasks (Chabris 2007). In individuals, this factor is called “intelligence.” For groups, this first factor is called “collective intelligence” or “c,” and it is a measure of the general effectiveness of a group on a wide range of tasks. Mathematically, this collective intelligence factor is a weighted average of the subtask scores, with the weights calculated to maximize the correlation with all the subtasks.

In addition to the tasks used to calculate c, each group also completed a more complex “criterion task.” In the first study, groups played checkers as a team against on online computer opponent. In the second study, groups completed an architectural design problem. Both of these tasks required a combination of several of the different collaboration processes measured by the individual tasks in the collective intelligence battery. As expected, c was a significant predictor of team performance on both these criterion tasks, and—surprisingly—the average individual intelligence of group members was not.

The researchers also investigated what characteristics of a group predicted c. They found that the average and maximum intelligence of individual group members was correlated with c, but only moderately so. So having a group of smart people is not enough, alone, to make a smart group.

But there were three other group characteristics that were also significant predictors of c. First, there was a significant correlation between c and the average social perceptiveness of group members, as measured by the “Reading the Mind in the Eyes” test (Baron-Cohen et al. 2001). This test measures people’s ability to judge other’s emotions from looking only at pictures of their eyes. Groups with a high average score on this test were more collectively intelligent than other groups.

Second, c was negatively correlated with the variance in the number of speaking turns by group members. In other words, groups where a few people dominated the conversation were less collectively intelligent than those with a more equal distribution of conversational turn-taking.

Finally, c was significantly correlated with the proportion of females in the group, with groups having more females being more collectively intelligent. This result, however, is largely mediated by social perceptiveness since, consistent with previous research, women in the sample scored better on this measure than men. In a regression analysis with all three variables (social sensitivity, speaking turn variance, and percent female), all had similar predictive power for c, though only social perceptiveness reached statistical significance.

Taken together, these results provide strong support for the existence of a general collective intelligence factor (c) that predicts the performance of a group on a wide range of tasks in a variety of settings, and a consistent relationship with social perceptiveness and equality of participation among group members as significant predictors.

Mechanisms of Collective Intelligence: Balancing Convergence and Divergence

The main aim of the second section of this chapter is to identify factors enabling collective intelligence in online groups. Equality of participation and social perceptiveness are two factors consistently related to collective intelligence in our studies, as described above. They are also essential to enabling the balancing of convergence and divergence in collectives more generally (Woolley and Fuchs 2011). Here we elaborate more on the role of convergent and divergent thought in collective performance, and then consider more specifically the tools and mechanisms that promote these properties in online collectives.

Balancing Convergent and Divergent Thinking

Some argue that collective intelligence emerges from the collaboration and competition of many individual entities. Research on collective intelligence has argued that central to collectively intelligent systems is the capability to engage in both convergent and divergent modes of thought, as well as to leverage the insights from reflection into course-correcting changes (Bloom 2000; Woolley and Fuchs 2011). Convergent thinking is thinking that proceeds toward or converges on a single answer. In contrast, divergent thinking moves outwards from a problem in many directions. Both convergent and divergent thinking are necessary to collective intelligence; convergence enables decisions and the possibility of moving forward, while divergence is critical for developing the wealth of insights and ideas necessary for true innovation. However, while traditional face-to-face groups tend to excel at convergent thinking, the literature on creativity suggests that divergent thinking is an area where groups often struggle (Thompson 2003).

Both convergent and divergent thinking require particular social interaction processes to occur successfully in collectives (Larson 2009; March 1991; McGrath 1984), whether those collectives are small groups (Woolley et al. 2010) or organizations. Convergence is fostered by increased quantity and intensity of interaction; the more information group members can transfer to one another, the greater the probability of arriving at a correct answer and one that all members will support (Siegel et al. 1986). Divergence requires just the opposite; groups generate the most divergent and creative sets of ideas when members work relatively independently (Brown and Paulus 2002; Thompson 2003) to enable everyone to participate more equally and fully in idea generation. Indeed, studies of innovation in organizations encourage the development of “skunk works,” as a means of protecting the pursuit of divergent modes of thought (O’Reilly and Tushman 2008) by keeping groups of individuals pursuing different ideas relatively independent of one another (Andriopoulos and Lewis 2009; Raisch et al. 2009; see also Rosenkopf and McGrath 2011). Doing so helps prevent one set of ideas from being crowded out by or subordinated to another.

Social perceptiveness and equality of participation are likely to play an instrumental role in fostering convergent and divergent thought in collectives. Social perceptiveness allows individuals to more effectively read the nonverbal signals of others, which is associated with the ability to tune’s one message in a manner that enhances consensus-building in groups (Elfenbein et al. 2007; Elfenbein 2006; Wolff et al. 2002). At the same time, equality of participation insures that all voices, including divergent voices, are heard, raising the chances that collectives will consider a broader range of perspectives (De Dreu and West 2001). Thus, collaboration tools that can be provided to collectives in online environments to foster social perception and equality of participation are likely to enhance convergent and divergent thought and, ultimately, collective intelligence.

Tools and Mechanism That Cultivate Collective Intelligence Online

So the question remains regarding how to encourage social perceptiveness and equality of participation in online collectives? Studies of technology use in online groups suggest some places to start, as conflicting findings regarding performance in online groups seem to relate to the type of task, its reliance on primarily convergent vs. divergent properties, and the ability of the technology used to foster the appropriate processes. For example, Sharda and colleagues (1988)—who observed a high level of performance in online teams—found that groups generated a greater number of ideas using email, a technology that enhances the independence of contributions and divergent thought. Burke and Chidambaram (1996)—who found no difference between online and face-to-face groups—measured decision quality resulting from the use of online discussion boards, a medium that enhances information exchange and convergence. The disparate findings have led to many studies examining ‘task-technology-fit’ where the researchers distinguish the type of technology best suited for different tasks (e.g., Goodhue and Thompson 1995). For instance, Majchrzak et al. (2000) found video conferencing better for managing conflicts while email was better for routine tasks such as analysis of data. Others have found that online groups that rely on a wide variety of different technologies are more satisfied and perform better than those that use more limited communications tools (Kayworth and Leidner 2001). These findings support the conclusion that different technologies cultivate different processes in groups, and when those processes are well-aligned with tasks demands, they help cultivate collective intelligence.

We propose here a framework for thinking about tools in terms of their role in enhancing social perceptiveness and equality in participation, which subsequently improve the quality of convergent and divergent thought in collectives.

Tools Enabling Social Perceptiveness

Traditional teams benefit from face time that enables social cues to be relayed and picked up by other members. Online collectives face the challenge of being deprived of face time and hence these social cues. This section primarily focuses on mechanisms that enhance the availability and interpretation of social cues within online teams.

  1. 1.

    Signaling to flag and communicate contextual information

    Participants in face-to-face groups can use a plethora of (often unconscious) nonverbal cues, such as facial expressions and body language, to facilitate communications which are not available to online groups. While we know that many tools exist to transmit intentional, spoken language in computer mediated settings, a new crop of tools is coming on the scene intended to amplify the passive signals people use to adjust the focus of their attention and effort on collaborative products. Systems can be further designed to provide some of these passive cues automatically, or else increase the ease with which group members can actively generate similar signals.

    These tools are inspired in part by research in the field of stigmergy. First introduced by Grassé (1959), stigmergy refers to the ways cooperative animals coordinate by leaving and sensing signs in a shared environment. A classic example is the pheromone trails that ants use to optimally route and distribute their foraging behavior. Similar examples occur in many other kinds of animal cognition (Bonabeau et al. 1999; Karsai and Balazsi 2002).

    The concept of stigmergy has also been developed in the fields of robot collaboration (Holland and Melhuish 1999) and human interactions (Marsh and Onof 2008; Parunak 2006). For instance, Parunak (2006) distinguishes between explicit (“marker-based”) and implicit (“sematotectonic”) signaling tools. Examples of explicit signaling tools are flags individual members can use to signal their current status (i.e., “busy,” “available,” etc.) or emoticons they might include in messages to convey their current mood or the emotional content of messages. Implicit signals can involve the automatic capture of activities which are translated into a signal for remote collaborators, to let them know when their collaborators are distracted, uncertain, etc. An example of an implicit signal would be the capture of the rate of cursor movement or numbers of additional windows open on listener’s desktop to signal remote presenters regarding the dissolution of attention being paid by listeners during a live online presentation. In collaborative problem-solving, implicit signals could be captured by measuring how long someone’s cursor hovers over or revises a part of a collective product, to highlight or change the color of areas where members are less certain of input provided. Such signals could help groups coordinate work by enabling remote others to see when attention is wandering (and thus move to reengage), or by attracting more workers to parts of problems involving greater uncertainty and needing additional input.

  2. 2.

    Status Updates on Profiles and Social Media

    With the advent of twitter, facebook and other social media, many organizations have adopted ‘enterprise microblogging (EMB)’, an off-shoot of the twitter model for short updates restricted to their company network, for internal communications (Zhang et al. 2010). Studies have shown that that EMB assists in (1) awareness creation, (2) task/meeting coordination and (3) idea generation & discussion (Riemer et al. 2010; Riemer and Richter 2010). In particular, Riemer et al. (2010) noted the usefulness of EMB for the gauging and sharing of opinions on current issues. Hence, such a tool could be immensely useful for capturing and amplifying the social cues that members of a computer-mediated collective might use for gauging the mood of the group.

    Furthermore, when members of online collectives update their status on various social networking media, they are able to better relay and share a social part of themselves that otherwise remains unknown to remote collaborators. Such status updates allow others to interpret otherwise ambiguous signals, such as not hearing from someone in response to a message or query, or receiving a shorter or different response than expected. By seeing their teammates outside of the ‘professional environment’ and interacting with them at a more social level, increased understanding develops amongst team members, and the social cues available for contextualizing other observable behavior become richer.

  3. 3.

    Check-ins

    Research on effective online team leadership has noted that ‘check-ins’ at the beginning of a virtual meeting leads to more successful online teams (Malhotra et al. 2007; Purvanova and Bono 2009). There are different types of check-ins that can facilitate teamwork. One of them is a round robin check-in where team members share either good news or ongoing progress at the beginning of a meeting. This check-in allows team members to connect and get on the same page before conducting a meeting or a task, and provides another source of contextual information for group members to use in interpreting otherwise ambiguous signals. Online collectives have the option of doing such check-ins synchronously or asynchronously; either approach can allow participants an opportunity to share good news and/or updates, information that otherwise may not be surfaced but may help the group.

Tools Enabling Equality in Participation

  1. 1.

    Multi-channel Chat

    A major impediment to creativity and divergent thought in groups is what is known as “production blocking” or the decline in the number and originality of ideas produced in interacting groups compared to the same number of individuals working alone (Thompson 2003). Production blocking stems from two different sources: first, as a result of the bottleneck that occurs in group conversation when people are hindered in sharing their ideas due to limitations on “air time” when only one person can speak at a time; and, second, when each group members’ ideas are influenced or inhibited as a result of hearing others’ ideas. A multi-channel chat room set up to facilitate private, public chats between members can allow equality in contributions to a collective product as well as avoid production blocking, as members can generate ideas independently in an initial step and then share them subsequently. Such a tool can facilitate the Delphi technique, a variant of the nominal group technique (Delbecq 1974), an approach in which individual ideas or inputs are developed independently and then pooled to create a final product. The Delphi technique requires a facilitator or a leader trusted by the team to aggregate responses from each member individually, ensuring equal input and avoiding production blocking. Online, a multi-channel chat allows the facilitator to privately chat with and gain input from each member. The private one-to-one chat can then be aggregated and shared in the public room, consistent with the recommendations of the Delphi technique, thus helping to foster equality of member inputs to collective products.

    A multi-channel chat room set up to facilitate different tasks (each ‘chat room’ is a ‘task’) allows members to work collaboratively or asynchronously as best suits the work at hand. By allowing members to divide the tasks amongst themselves and create subgroups focused on different areas of work, collectives can avoid the bottlenecks described above and integrate a broader array of inputs into the work of the collective.

  2. 2.

    Shared Online Documents and Social Media

    Shared online documents serve to create organizational memory as well as ensure equality of contributions to shared products. Like multi-channel chat, shared online documents enable members to be working simultaneously and to capture ideas and inputs as they occur, rather than having to wait for a turn to speak or to work on a more traditional document sequentially.

    As described above, the use of social networking media, such as facebook, google+ and twitter, in online collectives allows the members to ‘see’ other members and assist in developing social perceptiveness in the team. Members are able to express themselves, their thoughts, feelings as well as feedback. However, such sites can also serve as an online resource to create organizational memory. For example, using a facebook page where ideas can be documented in full view in addition to feedback to those ideas would prevent ideas being repeated and/or forgetting ideas all together. This eliminates waste, repetition and encourages new ideas to surface. Members have greater opportunity to contribute to and build those ideas, without being hampered or blocked while waiting for others to express themselves.

  3. 3.

    Electronic Voting Systems

    In conventional face-to-face teams, sometimes facilitators are used who are able to keep teams on track and help make resolutions by aggregating feedback from each individual. Online, one such tool that could be used for electronic facilitation is an electronic voting system. An electronic voting system allows each team member to provide feedback and determine outcomes based on the feedback of the majority of team members. This raises the probability that the ‘best answer’ can percolate to the top (Hertel et al. 2005). Electronic voting systems can also ensure equal participation amongst team members, and enable members to share views anonymously if they prefer to do so. This ensures no one particular team member can dominate or make decisions for the whole team.

  4. 4.

    Real-Time Feedback System

    Equality in conversational turn-taking can occur organically when smart teams engage all the team members or it can be facilitated using different tools. A tool to bring about awareness amongst the team members on their participation levels is the use of real-time feedback systems. These systems keep track of contributions, inputs, and level of communication from each member and display them in full view. As such, if any one particular team member begins to dominate the conversation, it becomes apparent in real-time. Conversely, if any team member is not contributing enough, that too can be flagged. While assisting equality in participation, this tool also prevents social loafing, or the tendency of individuals to put forth less effort when working in groups, another major threat to group creativity and productivity (Karau and Williams 1993).

Furthermore, a real-time feedback system allows members to gauge each other’s status and thus to assist in the development of social perceptiveness as well. Members with higher social perceptiveness can focus on the members on either extreme in terms of participation and help balance the team member’s contributions.

Another type of real-time feedback tool is an electronic chart that displays ranks of users by quantity of contributions. This is similar to the real-time feedback systems discussed above, but at the team level instead of individual level. This allows teams to see how they rank, in terms of their accomplishments, against other teams. Hence, teams have benchmarks to surpass or meet. This prevents underperformance and raises motivation, enabling teams to be more creative and productive (Paulus et al. 2013).

Conclusion

Collaboration with others via computer-mediation is becoming an everyday reality for more and more of us, and offers the possibility of increasing collective intelligence beyond what is possible for traditional face-to-face teams. Initial research on the performance of online teams was mixed, but more recent research on collective intelligence suggests that the work of such groups is fostered by the same qualities that foster the work of traditional face-to-face teams—namely, social perceptiveness and equality of participation. These group attributes enhance the quality of both convergent and divergent thought, and can be facilitated by a range of established as well as newer tools in online settings.

Convergent thought is critical to generating consensus and enhancing decision making, and is most directly fostered by social perceptiveness. Social perceptiveness can, in turn, be enhanced through tools that amplify what might otherwise be subtle signals, and provide group members more contextual information about others. The ability to signal status, to communicate about mood and other events that might impact members’ contributions to the group can help remote members sense what might otherwise be subtle cues, and adjust to these external influences. Beyond these explicit signals, integrating automatic or implicit signals to highlight areas of greater uncertainty or disagreement (such as text that is repeatedly revised, or data that has not yet been reviewed) is an example of how human-computer environments can be designed to further enhance collaborative work.

Divergent thought is essential to creativity and innovation, and requires the opposite conditions to those necessary for convergent thought to flourish in a group. Divergence is enhanced through periods of relatively isolated brainstorming and idea generation. It is directly fostered by the same tools that also enhance greater equality of participation among group members. Many readily available online tools serve to enhance this area of team collaboration, as online environments are well-designed for asynchronous work. Newer tools, such as real-time feedback about relative member contributions to group collaboration and collective products, can help groups preserve equality of contributions and divergent thought when collaborating in real time.

Newer areas of research in this area are inspired in part by work on stigmergy; that is, the ways cooperative animals coordinate by leaving and sensing signs in a shared environment. We know that humans sense all kinds of subtle, nonverbal cues from one another when collaborating, and that the ability to sense these cues also translates to online environments to facilitate collective intelligence in online groups. It is exciting to contemplate the ways in which computer-based tools and interaction platforms can compensate for the deficits that human teams frequently experience in sensing and interpreting such signals, to raise the level of collective intelligence even beyond what is normally observed in high performing, face-to-face groups.

The challenge for those of us who want to encourage the success of human-computer collectives is to understand how these tools can be honed to better facilitate the fundamental processes for collective intelligence. Doing so can enable an even broader level of participation, a trend that stands to benefit us all.