Designing collaborative multiplayer serious games

Abstract

The idea of Computer Supported Collaborative Learning (CSCL) is being investigated for more than twenty years. Since a few years, game-based approaches like video games for learning (Serious Games) offer new fields of application. The combination of game-based learning concepts and collaborative learning may enable new, game-based application areas of CSCL, like collaborative multiplayer Serious Games. Designing such games, however, is very challenging as it requires to take into account traditional single player game design concepts, concepts for multiplayer game design, and concepts for Serious Game design simultaneously. Only very few examples of such games exist today. In this paper we describe an approach for the design of game-based collaborative learning scenarios using multiplayer Serious Games. Our approach aims at combining design concepts from the fields of collaborative learning and (multiplayer) game design. Our approach takes into account the requirements of traditional single player games (fun, narration, immersion, graphics, sound), challenges of multiplayer games (concurrent gaming, interaction) and Serious Game design (seamless inclusion of learning content, adaptation and personalization). Furthermore, requirements of collaborative learning are considered, like group goals, positive interdependence, and individual accountability. Our design concept was used to create a collaborative 3D multiplayer game fostering collaborative behavior as a foundation for game-based collaborative learning in small teams. We performed a user study with eight gaming sessions and a total of 23 participants. Results showed that the game enables a collaborative gameplay and fosters collaborative behavior. This may allow us to use a game-based CSCL approach to combine the advantages of game-based learning with those of collaborative learning in future.

1 Motivation

Collaborative learning is a learning concept which is broadly accepted in various institutions of education today. For more than twenty years, the idea if using computers to support collaborative learning is being investigated. However, most of the research in the field of Computer Supported Collaborative Learning (CSCL) deals with e-learning applications or how to use (new) medias like the Internet or email to support collaborative learning. In recent years, game-based learning has become an alternative and a supplement to traditional learning concepts. Various research by e.g. Gee (2003) and Prensky (2001) has shown that Serious Games offer a new field of application which can be utilized to support learning in many fields (learning, sports & health, political education, etc.). Today there is a multitude of Serious Games for learning addressing different target groups. Yet most of those games are for single player use. Only a limited number of Serious Games have been designed with multiplayer support due to the lack of concepts for multiplayer Serious Games.

The combination of game-based learning concepts in multiplayer Serious Games with collaborative learning may enable new methods of CSCL. The design of such games, however, is challenging. The gameplay has to fulfill requirements of traditional single player games (fun, narration, immersion, graphics, sound), challenges of multiplayer games (concurrent gaming, interaction) and Serious Game design (seamless inclusion of learning content, adaptation & personalization). Furthermore, requirements of collaborative learning have to be considered, like communication and social skills or a proper group setup.

In this paper we describe an approach for collaborative multiplayer Serious Games which enable game-based collaborative learning. As a first step we developed a game design for multiplayer Serious Games fostering collaborative behavior among players. The game design takes into account both the design challenges of multiplayer games and of collaborative learning. Our approach attempts to fulfill the requirements for cooperative work as a prerequisite of collaborative learning while following the design guidelines for collaborative games found in literature.

We used this approach to design a 3D multiplayer Serious Game with a collaborative gameplay as a foundation for collaborative learning. We designed a collaborative game for four players including several tasks focusing at triggering collaboration. In order to accomplish those, the players need to successfully communicate and coordinate their actions. The whole gameplay is designed in a way such that the players always need to function as a team in order to succeed.

We performed a first evaluation focusing on playability and user experience. We also observed player behavior in terms of communication among the team members and logged relevant game data like player actions throughout the game. Our results showed that the game is well playable and fun to play. It also enables collaborative gameplay and fosters collaborative behavior like teamplay, coordination between the players and communication. We were also able to observe that teams with one person acting as a team leader during collaborative tasks performed significantly better than teams without a leader.

The rest of this paper is structured as follows: In Section 2, we discuss the state of the art, in Section 3, we describe our approach for game-based CSCL using multiplayer games. In Section 4, we present Escape From Wilson Island, a prototype for our approach, followed by a discussion of first results based on a user centered study in Section 5. We conclude this paper with a brief summary in Section 6.

2 Related work

2.1 Game-based learning

Game-based learning is one of the older fields of Serious Games. Prensky (2001) explained, why using games for learning can be a promising approach. He argues, that motivation is a key factor for learning, which games can provide. He further suggests to combine gaming technology with learning concepts. Gee (2003) argued that

“... schools, workplaces, families, and academic researchers have a lot to learn about learning from good computer and video games. Such games incorporate a whole set of fundamentally sound learning principles, principles that can be used in other settings, for example in teaching science in schools.”

Sandford and Williamson (2005) argued that young people are engaged in learning activities when they are playing computer games which are more complex and challenging than than their school tasks. They also provide an overview over several case studies of Serious Games used at school or for learning in general. They provide a list of characteristics of computer games and how those can be beneficial in school. Another interesting overview of games used in educational research is given by Squire (2003) who describes the various forms and genres of games already being used in classroom so far. Those are mainly ‘Drill-and-practice’ games, simulations, and strategy games. ‘Drill-and-practice’ games are mostly utilized for learning by enriching factual recall exercises in a playful way. Simulation games can be used to simplify complex systems, i.e. laws of physics, ecosystems (Sim Earth¹), or politics. On the other hand, high fidelity simulations can be used for realistic training scenarios as often used by military or e.g. flight simulators. Strategy games, in this context, can be viewed as low fidelity simulation games, where learners can observe, influence and control otherwise unalteralble variables.

Van Eck (2006) provides an overview over Serious Games used for digital learning compairing commercial off-the-shelf games used for Serious purposes with Serious Games designed for learning purposes. He emphesizes the importance to achieve a balance between the needs of the curriculum and the structure of the game. Otherwise either the learning outcomes might be compromised or a game might be used in a way it is not suited for. An exhaustive overview over the use of computer games for learning is provided by Mitchell et al. (2004). They provide motivation for the use of computers in education as well as examples and guidelines for it. Furthermore, they present recommendations for the design of educational computer games.

However, the number of professionally created Serious Games today is quite small. As Zyda (2007) states, “Today’s game industry will not build a game-based learning infrastructure on its own. It got killed in the early days of edutainment”. This may be true or not, according to IDATE,² the global Serious Games market in 2010 was only 1.5 billion €, whereas the entire video gaming market in 2009 was about 60 billion $. However, although a large number of today’s Serious Games are created in universities with naturally smaller budgets, according to IDATE “... we can expect to see the business worlds interest in serious games increase around 2013 ...”

However, the focus of educational games in the last decade, especially concerning learning games, was mainly on simple simulation games (TechForce³) or learning adventures (Geographicus,⁴ Winterfest⁵). Those games were created as a playful alternative to learning facts by heart or to provide a playful environment learning through trial and error (e.g. physics games). But most of these games are traditionally single player games, especially in the Serious Games sector. This means a limitation for the use of those games since they do not support collaboration.

2.2 Collaborative learning

The concept of collaborative learning is being discussed among educators for decades. Collaborative learning is used in schools today in various forms, like joint problem solving in teams, debates, or other team activities. According to Dillenbourg (1999), one definition for collaborative learning is “a situation in which two or more people learn or attempt to learn something together”. Roschelle and Teasley (1995) define collaboration as “a coordinated, synchronous activity that is the result of a continued attempt to construct and maintain a shared conception of a problem”. Compared to the definition of cooperation by Dillenbourg (1999),

“In cooperation, partners split the work, solve sub-tasks individually and then assemble the partial results into the final output”,

this is much more than just cooperation. Dillenbourg defines collaboration as follows: “In collaboration, partners do the work ‘together’ ”, Dillenbourg (1999).

The idea of collaborative learning is to make learners interact in particular ways such that certain learning mechanisms are triggered. Therefore, several mechanism to enhance the probability of these interactions to occur are currently being researched. These were initially defined by to Dillenbourg (1999):

• Setup of initial conditions: (group size, gender, same viewpoint vs. opposing viewpoint)

• Role-based scenario: problems which cannot be solved with one type of knowledge

• Interaction rules: Free communication vs. pre-defined communication patterns (see also Baker and Lund (1997))

• Monitoring and regulation of interactions: Need for specific tools for the teacher

Johnson and Johnson (1994) identified five essential elements which foster cooperative work in face-to-face groups. These are often cited as “five components that are essential for collaborative learning” (see Zea et al. (2009)):

• Positive interdependence: knowing to be linked with other players in a way so that one cannot succeed alone

• Individual accountability: individual assessment of each student’s performance and giving back the results to both the group and the individual

• Face-to-Face promotive interaction: Promoting each other’s success by e.g. helping, encouraging and praising

• Social skills: Interpersonal and small group skills are vital for the success of a cooperative effort

• Group processing: Group members discussing their progress and working relationships together

For the success of collaborative learning, both with or without the use of a computer, it is essential that the collaborative learning environment enables and fosters those elements.

2.3 Computer supported collaborative learning

The idea of using computers to support learning arose in the 1980s, with computers as tools mainly for writing, and organizing. In the 1990s, with the Internet and network technologies arising, new ways of communication and collaboration emerged. Intelligent tutoring systems were designed. However, the main task for a computer was being a medium for communication, in form of email, chat, forums, etc. (see Stahl et al. (2006)).

In recent years, many forms of CSCL have been designed and used in curricula at schools. Collaborative writing Onrubia and Engel (2009) is one such form, where learners collaboratively create a document using computer technologies. Furthermore, today many web technologies are used for CSCL, like Forums or Wikis (see Larusson and Alterman (2009)). Haake et al. (2004) provide an exhaustive compendium about CSCL environments, didactical concepts, and the use of CSCL in school and other learning scenarios.

2.4 Game-based collaborative learning

With the upcoming of virtual worlds like Second Life ⁶ or private virtual worlds like IBM Virtual Collaboration for Lotus Sametime,⁷ research also focused on using those as collaborative learning environments (Nelson and Ketelhut 2008). As they are very popular and often freely available, also Massively Multiplayer Online Games (MMOG) have been used as environments for collaborative learning scenarios. Delwiche (2006) held online courses in the MMOG Everquest ⁸ and Second Life teaching about videogame design and criticism. Hämäläinen et al. (2006) tried to find out whether the characteristic features of 3-D games can be used to create meaningful virtual collaborative learning environments. Zea et al. (2009) presented design guidelines enabling incorporation of features of collaborative learning in the videogame development process based on the five essential elements for collaborative learning stated by Johnson & Johnson. Voulgari and Komis (2008) investigated the design of effective collaborative problem solving tasks within MMOGs, and Rauterberg (2002) performed a test about collaboration in MMOGs finding out that communication is essential for effective collaboration. An approach for a 3D collaborative multiplayer Serious Game for learning with freely definable learning content is presented in Wendel et al. (2010).

2.5 Game design

Computer game design is a well researched field, with lots of literature available (Crawford 1984; Salen and Zimmerman 2004). Those books cover topics like game goals, how to create immersion, graphics, sound, and network technologies. In the field of Serious Games there are additional challenges to game design. Serious Games for learning not only have to fulfill the same requirements as other games, but they also have to equip the player with knowledge. To solve this problem, Wendel et al. (2011) proposed a set of guiding principles for Digital Educational Games design focusing on the seamless integration of learning content in Serious Games. Kiili (2005) proposed a gaming model for educational games based on flow theory by Csikszentmihalyi (1991). Said (2004) proposed a model for children, which presents five factors necessary to create an engaging experience for children. These factors are ‘Simulation interaction’, ‘Construct interaction’, ‘Immediacy’, ‘Feedback’, and ‘Goals’. Kelly et al. (2007) describe how to create a Serious Game for teaching focusing both on traditional gameplay questions and on the integration of learning tools. Manninen and Korva (2005) proposed an approach for puzzle design for collaborative gaming along with an implementation in the collaborative game eScape. Harteveld (2011) describes various aspects to be considered when designing Serious Games. The book covers gaming foundations, how to define the real world problem, the necessity to define the purpose of the game, and gives tips about interesting choices during Serious Games design.

A different approach for collaborative game design is made in Zagal et al. (2006). They analyzed a collaborative board game and identified important lessons learned and pitfalls when creating collaborative games. They finally try to convert their findings on the design of collaborative computer games:

• Lesson 1: To highlight problems of competitiveness, a collaborative game should introduce a tension between perceived individual utility and team utility.

• Lesson 2: To further highlight problems of competitiveness, individual players should be allowed to make decisions and take actions without the consent of the team.

• Lesson 3: Players must be able to trace payoffs back to their decisions.

• Lesson 4: To encourage team members to make selfless decisions, a collabora- tive game should bestow different abilities or responsibilities upon the players.

• Pitfall 1: To avoid the game degenerating into one player making the decisions for the team, collaborative games have to provide a sufficient rationale for collaboration.

• Pitfall 2: For a game to be engaging, players need to care about the outcome and that outcome should have a satisfying result.

• Pitfall 3: For a collaborative game to be enjoyable multiple times, the experience needs to be different each time and the presented challenge needs to evolve.

2.6 User experience

The term “User Experience” describes the whole experience a user has by playing a game. This includes aspects of cognition, emotion, physiology and so on. So user experience is a versatile construct. Often good user experience is connected to concepts like immersion or flow (Nacke 2009) or usability norms, but other aspects like autonomy and competence (Przybylski et al. 2009) are discussed, too. But an all-embracing definition of (good) user experience has still to be found.

3 Our approach

As shown in the previous section, the concept of collaborative learning is known for many years and has also been combined with computer technology (CSCL). Furthermore, many game-based learning applications or Digital Educational Games (DEGs) are being used for various fields in education. However, to the best of our knowledge, there are no multiplayer Serious Games for collaborative learning so far.

In our approach, we want to consider aspects from collaborative learning, Serious Game design, and multiplayer game design simultaneously. In order for collaborative learning to take place in a game-based learning environment, the Serious Game must fulfill the requirements for collaborative learning. As we showed in Section 2, there are several requirements which need to be met in order for collaborative learning to take place (e. g. requirements of cooperative working, see Johnson and Johnson (1994)). Furthermore, in a Serious Game, requirements regarding game design (c.f. Salen and Zimmerman (2004)) and learning content integration have to be met (c.f. Wendel et al. (2011)). Therefore, our combined approach integrates the requirements for cooperative working into the game design to create a Serious Game for collaborative learning based on one of the most popular Serious Gaming genres for learning: 3D virtual worlds.

Resulting from the literature review shown in the previous section, we developed the following concept ideas. They are designed in a way such that they match the necessary elements for cooperative working of Johnson and Johnson (1994): (Positive interdependence, Individual Accountability, Face-to-Face Promotive Interaction, Social Skills, and Group Processing). Furthermore, they take into account the lessons and pitfalls of Zagal et al. (2006) as described in Section 2 and the design guidelines stated in Zea et al. (2009).

• Common goal/Success: The goal of the game should be designed in a way such that success means success for all players. It should not be possible to win or lose the game alone.

• Heterogeneous resources: Each player should have one or more unique tools or abilities enabling him/her to perform unique tasks in the game which other players cannot perform, e.g. only the player with the axe can fell palms in order to get wood for building the hut, the raft or for fire.

• Refillable personal resources: In order to create a certain tension, there should be certain refillable resources (e.g. a health or a hunger value) which slowly deplete automatically or when players act dangerously. Furthermore, they should be influenceable in a way such that players can help each other (e.g. food could be gathered by one player and then be given to another player to prevent him/her from starving). This not only creates a general tension towards success, but also a tension between players and thus, a place for interaction and room for decisions.

• Collectable and tradeable resources: There should be resources in the game world necessary for the players to win the game. These resources should be tradeable between players in order to create space for decisions to negotiate or collaborate (e.g. giving a resource to another player for the common good of the team or trading resources between players).

• Collaborative tasks: If all tasks could be solved by one player, there would be no need to collaborate. So there should be tasks which are solvable only if players act together. Those tasks may include the heterogeneous resources described above to create a need for certain players to participate in team tasks. This may cause a need for discussion among players when the group depends on one individual.

• Communication: It has been shown that communication is vital for collaborative learning. So the game should provide a way for players to communicate (e.g. chat system, voice communication). While voice communication might be easier for most players, a text-based chat system might be easier to evaluate. Also a third party tool for communication like Skype, TeamSpeak or Mumble could be used. Apart from the technical availability of communication, the game design should enforce communication, i.e. through division of information (one player receives information which is important for another player).

• Ingame help system: The game should provide help to the players when they get stuck. The easiest way is a popup when players fail a task or it takes them too long to solve it. Furthermore, the help system should be triggerable by the players. A more sophisticated but also more immersive way (because it does not interrupt the game flow) is to include help in the game itself, e.g. by having ingame characters (NPCs) providing help when needed.

• Scoreboard: A scoreboard should show both individual efforts and team efforts at the end of the game. This may help players judge the overall success (e.g. by comparison with other teams or previous attempts) and each player’s contribution to the team performance. The individual score may function as a motivator for selfish actions which helps to make collaboration not self-evident.

• Trading system: Players should be able to trade items among each other. This creates space for decisions for or against collaboration. Especially regarding the scoreboard, there might be a motivation for and against trade at some times.

3.1 Reference to related work

As a next step we want to discuss our concept in relation to the lessons and pitfalls of Zagal et al. (2006):

• Regarding lesson 1: By having an individual score board for each player, we create a competitive element. Individual scores can sometimes be achieved by helping the group (e.g. when participating in solving a task together), or they can be selfish (e.g. when gathering resources).

• Regarding lesson 2: By the nature of the game (3D third person, open environment), each player may move and act freely. No player is forced to perform any action, although some actions are not possible without other players’ consensus.

• Regarding lesson 3: The results of decisions are always visible to the players as they are immediate. A player may e.g. decide to help solving a task or to gather resources. If he/she decides not to help, the group might not be able to complete the task, which will slow them down. The selfish player, however, will have a personal resource refilled.

• Regarding lesson 4: By providing players with heterogeneous resources (tools), each player has a different ability and responsibility.

• Regarding pitfall 1: The nature of a 3D third Person game makes it very difficult for one person to fell decisions for all players, however leader roles will certainly be possible and relevant.

• Regarding pitfall 2: As all players either win or lose together, each player should care about the outcome assumed he/she has the proper motivation to play at all. Such a motivation is provided by the narrative background story, which creates a similar motivation to ‘win’ the game as in any other game.

• Regarding pitfall 3: Although the core game itself will not change, when played repeatedly, the free game world and the free sequence of available actions can create completely different progressions of the game in different runs. Also, playing with different players will be a completely different experience for a player than a previous game. Furthermore, the difficulty of the game can be influenced by a teacher/trainer both before and during runtime, so that more experienced players will still find the game challenging.

Finally, we discuss our concept in relation to the cooperative working requirements by Johnson & Johnson (1994):

• Positive interdependence: As many tasks are only solvable if players work as a team, we think players will realize quickly that they cannot succeed alone.

• Individual accountability: By introducing an individual scoreboard, the game can assess each players personal performance. As the scoreboard is visible to the whole group, the results are given back to both the group and the individual.

• Face-to-Face Promotive Interaction: Although the game itself does not encourage promoting behavior like encouraging or praising fellow players, the game enables players to do so. Players can help, encourage, or praise other players via chat. Furthermore, they can help other players through their actions. A player who decides to help his/her fellow players, will significantly improve his/her chances of success.

• Social Skills: We believe that interpersonal and small group skills can be trained by use of this game if observed and guided by a teacher. The game provides lots of opportunities for practicing social skills both in speech (chat) or behavior (gameplay).

• Group Processing: Again, this is not a process created through the game itself, but the game provides the players with the possibilities to discuss their progress and relationships (chat) and to reflect on them (scoreboard).

4 Escape from Wilson Island

4.1 Game properties

Escape from Wilson Island (EFWI) was created using the Unity3d⁹ game engine. For the first prototype we used standard assets which are freely available for Unity3d. The game is scripted in C#.

The game can be classified as a 3D 1st/3rd person action adventure game. The focus of the game is the collaborative gameplay itself, including social skills like teamwork, coordination, or communication. In Fig. 1, the player’s view is shown including the GUI elements. The GUI contains a chat window, and a minimap to the left. To the right, the relevant player attributes ‘health’, ‘saturation’, and ‘fitness’ are shown with the player’s inventory below. The player’s avatar is in the center of the screen.

Fig. 1

Player’s screen with GUI elements of Escape From Wilson Island

We created a collaborative game for a small group of up to four players. Therefore, we chose a secluded game world in form of an island. This allows us to create natural looking boundaries (the sea) for the players’ ‘playground’. Most of today’s Massively Multiplayer Online Role-Playing Games (World Of Warcraft, Guild Wars,¹⁰ etc.) or Virtual Worlds (Second Life) are played in a 3D 3rd person perspective. So many players are used to this perspective. For this reason, we decided to use this perspective in Escape From Wilson Island.

As a narrative background, we chose a well known scenario (‘Robinson Crusoe’) which creates a setting that motivates the players to collaborate: The players stranded on a deserted island and have to escape from there. Therefore, they have to reach a neighbored island with a high mountain to ignite a signal fire there. The starting island contains resources like trees for wood, bushes with berries for food, or NPC herons running around randomly. Furthermore, the island is designed in a way such that it seems realistic, e.g. sand strands with palms near the water, hills and other trees inland. The island has both green meadows and rocky spots which look rather dangerous. In order to reach the other island they first have to build a raft for which they need wood and other items. Before they can start to build the raft, they have to ensure that they can survive on the island. Hence, they have to build a hut to sleep and gather berries or hunt herons for food.

The narrative background is told to the players in a short intro showing how the players strand on the island. At their starting point, they are welcomed by a Non-Player Character (NPC) which tells them their goal (leave the island) and gives hints on how to reach it (build a hut, gather wood for a fire, gather berries and hunt herons to have food, finally build a raft and reach the neighboring island).

To provide an ingame help system for the players, the NPC can be found at every time on the island and players can ask a set of predefined questions in form of dialogues. The NPC is also integrated in the game in form of a person who gives quests to the players (as additional tasks to be solved) when the game develops.

The game can be played in three camera views: 1st person, 3rd person (camera following the player) and isometric (top down). We decided to add different camera views as different players have different preferences and in various situations it can be helpful to change the camera view. The game controls are standard ‘W-A-S-D’ for movement with mouse cursor for the direction. Camera views can be switched by use of hotkeys. Interaction with game items is done by mouse click (mouse cursor changes whenever an action with an item is possible when the mouse is moved over the respective item).

In the following list, we show how we applied our design guidelines:

• Common goal/Success: Players can only escape together, not one player alone. An outro will be played at the end of the game as a reward if the games was finished successfully.

• Heterogeneous resources: Each player has one unique tool (axe, map, whistle, hunter’s badge) enabling him/her to perform unique tasks in the game which other players cannot perform, e.g. only the player with the axe can fell palms in order to get wood for building the hut, the raft or for fire.

• Refillable personal resources: Need for food, health, and fitness; The players’ avatars need to eat from time to time in order not to starve. Furthermore, they have a fitness value which regenerates when the avatars are sleeping. The fitness value is influenced by running and working. A lack of fitness slows players down. The health value is negatively influenced when players are starving or when they are drowning. It is regenerated when eating or sleeping.

• Collectable and tradeable resources: Wood, Berries, Meat; There are two ways to get food: A player can gather berries from a bush which restore a small amount of saturation, or the players can hunt a heron, which will give them some pieces of meat that can be cooked. Each piece of cooked meat restores a large amount of saturation, making heron meat a lot more valuable than berries. Wood can be gathered from palms if they are chopped.

• Collaborative tasks: Some tasks are only solvable if players act together (see Section 4.2).

• Communication: Players are able to communicate with each other via an integrated chat. We integrated a simple chat window for the players where a player can select the listeners. It is possible to chat with only one other player, with a set of other players or with all other players. The chat window is always visible in the lower left corner. Of course, it is also possible to allow players in the same room to talk with each other or to use a third party tool for communication like Skype, TeamSpeak or Mumble.

• Ingame help system: We integrated an NPC, which is living on the island. The NPC’s task is to guide the players through the game, giving them hints when necessary and answering some game related questions. The NPC communicates with the players via a predefined structured chat system or can be controlled by another player, e.g. a teacher. It then is able to talk to the players via a chat system or the teacher can design structured dialogs ad-hoc at runtime.

• Scoreboard: At the end of the game, each player will have an individual score that is visible to the whole group. The score depends on the number of (potentially) helpful actions performed during the game like gathering berries, carrying wood, building the raft, helping to catch a heron, etc.

• Trading system: Every player has a personal inventory where he/she can place items gathered throughout the game. Those items can be given to other players by placing them in a community box accessible by every player. Access to the chest is trust based, there is no control mechanism to prevent someone from taking something out of the chest.

To make the ingame feeling more realistic and to increase immersion we included some sound effects (footsteps, sounds for opening boxes, etc.). Additionally, the NPC dialogs are spoken by a real human voice simultaneously to the text. In Fig. 2, players are shown during a collaborative action (carrying a palm) while another player is felling another palm.

Fig. 2

Players carrying a palm; another player felling a palm

4.2 Collaborative tasks

Felled palms can only be carried in team (dependent of the size, this requires 2–3 players). Players need to position themselves at one of the ends of the palm or in the middle (see Fig. 2). If all positions are filled, the players can lift the palm. In order to carry it, the need to synchronize their movement. Therefore, no player me leave a certain cone around its position while moving. If one player steps out of his/her cone, i.e. he/she moves not in step with the other carrying players, the palm is dropped.

Herons can only be hunted in team, as they have to be surrounded, which needs at least 2 players but is easier if more players take part in it. One player (having a special item) can see possible escape routes. Players need to close those by clever movement while trying to approach the heron. This way they surround it. This task requires a lot of communication and coordination among team members.

The raft can only be steered when all players are participating, as each player can only sit in one corner, steering the raft towards his/her corner when paddling. If the players want to move the raft straight forward, only the two front players may paddle and they have to paddle at the same time. If they want to move to the right, only the player sitting in the front/right position may paddle. As the route from one island to the other is full of currents which only the player having the sea map can see, players need an extraordinary coordination to solve this collaborative task (see Fig. 3).

Fig. 3

Four players steering a raft together. Currents shown as arrows on the water

5 Results

5.1 Setup

For evaluation of EFWI, we made a user centered study with 23 participants, age 19 to 38 years ($m = 21.38/\mathrm {sd} = 3.98$), 18 females, 10 playing digital games at least 1 hour per weak, all of them were students. We used the following setup: After a 5 min introduction into how to play the game (goal, controls, etc.), games were played in groups of four players per game (one group with only three players). All players were located in one room, observed by two members of our team. Each gaming session took 30 min after which the participants had to stop playing and answer a questionnaire. The questionnaire retrieved information about User Experience (UX) and game design. Furthermore, the participants were asked to freely add comments about what they liked, disliked or missed. In addition, we logged game relevant information about player performance, success, and player behavior. We set up goals for all teams requiring collaboration to be reached: survive, build a hut, hunt a heron, and build a raft. In order to build the hut, one player had to fell palms, and three players had to carry the palm to the destination (which is difficult as players have to coordinate their movements in order not to drop the palm). To capture the heron, at least three players had to surround it while pushing it towards a cliff. One player (the one with the hunters badge) can see possible escape routes of the heron which players have to close by moving cleverly before the heron runs away. This requires a significant amount of coordination. Also, before the players can surround the heron, the player having the whistle has to attract it. The whole process is only possible if the team works collaboratively.

5.2 Questionnaire

To evaluate the effect of the game on the user we used a questionnaire measuring the user experience and the game design quality after playing the game for 30 min. This questionnaire abstracts different aspects of user experience and usability (for a summarization of aspects of flow theories see Nacke (2009)). So the user experience score contains seven subscales of user experience (negative emotion, positive emotion, cognitive load, motivation, immersion, flow and arousal), while the game design score contains ten subscales (quests, environment, effectance, curiosity, personalization, interface, feedback, social needs, storytelling and structure). Each subscale includes tree questions to the topic of the subscale (e.g. boredom, frustration and anger for negative emotion). Each question-item has to be answered on a 1 to 10 point scale. A first evaluation of 145 questionnaires shows a Cronbach’s Alpha = .93 for the overall user experience score of this questionnaire. This means the overall user experience score (the mean of the 21 user experience questions) seems to build one homogeneous factor. This user experience and game design measurement followed an explorative approach. So the main goal was to detect weaknesses in the game, which could be improved.

Following Nacke (2009) (p. 146), we define immersion as

“immersion in the game world derives from the player becoming the game character, in the sense of the player having the experience of acting within the game world”

and as Przybylski et al. (2009) suggest, perceived autonomy and competence may be an important source of positive emotions.

Believing that there could be a link between the game design and the UX, we split the questionnaire in items testing elements of game design (like “ability to choose between different tasks on his/her own”) and items testing elements of UX (like positive and negative emotions) to have a closer look on the possible link between UX and game design. Seven dimensions of UX and ten dimensions of game design were asked in the questionnaire. Every dimension includes three items, which could be answered on a ten point scale (10 = “I agree”, 1 = “I don’t agree”). The mean of the 21 items (see Table 1) of the seven UX scales can be interpreted as a value of the UX, showing Cronbach’s Alpha = 0.82 in this study. So the questionnaire seems to measure one construct (User Experience). The 30 items (see Table 2) of the ten game design scales can show specific weaknesses of the virtual world.

Table 1 User experience questions

Table 2 Game design questions

5.3 Critics and suggestions

In addition to the questionnaire, we asked the participants to give feedback in form of critics or suggestions for improvement. The most frequently mentioned problems were:

• Improvement of character control (12 / 23)

• Need for a minimap (5 / 23) (was not implemented at that time)

• More camera views (5 / 23)

• Improvement of graphics (4 / 23)

• More interaction with game world / more tasks (4 / 23)

• More ways to differentiate the own avatar from others (4 / 23)

5.4 Game relevant logged data

In order to measure what and how much the player groups achieved during the 30 min of play, we logged several actions: berries gathered, berries ate, palms felled, palms carried, fires ignited, wood added to fire, meat chopped, meat roasted, dialogues with NPC, herons caught. These data was logged for each player and for the group. Additionally, we logged the player chat, so that we could see, which player chatted with which other player(s), and how often. The logged data was saved to log files after the gaming sessions ended.

5.5 Discussion

As shown in Table 1, the participants perceived the game as fun, engaging and motivating through its development. They also were in a convenient condition, felt a loss of time and paid a lot of attention to the game. These are indications for a high perception of immersion. The most negative value was ‘Relief due to failure’, which we interpret like follows: Usually this emotion occurs when a player is in a state of high tension (like in a shooter, or a horror game). Many players feel better immediately after successfully solving the task, because the tension is gone. In EFWI, we have no ‘life threatening’ to the character circumstances, so there is hardly any failure which could reduce a form of tension, which then could cause a form of relief to the player. Also players did not really identify with their avatars and found the game a little frustrating from time to time which we can credit mainly to the difficult controls (see later in detail) and some bugs.

Table 2 shows mostly average or slightly positive values. However, there is a number of very low values, which we discuss here. ‘Own identity’ has an average value of 4.22, which means players do rather not think that they can create an own identity with their character. In fact, there is currently no option to personalize the avatar in an optical way (skins, pieces of cloth, etc.) or by development of virtual character skills as usual in role playing games. Also, the bad value for ‘Interesting characters’ may be explained by this fact. Music is rated very low which is not surprising as there is currently no music in the game. The players seem to evaluate this very negatively which indicates a demand for a background music. However, it is interesting that some players did obviously not miss music as they crossed a value of 5, which is rather neutral. ‘Acknowledgment and status’ is rated with 4.22 on average, which means that players did rather not feel they received the proper acknowledgment for their effort in the game. This may be due to the fact that the scoreboard is only displayed after the game. Also currently, there are no achievements implemented which could display a certain progress to the player.

So, we can assume that all of our player groups had fun at playing, many of them wanted to play on for ‘just a few more minutes’ after the 30 min of play. The good values in the UX part with a mean value of 6.95 and a standard deviation of 1.10 support this hypothesis. The Cronbach’s alpha of the UX part of the questionnaire is 0.82 which indicates that the UX question set belongs to one construct. However, the players criticized some gaming parts like controlling the character or graphics. These are implementation details which we will have to address. The need for a minimap for a better orientation is a real deficiency which we will have to solve. Also more interactions with the game world should be necessary in order to keep the game interesting for a longer time. Additionally, we will have to include more sounds and a meaningful background music.

Although we did not measure success by numbers or recorded the discussions between the players, we can state some interesting observations: All teams were able to achieve at least some if not all of the goals given by us in 30 min for which they had to collaborate. This indicates that it is possible to design collaborative tasks in a computer game for training of collaboration. We could observe the players to talk to each other about problems to be solved in the game, thus discussing their working relationships, helping and promoting each other’s success. Although we did not measure any improvement of social skills like teamwork, coordination, or communication, we observed that all of those skills have been used by the players throughout the game. This indicates that it is possible to specifically improve those skills using a proper game design.

During the gaming sessions, we noticed two different types of groups. Some groups had a clearly visible team leader while others did not have. Teams with a team leader seemed to perform better than teams without. Those teams seemed not to collaborate very good. We noticed that those teams mostly fulfilled solo tasks like gathering berries or felling trees, but it took them very long to carry a palm to the place where the hut was to be build or to hunt a heron. Some teams did not achieve to hunt a heron at all. We intend to further investigate this in a follow-up evaluation centered on team behavior and team leader behavior.

Regarding our design guidelines, we observed that the common goal was clear to the players. The heterogeneous resources made players coordinate their actions and the refillable personal resources (e.g. hunger value) made players help each others. We observed players gather food (berries) for other players, so that those could concentrate on other tasks. Therefore players used the collectable and tradeable resources (like berries or wood for fire). Furthermore, we could observe players to solve the collaborative tasks, which could only be solved in a group (i.e. carrying a palm). All teams significantly improved at performing this task throughout the game which indicates that they improved their coordination (for this task) during play. As all players were seated in one room, they were able to communicate verbally with each other. The ingame help system in form of an NPC was used very intensely by some groups, whereas other groups failed to use it. This indicates that another help option like an always present help button may be useful, too. Due to the time restrictions of the evaluation, the game could not be played to the end, so that no scoreboard could be shown to players. The trading system was used by players, but without an individual goal, there was no motivation not to collaborate, so no real trading could be observed.

Comparing the values of the subscales of the user experience questionnaire and the qualitative answers of the participants it seems obvious that the game design part of the questionnaire covers the most important game elements which were missed by the participants. Comparing the questionnaire used in this study to other existing work (e.g. Nacke (2009)), it seems also clear that even if there are diverse ideas of what user experience is, the core constructs (e.g. positive emotion, negative emotion, flow) are the same.

Our results indicate that it is possible to design and create multiplayer Serious Games which meet the requirements of collaborative learning principles by using principles of collaborative gameplay and to practice collaborative learning in a game-based learning environment, thus combining the advantages of game-based learning with those of collaborative learning. We think that it is easily possible to extend the game(play) of EFWI in a way such that players learn something in the game beyond teamplay and social skills.

6 Conclusion

In this paper we proposed an approach for collaborative gameplay in 3D multiplayer Serious Games as a foundation for game-based collaborative learning. Resulting from an extensive literature review, in our approach we tried to combine game design guidelines for collaborative games with the requirements of cooperative learning. Our resulting approach was a game design for a 3D multiplayer game for 3–4 players with a collaborative gameplay as a foundation for collaborative learning. We implemented a prototype using our game design approach with a 3D game engine and performed a user centered study for evaluation of user experience and game design issues. Results of the evaluation are promising. Players had a lot of fun playing the game despite some minor drawbacks like poor graphics or difficult controls. We could observe the players to be able to solve tasks for which they had to collaborate. We also could observe players making use of social skills like communication skills, teamwork, etc. while playing. Furthermore, we recognized an influence caused by the presence or absence of a team leader. This indicates that collaborative multiplayer games supporting collaborative learning can be promising alternative to traditional CSCL.

Next steps will include the implementation of the demanded missing features in Escape From Wilson Island. Furthermore, we will extend the gameplay with more tasks. Also, it will be necessary to include subject specific tasks to be able to evaluate the learning outcomes of the game (e.g. in form of a pre-post-test). One major feature will be the inclusion of a Game Master component for a teacher/trainer to be able to oversee, control and adapt the game according to his/her professional opinion at runtime. The Game Master can either play a leading role or be a passive and invisible control instance. In a follow-up study we want to evaluate which effects on collaboration the presence of such a Game Master can have and if it can improve the learning performance of players in a collaborative multiplayer Serious Game.