Keywords

1 Introduction

Gamification is a recent strategy used in several contexts where the human factor is a key for the success of the different initiatives. In particular, gamification is an alternative to increase the motivation and commitment levels in work teams.

The propose of use gamification in software process improvement initiatives is achieve high levels of motivation to work with team members, integrate participants to increase technical knowledge and increase progress indicators of project teams [1]. However, the procedures and methods for creating gamified environments are useful tools but is important have mechanisms to verify the incorporation of the principles, fundamentals and gamification elements in new gamified environments. Such verification is fundamental for assure the transformation from a traditional environment to a gamified environment.

Therefore, we develop a proposal for the assessment of gamified environments for assure to meet the purposes, goals, principles and elements that are defined as gamification fundamental components [1]. Our proposal is oriented towards the definition of an assessment framework for gamified environments that comprises the identification and adoption of gamification principles, as well as a design method for gamified activities [1].

In this paper, we present a gamification assessment model for implement the framework looks for calculate the level of gamification of an environment as an indicator of the level of incorporation of gamification fundamentals in such environment. The proposed model is applied in an environment called Video Scrum.

Video Scrum is a simulation of the Scrum framework. This simulation is considered a game very effective for teaching and demonstrating main Scrum concepts.

The structure of the paper is as follows: Sect. 2 presents the main concepts which fundaments the proposal. Section 3 contains a brief description of the assessment framework proposal. Section 4 presents the gamification assessment model. Section 5 describes the application of the gamification assessment model in the case study Video Scrum and the conclusions and future work are presented in Sect. 6.

2 Background

2.1 Gamification

Gamification is a growing trend where the human factor is fundamental in terms of motivation and engagement of the work teams [2,3,4]. Such trend can promote in individuals’ social interaction, quality and productivity of his actions [5].

Gamification can be defined as: (i) an approach that involves the same psychological experiences of the games [4]; (ii) the application of the same principles and environments of the games, regardless of expected results [2] and (iii) a process implementing motivational affordances for obtain psychological and behavioral outcomes in individuals [6,7,8].

The growing interest in gamification as a strategy for process improvement in organization is evident because: (i) the growth of computer game industry over the last two decades motivating individuals to play; (ii) the pervasiveness of social media and mobile and web-based technologies is changing the way how individuals interact inside organizations [9]; and (iii) organizations are continually looking new ways for influence the behavior of employees and clients.

However, there are a few well-established theoretical guidelines for design gamified experiences. This problem is the main motivation of this work oriented to propose a framework like a set of tools for: (i) create gamification environments guaranteeing the application of gamification principles and the achievement of the specific goals and, (ii) verify the accomplishment of gamification fundamentals in an existing gamified environment.

2.2 The MDE Framework

MDE Framework is an approach to design games emphasizing the need to understand games mechanics, dynamics and emotions for obtain an improvement in commitment levels from team members, employees and customers [10].

The elements of MDE framework are:

  • Mechanics. Are related with the designers’ decisions for gamify a non-game context such as goals, rules and the guidelines that stablish the boundaries of the gamified environment. Game mechanics are known before the gamified experience begins and are the same for all game participants.

  • Dynamics. Is a result of how the players follow the game mechanics. In other words, game dynamics emerge during play [11]. In the case of gamified environments, the interaction of team members can lead to dynamics like cooperation, competition or conspiracy.

  • Emotions. Refers to the mental affective state and reactions promoted in players when they participate in a gamified environment. In this sense, the main goal for gamification is create the player enjoyment for guaranteeing the continuity of the gamified experience. Some positive emotions promoted by games are excitement, surprise, wonder and amazement.

2.3 Gamification Principles

Oprescu et al. propose ten principles for implement gamification experiences [12]. Table 1 presents the description of each principle.

Table 1. Gamification principles proposal by Oprescu et al. [12]
Full size table

2.4 Assessment Proposal

Gamification is a trend that receives increasing attention from researchers as well as practitioners. For this reason, exists different approaches to faces gamification projects [13] including the phases business modelling, design, implementation and monitoring, and adaptation [15]. However, the activity less mature is the monitoring of the gamification designs. For this reason, Heilburnn et al. [14] suggest the use of gamification analytics tools to measure the success of gamification designs allowing to quantify and understand user behavior improvements [14]. Moreover, they propose the use of semi-structured interviews as a mechanism for obtain a direct feedback of user satisfaction and additional requirements for the gamified experiences.

In [15] the authors identify relevant software solutions for measure the fulfillment of user requirements in gamification projects.

In such proposals, the assessment is centered in collecting data for software applications about user behavior using gamification analytics. However, our proposal is oriented to a generic gamification environment, supported of not by software artefacts. For this reason, the data of participants’ behavior will be collected directly for users through neuroscience equipment. This alternative will allow discover at first-hand how participants react to the different elements of the gamified environment.

3 Assessment Framework Proposal

Software process improvement is an approach used by software industries for increase the productivity and quality of their processes and products. One important issue is the social interaction among the team members for improve motivation and engagement levels in such team members. This situation could be address using gamification like a strategy to improve the social factors related to the software development process. In this sense is necessary to define methods, guidelines and frameworks to assurance the proper implementation of the gamification strategy.

Our framework is a layered structure and is the result of the analysis of different studies, cited in the previous section, which describes gamification elements, components and principles. Such framework is composed by layers containing basic components (fundamentals, methods, procedures) looking for obtain the expected results with the incorporation of gamified environments in SPI initiatives (Fig. 1).

Fig. 1.
figure 1

Assessment framework for gamified environments (Color figure online)

Full size image

This framework is designed with two purposes:

  1. (1)

    Inductive: the creation of gamified environments starting from the identification and adoption of gamification elements to the assessment of the gamified environment.

  2. (2)

    Deductive: the verification of the accomplishment of gamification fundamentals in an existing gamified environment.

The framework begins with a principles identification layer as a conceptual support of the principles adoption layer. The first layer allows to reflect the elements necessary to guarantee that gamification is using adequately as a strategy to improve the participants’ behavior. Starting with the gamified environment design, the validated assessment is carried out based on principles identified, adopted and reflected in the design.

Finally, the framework improves through constant feedback including the four-layer conforming it. Moreover, Fig. 1 shows the way of framework is used according with the purposes mentioned above. The yellow arrow shows the sequence for the inductive purpose, in other words, when the goal is create a gamified environment from scratch.

The orange arrow shows the descending sequence for use the framework (deductive purpose). In this purpose is necessary begins with the environment design layer looking for identifying the game elements in the existing gamified environment. As indicate the orange dotted line we should continue with the procedure indicated in the assessment layer.

The four-layer conforming the framework are: (1) principles identification that will contextualized the gamified environment; (2) principles adoption identified at the previous layer; (3) gamified environment design and (4) assessment of gamified environment validating its design from the principles adopted at previous layers.

The framework layered-structure is defined identifying for each layer three key elements: (a) base component, (b) procedural component and (c) results component.

Base component of a layer includes the fundamental aspects for the use of the framework. Each layer will have a basic starting point, so that the framework user can establish the application basis of each layer.

Procedural component of a layer is equivalent to methods, procedures and guidelines allowing obtain each layer goals and expected results.

Results component of a layer includes templates for indicating to framework user the structure of expected results in each layer.

The results component is a unique template that will be built layer by layer. The construction of such template is the result of the applications of the instructions of the procedural component of each layer, so in each results component the template is improved with respect to the previous layer [17].

4 Gamification Assessment Model for Implementing Framework

The framework includes an assessment gamification model with the purpose of determine the level of gamification of an environment. The goal of this model is implements each framework layer providing variables and a set of equations for obtain gamification level indicators.

The model has the following components: (a) terms and variables definitions, (b) equation set, (c) measurement rules and (d) results reports template. The relationship between components is shown in Fig. 2.

Fig. 2.
figure 2

Model components

Full size image

4.1 Terms and Variables Definitions

This component describes the terms associated with the model as well as the procedure for calculate the model variables (Fig. 3). Besides, such component include the definition of variables and constants (Table 2) and the Likert Scale used for determine the evidence grade value (Grid) necessary to measure Principle Weight.

Fig. 3.
figure 3

Model procedure

Full size image
Table 2. Variables and constants definitions
Full size table
Table 3. Likert scale for grade value evidence (\( Gr_{id} \))
Full size table

4.2 Equations Set

This component presents the equations of the model. Each variable represents a measure necessary for obtain the value of the variable Principle Weight (\( W_{\text{id}} \)) required for apply the established measurement rules. Table 4 presents the equations of the model.

Table 4. Equations set
Full size table

4.3 Measurement Rules

This component describes three rules for determine the gamification level. Such rules are defined from the variable \( LW_{id } \) which corresponds to the percentage value of Principle Weight (See Table 5).

Table 5. Measurement rules
Full size table

4.4 Results Report Template

This component shows the suggested template for report the results of all framework layers. This template shows gamification principles and evidence identified, and the calculation of gamification level for the environment under assessment (See Table 6).

Table 6. Template for result component of gamification assessment model
Full size table

  • Column 1 refers to the principle identifier (ID) when the principles of framework identification layer are used [16]. According to the procedure of this layer, if exist more principles identified its suggested use the same consecutive.

  • Column 2 corresponds with the principle name included in framework identification layer [16].

  • Column 3 refers to the variable \( Gr_{id} \) defined according to a Likert scale. See Table 3.

  • Column 4 is an initial identification of the presence of the principle in the gamified environment.

  • Colum 5 refers of the MDE Framework element related with each principle.

  • Column 6 includes a brief description of each principle in the environment under assessment.

  • Column 7 presents the identification of environment elements related with each principle.

  • In Column 8, the Principle Weight \( W_{\text{id}} \) is registered (See Eq. (1)).

The procedure of the gamification assessment model ends with the calculation of gamification level for determinate if the environment incorporate the gamification fundamentals totally, partially or not at all according to the measurement rules.

5 A Case Study Video Scrum

In this section, we present a case study of the framework application using Video Scrum as a gamified environment. The goal of this case study is apply the assessment framework for gamified environments in a deductive way. In other words, use the framework for verify and stablish if such environment can be considered as a gamified environment. This goal is obtained thanks to the application of the assessment gamification model in such environment.

Table 7 shows the resultant template for the assessment layer for this model. Such table include the results of the execution of the procedural components of each framework layer.

Table 7. Template for gamification assessment model
Full size table

According to the procedure defined for the gamification assessment model (See Fig. 3) the next step is calculate the level of gamification with the data obtained in Table 7, for obtain the comparison data and determine the gamification level according to the measurement rules.

The application of Eq. (1) for Video Scrum results in (1) \( W_{\text{id}} = 668.75 \) and according to Eq. (2) \( LW_{id } = 13.375 \). The comparison of \( LW_{id } \) with the rules previously presented indicates that Video Scrum is a partially gamified environment.

According to our proposal and model results, Video Scrum simulation applies basic principles gamification and use gamification elements established in framework MDE. It is possible to demonstrate that Video Scrum is a simulation that responds to principles such as those defined by Oprescu and gamification elements established in frameworks like MDE. In fact, Video Scrum structure facilitates identification of components of a gamified instrument defined in framework design layer [17].

6 Conclusions and Future Work

This work proposes an assessment framework for gamified environments aims to facilitate the verification of principles and gamification elements identified and adopted as a part of the design of gamified activities included in a gamified environment for software process improvement.

In this work, we present a gamification assessment model for implement the framework with the following characteristics: (1) a set of variables and constants, (2) a set of equations for calculate each variable value, (3) measurement rules for determine if the environment incorporate gamification partially, totally or not at all, and (4) a results report template that is the result of the execution of the procedural component of all framework layers. The value of the variables is determined for evidences of the incorporation of gamification principles and the existence of game elements associated with each principle. This model is a first approximation for calculate the gamification level of an environment existent.

As a future work, we suggest: (1) apply the gamification assessment model of an environment built from the scratch using the framework, (2) refine the framework assessment layer including not only the gamification assessment model but also HCI measures and approaches for assessing the users’ behavior and (3) evaluate results obtained for Video Scrum case study respect to other gamification proposals.