Introduction

The term ‘Gamification’ came into being around 2008, with the creation of products like Bunchball and Ringorang (Olen, 2008). However, at this time, gamification mainly referred to the practice of including the element of rewards, in a non-game context like customer loyalty (Mangalindan, 2010). Since then, researchers have proposed gamification as a solution to improve engagement, influence learner attitude and behaviours; and thereby improve learning outcomes (Landers, 2014). Traditionally, research has focused on studying gamification in the context of education. However, the increasing presence of technology in the workplace and the challenges posed by the current environment, provide a reason to analyse the use of gamification in training activities. Although there exists ample theoretical research in the area of gamification, empirical evidence supporting gamification is still sparse.

The current study attempts to fill this gap by providing empirical evidence for the effectiveness of gamification. The participants for the module were chosen from organizations in India and the researchers chose to use a Solomon four-group design. The design allowed the researchers to understand if the pre-test had any impact on the post-test scores, adding to the validity of the experiment. As organizations look to evaluate and adopt technological solutions, the findings of the current study would help organizations evaluate gamification as a potential solution or tool to fulfil their training needs.

Review of Literature

Gamification has been an area of interest since the idea was born in 2008. Deterding et al. (2011) defined gamification as “the application of game elements to non-game contexts” (p. 9). The definition brings into focus three main characteristics of gamification. Firstly, the concept of gamification deals with game elements. Game elements are like the building blocks of a game. They can be found in all games and every game can be broken up into the game elements that they are made of. According to Kapp (2016), game elements include story, rules, chance, competition and collaboration. Some other elements that have been popularly included in gamification literature are leader boards, badges, points and timer.

The second part of the definition talks about a process of application. Thus, gamification is not an end product but a process. This is one of the key characteristics of gamification that sets it apart from similar concepts of serious games and games in general. Landers (2019) encouraged the borrowing of terms and taxonomy but stressed the need to differentiate between the terms, so as to allow the concept of gamification to grow and be understood.

The last part of the definition talks about the application of the elements in a non-game context. This suggests that gamification applies to a particular type of context. ‘Non-game context’ refers to any scenario that is meant for purposes other than fun. Thus, while gamification is supposed to add fun to a scenario, it must not dilute the primary objective of the activity (Werbach, 2013).

In the organizational context, gamification was found to improve learning (Andriamiarisoa, 2018; MacKinnon et al., 2015; ONeill et al., 2018; Trimblett, 2016). A study by MacKinnon et al. (2015) used a pre-test, post-test control group design to explore how gamification improved learning and learner motivation. Their findings suggest that gamification led to a significant increase in learning outcomes. The use of game elements was also found to help bring about the desired changes in employee behaviour (ONeill et al., 2018; Suh & Wagner, 2017;). Suh and Wagner (2017) used surveys to understand if gamification would increase knowledge sharing in the organization. They evaluated the impact of gamification on the perceived hedonic value of an Enterprise Collaboration System or ECS. Their study suggests that the application of game elements to the ECS would increase the perceived value of the system and encourage employees to participate in knowledge sharing.

Sargent (2017) explored how gamification improved engagement in online corporate training programs. The evaluation was done by comparing the results of a control group and a test group. The findings suggested increased engagement in the group that participated in the gamified module. Sargent (2017) also found that the test group had higher levels of motivation to participate in the program, implying that the use of game elements increased participant motivation.

Theoretical Framework

While literature was key to deciding the methodology of the study, the hypotheses of the current study are primarily based on theory. This section briefly describes some of the theories that support the proposition that gamification would have a positive impact on learning outcomes.

One of the first macro theories to support learner-centric methods was the Self Determination Theory by Deci and Ryan (1980). The Self Determination Theory or SDT was crucial in bringing into focus the role of motivation. In addition, the theory highlighted two dimensions of motivations; autonomous motivation and controlled motivation. Autonomous motivation included the concept of intrinsic motivation and a category of extrinsic motivation that arose from the understanding that there was value to be gained from engaging in the activity. On the other hand, controlled motivation stemmed from the expectation of rewards or punishment. The theory proposed that in order to engage the learner, one had to aim at autonomous motivation and that autonomous motivation was driven by the psychological needs of competence, autonomy, and relatedness. The universal nature of these needs has been explored and their relation to performance and physical and psychological well-being has also been established (Meyer and Gagne, 2008; Ryan & Deci, 2000;). This brought into focus the need to evaluate the impact of the gamified module on learner motivation.

The hypotheses of the current study are primarily based on the Theory of Gamified Learning (Landers, 2014). The Theory of Gamified Learning draws from the Input-Process-Output model proposed by Garris et al. (2002). Accepting the role and importance of learner motivation, Garris et al. (2002) examined the process of creating a motivated learner. According to Skinner and Belmont (1993), while it was easy to identify a motivated learner, it is far more difficult to find and create a motivated learner. The model proposed by Garris et al. (2002) identifies three key steps of the learning process; that would lead to increased learner motivation. According to them, the key was to motivate the learner to revisit the content and thereby improve learning. This could be achieved through the use of game-like features.

This proposition was further refined by Landers (2014) and presented as the Theory of Gamified Learning. The theory is diagrammatically presented in Fig. 1.

Fig. 1
figure 1

Representation of the Theory of Gamified Learning

Full size image

The Theory of Gamified Learning makes five propositions. The first proposition suggests that the instructional material can directly influence learning outcomes. Sufficient empirical evidence exists for the same (Arthur Jr et al., 2003; Seidel & Shavelson, 2007). The second proposition suggests that the attitude and behaviour of the learner can impact the learning outcomes. Rikers and Paas (2005), provided evidence for the same and found that the learning was directly proportionate to effort. Zhao and Kuh (2004) also found that the level of learner engagement would directly impact the learning outcomes.

The third, fourth and fifth propositions are directly related to gamification. The third proposition suggests that the application of the game elements would impact learner attitudes and behaviours. Evidence for this relationship can be found in literature for serious games (Wilson et al., 2009). The fourth and fifth propositions refer to the role of the game elements in the relationship between the instructional material and the learning outcomes. The fourth proposition is that the use of game elements would moderate the relationship between the instructional material and learning outcomes. However, as the term moderating suggests, the elements can only strengthen or weaken the initial relationship and the quality of the original material continues to remain key to the process.

The last proposition refers to the direct role of gamification. According to Hamari et al. (2014), this mediating role is the primary role of gamification in the learning process. Landers and Callan (2011) used the elements to encourage certain behaviours and found that the change in behaviour improved the academic performance of the students.

Drawing from the Theory of Gamified Learning and reviewed literature, the researchers propose the following hypotheses;

  • H1: Learning would be greater for potential learners experiencing a gamified module as compared to a module without game elements.

  • H2: Potential learners would react more positively to a gamified training module as compared to a module without game elements.

  • H3: Learner motivation would be higher for participants of a gamified module as compared to the participants of a module without game elements.

  • H4: Learner motivation would mediate the relationship between gamified instructional design and learning.

  • H5: Learner motivation would strengthen the relationship between gamified instructional design and trainee reaction.

Methodology

The researchers opted for an experimental research design. The main objective of the study was to evaluate the impact of a gamified training module on learner motivation, reaction and learning. To understand the impact, the researchers chose to adopt the Solomon four-group design. The Solomon four-group design allows us to evaluate any impact the pre-test may have on the post-test scores. While the pre-test is crucial in determining the starting point or baseline of the participants, it may increase participant sensitivity to the experiment. To overcome this problem, Solomon proposed the four-group design in 1949 (Navarro & Siegel, 2018). Figure 2 provides a diagrammatic representation of the Solomon four-group design.

Fig. 2
figure 2

Solomon four-group design

Full size image

Source: Flannelly et al. (2018).

The analysis of the data was carried out using the flowchart provided by Braver and Braver (1988) and provided in Fig. 3. All tests were carried out on IBM SPSS.

Fig. 3
figure 3

Analysis of data

Full size image

Participants

The researchers aimed at evaluating the impact of gamification on learning outcomes in the organizational context. For the same, the researchers approached organizations in India. Each group consisted of 15 participants, bringing the total number of participants to 60. The final set of participants included employees of two firms; a privately owned organization and a public sector firm in India. 57% of the participants belonged to the public sector organization and 43% belonged to the private sector organization. The majority of the participants were males (78%). The participants belonged to different levels of management. Junior management formed the majority with 60%, followed by middle management (22%) and senior management (18%). In order to participate in the program, the trainee had to be able to comprehend basic English and had to be employed in a managerial role in the organization. In addition, access to laptops and the internet was a must.

The gamified module was tagged as Module A and the regular module was tagged as Module B. Detailed instruction sheets, one for Module A and one for Module B, were provided to the Human Resources (HR) team, explaining how the participants may register, log in and complete the program. The HR teams then created four groups and randomly assigned participants to each group. Random assignment was possible, given that HR teams did not know which module was gamified and which was not. The first two groups were asked to complete the pre-tests, prior to accessing the training, while the third and fourth groups were allowed to access the program, without completing the pre-test.

Training Module

The training modules were designed on Moodle Cloud. The modules covered the basics of cognitive behavioural training and aimed at helping individuals understand the power of thoughts and how they can be harnessed to achieve personal balance and professional success. The module consisted of videos that the trainee had to go through and learning was assessed through quizzes. The material for both the modules was identical, however, the modules were structured differently.

In the gamified module, the course was divided into levels (Novice, Amateur and Expert). To clear a level, the trainee had to complete a short, timed assessment. On completing a level, the trainee would receive a badge that was displayed in their profile. Figure 4 provides screenshots of the badges page and the different levels (Novice, Amateur and Expert). The gamified module also included timed activities, funny memes, leader boards and powerups. These were included by linking the module to the Quizzes application. Leader boards displayed how the trainee was performing as compared to colleagues. Powerups included bonus points, extra time, bonus questions and other things that the trainee could win by completing tasks correctly and quickly. Figure 5 presents screenshots of a sample leader board and examples of the memes and powerups that were employed. The average time the participants took to complete the program was three hours, approximately.

Fig. 4
figure 4

Levels in the Gamified Module and the associated badges

Full size image
Fig. 5
figure 5

Examples of the leader board, memes and powerups

Full size image

Measurement and Scaling

Learning was measured through the pre and post-test scores. Both the tests consisted of ten multiple-choice questions each. Every correct answer would earn the participant 1 point and an incorrect answer would result in a 0. The pre-test was administered at the beginning of the module and the post-test after completion of the training program.

The instrument to measure reaction was adopted from Brown (2005). The instrument included three items that measured satisfaction with technology, two items that measured enjoyment and two items that measured perception of relevance. All items were measured on a 5-point Likert scale. The questionnaire to measure reaction was administered after the participant completed the module. The instrument had high reliability, as indicated by a Cronbach’s alpha score of 0.930. The questionnaire contained statements like, “The technology interface was easy to use”, “The lecture was relevant to my work” and “Learning this material was fun”.

Learner motivation instrument was adapted from the Training Valence, Instrumentality, and Expectancy Scale or the T-VIES-it by Zaniboni et al. (2011). The instrument consisted of three items that measured Valence, three items that measured Instrumentality and three items that measured Expectancy. Instrumentality reflects the trainee’s belief that performance in training would lead to successful job performance. Expectancy reflected the belief that the effort put into the training would lead to successful training performance and Valence reflected the trainee’s perception of the value to be gained by participating in the training program. All items were measured on a 5-point Likert scale. Learner motivation was assessed midway through the program. The T-VIES-it returned a Cronbach’s alpha score of 0.957, indicating high reliability. The scale included statements like “Attending training activities, I want to improve technical/practical knowledge in my job”, “If I am involved in training activities, I am confident I can improve my ability of initiative” and “Acquiring new skills thanks to training activities, positively influences my performances”.

Results

The descriptive statistics for each variable are presented in Table 1. The skewness and kurtosis values were analysed to check if the data could be considered normally distributed (Kline, 2005).

Table 1 Descriptive statistics
Full size table

The mean values for reaction (R) and learner motivation (M) were found to be close to 4. This indicated that most participants reported high levels of learner motivation and a positive reaction to the training program. The mean value for the test score (T2) was found to be 6.7; thus, the majority of participants scored greater than 50% in the post-test. The skewness and kurtosis values were found to be within the prescribed limits, suggesting that the data could be considered normal. This allowed the researchers to use parametric tests for the testing of the hypotheses.

  • H1: Learning would be greater for potential learners participating in a gamified module as compared to a module without game elements.

For the first hypothesis, the researchers followed the analysis flowchart provided in the methodology section. The first step involved the evaluation of the level of pre-test sensitization. This was checked by analyzing the significance of the interaction between the treatment (Gamified) and pre-test (Pretest). This was carried out using a two-way ANOVA to check for the impact of the interaction term on the post-test scores (T2). The result of the analysis is provided in Table 2.

Table 2 Test of between-subject effects
Full size table

The model summary suggests that the interaction term did not have any significant impact on the final test score. Since there does not appear to be any pre-test sensitization, the researchers proceeded to examine the main effect of the treatment.

The results of the analysis of the difference in the post-test scores (T2) of the experimental (E) and control (C) groups have been provided in Table 3. The results indicate a significant difference in the mean values of the post-test scores (p < 0.05). In addition, the negative mean difference suggests that the mean score of the experimental group (mean = 7.60) was significantly higher than that of the control group (mean = 5.37). Thus, the results suggest that H1 is to be accepted and that learning is higher when learners experience a gamified training module.

Table 3 Independent samples test for learning
Full size table

  • H2: Potential learners would react more positively to a gamified training module as compared to a module without game elements.

The second hypothesis was tested by comparing the mean scores for reaction. An independent samples t-test was used for the analysis. The results of the analysis have been provided in Table 4. The mean scores for the three dimensions of reaction; reaction to technology (Rt), enjoyment (Re) and relevance (Rr) have also been presented.

Table 4 Independent samples test for reaction
Full size table

The results of the analysis suggest that the participants of the gamified module (mean = 4.12) reacted more positively to the training module, as compared to the participants of the regular module (mean = 3.61). The difference in mean scores was found to be significant for all dimensions of the reaction scale, i.e., Reaction to Technology (Rt), Enjoyment (Re) and Relevance (Rr). Thus, participants of the gamified module reported to be more satisfied with the learning experience. This led to the acceptance of H2.

  • H3: Learner motivation would be higher for participants of a gamified module as compared to the participants of a module without game elements.

The third hypothesis was tested by comparing the mean scores for learner motivation (M) and the dimensions of valence (V), instrumentality (I) and expectation (Ex). An independent samples t-test was used for the analysis. The result of the test has been provided in Table 5.

Table 5 Independent Samples Test for Learner Motivation
Full size table

The mean scores for the dimensions of valence and instrumentality were found to be significantly higher for the experimental group (V = 4.29, I = 4.11) as compared to the participants of the control group (V = 3.82, I = 3.63). However, in the case of expectancy, the difference in the mean scores was not found to be significant (p < 0.05), although the mean score of the experimental group (mean = 4.11) was higher than that of the control group (mean = 3.62). The difference in mean scores for overall learner motivation (M) was found to be significant (p < 0.05); with the experimental group reporting a higher score (mean = 4.16) than the control group (mean = 3.69). Thus, H3 was accepted.

  • H4: Learner motivation would mediate the relationship between gamified instructional design and learning.

The mediation analysis was carried out using Model 4 of the PROCESS macros (Hayes, 2018). The PROCESS macro is a relatively simple and easy to use tool that can be used to analyse a range of models, including mediated, moderated, mediated moderation and moderated mediation models. The macro also provides a bootstrapping option to check for out-of-sample validity.

The results of the mediation analysis have been summarised in Table 6. The analysis involved three steps. Step 1 involved the analysis of the relationship between the independent variable (Gamified) and the mediating variable of learner motivation (M). The independent variable was found to significantly predict the mediating variable (p < 0.05), with an effect size of approximately 0.5. The next step analysed the impact of the independent variable on the dependent variable, i.e., Gamified on the dependent variable of learning (T2). This relationship was also found to be significant with an effect size of 1.69 and p < 0.05. The last step involved evaluating the mediating role of M. The results of the Bootstrap analysis suggest that the mediating effect is significant, with the lower and upper limit found to be greater than 0 (0 < LLCI<ULCI).

Table 6 Results of the mediation analysis
Full size table

Since the direct and indirect paths were both found to be significant, there is statistical evidence to conclude that learner motivation partially mediates the impact of gamification on learning. This led the researchers to accept H4. The model was found to explain 22.6% variance in learning from the training module.

  • H5: Learner Motivation would strengthen the relationship between gamified instructional design and trainee reaction.

Output for the second mediation analysis has been presented in Table 7. As for the previous analysis, the mediation analysis followed three steps. The results suggest that the independent variable (Gamified) had a significant impact on the mediating variable of learner motivation (M)(p < 0.05). The direct effect of gamification on reaction was found to be not significant (p < 0.05). However, the results of the bootstrap analysis suggest that the indirect relationship is significant and learner motivation fully mediates the relationship between gamified and reaction (0 < LLCI<ULCI). Thus, as there is statistical evidence to suggest that the effect of gamification on reaction is significantly strengthened by learner motivation, H5 was accepted. The model explains 29.3% variance in the participant’s reaction to the training module.

Table 7 Results of the mediation analysis
Full size table

Discussion

The current study is based on the Theory of Gamified Learning by Landers (2014). In the current study, the researchers measured learner motivation of the potential learners and found that the level of motivation in learners participating in the experimental group was higher than the level of motivation of learners in the control group. Thus, the gamified module resulted in higher learner motivation. Learner motivation was measured through the dimensions of valence, instrumentality and expectation (Zaniboni et al., 2011). The mean values of the dimensions of valence and instrumentality were found to be significantly different for the control and test group. However, expectancy was not found to be significantly higher. Thus, the trainees participating in the gamified module were more likely to believe that there was value to be gained from the training program and that training performance would result in successful job performance. The findings are in line with those of Sargent (2017). In a similar study, Sargent (2017) explored how the use of game elements would impact the levels of motivation in a corporate training program. They compared the results of a control group and a test group and found that the test group showed higher levels of motivation to participate in the program.

The fifth proposition of the Theory of Gamified Learning, looks at the role of the game elements in directly and indirectly influencing the dependent variable of learning outcomes. Landers and Callan’s (2011), in their studies, used game elements to encourage a certain behaviour and found that this improved academic performance. Armstrong (2015) evaluated the impact of gamification on learning outcomes in the context of the classroom. The findings of his study suggest that the use of game elements had a significant impact on learner’s reaction to the learning activity and the increase in knowledge. The participants of the gamified module reacted more positively to the program and their post-test results were found to be higher than the control group.

The findings of the current study were also found to be in line with previous literature. In the current study, the direct impact of gamification on learning outcomes was evaluated by analysing the mean values for learning (T2) and reaction (R). The researchers opted for the Solomon four-group design to ensure that the effect of the intervention is not sensitized by the pre-test (Navarro & Siegel, 2018). The initial analysis revealed that the pre-test did not influence the post-test results. The findings of the current study suggest that the learning through a gamified module is significantly higher than the regular module. Learner reaction to the training module was also found to be more positive for the gamified module. Trainees participating in the gamified module were found to rate the program higher on all dimensions of reaction; namely, reaction to technology, enjoyment and relevance.

The study further evaluated the indirect relationship by analysing the role of learner motivation as a mediator. Learner motivation was found to be a significant mediator in the relationship between the gamified instructional design and reaction to training and the proposed model was found to predict 45% variance in reaction. Learner motivation was also found to significantly mediate the relationship between gamified instructional design and learning and the proposed model was found to predict 30% variance in learning. The findings provide empirical evidence that supports the fifth proposition of the Theory of Gamified Learning (Landers, 2014).

Limitations

The researchers employed the Solomon four-group design for the current study, in order to control, to the extent possible, the threats to validity. However, the Solomon four-group requires a sufficient number of participants in each group. The current study faced problems in attracting participants. While a large number of organizations were approached, only a few organizations responded positively. Given that most organizations are going through a difficult time, as a result of the global pandemic, constant follow up, scheduling, rescheduling and coordination with trainees, based in different parts of the country, were some of the challenges that the researchers faced. The study would have benefitted from a larger sample size.

In addition, the instruments for Reaction and Learner Motivation were self-assessment tools. Additional independent measures to support the self-report element would lend support to the findings of the current study and this could be an avenue for future research.

Implications

The findings of the study suggest that gamification can impact the training outcomes of reaction and learning, in a positive manner. The findings also suggest that the use of game elements can improve learner motivation. These findings are particularly important for organizations looking for innovative training techniques during the current pandemic. The current crisis has brought into focus the need to be flexible and innovative. Gamification allows one to enhance a module through the use of game elements and thereby increase its effectiveness. Theoretically, the findings provide support for the propositions of the Theory of Gamified Learning. The study also adds to literature in the area of gamification by presenting evidence for the impact of gamification in the organizational context. The study is also unique as it is based on an Indian sample.

Conclusion

Gamification has been discussed and evaluated for more than a decade. As organizations search for innovative and effective learning solutions, researchers propose that gamification is one such technique that could enhance training and learning activities. However, Landers (2019) also warns against adoption without a complete understanding of the concept. The current study attempts to provide evidence to support the adoption of gamification. In designing the study, the researchers have ensured the usage of game elements and not complete games. This ensures that the focus and validity of the study is not compromised.

The findings of the study support the use of gamification in the context of training in organizations. The participants responded and reacted more positively to the gamified module. The participants of the gamified module were also found to be more motivated. Analysis of the relation suggested that, as proposed by theory, the use of game elements influenced learner motivation and thereby improved learning outcomes.