Gamifying Digital Learning Platform for Information Security Awareness

Abstract

Gamification, the process of integrating the “fun” aspect commonly found in a videogame to mundane processes like learning to make it more interesting. In this research, an undergraduate university student in Thailand is assigned into two separate groups: A and B. Group A designated as “Gamified Group” is to study the topic of information security via educational videogame. Group B designated as “Traditional Group” is to study the same topic via provided paper-based material. After that, both groups are to test their understanding of the topic by doing quizzes. We found that while gamified group expressed more interest and more actively participate in this experiment compared to the traditional group, the average test score from both groups is comparable to each other with further experiments shown that undergraduate students in their senior year from both groups achieved higher average score than first-year student likely due to their familiarity with information security from coursework they took during years of their study. This research proved that videogames can be used as a tool in a higher education environment, with the possibility of developing better education videogames with increasing efficiency and make them more appealing to the masses.

1 Introduction

In the digitalization era, information has great importance and value. Thus arise the need for those who work in the information technology and banking field to equip themselves with appropriate knowledge and present-day extensive digital learning network has seen popularity as a platform to raise security awareness both for an individual and corporate environment. But learning can be monotonous especially when the lesson in question is not align with your current knowledge, experience, or interest. New learners could spend a long period just to acquire a basic glimpse and need a high degree of motivation to stay focused, many are discouraged by its tedious nature. Gamification has proven to improve learner’s motivation and they are willing to participate in an activity. In this research, we provide the description of the experiment of using gamified learning platform as a tool to engage learners in the subject of information security awareness and compare the result of the experiment to those who learn the same subject via traditional paper-based means. The hypothesis for this research is that gamifying learning process will improve the motivation of participants which will lead to improvement in test scores for security awareness tests comparing to those who study via traditional means.

1.1 Objectives

Steam customer purchased of videogame from 2020 onward highlight general public’s interest in videogame and possibility for gamification to motivate user in a topic of information security. We are interested in two hypotheses: whether the gamified platform can complement traditional learning methods and whether the gamified platform is suitable for learning advance or complex topics.

1.2 Significance of the Study

First, there are several studies on gamification. However, our study of Thailand's context specifically focuses on university students in a computer science field. Second, the use of video games as a platform for academic purposes, while customary, is mostly concentrating on kid’s education or elementary lessons. We hope to gain a better understanding of how gamification can help in a higher education environment.

1.3 On the Definition of Gamification

According to Wikipedia [1], Gamification is a strategic attempt to enhance activity to create a similar experience when playing a videogame with objectives to motivate and engage users, be the use of game design and principle in a non-game context so there might be an argument that the goal of gamification is not “having fun” but about value creation. However, in this research, we would like to provide a different perspective, in that “fun” is what makes a videogame a videogame. “Game can be fun to play, and fun alone is the approved reason for playing them” is a sentence that sociologist Erving Goffman uses in his essay “Fun in Game” [2] citing that we can include as many rules and design strategies we want but a “fun” videogame is what motivate and increase engagement while a “not fun” videogame does exact opposite. We agree with Goffman’s argument based on our reasoning, personal experience in a lifetime of playing a videogame, and statistical proof between well-received game and badly-received game rest on one essence, Do you think your game is fun? And do those who play your game think so?

1.4 Difference Between Cyber Resilience, Cyber Security, and Information Security

While these three words seem to use interchangeably, they are different in perceived objectives. For cyber resilience and cyber security, both are related byword ‘cyber’ and are both forms of protection against cyber threats. However, cyber resilience recognizes the situation where the defensive line, cyber security, failed and focuses on how an organization can remain operational in a critical situation [3].

Information security can be described as the prevention of unauthorized access and modification of data in any given state of transferring and storing created to cover three objectives of confidentiality, integrity, and availability known as the CIA triad. Be those data be physical or digital, personal, or organizational data [4].

In conclusion, we can think of information security as the smallest parts of the three, perhaps most important as shown in Fig. 1, and related to the daily use of technology and how a person can prevent a situation where they are a target of cyber threats. If an attack does happen, cyber security acts as a defensive line to prevent further damage spread but if all else failed. Then, it will rely on cyber resilience as a fail-safe plan to survive the attack and recover to pre-attack state as much as possible. Fixing the system after the damage was done was counter-intuitive and the best approach is to prevent the attack from ever happening in the first place thus by raising awareness of individual people we can be certain that will reduce the incidents of cyber security breaches and cyber resilience must be used in the future.

Fig. 1.

Cyber security model

2 Literature Review

According to R.K. Dixit et al. (2008) [6] cybersecurity is the challenge of the digital era particularly when organizations want to comply with security policies such as Confidentiality, Integrity, and Availability also known as “CIA Triad”. The paper quotes the integration of gamification with the traditional teaching model as means to help reach the learning objective, the experiment is carried out for university students in their final year. In the first part of an experiment, the lecturer would teach the subject about cryptography bound to the story/example concept. Using imaginative scenarios such as “Alice wants to prove to Bob that she solves the Rubik cube without actually solving. How can Alice convince Bob?” when students provided the answer for Alice, they also answer the key question of cryptography “How sender of cryptography prove that he/she is authorized sender without sending key?”. The second part of the experiment is to use a riddle instead of a direct question with the sole objective to assess the student by creating more interest. In this case, the riddle is about how to cipher local language into a known language (e.g., English). The observation shows that students become interested in solving the riddle and finding a hidden answer. The process then linked to the coding lesson that the lecturer aimed to teach. The research concluded that gamification can be applied with traditional teaching with the result shown 80% of observed student participated in the experiment have increased engagement and attention span thus make the class more active.

In a journal by Ulrike Hammerschall (2019) [7], while agreed that gamification can be applied, pointed out that not all experiments in gamification ended successfully. Focusing on the motivation aspect of a participant, this research introduced a gamification framework for long-term engagement based on two theories: A Self Determination Theory (SDT) and Transtheoretical Model (TTM). The first theory, SDT, is an approach to human motivation and personality. It identifies three psychological “needs” that enhance motivation when satisfied, that is the need for competence, the need for relatedness, and the need for autonomy. Those three “needs” will determine how motivated the person is based on three stages: Intrinsic or highly motivated, Extrinsic or mildly motivated, and Amotivated or unmotivated. This research further clarifies that for someone to be interested or start a new activity, they need to be at least intrinsic about it and by using gamification, the aspect that makes videogame fun and engaging, will stimulate a person’s motivation between intrinsic or extrinsic through reward, feedback, or fulfillment. The second theory, TTM, defines a model of behavior change developed by James O. Prochaska et al. (2005) [8] into 5 stages: Pre-Contemplation the stage with no intention to change; Contemplation the stage of thinking about change; Preparation the stage of preparing for change; Action the stage of changing and Maintenance the stage of keeping the changed behavior. The research notes that the first three-stage is the most important but using gamification in this period is useless, only during action and maintenance stages that the strength of gamification comes into play. The reason is that the player will have the autonomy to do the change and for that he/she needs to believe that one’s effort is crucial for success.

Believing that, to complete the learning process, it is necessary to combine both scientific and everyday practices. Maciej Laskowski (2015) [9] experimented to find that students should learn by finding the practical solution to the problem instead of finding the answer in the textbook thus increasing students’ involvement. Maciej also pointed out that the experiment is beneficial not only inside the education field as all “learners” regarding his or her professions, and lesson of intent are considered students. Gamification can split into two areas: 1.) Educational game and 2.) Gamified Classroom, Maciej chooses the latter to experiment on a group of IT students in their later year of bachelor’s degree, called experiment A, and another group of master’s degree students, called experiment B. The result has shown that, for experiment A, students who study in traditional class score better academically than students in the gamified classroom but have less participation rating. The student in the gamified classroom, where the leaderboard system is used, reported that a portion of students who ranked lower in the leaderboard has decrease motivation compared to the student at the top of the leaderboard. For experiment B, the gamified classroom has more participation rating and homework turned-in than traditional class but score lower academically than the student in the traditional classroom. It is theorized that because master’s degree student only focuses on passing the class and generally do not mind about grading if they pass. A mentality that vastly different from bachelor’s degree students. The journal ended with a conclusion that gamification in the academic field can improve attentiveness and participation of students but has almost no improvement academically.

Chee-Ken Wong and Chien-Sing Lee (2016) [10] developed a website that focused on the effect of gamification for learning Science, Technology, Mathematics, and Engineering (STEM). The participants of this experiment are diverse in their knowledge of the subject. As a means to find out that their gamified website succeeds, a survey was used to collect user’s opinions on a website that would incorporate gamification as a base for future enhancement. The method of using a survey is what inspires to be used as one of the main features in this paper.

A dilemma of overspecialization is, according to Wikipedia (2020) [11], is when a person works in an excessively narrow occupation or field. Most time in a professional environment, specialization can be seen as beneficial as a person have in-depth knowledge and can handle more difficult tasks. The downside however is if a person is too narrowly over-specialized it can lead to poor training, unnecessary workflow, and risk their specialization become outdated as time went by, especially in information technology where technology advance rapidly. Marc E. Pratarelli (2007) [12] went into detail that as more and more innovation is created so need the specialist to maintain them leading to favor of specialization of specific IT field. Modern academics tend to prepare people to become specialists to satisfy job-market, companies and actively sought out a specialist for tasks/projects they currently handling but once those technologies are replaced by more advanced forms and become obsolete, thus lead to these specialists that have a harder time adjusting than their more flexible general-purpose counterpart. The research concluded that while specialist indeed is essential in the function of society, they are not to be replaced general-purpose workforces. Academics need to train these two types of workforces in equal ratios. And people who choose to become specialists need to realize their fatal weaknesses and dilemmas that one day, the knowledge they have mastered will be replaced and it is better to have a mindset of being flexible and ready for changes. For this paper, using a survey to gauge the understanding of participants on the topic of information security, we find this as an opportunity to discover how many university students suffered the same dilemmas which will grant us an understanding of the current academic situation in Thailand.

Yevgeniya Daineko et al. (2008) [13] has developed an educational software based on the game engine Unity 3D for studying physics. Believing that new technologies in education will have a significant effect on the process of interaction in the context of transferring knowledge. The author draws inspiration from various previous works that make use of virtual environments such as Virtual Star Laboratories by MIT, Project Eyes on Earth by NASA, A 3D model of a chocolate factory by FX Palo Alto, and Virtual Game Lab which is an experimental project by psychologist and engineer aims to study cognitive abilities and behavior of human to solve various mental diseases. The author outlined and went into detail of many potential game engines that are available such as Unity, UDK, Cry ENGINE, Torque 2D/3D, and HeroEngine, some engines are more specialized in developing genre of video game for example HeroEngine is commonly used by MMO (Massively Multiplayer Online) videogame developer. Unity is regarded as the game engine with a wide range of possibilities and an easier learning curve as well as its multiplatform allows for easy and quick porting of games into various mediums. Using game engines, the research conducted many scientific experiments inside virtual environments with the result showing that the execution is much easier and faster. It is also trivialized more dangerous experiments in which scientists can continue their work uninterrupted without fear of hazardous elements. We will go into more detail about the process of picking a game engine suitable for this paper.

Laura Alejandra Martinez-Tejada et al. (2020) [14] discussed many genres of videogames to choose for the experiment. It was decided that the game must be an easy game with flexibility for future development/improvement, a 2D platforming game was selected. The research also went into detail about the experiment of using various graphic aesthetics, game difficulty, sound, etc. to find out if any of those aspects affect the interest of the player and to keep them engaged. The result of this experiment concluded that what affects players’ interest most is gameplay and difficulty, following by graphic aesthetic and game length, then the sound effect. Interestingly that while the aesthetic of the game affects the interest of players, a further experiment where players encounter repeatedly the same thing can considerably bore them, also many players reported feeling bored because intermission (between stages) is too long. The conclusion given is that for a game to keep players’ interest it needs to have an adequate game length, balanced active game-time and idle time, do not repeat or reuse aesthetic which serves as a guideline for designing an engaging game for this paper.

In an article published by Venturebeat (2020) [15]. Steam, a video game digital distribution service by Valve, reported a 21.4% increase in games purchased by a customer in 2020 in comparison to 2019 with Valve summed up the growth in its blog post saying, “While Steam was already seeing significant growth in 2020 before COVID-19 lockdowns, video game playtime surged when people started staying home, dramatically increasing the number of customers buying and playing games,”. While Steam has yet to disclose the purchased data of 2021, Analysts predict an increase in game purchased to continue rising with the current ongoing pandemic and recently announced Steam’s 2021 Winter and Summer sales. Gameindustry.biz, United Kingdom videogame media, tracking digital download game sales from 16 major game companies across 50 European, Middle-East, African and Asian countries reported 4.3 million games were sold during March 16–22, 2020 [16] when the new pandemic first spread globally, a 63% increase compared to amount sold pre-pandemic. The report also points sharp increase in a country where pandemics hit the hardest and longest, Italy, which saw a 174.9% increase in digital game sales when the Italian government first announce a lockdown. Physical game sales also soar with about an 82% increase likely contributing to the abundance of logistic and delivery services born during the pandemic. This data is consistent with what other news media found such as CNBC, the Conversation, and TechSauce discussed below.

An article by TechSauce [17] cited an increase in gaming-related conversation on Twitter application by 97 percent in Thailand during pandemic ranking 4th after Japan, United States, and South Korea as countries that tweeting most about gaming. Twitter data analysts outline three key insights that are: Socialization, Leisure and stress relief, and Videogame popularity among Gen-Z and Millennial.

In a journal by Zhu Lin, with the release of Nintendo’s “Animal Crossing: New Horizon” successfully get everyone’s attention, the game’s dreamland-like aesthetic and relaxing gameplay provide “temporarily escape” from harsh reality for many people who suffered anxiety and isolation during COVID lockdown. [18] The game’s social platform is a perfect solution where people may meet up online. On the other side of the spectrum, id Software’s action First Person shooter “Doom Eternal” released alongside Animal Crossing also gain an increase in popularity as a fun shoot-em-up game that provide “stress relieve” through the sheer amount of violence and gore presenting in-game with one videogame reviewer cited in the article that Doom Eternal help make them “less angry” [19].

3 Methodology

For this research, there are four stages of progression: First, Picking a target group. We choose Computer Science undergraduate students equally divided into two groups: Group A, “Gamified Group”, and group B, “Traditional Group”. As for experimentation. There are two tools in use:

• A paper-based document containing all the lessons the participant needs to learn about security awareness before the examination. This will give to the only participant of group B. (See Appendix)

• A gamified lesson in form of a simple videogame. The detailed lesson in-game is kept the same as the document given to group B with some alterations to suit the flow of gameplay. The game will be played only by the participant of group A.

The first group learns solely via the gamified platform and the second group solely via traditional paper-based means. Both groups are given approximately 30 min to study their assigned lesson. Once both groups finished studying, we will move to another stage, the Examination.

Second, Preparation for the gamified lesson. Taking lessons learned from an experiment by Laura Alejandra Martinez-Tejada et al. (2020) [14] it is safe to assume that one does not need to develop a complex videogame to complete the intended objective. Rather, the simpler the game the easier it to determine the effectiveness of gamification. Any result will be used for future research into more advance and complex videogames. We decided to choose Unity for developing our gamified platform as it is one of the most well-known game engines in the current market, due to its free-to-use and consumer-friendly policy, lower learning curve, versatility, low system requirement, the engine provides a vast library of materials and established online community. The finished gamified platform resembled a complication of minigame comprises into one single game.

Another notable mention is auxiliary applications, that may or may not be categorized as game engines like Torque, Blender, and Adobe may be used to help create models, artwork, or other functions needed in the finished game.

Unlike traditional learning methods that rely on repetition and memorization, Gamification can provide the student with an interactive lesson in similar manners to videogame use its storytelling and gameplay to captivate the player. For example, in one minigame, a student(“player”) assumes the role of “Mario” from the well-known classic platforming game “Super Mario Bros.” shown in Fig. 2. We combined the story of Mario in his quest to save Princess Peach from the evil Bowser with a lesson we want a player to learn, in this case, about the common delivery method of computer viruses.

Fig. 2.

A screen from the minigame resembled Super Mario Bros.

Another reason we choose to develop a minigame with Super Mario Bros. theme is that the game itself is an established videogame title, publicly well-known even to those who do not play videogame have heard of Mario. Thus, by role-playing as Mario, we created a sense of objective for the player. “To save the princess by choosing the correct answer for Mario”. There, we continue into the gameplay section of minigame, to control Mario and hit one of the coin blocks that contain the correct answer shown in Fig. 3. In any case that, the student-controlled Mario chooses the wrong answer that coin block will become inactive with a message asking a player to choose another coin block. The process will repeat until the player finally chooses the correct answer.

Fig. 3.

Player-controlled Mario to hit one of the coin blocks that contain the correct answer. Choosing wrong coin blocks resulted in those blocks become inactive with the message “Wrong answer” shown to a player.

However, the Mario character and the assets associated with the Mario series are the product of Nintendo and are protected by intellectual property law. An ‘asset flip’ or using copyright-free assets instead of Mario is considered. The change will only affect aesthetics and not the core gameplay of the game.

Another minigame example is a simple matching game, where the player assumes the role of an electrician to correctly connect all electrical wires shown in Fig. 4. The gameplay share similarity to choosing coin blocks in Mario minigame but different aesthetically. This is to prevent repetitive gameplay experiences and provide the player with something new to not bore them, a strategy often uses by many videogame developers to keep the gameplay loop fresh. Choosing the wrong wire resulted in that wire retracing to the original position, letting the player know that their answer is incorrect and need to choose another answer.

As both students from groups A and B are given equal time of approximately 30 min to learn about information security. It is crucial to developed videogame based on this factor. Since, unlike reading material from texted paper, the playtime of each player can vary extensively and without reserving, some students might not be able to finish their game within the assigned 30 min. After both groups finished studying their assigned lesson, we then continue into the third phase. Taking quizzes.

Fig. 4.

An electric wiring minigame

Third, A Quiz. A test was given to participants of both Group A and B after they completed their respective lessons. The participants are instructed to complete the test within 30 min and are not allowed to use any means of assistance during this period. An exam has 3 parts with a gradually increase in difficultly starting from the most basic knowledge of information security in part 1 (easy), to intermediate in part 2 (normal), to advanced knowledge in information security in part 3 (hard), each part contains 10 questions with a total of 30 questions combined, each question uses a traditional 4 choices that participant must choose the only one which they think is most true. The basis for questions in the exam based on ISACA CISM’s information security tests (2020–2021) [20, 21]. Once all participant from both groups completes their test. We will analyze the score of each group to compare the following:

• Which groups fare better academically in terms of scoring the most correct answers out of 30 questions.

• Comparing how many students from each group pass the easy difficulty.

• Comparing how many students from each group pass the easy and normal difficulty.

• Comparing how many students from each group pass all the difficulties in the exam, scoring a minimum of 18 scores with a minimum of 6 scores for each difficulty.

As the test using a linear progression of questions becoming gradually more difficult every 10 questions. It is an opportunity to see whether student from each group has the adequate knowledge to progress into more advance subject or they only specialized at parts of the exam. For example, one student might have intensive knowledge of advanced information technology but lack a basic understanding of information security. Another student, on the other hand, might have knowledge in basic and intermediate information technology but lack understanding in advanced knowledge. The passing line is more than 60% correction rate (6 out of 10) for the student to pass each part.

Forth, Result Analysis. The test result of both group A and B will be totaled and compared to prove that the hypothesis of this research that is: “By gamifying learning process will improve the motivation of participant which will lead to improvement in test score for security awareness exam comparing to the test score of those who study by traditional means” holds true. From there, depending on which hypothesis held true or not, we will analyze the factor that contributed to the result of the experiment, examining possible improvement(s), and reach the conclusion of this experiment.

Due to the ongoing COVID-19 pandemic and subsequent lockdown mandate, we made a backup plan in case the original plan to conduct experiments with a gathering of people together may not be possible. Paper examination and reading material will be provided via online form and file transferring. The videogame will be played by sharing the application and played in a limited timeframe as planned. Additionally, the target group may change to those with Computer Science knowledge or related fields with at least a bachelor’s degree and similar age.

4 Result

The test was conducted on two separate occasions, both tests divided participants equally into two groups: A and B.

In the first test, with a student from the same coursework and academic years as a participant. Observation of students from Group A (“Gamified”) showed a positive response and overall interest toward gamification platform with approximately a third of the group express the idea of binding difficult topic with videogame interesting. Nevertheless, the average scores of both groups shown in Fig. 5 are comparatively similar with students from both groups achieved passing scores (more than 60%) on Part I and Part II of the test, while retaining between 40–60% average score on Part III of the test. Both groups have shown great understanding of the material covered in Part I with an average score of 75% with a declining score continue Part II of 60% and III of 40%.

Fig. 5.

An average test score of both groups in the first test

In the second test, a student from the same coursework but varied academic years as a participant. Observation of students from Group A (“Gamified”) showed an identical positive response and overall interest as the first test although about half of the group express the binding of a difficult topic as interesting with other half commented that the gamified platform might not be able to provide enough material to an otherwise elaborate topic as information security. As a matter of score, the average scoring of both groups shown in Fig. 6 is comparatively similar, in the same manner as the first test. However, both groups showed an increase in average score across all three parts, with an average score of 80% on Part I and around 60% on Part II and III. The reason likely contributed toward the presence of the senior student in both groups who potentially have a prior lesson in topics covered in this test.

Fig. 6.

An average test score of both groups in the second test

5 Conclusion

Through this experiment, it is observed that while most students found using the gamified platform as a great and interesting learning tool for information security, it offers limited benefits in terms of improving student scores when compared to traditional teaching. Mayhap due to the style of the game used in this research did not reflex all participant fondness and are more streamlined than lessons found in paper-based materials, which called for future research into developing more generally accepted gameplay and how to integrate more lessons into videogame while at the same time balancing the fun aspect. However, if we take into consideration other factors such as enjoyment and attentiveness. The gamified platform has achieved much more compared to the traditional learning method in both educational and fun. Overall, the result has shown that gamified platforms can be applied and use as an effective tool in a higher education environment to increases student engagement and study efficiency.