Keywords

1 Introduction

With the diffusion of household internet connectivity, it has become possible for anyone to make casual statements over the internet. What’s more, the proliferation of smartphones has enabled easy access to the internet even among those unable to use a computer, contributing to an internet user base of ever lowering age. In recent years, inappropriate speech and flaming on social networking services (SNS) have become a problem. On SNS, flaming describes a situation in which a user who has engaged in inappropriate speech is criticized by a large number of other users. Flaming has resulted in many cases in which the offending user is identified, or—based on the content of the speech—has evidence of his or her actions submitted for prosecution. Posters of such inappropriate speech are frequently junior and senior high schoolers, leading some to call for restrictions on their smartphone ownership and SNS usage, while others call for pertinent education. In this study, we followed the latter way of thinking and developed an effective SNS education system. Knowing that smartphone ownership increases sharply in fourth grade, and that smartphone-related trouble often occurs shortly after purchase, we decided that fifth and sixth grade students could be educated most effectively. We then developed a serious game that allows children of this age range to experience flaming in a fictitious setting, with the purpose of teaching information literacy [1–4] and online manners.

Our goals in developing this serious game [5–7] were to give children a simulated experience of actual events, and bring resolution to the societal problem of flaming. Education was the main objective, not simply entertainment. Serious games are in use at schools and other educational settings in England, where—despite their status as games—they have been demonstrated to be educationally useful. Related studies include one which used a roleplaying game for experiential learning of information ethics [8], and another which evaluated the use of both a serious game and worksheet [9].

One advantage of employing a serious game is that it allows children to actively use a mock version of Twitter to experience realistic flaming. Also, it visualizes the complicated mechanisms of SNS, making them easier to understand. Furthermore, we by allowing children to make mistakes and experience flaming—something that had not yet been possible in education—we believed they would be better able to recall what they had learned due to the Zeigarnik effect [10, 11]. The Zeigarnik effect is a phenomenon in which people remember uncompleted tasks better than completed ones. We believed that active learning via a serious game would be a highly effective device for learning about information ethics, and our goal was to get children to understand SNS and flaming by experiencing them within such a game. As it is impossible to educate children on SNS when they have not used them before, we thought that a serious game in which they use a mock SNS and get instant feedback on their actions would be the most suitable form of education.

2 A Serious Game for Learning SNS Manners by Flaming Experiences

We developed the main system using GameMaker: Studio, as it readily lends itself to event-driven programming. GameMaker: Studio is a piece of game development software that was developed by YoYo Games.

As seen in Fig. 1, the Twitter screen is displayed on the left side of the game screen, and the virtual world is displayed on the right. The protagonist’s tweets are those with the cat thumbnail. On the top right is the Navigator, a character that teachers the user about Twitter and the system, and shows the user the number of tweets he or she has posted so far.

Fig. 1.
figure 1

The game screen.

Full size image

In this exploration RPG, the protagonist and three of his or her friends are lost in the world of Twitter, and to return to the real world, they must master tweeting. The Twitter world contains a number of sparkling spots, which the user must search for to progress through the game. When the user finds such a spot and examines it, an event is triggered which the user can then tweet about. A total of 15 tweets have been prepared, 4 of which are inappropriate. Tweeting triggers another event, which has been carefully designed to convey—in a way that is easy for children to understand—the fact that the user’s tweets are being viewed by both friends and a large number of strangers. If the user posts an inappropriate tweet it will trigger flaming and a game over, an event that makes the user aware of his or her wrongdoing. In addition to the four characters lost in the Twitter world, there are two other friends in the real world, and these characters can communicate with the protagonist via Twitter. The Twitter used in this study’s system is not the real thing, but a mere imitation that can only be used within the system itself. If the user tweets, that tweet will not be posted online.

2.1 Inputting

We anticipated the game being played on an iPad, so we made input entirely touch and slide-based. At the start of the main system the user selects a sex and inputs his or her last name, Twitter user name, and password. This allows the user to control a protagonist of the same sex and name, who tweets under the username he or she selected. If flamed, the user will be identified and his or her last name spread across the internet. Password selection was enabled for an event in which the user, having revealed his or her password to a stranger, has his or her account hijacked. The purpose of this personalization was to increase the sense of immersion and realism, and cause the user to feel remorse after being flamed.

2.2 Tweeting

The user does not write the in-game tweets; he or she tweets messages that have been prepared in advance. The user examines sparkling spots scattered across the Twitter world, triggering events which he or she can tweet about. For instance, if the user examines an ambulance, he or she will be able to tweet about that ambulance. When a tweeting event is triggered, the user will be asked if he or she would like to tweet. If the user selects “no,” the event will be suspended. If the user selects “yes,” he or she will be able to tweet, the event will progress, and the sparkling spot will disappear. Reactions to the user’s tweet will appear on his or her timeline, and the user’s surroundings will change. For example, if the user tweets about an ambulance, reactions to the post will appear on his or her timeline, and other users who saw the post will gather in the area. There are three possible types of tweet: text-only tweets, reply tweets, and tweets with a picture attached. The following tweets have been created for the system.

2.3 Flaming

Of the game’s events, the following four involve the user posting a tweet that leads to flaming:

  • The user tweets a picture of him or herself inside a patrol car

  • The user tweets a picture of him or herself inside of an ice cream freezer case

  • To find the owner of a lost smartphone, the user tweets a photo of the phone’s home screen picture (the owner’s face)

  • The user tweets a picture of him or herself standing on railroad tracks

If the user posts any of these tweets, he or she will be praised by friends, but shortly thereafter will be bombarded with third-party criticism, resulting in flaming. The user’s timeline will be inundated with criticism, and the Twitter world screen will turn red and then to an image of white-noise-like static. Afterward, the police will come arrest the user, and the scene ends in a game over. All of these events are actual incidents from the past.

  • The protagonist and friends have a picture taken of themselves inside a patrol car, believing they will be praised by their other friends. To share the picture with said friends, the protagonist casually posts it on Twitter.

  • Caught up in the moment, someone climbs inside of an ice cream freezer case, and a friend, thinking it is funny, snaps a picture. The friend, wanting to share the picture with other friends, decides to tweet it.

  • The protagonist and friends come across a lost smartphone. Wondering how they can return it to the owner, they check the home screen and find that it has a picture of what appears to be the owner. Following the suggestion of a friend, the protagonist tweets a picture of the home screen, believing that it will allow them to locate the owner quickly.

  • The protagonist and friends, at a loss for what to tweet about, come to a train station. They take a picture on the railroad tracks and talk about how their friends would praise them if they were to tweet it. The protagonist decides to act on this idea, and tweets a picture of him or herself standing on the tracks (Fig. 2).

    Fig. 2.
    figure 2

    Wanting to share the picture with other friends.

    Full size image

  • Following a game over, the screen transitions to a description of the inappropriate tweet posted by the user (Fig. 3). This screen shows the actual real-life tweet and explains what happened to the person who tweeted it and what about the tweet was inappropriate (Fig. 4). After reading this information, the user will be returned to a point just before the flaming incident and will have another chance to play the game.

    Fig. 3.
    figure 3

    The police coming arrest the user.

    Full size image
    Fig. 4.
    figure 4

    Game over Screen. Actual incidents from the past are shown.

    Full size image

The main system includes another event involving not an inappropriate tweet, but a hijacking of the user’s account. In this event, a suspicious fortune teller promises to tell the user’s fortune after hearing his or her username and password. If the user divulges this information, his or her account will be hijacked and used to post tweets. Changing the password will stop these tweets from being posted. This event was inspired by an incident with a suspicious site onto which people would enter their Twitter user name and password. Subsequently hijacked user accounts were used to tweet advertisements for suspicious websites (which included the sites’ URLs), replies to users’ friends asking them to buy prepaid credit cards, unintelligible messages posted when a company’s account was hijacked, and more.

2.4 Strategy for Getting Children to Tweet

There exist gates scattered about the Twitter world, each of which is opened by posting a certain number of tweets. Therefore, users must tweet in order to play the game. Advancing deeper brings the user to a teleportation machine, which can be used to return to the real world and beat the game.

Elementary school level SNS education (including Twitter) is challenging because of children’s difficulty in grasping abstract concepts like SNS, as well as the difficulty in making children aware of the people on the other side of the screen. So it is beneficial to portray SNS in a concrete, visual manner. To accomplish this, we had the protagonist and his or her friends get lost in the Twitter world, where tweeting would cause changes to their surroundings. In other words, we expressed the various effects of tweeting visually. This simulated experience gave children an understanding that their tweets are constantly being viewed by both acquaintances and complete strangers. This is extremely important, as a misunderstanding of the breadth of the internet, such as the belief that one’s tweets will only be seen by one’s acquaintances, is one of the reasons given for inappropriate tweets.

2.5 Experimental Control System

To demonstrate the learning effectiveness of the main system, as imparted by the Zeigarnik effect, we developed a control system with which to compare it to. It is different from the main system in that when the user attempts to make an inappropriate tweet, he or she is stopped by friends that are also lost in the Twitter world. Afterward, the friends and the Navigator warn the user about the danger of the tweet: while the tweet may gain the user praise from friends, there will be a flood of third-party criticism, and in the worst-case scenario, evidence of his or her actions will be submitted for prosecution. The following is an excerpt of such a warning used by the system: “If you use tweets to brag about reckless horseplay or other behavior that causes trouble, you’ll be attacked by many people and may even be arrested.” After the warning, there is no game over, and the user is able to continue playing the game. Everything else is identical to the main system.

3 The Experiments

In this study, we performed a preliminary experiment followed by two full-scale experiments. Before conducting the experiments, we performed a preliminary survey testing the depth of children’s knowledge on smartphones and SNS. 19 fifth graders (9 boys and 10 girls) from Akagawa Municipal Elementary School and 81 sixth graders (41 boys and 40 girls) from Showa Municipal Elementary School participated in the survey, for a total of 100 participants. Both schools are located in the City, which cooperated in the subsequent main experiments. We then analyzed the children’s knowledge base, and used the information to improve the main system’s design.

In the preliminary experiment, 47 students from a High School, aged 14–17, used the main system or watched it in action, and evaluated whether it was an accurate portrayal of flaming, and whether the system could be used for learning. When asked if the main system had accurately portrayed flaming on Twitter, 96 % of students replied that it was an accurate portrayal, while 4 % replied that while it was an accurate portrayal, there needed to be more innovative elements. We used this feedback to improve the main system’s design, including its innovative elements and UI.

The first full-scale experiment had 19 fifth grade subjects (9 boys and 10 girls) from Akagawa Elementary School, and used 2 kinds of educational tools—the main system and control system. The purpose of this experiment was to verify whether making mistakes and experiencing flaming would increase learning effectiveness.

The second full-scale experiment had 81 sixth grade students (41 boys and 40 girls) from Showa Elementary School, and used 3 kinds of educational tools—the main system, control system, and video materials on flaming created by the IPA (Information-Technology Promotion Agency, an Incorporated Administrative Agency). The goals of this experiment were to verify whether learning effectiveness was increased by making mistakes and experiencing flaming, to verify whether learning effectiveness was increased by engaging in active learning, and to verify the difference in learning effectiveness between the main system and the video-based teaching materials. A simple test was used for this verification. In creating this test, we referred to and cited test questions found in information ethics teaching materials [12] created by GREE for junior and senior high school students. Questions are as follows. Q1: Posting your wrongdoing on the Internet stays on the right side of the law. (Y/N), Q2: Posting your comments on the Internet cannot be seen anyone except your friends. (Y/N), Q3: It will be flaming on the Internet only if someone perpetrates a crime and runs afoul of the law. (Y/N), Q4: We can post photos on the Internet without prior consent by a person in a picture. (Y/N), Q5: We can post photos of a close friend on the Internet without prior consent by a person in a picture. (Y/N), Q6: In the case of flaming on the Internet, if you erase your posts the problem will be solved. (Y/N), Q7: Even posting under anonymity, your real name sometimes come out. (Y/N), Q8: We have to observe the rule and know manners on the Internet as actual life. (Y/N) Q9: What is the twitter? (written questionnaire) Q10: What kinds of tweets will be flaming? (written questionnaire).

Subjects were divided into two groups (groups one and two) during the first full-scale experiment and three groups (groups three, four, and five) during the second full-scale experiment. These groups and the teaching methods employed on them are described in the table below (Table 1).

Table 1. Groups and conditions
Full size table

We first held a simple class on SNS for each group, and then taught them in greater detail using the prescribed educational tools (main system/control system/video materials). Afterward we gave each group an identical post-test to measure the learning effectiveness of the educational tool used.

4 Results

The average scores for the post-test administered to each group at the two elementary schools are listed in Table 2. In terms of average score for multiple choice questions, there was not much of a difference observed between groups. In fact, when comparing each of the groups to one another, no significant difference was observed in most cases. However, large differences were observed between groups when comparing average scores for written response questions.

Table 2. Average scores for post-test
Full size table

To determine whether there was a statistically significant difference between the average scores of the groups that used the main system and the groups that used the control system, two-sided t-tests with a significance level of five percent were performed on groups one and two and groups three and four (Table 3). The average scores were found to be significantly different, with the groups that used the main system scoring significantly better.

Table 3. Two-sample test of groups that used the main system and groups that used the control system
Full size table

Next, to determine whether there was a statistically significant difference between the average scores of the groups that used the main or control system and the group that used the video-based teaching materials, two-sided t-tests with a significance level of five percent were performed on groups three and five and groups four and five (Table 3). The only significant difference was found in the written response question scores for groups three and five, in which group three (a group that used the main system) scored significantly better than group five.

5 Discussion

In this study we developed a serious game for experiential learning of SNS manners, and performed experiments comparing the main system with the control system and video-based teaching materials.

In terms of average scores for multiple choice questions, there was not much of a difference between groups, with no significant difference observed in most cases. However, large differences were observed between groups when comparing average scores for written response questions. We believe this is because the multiple choice questions were easier and could be solved with less knowledge, while the written response questions were difficult to solve without acquiring the proper knowledge. The average scores for written response questions, when arranged in order of highest to lowest, were as follows: all groups taught using the main system came in at the top, followed by groups taught using the control system, and the group that was taught using the video-based teaching materials came in at the bottom. From these findings it can be said that the main system, by having children experience flaming as part of their education, was the most effective learning device. Furthermore, judging from the fact that the control system reaped higher average scores than the video-based teaching materials, it can be said that active learning and the visualization of Twitter’s mechanisms are more effective learning devices than passive learning.

When comparing the groups that used the main system with those that used the control system, it was found that the main system had significantly higher learning effectiveness. We believe this is because subjects, stimulated through their experience with flaming, endeavored to gain a precise understanding of Twitter—an inference which is supported by evidence. The majority of children who (under the control system) did not experience flaming exhibited a lack of understanding of not only flaming and inappropriate speech, but of Twitter itself. The only difference between the main system and the control system was the presence or lack of flaming; all other aspects of the systems, including their explanations of Twitter, were identical. What’s more, it was observed that groups using the main system frequently performed slide-based operations on the left-hand Twitter portion of the screen, while groups using the control system rarely used these slide-based operations and instead just watched the right-hand virtual portion of the screen. It is possible that the children’s flaming experience stimulated an interest in Twitter, causing them to carefully watch that portion of the screen.

We were worried that in performing a preliminary test (administered before the main and control systems were used) the children would inadvertently learn about inappropriate speech, rendering accurate measurement of each system’s learning effectiveness impossible. Thus, we only administered the test to groups three and four. However, there was no significant difference observed between these groups and the group in which the preliminary test was not administered, suggesting that the preliminary test did not have an influence on the experiment. In comparing the results of the preliminary test and the post-test, no significant difference was observed in the multiple choice questions, while the average score of the written response questions was found to be significantly higher in the post-test, demonstrating the learning effectiveness of the systems.

Regarding Table 3, no significant difference was observed in the average scores of written response questions, while a significant difference was observed in the average scores of multiple choice questions. This indicates higher learning effectiveness for the sixth grade group. This could be related to a difference in the knowledge base of fifth and sixth graders.

The test results for the written response questions differed greatly between the group that used the video-based teaching materials and the group that used the main system. The former lacked an understanding of Twitter itself, and exhibited less variation in its answers to written response questions. A possible reason for this is the fact that, compared to the video-based learning materials, the main system provided subjects with an opportunity to see a greater variety of tweets, including inappropriate tweets. These results show a large difference in the learning effectiveness of active learning and passive learning, suggesting that active learning may be more effective. It can also be said that by visualizing Twitter, the main system succeeded in making the SNS easier for children to understand, enabling them to learn more about it in a short amount of time.

6 Summary

In this study we developed a serious game for experiential learning of SNS manners. Under this system, subjects used a mock version of Twitter to actively learn about information ethics. To confirm the learning effectiveness of this main system, we performed experiments comparing it with a control system and video-based learning materials, with fifth and sixth grade elementary school students used as subjects. The main system was found to have higher learning effectiveness than both the control system (in which children were not allowed to make mistakes or experience flaming) and the video-based learning materials (a passive method of learning).