Keywords

1 Introduction

In this study, we developed a digital storytelling system that generates stories from hypothetical questions. The digital storytelling system is capable of developing stories in a non-linear order, and users can actively experience the stories [1]. In this system, a story is represented by expressing it in a knowledge base in “If Then” format. Users can create a story with a synopsis different from the original story simply by changing the “If” part. For example, if the user changes the appearance and personality of a character, the outline of the story will change. It is therefore possible to generate different stories derived from the original story.

In the What-If Machine [2], a considerable amount of knowledge and fictitious ideas are necessary for story generation, and stories are generated by using questions and answers. By choosing the answer to a given question such as “What” or “Where,” it automatically generates and displays the question “What If ~ then.” For example, when the user sets Who (he) and Where (park), the trigger of the story of (play) is displayed. In this system, the user creates those settings, and then the subsequent deployment is also generated.

Scheherazade automatically generates continuing developments to the question. Scheherazade searches for answers to questions on the Internet and incorporates the answers into stories [3, 4]. In our proposed research, we do not use information on the internet but instead use commonsense knowledge offline.

In the proposed system, the user can generate various stories by setting and changing the “If.” By so doing it is possible to improve the creativity of the user by creating patterns of various stories and imagining the future of the story.

Our objective is to realize a system that can help users to visually recognize the story as an image and allow the user to experience the story interactively. Therefore, in this system, we decided to add an animation as well as text information with generated story.

There is a research [5] to convert text information into animation by performing semantic analysis for each sentence using natural language processing. Interactive e-Hon [6] generates stories with animation by analyzing the meaning of text information and combines animation materials. In WordsAnime [7, 8], users can set the storyline by guiding the system according to the sentence construction method, and animation according to each storyline is displayed. By expressing hints for storyline creation, we support the user’s story generation by knowledge base of the story connecting storylines and storylines. We adopted the method of assigning animation in this system from the text information in these studies.

In this study, we developed a digital storytelling system that generates stories from hypothetical questions, and verified that creativity is enhanced by using the developed system.

2 An Interactive Digital Storytelling System with “What If” Functions

In this system, the user defines the personality and appearance of characters based on the existing story. Based on that definition, a new story is generated by firing a knowledge base of behavior patterns of characters.

2.1 System Overview

In the system, the properties of the characters can be changed by using a production system called CLIPS. This system writes the execution history to a text file that can be read by an animation system called Unity. By so doing, the animation and narrative of the stories can be displayed on the screen. This process is repeated for every branch of the story.

2.1.1 What If Functions

In this section we explain What If functions. There are many branches where the flow of the story changes in advancing a story in this system. The user can change the definition of “If” of “If Then.” In one branch, the story advances based on the definition of the character set for the user. In addition, the user can select and modify not only the characters but also the definitions of the objects related to the story.

For example, in Snow White, the user can change definitions such as “if the appearance is beautiful,” “if the personality is evil,” “if the poisoned apple is a poison banana.” This function can result into multiple stories.

2.1.2 Story Development Using Commonsense Knowledge

In this system, the kinds of actions based on various properties are accumulated in the knowledge base. We set the behavior patterns of characters using this commonsense knowledge. Table 1 shows excerpts of the commonsense knowledge accumulated. Table 2 lists examples of what actions are actually taken based on Table 1. In order to point to what kind of character it is in the column of the rule, we define in advance what the character will be if the characters are pure, mean, or cowardly. The “If” part shows conditioning. The “Then” part defines what action to take in the case of “If.” X and Y mentioned here also refer to the characters. Table 2 gives examples of behaviors that made the characters Snow White. From the above, a story is generated by applying multiple rules that define what kind of behavior to take depending on the character and appearance of the character.

Table 1. Commonsense knowledge
Full size table
Table 2. Examples of behaviors that applied commonsense knowledge
Full size table

In CLIPS, “defrule” allows the user to define the rules of actions according to the appearance and personality of characters. It is possible to display the execution history as a synopsis. In “assert,” it is possible to add or change the definition of the character and save it to a text file. In the “rule” in Fig. 1, if the personality of a character is beautiful and its appearance is evil, it is a flow that shows a synopsis is shown with a “printout.”

Fig. 1.
figure 1

System screen (Snow White)

Full size image

2.2 Example of Story Generation

We developed a system based on a story. “Snow White” is selected because it is famous worldwide in Grimm’s fairy tale. It is easy to think of points in these stories where branching can occur.

We developed a system based on “Snow White” as a theme. Figure 1 shows the execution screen of this system. In the lower right corner there is an “Next” button. By pressing the button, text information is read and animation is given. In addition, in the window on the left side, it is possible to confirm the definition of characters currently selected by the user. Finally, the lower left side shows a synopsis in Japanese notation. For example, when the user encounters a branch, “The princess visited a house in Snow White,” the user sets the definition of Snow White with “cautious character” and CLIPS. Then, based on the definition, it conducts actions that conform to the definition, and it is the trend that the story advances by taking action such as “close the door without getting an apple.” When this sequence of flow is applied to the condition of “If then,” the “If” part becomes “if Snow White is a prudent personality” and the “then” part becomes “close the door without accepting the apple.” In addition, if Snow White’s definition is “pure personality,” actions such as “Open the door and eat apples from the woman and go into a coma” are selected. If this example also applies to the condition of “If then,” the “If” part becomes “if Snow White is pure personality” and the “then” part becomes “she opens the door and gets an apple from the woman and eats it and enters a comatose state.” The action of “then” obtained from commonsense knowledge is written out to a text file, its behavior is read in Unity, and animation is applied. There are multiple actions, but we applied only one this time. A new story is completed by repeating a branch point of such a story a plurality of times.

3 Experimental Evaluation

In this experiment, we asked 21 subjects of elementary school fifth graders to use this system and distributed a questionnaire in order to determine whether the system increases creativity. Creativity is increased by using the questionnaire. Creativity is defined by Ito as “the discovery of a new viewpoint that enables it to transform to a new theory, new combination of materials, solving problems” [9]. In this study, we defined that “Creating is to act of combining and expressing new ideas and perspectives as one’s own” and used as evaluation indicators. By using these indicators, we verified how much creativity was enriched before and after the system was used. We first conducted a pre-test first, and then asked the subjects to observe images projected by a dome-type projector. A dome-type projector is a half-circle assembled with cardboard. We carried out a pre-test before using this system and carried out a post-test after using the system. The experiment took about 45 to 50 min in total. In the pre-test, it was assumed that the material of the creativity test [10] proposed by Yumino et al. (2001) was revised to ensure that it was consistent with the fairy tale. In the creativity test, the experimenter showed a picture, asked the subjects themselves to talk about the continuation of this picture, and then had them write it on paper. We taught the subjects that it is okay to create as many sentences as desired. The pre-test involved the continuation of the scene in which Momotaro was born. (“Momotaro” is a famous Japanese fairy tale.) The post-test involved the continuation of the scene in which Momotaro was born and the ant is working and the grasshopper is sleeping in ants and grasshoppers of Grimm ‘s fairy tale. In addition, the questionnaire contained seven questions asking about the contents of the system, such as whether it was interesting using this system or the system was easy to understand. Three to four of the subjects were in the dome at any one time, and they had this system using Snow White as their theme. All subjects selected What-If, and the experimenter manipulated this system while commenting.

4 Results

For the results of the experiment using “Snow White,” we used the score in the creativity test to measure creativity. The average score for the first question in the pre-test was 1.33, and the average score of the first question in the post-test was 3.10: thus, the score increased significantly (t = 4.15, df = 20, p < 0.05). Because there was no pre-test in the second question, we compared points compared with the existing story. It was assumed that the average score of the pre-test is zero point. The average score of the post-test was 1.86 points and the score significantly increased compared with the existing story (t = 5.83, df = 20, p < 0.05). Further, in the system questionnaire, at least 90% of the subjects gave the highest grade, grade 5, to the questions, “Is this system was interesting?” “Do you want to do this system?” and stated that “Is this system easy to understand?”

5 Discussion

We developed a digital storytelling system that automatically generates narratives from hypothetical questions, and examined whether the creativity of children improved based on the experiments conducted with them as subjects.

The post-test of the average points on the creativity tests were improved by 1.77 points from post-test for the first question, and improved by 1.86 points from post-test for the second question. From this, it is considered that there is a possibility that this system will increase creativity.

In response to the creativity test, many of those who wrote responses applying the storylines displayed in this system were observed, and the effect of this system was considered. For example, the users chose “gentle personality” or “evil nature” in the system story. As a result, there were many cases in which subjects applied the character “gentle” or “evil” in the post-test, and created new stories using that personality.

In the questionnaire, “90% or more of the subjects gave the highest grade (5) to the questions,” “Was the system interesting?” “Do you want to do it again?” and “Was this system easy to understand?” The reason cited by many subjects was the fact that “many different stories can be made.” It is thought that it is interesting because the user to create his/her own favorite story. In addition, there were many statements that “Pictures and sentences are written and easy to understand.” Based on this, it can be said that displaying the narrative of the story as well as the animation makes the system easier to understand.

We also asked the question, “When you tried using the system, what were the bad points?” The users indicated that as a good point that “a variety of stories can be made” were many, but also indicated that as a bad point such as “it is difficult to see with bugs or blurry options.” It is may be necessary to separately adjust the layout of the screen to be displayed when projecting onto the dome-type screen. There is the opinion that “the screen is large and it was easy to see.” It could be thought that it is easier to see when projected 180° with a screen or dome type than having the system on a personal computer screen. In addition, because some subjects thought that “the length of the story is short,” we would like to take a long story into consideration. In future work, as there is still little commonsense knowledge in this system, we would like to increase the commonsense knowledge in the knowledge base.

6 Conclusion

In this study, we developed a system that generates and animates stories by the function of expanding stories from questions and definitions based on hypotheses. Users were able to create a story with a synopsis different from the original story by selecting and changing the definition of the story in the knowledge base system. In evaluation experiments conducted, children’s creativity improved as a result of experience with system we developed.