Keywords

1 Introduction

Mental illness has a long history of being stigmatised in societies around the world. However, only a small percentage of people with mental illness actively seek professional help for their illness [1]. A form of therapy called Cognitive Behavioural Therapy (CBT) is considered the gold standard in the treatment of a wide range of mental health issues [2]. A serious game is a game that does not solely rely on pure entertainment but instead on honing a specific skill set. Serious games encapsulate various genres. They have been applied for different purposes, such as education, military training, and healthcare [3]. By extending the reach of CBT to a serious game-based product, the possibilities become intriguing. Serious games can increase the impact of therapy interventions via three processes [4]. Firstly, it extends the reach of digital therapy to those who might not otherwise use it. Secondly, it improves the player’s immersion through both motivational and game-based mechanics. Thirdly, utilising numerous techniques for change, including therapy techniques and game features. It is considered feasible to translate forms of therapy, such as CBT, to games by exploiting features of games for therapeutic change [4]. This paper focuses on the application of serious games in healthcare.

The goal of serious games in healthcare is to create a simulated world in which the regular path is reversed, meaning that when the player tackles challenges in the virtual world, they gain skills and build the right mental fortitude in the real world [5]. These games allow the player the freedom to fail in a non-threatening environment where choices relate to clear consequences. However, an issue with these types of games arises since there is a noticeable decline in their use over time, and it is perceived that players lose interest in them within a short period. This issue is due to the lack of an engaging storyline [6]. Thus, an interactive story in a serious game can help add the hook that will pull the player in, achieve immersion, and encourage them to continue playing [7]. Also, confining the user to one fixed story is another issue because it will always be the same story every time. Therefore, the availability of a selection of stories can keep the player engaged and provide continuing support for their mental health and well-being. The work reported in this paper addresses both issues by exploring how CBT can be incorporated in serious games.

The remainder of the paper is organised as follows. Section 2 provides a literature review. Section 3 describes the target platform for the story analysis presented in this paper. Section 4 presents the story analysis based on a case study, The Selfish Giant by Oscar Wilde. Section 5 discusses the implementation of a story app that provides a target platform for stories incorporating CBT resilience activities in the storyline. Finally, Sect. 6 presents a brief discussion and concludes the paper.

2 Literature Review

2.1 Mental Health and Cognitive Behavioural Therapy

Everyone experiences some degree of anxiety; it is a natural response in our human nature. Stress helps the individual to recognise and respond to a threat by activating fight or flight reflexes. A regular amount of anxiety can help enhance people’s performance and trigger actions in response to dangerous situations [8]. However, persistent stress can develop anxiety disorders, phobias, panic attacks and obsessive behaviours [8]. An anxiety disorder causes a great deal of worry and stress intensely and frequently. It also hinders the individual’s ability to relax and experience a sense of enjoyment and well-being [9].

CBT has proven to relieve symptoms of anxiety and depression much more rapidly than traditional drug therapy or psychotherapy [10, 11]. The premise of CBT is that the individual’s thoughts affect how they feel and how they behave. In other words, it is not the situation that determines how one feels but one’s perception of the situation. CBT helps individuals identify their thought patterns and challenges their negative cognitive distortions of situations and events to improve mental regulation and reinforce a coping mechanism that targets analysing and solving current problems. CBT provides a way of prevention. However, it is not a cure. Nevertheless, continual use of CBT in the long term builds the individual’s resilience and ability to cope with adversity [10].

2.2 Storytelling and Interactivity

Every culture has told stories throughout its human history. Although different cultures have produced their unique way of storytelling, many aspects of storytelling are universal across all cultures [12]. In [13] the authors argue that, according to German scholar Gustav Freytag, a story plot comprises five essential elements [14]. The first element is the introduction and exposition, where an author would introduce story characters, establish the setting and present a first glimpse of the conflict to come. The second element is the rising action; this usually holds a series of events that build up to the story’s central conflict. The third element is the climax, which is the pivotal point of the story where everything converges. Typically, the main character would have to make some life-changing decisions and face the primary opponent of the story. The fourth element is the falling action, which shows the events and complications resolving; the consequences of the main characters’ choices during the climax are revealed to the readers. The fifth and final element is the resolution, which is the conclusion of the story and the consequence of the protagonist’s decision throughout the story.

Interactive storytelling gives the audience two essential tools that distinguish it from passive storytelling: choice and control. The audience gets to choose how to act within the interactive world, and their choices impact the story’s progression [15]. Taking first-person control of story events and directly taking part in the story are vital elements of a story’s interactivity. They induce a sense of belonging and competency in the individual when they feel responsible for the story unfolding. Individuals usually gain skills by witnessing cause-effect events and analysing their relationships. Thus, behaviour development can be improved when these stories address behaviour development issues, and the lesson to be gained from the tale encourages health [16].

2.3 Applications of Mental Health and CBT in Serious Games

The representation of mental health in digital games has been the subject of research. [17] and [18] provide a recent review of games that depict mental illness with varying degrees of implicit to explicit representation. Some of these games are: Hellblade: Senua’s Sacrifice, Night in the Woods, and Celeste [19, 20]. Moreover, several researchers have explored the incorporation of CBT into serious games [19,20,21,22] and [23]. These are described below.

In [21] and [22] the authors discuss a game called Please Knock on My Door, which is entirely based on its own developer’s real-life struggles with depression [24]. It articulates how the mental stability of the main character in the game interferes with his daily routine while struggling with severe depression. The role of the player is to help the in-game character overcome their depression. This can include the player tasking the in-game character with mundane activities, such as having food and going to work. The game does not offer a therapeutic intervention. However, it gives insight and awareness to the player of what goes on in the head of a person who is depressed or experiences social anxiety.

In [23], a platform consisting of an app and a website is discussed, called SuperBetter, which aims to engage individuals on a journey to pursue goals and address mental health challenges. The platform adopts a resilience model consisting of four resiliencies: mental, physical, social and emotional. Each type of challenge in the app corresponds to a specific type of resilience. Building resilience is one of the cornerstones of CBT. SuperBetter’s resilience model is adopted in this paper as the central approach in designing CBT stories for serious games.

3 The Target Platform

In the work reported here an app was proposed and developed as the target platform for playing CBT-based stories. The focus of the app is to play interactive fiction that incorporates CBT within the storyline as a way of providing therapeutic intervention for the players, who might be experiencing symptoms of mental illness. The interactive nature of the story adds a hook that can pull the player into the game, achieve immersion, and encourage the player to continue playing to push the story along [7]. It also provides support for playing multiple stories. This provides the availability of a selection of stories and storylines that can keep the player engaged in activities that support their mental well-being.

The CBT techniques incorporated in the app follow the resilience model described in Sect. 2.3. The resilience model targets the individual’s resilience skill, focusing on four types of resilience: emotional, which describes how well a person copes emotionally with stress and affects the individual’s ability to be imaginative and creative; mental, which is referred to as mental fortitude, is the ability to withstand or adapt to adversity and helps develop coping strategies; physical, which is the body’s ability to maintain stamina and recover quickly; and social, which helps reinforce positive human contact and social connections.

The work reported here also proposed but did not implement a story editor to support the creation of stories that incorporate CBT. A story is either composed from scratch or selected from a set of existing stories. The story is then analysed to look for ways to incorporate CBT resilience activities. The story editor supports the incorporation of the resilience activities into the storyline, encoding the story as an XML document with associated interactive media into a file that can be downloaded into the app where it can be rendered as a game.

Since story analysis is central to developing CBT stories, this paper focuses on story analysis and offers an illustrative example and a set of guidelines for incorporating CBT resiliencies into serious games.

4 Story Analysis

The authors approached the story analysis for interactive storytelling by selecting a short story and analysing it to discover the scope for the player to carry out CBT activities. The analysis comprised five steps: scene breakdown, character identification, emotions and personality traits, states, and activities. The story adapted for this approach was The Selfish Giant by Oscar Wilde [25]. The story turned out to be very appropriate in the context of mental health since it is full of meaningful and moral undertones. The Giant pushes people away and barricades himself inside his castle, wallowing in sadness and loneliness. This can be interpreted in the same way that people with mental illness deal with their mental health issues by shutting people out and putting up barriers around themselves to stop others from helping them. The following describes each of the analysis steps.

Scenes.

The first step was to break down the story into scenes. Freytag’s Pyramid plot structure was adopted, stating that a story consists of 5 main plot elements. After breaking the story down into those elements, the scenes could be identified easily. A new scene is identified whenever a new location is presented, or the story moves from one plot element to another. For example, consider the following excerpt taken from The Selfish Giant:

“One day, the Giant came back. He had been to visit his friend the Cornish ogre and had stayed with him for seven years. After the seven years were over, he had said all that he had to say, for his conversation was limited, and he determined to return to his own castle. When he arrived, he saw the children playing in the garden.” [25]

By analysing this excerpt, it is clear that this paragraph can be split into two scenes since this section takes place in two different locations.

Characters.

The next step was to extract all characters that play an active part in the story. These include the Giant, the children, birds, trees, snow, frost, north wind, hail and the Little Boy. Sometimes it is unclear who the characters in a story are since they can be whomever the author wishes. Consider the following excerpt:

“The poor Tree was still quite covered with Frost and Snow, and the North Wind was blowing and roaring above it. “Climb up! Little Boy,” said the Tree, and it bent its branches down as low as it could, but the Boy was too tiny.” [25]

This excerpt shows that characters can be identified by seeing whether they had any dialogue or interactions with other story objects. If yes, then that story object can be considered a character, as seen when the Tree speaks to the Boy.

Emotions and Personality Traits.

The next step was to record the emotional state of the characters in each scene. Emotions are critical for storytelling because it makes the product seem less computerised and more natural. Emotional states were the focus since emotions play an essential role in the mental stability of an individual.

A psychologist called Robert Plutchik theorised that there are eight primary emotions: fear, anger, happiness, disgust, surprise, sadness, anticipation, and trust [26]. Plutchik then created the wheel of emotions to demonstrate the various overlap and levels of emotions [26]. It was decided in the current work to focus on the primary eight emotions.

Then it was time to analyse the effect that different personality traits can have on a character’s actions. The most widely used model of personality traits is called The Big Five, a set of five broad, bipolar trait dimensions. Each dimension includes characteristics that can either enhance or mar mental health [27]. These traits are [28]: extraversion, openness, agreeableness, conscientiousness, neuroticism. These traits were analysed to identify the reasoning behind the characters’ actions in the story. For example, throughout the first part of the story, the Giant is identified as selfish. Selfishness is associated with the lack of consideration of other people. It could also be associated with withholding one’s possessions from other people and not sharing them. This can be very clearly seen from the following excerpt:

“When he arrived, he saw the children playing in the garden. ‘What are you doing here?’ he cried in a very gruff voice, and the children ran away. ‘My own garden is my own garden’, said the Giant; ‘anyone can understand that, and I will allow nobody to play in it but myself.’” [25]

The Giant refuses to share his garden with the children for no other reason than his selfishness. Therefore, it is seen that the Giant’s personality falls on the low end of the agreeableness spectrum. Thus, to help the Giant reach a good emotional state, he would have to be more agreeable.

Identifying personality traits also helps when trying to incorporate a more personalised CBT experience to the player. Since CBT identifies thought patterns and behaviours, its efficacy can be drastically enhanced by considering personality traits [29]. Moreover, identifying these traits adds another level of analysis that reflects the complexity of the real world and its complications.

States.

The next step was to record the location and emotional state of every character in each scene. Then, analyse how their emotional state changes with each scene and trigger that emotional state transition. In other words, a finite state machine would be created to represent the different states of the story. This approach was chosen because it is a straightforward way of keeping track of current story events and how to move forward from any given point by triggering an appropriate transition.

Firstly, an initialisation state was created to initialise all the characters states, with their attributes set to null (Fig. 1). Null means the character’s attribute at this stage is undefined because the character has not come into the scene yet.

Fig. 1.
figure 1

Initialise states

Full size image

Then a composite state for each scene was created. Each scene state includes all the currently active character states and their attributes. Then the transition from each scene to the next is either by a CBT activity or a simple scene transition.

Two different ways were created to represent the state machine to enhance the clarity of the transitions and the different emotional states. The first representation shows the complete transitions throughout the story, including scenes and the characters’ emotional state and location. This type of representation shows all the states of the characters, even if their state is still undefined or has not changed from one scene to the next. Figure 2 is a sample taken from the complete transition state machine diagram.

Fig. 2.
figure 2

Partial state transition sample

Full size image

The second representation shows a single character’s transition at a time, choosing to focus on the character’s arc at any given scene instead of including every character. Figure 3 is a sample taken from the Giant’s transition state machine diagram.

Fig. 3.
figure 3

Single character transitions

Full size image

Activities.

The process of identifying the resilience activities included analysing the story sentence by sentence in search of keywords and specific plot points that indicate where an activity can be integrated to push the story forward. Each type of resilience was mapped to an activity based on the following assumptions:

  • The character conversing with other characters through dialogue interactions can suggest a social resilience activity.

  • The character experiencing a strong emotion can suggest an emotional resilience activity.

  • The character being inquisitive or investigative of their surroundings suggests a mental resilience activity.

  • The character engaging in any physical activity, i.e., walking, going up the stairs, can suggest scope for a physical resilience activity.

The following excerpt shows an example of a resilience activity:

“‘What are you doing here?’ he cried in a very gruff voice, and the children ran away.”

This scene gives scope to implement a social resilience activity. The player’s role would be to verbally encourage the Giant to interact with the other characters in the most socially appropriate manner. Additionally, that same scene can allude to an emotional resilience activity since the tone of the Giant’s speech indicates anger, which is a strong emotion. In the emotional resilience activity the player would reflect on emotionally soothing visionary things, e.g., happy pictures and nature scenes.

The following excerpt shows another example that suggests a resilience activity:

“It is your garden now, little children,’ said the Giant, and he took a great axe and knocked down the wall.” [25]

This scene holds scope for implementing a physical activity since the Giant knocked down a wall. In this scene, the player’s role would be to do a physical resilience activity to encourage the Giant to knock down the wall.

The following excerpt shows scope for implementing the last type of activity, mental resilience:

“I cannot understand why the Spring is so late in coming,” said the Selfish Giant, as he sat at the window and looked out at his cold white garden; “I hope there will be a change in the weather.”

This scene shows the Giant looking out the window and observing his surroundings. This indicates scope for a mental resilience activity, where the player should be inquisitive and investigative to answer questions relating to the in-game surroundings.

When implementing these activities, it was decided to add additional dialogue cues and CBT activities. However, it was necessary to be respectful towards Wilde’s writing and remain as close to the original text as possible. It was then observed that it is possible to integrate the resilience activities within the state transitions created from the previous step in the analysis.

Screenplay.

Following the above analysis steps, and to aid the analysis, the story is set out in the form of a screenplay. The traditional screenplay format was adopted, also known as The Master Scene Format. This format typically consists of six main elements: scene headings, the action, character names, parentheticals, character dialogue, and transitions [30].

The first element, the scene heading, known as the slug line, is typically written in all caps. It would include INT or EXT for interior or exterior. Then it is followed by the location where the scene takes place and the time of day. The second element is the action. The action describes any action that can be seen or heard. Any sound effects should be written in all caps [30]. The third element is the Character name. The Character name should be written in caps and located in the centre of the page. A character who is speaking in voiceover or is off-screen is designated by V.O or O.S, respectively. The fourth element is Parentheticals. Parentheticals should describe physical or emotional action. It details how the character is speaking the dialogue. The fifth element is character dialogue. Dialogue should be centred on the page; it is what the character says. The sixth and last element is the Transition. The Transition indicates the end of a scene. It is usually written in caps [30]. Going through these steps made it possible to integrate the interactive CBT based storyline with the screenplay format. Figure 4 illustrates, using annotations, how the CBT based activities can be incorporated.

Fig. 4.
figure 4

Story screenplay

Full size image

Summary of Analysis.

In sum, analysing a story to implement therapy intervention techniques is quite challenging at first. However, it is possible to do so by following a number of straightforward steps after choosing a story.

The first step is to break down that story into sections; these sections represent different scenes in the story. The second step is to try and identify the characters in the story and identify their impact on the story. What are their roles? Are they a hero, villain, or supporting character? In the third step, the author should identify all the characters’ emotions and personality traits: to be able to understand how their personality affects their behaviour and to add a level of depth to the reasoning behind their actions. The fourth step is to create states for the characters: to monitor their emotions in each scene, to analyse what triggers their emotions to change, to understand what impacts their emotions, and to find opportunities to induce a change. The fifth step is to identify what CBT activities to incorporate within the story and the most appropriate place for the activities to be incorporated naturally while keeping the flow of the story consistent. After the analysis is done, the next step is to compose the story following the proposed screenplay format.

5 The Story App

The development of the CBT-based serious game consists of three stages. First is composition: the story editor will enable the author to compose a story. The author will be able to choose a visual representation for the characters and scenes in the story and choose to add resilience activities that integrate CBT within the story. The second stage is story mapping, where the screenplay composed by the author is compiled into a jar file. Each resilience activity is mapped to an XML representation in the Editor and a set of java classes in the story app. That jar file includes the screenplay and all media incorporated by the author in the story. The third stage is animation, where the interactive story incorporated in the jar file is played through the story app. The story app can hold an archive of stories composed and uploaded by authors. The player would be able to pick and choose stories to download to the app and play. The story app has been developed and is now a working android application.

5.1 Implementation

The application was developed using Android Studio IDE, which uses the Java programming language and XML. The interface comprises simple graphical elements such as buttons, images, and text fields to avoid overwhelming the interface with different features. A colour scheme was selected for the app based on the psychological properties of colours. In isolation, a single colour on its own has psychological properties, but when combined with other colours, it can produce an emotional response from the user [31]. The right colour combination can act as an unspoken guideline to the user for how to navigate the app [32]. Therefore, it was an important aspect to take into consideration during the implementation. An XML document was created as a representation of The Selfish Giant story and its assets. This document was created using XML Notepad. The parsing of the XML document was implemented through the application’s programming interface API: Xml Pull Parser (XPP).

5.2 Walkthrough

The main interface includes guidelines for the player on using different aspects of the app (i.e., resilience activities) and downloading new stories. Furthermore, a scrollable view at the bottom shows the archive of stories (Fig. 5. A). For the present, only The Selfish Giant story is playable. When the player chooses a story, they are taken to that story’s interface and prompted to start their adventure (Fig. 5. B). All illustrations used in the story were adopted from the illustrated version of the book [33]. The story’s interface is also equipped with a state window that shows the emotions of all in-game characters in any given scene (Fig. 5. C).

Fig. 5.
figure 5

GUI samples

Full size image

The user can also infer the characters’ emotions from the colours used in the interface. For example, when the Giant is happy, the colour of the text view changes to yellow. If he is angry, it changes to red. Moreover, each type of colour representation is accompanied by an animation of the text view that intensifies that feeling, i.e., when the emotion is happy, the whole text view is animated to jump up and down as if it is replicating being happy. If it is angry, then the text view shakes aggressively from side to side.

The player’s goal is to maintain the characters’ emotional state and the overall state of the game world to achieve harmony. The emotional state is affected by four resilience skills: emotional, mental, social, and physical. Each type of resilience maps to an activity. If the player completes an activity, they will progress through the story. The following figure shows samples of each resilience activity (Fig. 6).

Fig. 6.
figure 6

Resilience activities

Full size image

6 Discussion and Conclusion

The research reported in this paper delivered the following:

  • A guide on analysing a story to incorporate CBT resiliencies.

  • A modified story script of The Selfish Giant, following the proposed guide.

  • A set of requirements for the development of the story editor.

  • A fully functional android application that can play CBT stories.

Analysing a prewritten story proved to be especially challenging. It was challenging to identify where to bring in the CBT activities within the story and finding ways where the transitions would fit in naturally and not seem out of place. However, the authors were able to incorporate at least one of each type of activity within the story. It was concluded that the activities would need to be more diverse and inclusive to work with any given story. It was also challenging to give feedback to the user without being patronising, so it was decided to substitute the typical levelling up mechanism within digital games with a more implicit approach using sound effects and colours.

Also, currently only a single story has been analysed and implemented in the app. Therefore, as part of moving forward, a range of stories will be analysed using the proposed analysis steps to prove and refine the proposed analysis process and the app.

In addition, a further level of complexity will also be explored by offering a branching storyline. This means the story does not have a linear sequence of events, but multiple different directions and endings to explore, offering freedom of choice throughout the story. Creating branching stories would also provide a more personalised user experience.

In conclusion, the work reported in this paper offers a means for expanding the provision of CBT within serious games. The proposed story editor offers scope for authors to create CBT based stories for serious games and the implemented app offers scope for downloading and playing these stories. The work aims to contribute towards breaking the stigma surrounding mental illness and providing help to those in need. However, studies under the guidance of mental health experts are required to evaluate the impact of the approach on mental health.