Introduction

Mindfulness is becoming increasingly popular, and the number of mindfulness interventions is increasing. At the same time, scientists criticise the down side of this movement, realising that “scientific literature is plagued by conceptual and methodological shortcomings [and] divulged in a sensationalist way” (Farias and Wikholm 2016; p.329). Of particular interest are the digital interactive mindfulness interventions, which lack scientific rigour and formal evaluation. Plaza et al. (2013) conducted a literature and software review of mindfulness-based mobile applications (apps) and concluded that there is “a complete lack of evidence to support the usefulness of those apps” (Plaza et al. 2013, p.13). To date, very few original apps exist that were specifically designed and tested for mindfulness (excluding guided-meditation programs). The AEON app was developed to improve the mindfulness factor Decentering (Chittaro and Vianello 2014), while a modified version of the Positive Activity Jackpot was used to cultivate the mindfulness factor Openness to experience (Sliwinski et al. 2018b). Yet, there is no digital interactive intervention that is aimed at training mindfulness by means of targeting all of its known evidence-based factors. This study fills the gap by taking a systematic research-based approach to the development of a digital game for mindfulness. The game medium was chosen because of its interactive nature and immersion of experience, which allows the cultivation of experiential knowledge at the same level as mindfulness meditation. Furthermore, games are designed to be motivating, engaging and entertaining, while meditation alone is sometimes perceived as a daunting task (Lomas et al. 2015).

The first challenge is to define mindfulness. There is no consensus about which, or, how many factors the concept of mindfulness includes. Indeed, there are numerous questionnaires and definitions (Bergomi et al. 2013; Malinowski 2008), which so far have consistently failed to capture the concept of mindfulness adequately (Chiesa 2013; Park et al. 2013). The game discussed and evaluated in this study is based on the factors elucidated by the Comprehensive Measure of Mindfulness Experiences (CHIME; Bergomi et al. 2014) questionnaire, as it assimilates available operationalisations and their formalised mindfulness factors to create a more complete definition of mindfulness. The CHIME defines mindfulness as a quasi-trait that is an inherent capacity in human beings that can be trained (Bergomi et al. 2013). The CHIME is in line with the operationalisation of mindfulness by Bishop et al. (2004), while integrating available research evidence from prior studies about the nature and measurement of mindfulness. Before construction, the authors carefully reviewed the most popular questionnaires and investigated the factors that they measure through a literature review of their theoretical basis as well as a comparative analysis of established mindfulness factors (Bergomi et al. 2013). Nine aspects of mindfulness were identified by CHIME that are covered by previous operationalisations of mindfulness and reduced to eight factors using factor analysis (Bergomi et al. 2014).

The final definition of mindfulness and resulting measurement instrument consists of eight individual mindfulness factors, which are (1) Awareness towards inner experiences; (2) Awareness towards outer experiences; (3) Openness to experience (non-avoidance); (4) Decentering (non-identification and non-reactivity); (5) Acceptance (accepting, non-judging and (self-)compassionate orientation); (6) Relativity of thoughts and beliefs; (7) Insightful understanding; and (8) Acting with awareness (being in the present moment). The comprehensibility and high content validity of this questionnaire make it an appropriate tool for capturing a more complete view of mindfulness (as defined and recognised in the scientific literature). The CHIME instrument is used in this study to disaggregate mindfulness into eight distinctive factors, permitting an investigation into mindfulness, its individual parts, and their relationships.

A thorough review of existing digital interactive approaches that are feasible for training each of the eight CHIME factors was conducted (Sliwinski et al. 2015), and from the results, design implications were derived in the form of practical mechanics, dynamics and aesthetics (Hunicke et al. 2004). To further validate the software choices, experts from the domains of mindfulness and games research were asked to state the feasibility of each approach to its associated mindfulness factor (Sliwinski et al. 2018a), confirming the selection of interactive approaches. Three mindfulness factors lacked a research-based digital mindfulness intervention, for which suitable games were evaluated (Sliwinski et al. 2018a) and their effective mechanisms of action identified. Based on the gained insights, a design framework was created, which provides a way of understanding and organising the key concepts and effective training aspects to aid the development of mindfulness games (Sliwinski et al. 2018a).

This ground work was used as a framework to design the Interactive Mindfulness Program and Virtual Evaluation (IMProVE) game, an HTML5-based game in the infinite runner genre, designed as a casual game, which uses embedded game metrics to analyse the player’s mindfulness state. IMProVE was developed following user-centred design principles, including persona development, heuristic analysis and requirements engineering, as well as integrating knowledge acquired about designing effective game elements to train mindfulness.

The IMProVE digital game was developed as a complementary support tool for practising mindfulness, i.e. a more beginner-friendly gateway into mindfulness practice. Acknowledging the full value of mindfulness meditation, IMProVE is not aimed to be a substitute or “meditation 2.0”. Additionally, IMProVE includes embedded analytics, which can be used to measure the player’s level of mindfulness, which is derived from the personal play style and in-game performance. Developing an objective measure of mindfulness alongside digital interactive mindfulness training provides a long-awaited novel and original contribution in the field of human-computer interaction and mindfulness research (Baer et al. 2006; Bergomi et al. 2013; Bishop et al. 2004). The aim of this study is to evaluate IMProVE, which we hypothesise to have a positive effect on mindfulness and the individual factors that it comprises. Furthermore, we investigate whether metrics embedded in IMProVE can provide a method to assess the player’s mindfulness, and which should correlate significantly with self-reported mindfulness.

Method

Participants

A total of 586 participants were recruited from relevant social network groups, online forums and newsletters, of which 94 participants completed the game and the post-test. Data collection was performed between 20 March and 9 May 2017. Most participants who did not complete the game, did not start it at all and left the website on the welcome screen (of which some were presumably web crawlers and other internet bots) or left during the first questionnaire (which probably deterred them from continuing the process).

The participants’ median age was 34.5 (range 19–70) of whom 46 were females and 48 males. 77.8% had English as browser language, while 61.7% were identified as US English, which is used as a proxy for identifying the participant’s nationality. The lowest browser resolution was 1024 × 768 pixels, which was sufficient to play the game that was built with a native resolution of 640 × 480 pixels (adapting to higher resolutions to reach full screen). Twenty percent of all participants played IMProVE in the presence of other people.

Following the study design by Lau et al. (2006), meditation experience was assessed with two questions (“Have you ever attended any course and/or do you practice any form of meditation, e.g. mindfulness, yoga, tai-chi, qigong?” and “Do you practice or have you ever practiced mindfulness meditation on a daily basis for at least 8 consecutive weeks?”). Stating “yes” on either question indicates some mindfulness experience, and participants stating “yes” on both questions were coded as experienced meditators. The duration of 8 weeks was chosen, because this is the typical duration of mindfulness-based treatments, which could be argued as the minimal amount of practice to learn the skills necessary for cultivating mindfulness. However, it is acknowledged that the term experienced is open for interpretation and might also entail much more practice. The sample distribution regarding meditation experience consisted of 21% with no experience, 30% some experience and 49% high experience. Gaming experience was measured by two questions (“How many days per week do you play digital games?” and “How many hours per day do you play digital games?”) that were used to classify participants into the groups of no experience (44%) and some experience (at least 1 h/week; 56%). The study was completed with ethics approval from the University of the Sunshine Coast (S/16/992). Participation was incentivised with a surprise gift for the player who achieves the highest score.

Design

A one-group pre-test-post-test design was applied, where each participant played the IMProVE game for 15 minutes. State mindfulness was measured before and after the intervention. Additionally, mindfulness was measured by means of analysing the player’s game performance.

Measurement

State Mindfulness Scale (Tanay and Bernstein 2013)

With its two factors, State Mindfulness of Mind and State Mindfulness of Body, the State Mindfulness Scale (SMS) is conceptually aligned to the two-factor definition by Bishop et al. (2004), both taking traditional Buddhist scholarship as a conceptual foundation. The 21 questionnaire items were generated based on the feedback from mindfulness researchers and instructors (Tanay and Bernstein 2013). Participants have to state their agreement with each statement on a 5-point Likert scale (i.e. “1” not at all, a little, somewhat, well, “5” very well). Exploratory and confirmatory factor analyses were based on 353 adult participants from three different samples. The model revealed to have a good fit and demonstrated robust item-factor, and sound internal consistency with a Cronbach’s α of .95 (Tanay and Bernstein 2013). Exemplary items are “When my mood changes, I notice it right away” and “I see my mistakes and difficulties without judging myself”. The SMS instructs participants to focus on the past 15 minutes. A reliability analysis on the current sample revealed a Cronbach’s α = .96 for the SMS total at both pre-test and post-test. The internal consistency for the sub-scales was for the Mind factor α = .94 (pre-test) and α = .95 (post-test), and for the Body factor α = .9 (pre-test) and α = .89 (post-test).

Comprehensive Inventory of Mindfulness Experiences (Bergomi et al. 2014) Visual-Analogue Scale

The CHIME scale measures eight individual mindfulness factors with 37 items to which participants have to answer on a 6-point Likert scale (i.e. “1” almost never, infrequently, somewhat infrequently, somewhat frequently, frequently, “6” almost always). The questionnaire demonstrated an internal consistency of Cronbach’s α = .93 and was successfully applied in healthy individuals (Bergomi et al. 2014), which is the target population of this study. An acceptable sensitivity has been shown, especially for participants without meditation experience (Bergomi et al. 2014), which corresponds to beginners in mindfulness practice as targeted by this study. The scale has been shown to be stable and shows no systematic variances for different age or sex (Bergomi et al. 2014).

CHIME was used to develop IMProVE as it provides the most comprehensive operationalisation of mindfulness to date. To measure the eight individual mindfulness factors, as defined by CHIME, on a state level, we developed a visual-analogue scale (VAS). The CHIME VAS consists of eight items, one for each mindfulness factor, which were formulated based on existing CHIME items of the full scale. For this purpose, the items, which are formulated as statements, were synthesised and rephrased as one question per factor (see Table 1 for all items). Anchors for each items were “Never” and “Always”. Participants answered the scales by means of sliders, which coded their position as values from 0 to 100. To investigate the reliability of the CHIME VAS, a reliability analysis was conducted, demonstrating a Cronbach’s α of .62 at pre-test and .7 at post-test.

Table 1 Items of the CHIME visual-analogue scale
Full size table

IMProVE Game Metrics

The player’s performance and play style were used to measure the level of mindfulness. Relevant variables derived from the game were the number of successful and failed key breaths, the frequency of mindfulness items picked up, and the total game score, which includes breaths tracked and all items picked up (see Table 2).

Table 2 Integration of mindfulness metrics in IMProVE
Full size table

User Experience Questionnaire (Laugwitz et al. 2008)

To assess the user experience of IMProVE, the User Experience Questionnaire (UEQ) consists of 26 items, formulated as opposite adjectives, which are scored on a 7-point scale. The items correspond to six dimensions, namely Attractiveness, Perspicuity, Efficiency, Dependability, Stimulation, and Novelty. The UEQ was constructed by generating an item pool (initially 80 items) by user experience and usability experts (Laugwitz et al. 2008). A factor analysis revealed the structure of the UEQ. Items with the highest loadings on each factor where selected to be included in the final version. The UEQ demonstrates satisfactory levels of reliability and construct validity, with the Cronbach’s α for the sub-scales ranging from .60 to .88. Exemplary items are “boring-exciting” and “complicated-easy”.

Procedure

The game process commenced with an introduction screen to brief the participant, with a link to the research information sheet. After that, participants filled out a demographics questionnaire, which asked age, gender, prior game and mindfulness experience, as well as asked for a thought, which the participant wanted to distance from (relevant for the Decentering game element). Next, the SMS and CHIME VAS were applied to identify the participant’s level of mindfulness at baseline. On the next screen, a short story introduction formed the transition into the game (“You are in the role of a mindfulness student, stranded on the sea with your kayak. Your mission is to navigate through the water to reach the mainland”), which started with a tutorial that explained the controls. Participants played IMProVE for 15 minutes, after which the post-test was triggered. CHIME VAS and SMS assessed the state of mindfulness after the game. To gather insights on the participant’s user experience of the game, the UEQ was applied, accompanied by an open question (“Please share your feedback about the game (comments, remarks, suggestions, etc.)”. Finally, players were thanked for their participation in the study (Fig. 1).

Fig. 1
figure 1

Study procedure with timestamps for individual events

Full size image

Game

IMProVE was developed as a browser-based game in the infinite runner genre, meaning that the player’s avatar is constantly on the move, while items and obstacles move towards them (see Fig. 2). The player is in the role of a mindfulness student, stranded on the sea with his/her kayak and the mission to navigate through the water to reach the mainland. This theme was chosen because water has a naturally calming effect and because controlling a kayak translates well into the mindfulness mechanic of breath awareness, i.e. using the metaphor of alternate paddle strokes for in- and exhaling. The player moves automatically forward, though can increase speed by using the paddles (i.e. conscious breathing). Fish and scenery in the background were designed to facilitate the feeling of movement and depth. In dynamically adjusted intervals, lotus flowers are spawned by the game (moving from a random top position to the bottom). Picking up a lotus flower rewards the player with points. Similarly, the player has to avoid collisions with ice bergs, which subtract points from the score. The game starts with a tutorial that explains in a step-by-step manner how movement controls work and how to collect items and avoid obstacles. Furthermore, the tutorial explains how the breath tracking works and what key breaths are, as well as how to submit them. All tutorial steps are experiential, meaning that the user must perform that required action to progress. All interactions (e.g. item pickups), movement (paddling) and environmental effects are accompanied with visual and audio feedback (for example, the kayak and score blink when bumping into an ice berg to visualise the point subtraction, and a thud sound is played).

Fig. 2
figure 2

Screenshot of IMProVE

Full size image

Additionally, a calm background music that fits the current setting is played throughout the duration of the game. To make the game more engaging, weather effects are triggered, including clouds, rain, thunder, storms and darkness of night. IMProVE is available at http://www.mindfulgaming.org/improve

To motivate the player to breathe consciously, the breath is directly linked with the kayak’s control. The game instructs the player to reflect on inhaling and exhaling by pressing and holding alternate buttons (“W” and “E”). These map to alternate paddle strokes. The method of breath counting was integrated into the game as the player must press a special key ("Space") on each and every 10th breath, which is then rewarded with additional points and encouraging audio-visual feedback (i.e. uplifting sound and a “Good Job!” message).

Mindfulness mechanics, dynamics and aesthetics were integrated based on available research evidence, as presented by a prior review and evaluation study about effective mindfulness elements by Sliwinski et al. (2018a). Table 3 outlines how effective elements to improve mindfulness were used per mindfulness factor, as defined by CHIME (Bergomi et al. 2014). For example, as breath counting was found to be an effective way to train the mindfulness factors Awareness towards inner experiences and Acting with awareness, it has been applied in this game by having the player breathe consciously through mapping inhales and exhales to different control keys and submitting every 10th breath with a different key to get more points. Openness to experience training was implemented by micro tasks of self-reflection, following the example of the Positive Activity Jackpot app modified for mindfulness training. Decentering was based on the model of the AEON app, where a player types in a thought and observes its disappearance, as a way to distance themselves from that thought. Positive accepting messages, triggered by items that can be picked up, were used to increase the factor Acceptance. Relativity of thoughts and beliefs is stimulated by challenging the established beliefs of how the game works through inverting movement controls.

Table 3 Integration of mindfulness elements in IMProVE’s game design
Full size table

Analysis

All data was gathered by the game and saved as JavaScript Object Notation (JSON), which was exported into a tabular CSV format. Analysis was conducted using SPSS Statistics 22 (IBM Corp. 2013). Descriptive statistics were used to explore individual differences between participants. Reliability statistics were conducted for all scales and sub-scales used. Experimental data obtained from mindfulness questionnaires were analysed by means of repeated measure ANOVAs. Post hoc tests were applied to highlight where differences occur. To validate the feasibility of IMProVE metrics as a method for assessing mindfulness, game variables were compared with obtained results from questionnaires. P values of multiple comparisons were corrected by controlling for false discovery (FDR) based on the method by Benjamini and Hochberg (1995).

Results

Repeated measure ANOVAs were applied to test the learning effects, i.e. impact of the game on mindfulness (Table 4). A statistically significant effect was found on state mindfulness, as measured by the SMS total scores, ANOVA, F(1, 93) = 10.13, p < .01, ηp2 = .1. This effect was present for the SMS sub-scale Mind, F(1, 93) = 28.29, p < .01, ηp2 = .23, but not Body, F(1, 93) = .504, p = .48, ηp2 = .01. No interaction effects were revealed for gender, age, gaming experience and mindfulness experience for any of the conducted tests.

Table 4 Repeated measure ANOVAs
Full size table

No significant effect was found for the CHIME VAS total score. Investigating each CHIME factor individually, significant effects were found for three of the eights factors, namely a negative learning effect for Awareness towards outer experience, F(1, 93) = 14.81, p < .01, ηp2 = .14; a positive effect for Decentering, F(1, 93) = 7.89, p < .01, ηp2 = .08; and a positive effect for Acting with awareness F(1, 93) = 4.6, p < .05, ηp2 = .05. No interaction effects were found for gender, age and gaming experience. Different effects were observed for participants with different levels of mindfulness experience. Visual comparison of the groups with different mindfulness experience suggests that those with high mindfulness experience receive the greatest benefits from the game, but the effect is only marginal, as no significance was reached when testing it.

A correlation matrix was computed between all applied mindfulness measures (Table 5). The SMS total scale correlated significantly with its sub-scales and the CHIME VAS total, as well as all the CHIME factors except for Relativity of thoughts and beliefs and Insightful understanding. The CHIME VAS total correlated significantly with all CHIME factors and all SMS factors. Two of the CHIME factors, Relativity of thoughts and beliefs and Insightful understanding, showed nearly no significant correlations to the other variables.

Table 5 Correlation matrix of applied measures of mindfulness
Full size table

The relationship between applied mindfulness measures and the game’s metrics was investigated (Table 6). The game’s total score and the number of successful key breaths was found to correlate significantly in the negative direction to the SMS and CHIME total scores, as well as to the SMS sub-scales and the CHIME factor Openness to experience. The number of mindfulness items picked up in the game showed no significant correlations to the total scores of mindfulness questionnaires. The four categories of mindfulness items (Acceptance, Decentering, Openness to experience, Relativity of thoughts and beliefs) were correlated individually with their associated factors, which showed one significant correlation only, for Relativity of thoughts and beliefs, r = .22, p < .05.

Table 6 Relation between 12 mindfulness measures and three game metrics (key breaths, mindfulness items, and score)
Full size table

To assess the user experience of IMProVE, a semantic differential graph was created based on UEQ scores (Fig. 3). The five most salient qualities of the game, which were identified by the items with the highest deviations from the neutral point, were the features of being friendly, easy to learn, slow, good, and easy. Additional qualitative aspects were attractive, organised, secure, clear, inventive, valuable, motivating, understandable, creative, interesting, supportive, pleasant, practical, efficient and meeting expectations.

Fig. 3
figure 3

Results of the UEQ semantic differential scale. The player was asked to rate the experience on spectrums between two opposing qualities; neutral ratings are represented by the middle line

Full size image

An open question after the UEQ gave players the opportunity to leave additional feedback. Overall, the game was perceived as very positive and effective: “The game had a generally good flow”, “Sound and atmosphere were very dense despite the simple art style.”, “I felt empowered by the activity and relaxed. I really like this!”, “It was an excellent demonstration of mindfulness and clearly coupled mindfulness with a video game. It was a pleasant experience.”, “It is very interesting and relaxing, made me aware, and focused on a present.”, “I can really feel the difference in my concentration and focus now.”, “Overall I feel calmer, more focused, and like I have a better understanding of the original negative/limiting belief.”, “I loved the experience. Really well done and innovative. I enjoyed it pretty much”, “I really appreciate this game. It forced me to be conscious of every action, the breathing, the space on the 10th exhale, sometimes the controls inverting, etc.”, “There was a moment where I became self-aware of what was happening and how gameplay mechanics prescribe and provoke certain reactions, couldn’t stop smiling since! Great use of the games medium for practical meditation. One-of-a-kind.”

The pace of the game was rated as challenging but appropriate; however, the amount of stimuli was rated by some players as overwhelming: “The controls felt overwhelming at first, but it was a fun new challenge to steer the boat while pressing keys according to the rhythm of my breath.”, “Having to do multiple things at the same time was confusing (e.g. breathe, move boat, think about answers to scales), but rewarding when I managed to do it. I think it was well paced, with periods of calm and then suddenly lots to do which made me aware of how easily distracted I was from the breathing movements.”, “Difficult to concentrate on moving the boat at the same time as focusing on the breath to move paddles.”, “I felt frustrated by the pace at first, but was glad to have the time to relax and contemplate my experience.”, “For me, there was too much going on the screen. It was very distracting.”, “There was a point in the game where I decided to just focus on the breath and ignore whatever else is going on.”

Many participants reported the breath tracking mechanic to be confusing: “It was unclear at the beginning whether or not I was supposed to breath 10 times and then hit spacebar, or to hit spacebar instead of the W or E key on the 10th breath. Took me a few minutes to figure it out.”, “The space pressing at every 10 breath was confusing. I thought after 10 breaths I have to press space and not instead of the 10th breath.”, “What annoyed me was the fact that I kept hitting (or not hitting W and E), and therefore missed most the SPACE times”, “my attention was geared at mastering the keys, counting my breath and noticing what was happening on the screen. That competed with noticing how I felt.”, “It wasn’t clear if I should just click on the space bar on the 10th breath or the E and then the space bar. Also, I count one breath as in and out, to that confused me at first, too.”

Another frustration was the randomness at which positions the items were spawned: “The spawning of lotuses and rocks was a bit too random. Sometimes both things were so close to each other and even overlapping, that I didn’t thought it was worth moving so close towards the rock.” Also, the symbolism of the items was confusing to some players, as well as the fact that they were not introduced in the tutorial: “It was confusing to me, that objects floated by, that weren’t introduced in the tutorial”, “Better explain what some icons mean (if they are negative of positive)”, “I thought the whirlwinds, which make your steering inverse, were saws and thus a danger to my kayak.”. Next, the experience identification task for Openness to experience was criticised: “The questionnaire thing was a sequence that didn’t really worked for me. Also it was more abstract then the rest of the game and thus pulled me out of the experience a bit.”, “I feel it got a little repetitive and there could have been some more scales for variation.”

On a more general note, some players expressed their concern as to how the game was linked with mindfulness: “My biggest gripe with this game though is, that score, the controls for breathing while steering and avoiding or hitting obstacles and lotus flowers felt ultimately meaningless”, “Linking the concept and practice of mindfulness to a points driven game seems counterproductive.” Also, some of experiences produced mixed results in the context of mindfulness: “I enjoyed the game, and found myself feeling quite disconnected from my body while playing. While I was more in tune with my breath than I might typically be while on the computer, I felt even less in touch with the rest of my physical experience.”, “I don’t think it helped me connect with my body but I did increase awareness of my frustration during the game.”

Feature requests included: “I would suggest having the user input three or more thoughts they are having, perhaps some positive ones too. That would serve to reinforce that positive thoughts come and go, just like negative ones.”, “Then at the end of the game, the user could be given some kind of score, and also given some indication of the regularity of their breathing (or lack of), and also how their breathing changes during different stages of the game.” Lastly, some players reported infrequent bugs as well as the odd situation, where the game stopped spawning any items.

Discussion

This study evaluated IMProVE, a novel game-based approach for mindfulness training and assessment. To investigate its efficacy for mindfulness training, two state mindfulness questionnaires were administered at pre- and post-test. To validate the feasibility of using the game’s metrics as mindfulness measures, these were analysed against the administered questionnaires. The game’s user experience was derived from players perceived qualities on a semantic differential scale, as well as one open ended question.

Feasibility for Mindfulness Training

Results from the SMS showed a significant improvement in obtained mindfulness scores after playing the game, which confirms the game’s suitability for mindfulness training. This effect was limited to the Mind factor, which focuses on mindful awareness of thoughts, emotions and sensations. No significant increase was observed for the Body factor, which related to mindful awareness of physical sensations only. This might be due to some items of the Body sub-scale, which might not fit well to the nature of the game-based mindfulness intervention, for example “I felt in contact with my body.” and “I changed my body posture and paid attention to the physical process of moving.” (Tanay and Bernstein 2013). However, the game did aim at increasing awareness of bodily sensations as well, by targeting the CHIME factors Awareness towards inner experiences and Openness to experience, so it could be also argued that IMProVE was unsuccessful in achieving a significant benefit in this regard.

To examine mindfulness comprehensively, all eight CHIME aspects were measured by means of a self-developed visual-analogue scale. Although the incremental validity of the scale slightly changed between pre- and post-test, it demonstrated an acceptable reliability (α = .74). Furthermore, the CHIME VAS was correlated significantly with the SMS. No significant changes were observed for the CHIME VAS total score after playing the game, which suggests the game may not be effective for training mindfulness, as based on the CHIME model. To obtain a more detailed picture of the game’s impact, individual CHIME factors were analysed. A significant positive effect was found for the Decentering factor, confirming the successful implementation of its game design element as proposed by the AEON app (Chittaro and Vianello 2014). A significant positive effect was measured for the factor Acting with Awareness, which suggests a beneficial effect and successful implementation of breath counting as the core game mechanic (Levinson et al. 2014).

A negative significant effect (decline) was reported for the mindfulness factor Awareness of outer experiences, which is interesting, since the game was designed to achieve the opposite. This phenomenon, however, might be rooted in the experimental design. The factor was measured by the item “How aware are you of your surroundings”, which was asked directly before and after playing the game. It might be that the player’s attention was fully captured, thus focused on the virtual world by the game’s engagement, and this in turn might have caused a lack of attention to outer surroundings. It could be that the participant would have scored differently if the question was asked at a later point in time, thus eliminating the direct effect of the game and letting the intended mindfulness learning effect sink in. Indeed, most studies investigating this mindfulness factor compared gamers with non-gamers in cross-sectional studies (Sliwinski et al. 2015), and not in an experimental study with pre- and post-measurement.

For the six remaining CHIME factors, no significance was reached, which implies that the applied game design elements that were tailored to these factors were not effective. The contradicting results between SMS and CHIME scores could be explained by the fact that the SMS is largely affected by the Decentering factor, as shown by its significant correlation with the Decentering sub-scale of the Toronto Mindfulness Scale (TMS; Lau et al. 2006) in its validation study (Tanay and Bernstein 2013). Hence, the positive improvement in Decentering from the IMProVE might have pushed the SMS into significance.

The key mechanic of the game was breath counting, i.e. controlling the player’s avatar by means of tracking one’s own breath. In the original breath counting exercise by Levinson et al. (2014), participants counted their breaths without any external help and submitted the 10th breath based on their mental calculation, which required sustained attention. In IMProVE, however, players are presented with a visual feedback indicator, which counts the breaths automatically. This user interface element was included after receiving user feedback from the prototype, saying that it was too difficult to count one’s breaths, especially when other things were going on in the game (i.e. collecting items and avoiding obstacles). Providing this visual breath counter made the game easier, but might have caused many of the player’s to not engage cognitively in breath counting, i.e. too much visual feedback for the breath counting and its resulting computational offloading might have prevented the allocation of cognitive resources. Because of this, the CHIME factor Awareness of inner experiences might have not been improved. This was in fact confirmed by the feedback that was received from the open question, as well as the ambivalent experience that the game was improving one’s awareness of the breath and being in the present moment (indicating mindfulness), but feeling disconnected to one’s body at the same time (indicating mindlessness). While breath tracking had increased attention and awareness to some degree, the immediate external focus upon in-game stimuli on the screen, as opposed to having one’s eyes closed, might have prevented its focus and deepening within the body.

For Openness to experience, an experience identification task was integrated in the game, which asked the player to describe their current experience on four different qualities. Acceptance was targeted by displaying positive affirmations, which was validated to be an effective method to train compassion through a mobile mindfulness app (Vacca 2016). CHIME items from the Acceptance sub-scale floated from top to bottom and stayed with a player for 30 seconds. The activation of game elements associated to a mindfulness factor was tracked by the player picking up the associated item. The mean frequency for picking up these items was 1.63 for Acceptance and 1.97 for Openness to experience, while Decentering was picked up 2.49 times on average and Relativity of thoughts and beliefs 2.55 times on average. A significant training effect was observed for Decentering after game play, which supports the efficacy of its associated game element; however, none was found for Relativity of thoughts and beliefs. No significant correlation was found between the CHIME factor Decentering and the frequency of activation of its associated game element, which is puzzling.

No specific game element was included for the factor Insightful understanding, since no specific strategy could be identified to singularly target this factor (Sliwinski et al. 2018a). It was hypothesised that this factor is an overall reflection of the other mindfulness factors, and as stated by the authors of the CHIME model, the eight factors of mindfulness are not to be seen as independent or non-overlapping, but rather meaningful descriptions (Bergomi et al. 2013).

A substantial body of evidence demonstrates the cognitive and health benefits of mindfulness (Sedlmeier et al. 2012), which justifies training of mindfulness skills for improving individual cognitive and emotion-regulatory abilities. However, caution is advised to not apply a reductionist view of mindfulness, for Buddhist tradition describes it as a cognitive transformative process (Dreyfus 2011), hence being more than just the some of its parts. While IMProVE was designed to train mindfulness comprehensively, where for the first time all factors were targeted simultaneously, the overall goal remains to provide a tool for beginners to easily experience mindfulness meditation and not be discouraged from the practice. Since contemporary psychotherapies (e.g. ACT, DBT) have integrated elements of mindfulness, this research may facilitate their digital evolution by providing guidance on how to design effective training elements for known aspects of mindfulness. Internet-based digital therapies (e.g. Lappalainen et al. 2014) and mindfulness interventions (e.g. Boettcher et al. 2014) are recognised, effective, and cost-effective training methods for improving psychological health and cognitive enhancement. The addition of interactive game-based elements may benefit established approaches not only regarding efficacy, but also by improving (i.e. decreasing) their attrition rate through increased enjoyment and engagement.

Validation of Mindfulness Game Metrics

Three game metrics were included in the analysis as to whether mindfulness can be reflected by the player’s in-game behaviour. Firstly, the core elements of breath tracking and breath counting were used to calculate the number of successfully submitted key breaths. Secondly, the number of mindfulness items picked up was counted. And thirdly, the game’s total score was saved, which included breaths tracked, successful key breaths, as well as mindfulness and non-mindfulness items picked up.

The number of mindfulness items picked up in the game showed no significant positive correlations to the scores of self-reported mindfulness measures (SMS and CHIME VAS), which disqualifies them as a valid measure in this study. Generally, all items were picked up two times on average, which is extremely low and too infrequent to have an impact. It also stands in contrast to earlier (personal) play testing, which resulted in seven pick-ups on average. This issue addresses a general limitation of IMProVE, which can be described as spreading its impact on all mindfulness factors too thinly. An alternative recommended approach would be to identify the player’s least-developed mindfulness factors first and target only very few, but in a more concentrated manner. Another principal approach worth testing is to create holistic mindfulness experiences, instead of aiming for many individual aspects that may reinforce the view of mindfulness as being a fragmented experience.

Both, the number of successful key breaths and the total game score, were found to correlate significantly with SMS scores, but in the negative direction. After visually inspecting the graphs of both metrics against SMS scores, no tendency or pattern could be identified; thus, it is reasonable to assume that the reported correlations are caused by error or unknown confounding factors. In summary, IMProVE was unsuccessful in measuring the player’s level of mindfulness based on play style and behaviour. Future endeavours to evaluate mindfulness by means of game metrics are advised to integrate measures, which are independent of other elements, such as those for the training of mindfulness.

User Experience

All qualitative aspects, as identified by the user experience questionnaire, were positive, which implies a successful execution of the applied user-centred design approach in the development of IMProVE. Especially, its most salient features of being friendly, easy to learn and slow support its compatibility with tailoring it for mindfulness practice. Gaming experience had no impact on the achieved mindfulness benefit, as measured by the SMS, which confirms the game as being easy enough to control for participants without any prior experience.

In support of the game being a positive experience were the players’ comments, which emphasised the qualities of being pleasant, relaxing and improving focus. Nonetheless, no significant effect was found for the CHIME factor Acting with awareness, which is described as being focused. An explanation for this could be found in the comments of other players, who complained that the game was too distracting. A recent study by Ahmed et al. (2017) explored the role of multi-modal feedback in relation to focus and relaxation for interactive meditation. The three senses touch, vision and audio, as well as their combinations were explored to find out their effect on focus and relaxation. These three human senses were shown to each play a different role and have different relevance for various contexts. Vision and touch can be helpful to trigger focus, while audio is more beneficial in eliciting relaxation (Ahmed et al. 2017). The appropriate configuration of those senses can lead the user into the mindfulness zone, as opposed to drifting away in into wandering (too much relaxation) or stress (too much focus). In a future version of IMProVE, boredom and stress could be detected by dedicated questions in the experience identification task (in the game element for Openness to experience) to then address the appropriate sense (with touch in the form of vibration in the next smartphone-based iteration).

This study revealed an interesting conflict between user experience and efficacy for mindfulness training. While initially breath counting was integrated as originally intended, for reasons of better user experience, additional visual feedback on the user interface was added, which might have reduced the impact of this game element and proposed benefits. Contributing to inefficacy of the breath counting element was its control mechanic, which was found to be the main frustration of the game. It was unclear to many participants how an inhale and exhale was defined (i.e. as one button or two buttons) and when exactly to press [Space]. Since this feedback was also received for the prototype of IMProVE, to make this mechanic more transparent and to give more guidance, a visual feedback indicator was integrated. Furthermore, the key breath submission was perceived as a hard stop, as the avatar stopped paddling when space was pressed or hold. To simplify the breath tracking, it is recommended to use a metaphor, which requires one button only, such as using a rowing boat instead of a kayak, where holding space is equivalent to the drive (i.e. propelling the boat forward) and releasing it reflects the recovery (i.e. reverse movement to start position).

Some participants complained about the tutorial, which did not introduce the mindfulness items or their intended functioning. A future iteration should include more information upfront to make the players more enthusiastic about the game and its intended training effects. Special care should be also devoted to rethink the symbolism of the items. The lotus flowers were seen as very stereotypic for mindfulness and the maelstroms made a negative impression, with some players trying to avoid them at first. The spawning of items should also be changed from random positions to a semi-random system, which keeps a minimum distance to another object, to prevent overlapping. For Openness to experience, an experience identification task was applied, which used four different scales, which are based on Motion, Mass, Temperature and Cohesiveness, as applied in Mindfulness-integrated Cognitive Behavioural Therapy (Cayoun 2011). The sequence of these four scales was perceived as repetitive and more varied scales were preferred, which would fit well with the suggested identification of the player’s internal state to detect the levels of relaxation and focus.

Finally, a mindfulness game should strive to provide intrinsic motivation to play it and not solely to achieve a high score. One participant admitted to have played the game not as intended, by paddling at a high rate, to achieve a high score. To prevent cheating, it is recommended to replace the total score with a more meaningful representation about the current mindfulness state. For this, a graphical rather than a numerical depiction is advised, such as a traffic light that changes dynamically, based on the game’s embedded metrics.

Limitations and Future Research

The evaluation of IMProVE is subject to considerable limitations. The experimental design lacks rigour, since there was no comparison with meditation or an identical regular (non-mindfulness) game. Follow-up studies are planned to include these groups for an adequate comparison of results. In addition, instead of one play session, it is recommended to test a mindfulness game as intended by design, which in this case would be regularly and for shorter times, thus similar to the usage patterns of casual games and mindfulness meditation. This study created the CHIME VAS as a tool to assess state mindfulness for all CHIME factors. Future studies using this measure should consider also including the full CHIME questionnaire to validate the VAS.