Keywords

Introduction

A brain-computer interface (BCI) is an input modality that can infer certain actions, intentions, and psychological (e.g., cognitive, emotional) states by analyzing the brain activity it captures. Besides its classical purpose of redressing the communication and mobility of disabled people (Wolpaw et al. 2002), BCI has been proposed as a candidate modality for a range of recreational HCI applications to be used by the general population (Tan and Nijholt 2010). Among these, BCI games (Plass-Oude Bos et al. 2010) lead the way. They attract the interest of researchers and developers from both BCI and game communities. However, a discrepancy between the BCI games developed by the two communities is observed.

Many of the BCI games developed by the BCI community aim at testing some psychological hypotheses or evaluating the performance of signal analysis and classification techniques. Thus, less attention is paid to game characteristics – such as challenges presented, which actions are available to overcome the challenges and what are the interaction mechanisms (Marshall et al. 2013) – than to technical aspects. Moreover, these games do not usually have any narrative or rich feedback or visuals and user experience or game experience, and payability evaluations are almost never carried out. This leads to BCI games that are reliable but often not enjoyable or entertaining from a gaming perspective. On the contrary, BCI games from the game community are developed with respect to the game design principles. However, the neurophysiology and signal analysis techniques they rely on are largely unknown, because these games mostly make use of the commercial BCI headsets that have their private technical details. This leads to BCI games that are potentially entertaining but unsatisfactory in terms of feeling of control. In either case, not only are the researchers and developers wasting their efforts in building incomplete, inconsistent BCI games, but also the community keeps on perceiving BCI games as futile applications.

The goal of this chapter is to transfer some knowledge from the games and the BCI communities into a shared framework to make both communities aware of each other’s research. From the game community, some game-playing motivations which can be satisfied by the features of BCIs are presented. From the BCI community, the current general interaction paradigms are presented, and the ways they can be used in games are discussed. This will contribute to bridging the gap between the two communities and promoting the development of entertaining, playful, and reliable BCI games.

Background: Brain-Computer Interface Application

A BCI application (or system) can be represented as three procedural blocks (Gürkök and Nijholt 2012) (see Fig. 1). The interaction block manages the high-level interaction between the user and the BCI. It is responsible for evoking or instructing the user to generate the brain activity required for the BCI application to operate. In return, it supports and satisfies the users with respect to their intentions and psychological states. If the application is an active BCI (aBCI), the brain activity that the user has deliberately generated is converted to a control command, such as providing a direction or making a choice. If it is a passive BCI (pBCI), then unintentionally generated brain activity, such as mental workload, is used to optimize the user’s well-being by, for example, changing the visuals of a game or adjusting the difficulty.

Fig. 1
figure 1

A brain-computer interface application model

Full size image

The acquisition block acquires the user’s brain activity. In human-computer interaction (HCI) applications, this is usually the electrical activity captured by an electroencephalograph (EEG), but there are examples with blood movement detected by functional near-infrared spectroscopy (fNIRS) (e.g., Girouard et al. 2010). An EEG is a device that measures voltage changes on the brain surface via electrodes in contact with the scalp and outputs digital signals. Being portable, plug and play, inexpensive, and capable of conveying fast-occurring brain activity, it is preferred in HCI applications.

The interpretation block interprets the digital signals generated by the acquisition block and outputs a prediction on user action, intention, or state. The extent and quality of this prediction are bounded by our knowledge on human neurophysiology. It is only known what happens in the brain with respect to specific, well-defined user actions and external events. For example, it is known what happens when one moves a limb or imagines moving it (called ERD, event-related desynchronization; Pfurtscheller and Lopes da Silva 1999). ERD or motor imaginary is a popular BCI game control strategy (Marshall et al. 2013) and one of the most studied BCI control signals (Coyle et al. 2011). The brain’s response when one looks at flickering light (called SSVEP, steady-state visually evoked potential; Herrmann 2001) is known; the frequency of the flickering light or stimulus is reflected in the visual region of the brain. If one pays attention to an infrequent-occurring target image in a sequence of images, the brain response is also known (called a P300 of P3; Polich 2007). So, if one designs a BCI application around these cues and assigns a unique meaning (e.g., a command) to each of them, the application can infer one’s actions or intentions within a predefined set. The situation is similar for psychological states. It is not known what happens in the brain when one feels sorry for someone in trouble, but it is known when particular pictures or sounds negatively affect people (Chanel et al. 2009). Thus, when using a BCI, one cannot know everything, but within certain specific contexts, one can give good predictions about someone’s actions, intentions, or psychological states.

Besides the unique opportunities BCI provides in sensing the human, which are explained above, it has weaknesses compared to other HCI input modalities. Some of these are not plug and play, use of gel or water-based electrodes, unaesthetic EEG caps, and its low accuracy. Concerning the latter, this is partly due to our limited knowledge about human neurophysiology and to the contamination in the measured brain signal, for instance, due to bodily movement, eye gaze, blinking, or other artifacts. The many ongoing brain studies, in particular source imaging, will keep enhancing the knowledge of the neurophysiology. Moreover, a line of BCI research is trying to remove or correct for contamination due to bodily movement (Fatourechi et al. 2007). Another reason for contamination is the spatial mixing of brain signals sourcing from different cortices (brain areas), masking the signal of interest. Another line of research is working on separating these mixed signals into source components using machine-learning techniques (Bashashati et al. 2007).

A Framework of BCI Games

In the framework a BCI game is represented using two descriptors. The first descriptor specifies the player motivation(s) the BCI game satisfies. In other words, it answers the question: Why is the game played? The second descriptor specifies the interaction paradigm(s) the BCI game is built upon. So, it answers the question: How is the game played? Next, these descriptors are discussed and some guidelines are provided for each. For the first descriptor, use will be made of research in the game community, while for the latter of research in the BCI community. Finally, the relation between the two descriptors will be discussed.

Motivations Satisfied by the BCI Game

People play games to experience positive affect (Johnson and Wiles 2003), and positive affect is significantly correlated with psychological needs, such as autonomy, competence, relatedness, pleasure, and self-esteem (Hassenzahl et al. 2010; Sheldon et al. 2001). Therefore, people play games which tend to fulfill their psychological needs. Indeed, there is a correspondence between some psychological needs (Sheldon et al. 2001) and some game-playing motivations (Rouse 2005), such as competence and challenge or relatedness and socialization. In this section, some example game-playing motivations (or needs) are provided in which BCI can make a difference, and ways to make the difference are discussed.

Challenge. When someone achieves a goal or when someone feels that he or she is doing something well, positive affect is experienced (Kubovy 1999). That is why people, and particular game players, enjoy doing things that challenge them. Challenge is one of the elements of flow , which is the optimal experience for any activity and described as “so gratifying that people are willing to do it for its own sake, with little concern for what they will get out of it, even when it is difficult, or dangerous” (Csíkszentmihályi 1990). Many researchers have shown the link between flow and games (Cowley et al. 2008). Sweetser and Wyeth (2005) proposed a model describing which elements a game should have in order to provide flow. Their model suggests that a game should offer challenges matching player skills and both must exceed a certain threshold. Similarly, Carroll and Thomas (1988) suggest that “examples of fun indeed must have sufficient complexity or they fall flat (jokes that are too obvious, games that are not challenging).” Moreover, “things are fun when we expect them to be of moderate complexity (interesting but tractable) and then in fact find them to be so (i.e., not too difficult or too easy).”

As pointed out in section “Background: Brain-Computer Interface Application”, BCI is an imperfect technology. Nijholt et al. (2009) suggest that BCI, even when it is still an imperfect technology, can make a perfect game. Players of a BCI game need to show continuous effort to prevent errors and even repeat their actions until they are understood by the BCI. Not only the purposeful repetition brings fun (Blythe and Hassenzahl 2003), but also the successful end result leads to a positive affect. Based on this point of view, one can think that challenge is inherit in any BCI game, because any game requires voluntary control (Sweetser and Wyeth 2005) and any BCI provides imperfect control. However, the technological shortcomings of BCI cannot provide a positively affective challenge if the game cannot conceal them through the tasks the players need to perform. The game should provide the players with the illusion that when they encounter an error, this is not simply a technological fault of BCI but rather an expected situation or challenge given the player tasks and game environment. Players should not be frustrated by the incapability of BCI but should see it as challenge which can be met. This way, they should be motivated to repeat the tasks until they learn how to generate the desired brain activity. Meanwhile, the BCI should also learn how to interpret the generated brain activity.

A similar idea is exploited by the Nintendo DS game Nintendogs in which the players teach dogs some commands, such as sitting, using the built-in microphone. In reality, it is not unusual that it takes multiple trials until a dog learns a command. Moreover, the commander also needs to show effort to provide standard, consistent, and clear commands. So, unsuccessful commands in the game would not frustrate the player but rather motivate and also guide them to provide better commands.

While posing challenges, BCI performance should not be completely neglected. Players should experience uncertainty and curiosity (Malone 1982), but they should be able to predict, to some extent, the game response. Continuous recognition feedback plays an important role in this. Also, the challenge posed by the game should be dynamic. That is, “the level of challenge should increase as the player progresses through the game and increases their skill level” (Sweetser and Wyeth 2005). It has been shown that people can manipulate and learn to improve their voluntary mental actions as well as involuntary reactions as they keep interacting with a BCI that provides accurate feedback (Wolpaw et al. 2002). So, BCI control can be regarded as a skill.

Fantasy. Games let players do things that they cannot do – at least safely or without being criticized – in real life, such as flying or smashing cars. However, in a virtual world, it is not trivial to provide the very same sensation resulting from doing something in the real world. Such a sensation is known as presence and defined as “the perception in which even though part or all of an individual’s current experience is generated by and/or filtered through human-made technology, part or all of the individual’s perception fails to accurately acknowledge the role of the technology in the experience” (International Society for Presence Research 2000). Riva (2009) claims that rather than our perception, it is our chain of actions that create the presence. He explains that a user “is more present in a perceptually poor virtual environment … where he/she can act in many different ways than in a real-like virtual environment where he/she cannot do anything.” Actually, ‘to act’ is not the ultimate goal, the aim is ‘to be’ in the virtual world and to act is one way of satisfying that aim. So, a player is more present in a virtual world in which he can represent himself more. At this point, the means or actions with which the player can drive the game become crucial.

One of the immersive game activities is role-playing, that is, feeling like oneself and game character are one (Lazzaro 2004; Yee 2006). In role-playing games, the amount and, more importantly, the quality of the self-representation the player can convey to the game are of utmost importance.

Traditional game controllers, such as a game pad or joystick, restrict the information flow from the player to the game. Firstly, the number of buttons or degrees of freedom provided by these controllers is insufficient to satisfy the infinitely large amount of information that could be transferred from the player. And secondly, the idea of representing oneself using buttons or a joystick is not an intuitive one since the player has to spend an effort to learn and memorize the mapping of his or her intentions to controller actions. Tremendous research and development has been going on to alleviate this HCI bottleneck (Sharma et al. 1998). One example is the work on motion-capturing techniques and devices (such as a Kinect), which enable one-to-one correspondence between player actions (as well as reactions) in the real world and those in the game world.

There are times when the players may need a deeper representation of themselves, rather than their overt actions; consider a life-simulation game, The Sims. In this game, the player controls the life of a character (or several characters) that can be customized to look like the player in terms of outfit or bodily and facial features. The character is also autonomous, and its behavior is influenced by the personality assigned to it by the player at the beginning of the game. It is inevitable that, at some time during play, this virtual look-alike of the player will not act or interact with other characters in congruence with the player’s feelings or thoughts, because of either the inaccuracy of the player’s personality assignment or the imperfection of the game to produce a desirable action. Consequently, this incongruence will hamper the player’s sense of presence. In cases such as these, BCI can provide a translation between the psychological state of the player in the real world and the dynamics of the game world, just as a Kinect provides correspondence between real-world and game-world actions. So, the additional inner state information can strengthen the feeling of presence.

Sociality. Some people enjoy playing computer games with other people (Rouse 2005). They play not necessarily for the challenge but just to be with others. They enjoy spending time with friends, seeing their reactions and expressions, and gloating or feeling proud upon winning (Sweetser and Wyeth 2005). Any multiplayer version of a BCI game can provide such an interactive environment. Players may cooperate or compete using BCI, or they can share their experiences, such as difficulties or enjoyment with control, while playing the game. These are, of course, not specific to BCI games. But, there are other ways in which BCI can provide sociality and which cannot be replicated easily or at all by other controllers.

Many social actions are related to expressing and perceiving emotions. Previous studies have shown that communication of heartbeat, which is a reflection of emotional activity, can improve the copresence (Chanel et al. 2010) and intimacy (Janssen et al. 2010) of players. Heartbeat is certainly not the only nor the best indicator of emotion. BCI can recognize certain psychological states and let us share them. According to neurobiological emotion theories (e.g., LeDoux 1995), the brain is involved in the production and conscious registration of emotional states. So, BCI can provide quick and direct information about our emotional state. Since involuntary brain activity, such as emotional response, is not easily controllable, BCI can provide objective information about our emotional state. For this reason, BCI can also be used in game situations where players would like to hide their psychological states from each other. For example, in a bluffing game, players can restrict their bodily movements and to some extent even their physiological activity but not their brain activity. So, BCI can be used for emotion awareness or, more generally, psychological awareness in two opposite game logics.

Going one step further than emotional awareness, emotional contagion theory states that people tend to converge emotionally and, for this, they tend “to automatically mimic and synchronise expressions, vocalisations, postures, and movements with those of another person” (Hatfield et al. 1994). Research has confirmed that synchronization contributes to coherence (Wiltermuth and Heath 2009) and can be used as a measure of the intensity of the interaction between people (Hatfield et al. 1994). It has therefore been used in some game experience research (Ekman et al. 2012). This suggests that synchronization games can strengthen the interaction between players. In such a game, BCI can enable synchronization of psychological states, in the form of emotional synchrony (Kühn et al. 2011) or mental synchrony (Sobell and Trivich 1989); this adds another dimension of synchrony and can provide a deeper and personal interaction between two players compared to physical synchrony.

Interaction Paradigms Used by the BCI Game

BCI applications rely on brain signals originating from player actions and reactions to events. These actions and reactions are called interaction paradigms and are divided into three categories: mental state regulation, movement imagery, and evoked response generation. In this section an overview is given of BCI games in order to analyze the interaction paradigms used in the interaction design. An extensive survey of BCI games and applied BCI interaction paradigms can be found in Plass-Oude Bos et al. (2010) and Marshall et al. (2013).

Mental State Regulation. Mental state games are usually played via two activities: relaxing or concentrating. These activities stem from clinical practice, such as relaxing to reduce anxiety or concentrating to reduce attention deficiency, but they are used in BCI games for very different purposes. Most of the mental state games allow players to move physical (Hjelm 2003) or virtual (Oum et al. 2010) objects, but there are other mechanisms such as changing the game avatar (Plass-Oude Bos et al. 2010).

Relaxing is a preferable activity in a game as it leads to a positive affective state that players would like to reach while playing games (Lazzaro 2004). Therefore, even if the game environment is not an affective one, people may play such games for the end effect of being relaxed. Moreover, they might easily refer their acquaintances and even children to play such games.

Concentration is also a preferable game activity due to its absorbing effect. According to the flow (Csíkszentmihályi 1990; Sweetser and Wyeth 2005) and immersion theories (Brown and Cairns 2004), concentration is the key to successful games. Therefore, games requiring concentration or paying attention, which is one of the activities leading to concentration, ought to provide a positive play experience. It should be noted that this does not immediately equate the concentration required by a BCI game to that naturally occurring during flow or immersion. However, it suggests that the former can contribute to the establishment of the latter, provided it is integrated into the game in a natural manner. The concentration-related activities should not isolate the player from the game world, but only from the real world. They should make sense in the game context and the story line. The flow should not be broken when the concentration-related activity ends.

Mental state regulation games should either be slow paced or in these games BCI should be used as an auxiliary controller along with a primary controller which is faster than BCI. The speed with which one can change his or her state of relaxedness or concentration is much slower than the speed with which one can press buttons or use any other modality. Mental state games usually allow only binary control. For example, in a relaxation game, players can either be relaxed or not relaxed so they can communicate a maximum of two discrete commands. It is possible to fit a continuous scale between these two states but validating such a scale is nontrivial. Therefore, mental state regulation is less suitable for games that require large numbers of distinct commands. In mental state regulation games, positively affective activities should be preferred. Otherwise, long durations of play may change brain functioning in an unwanted direction.

Movement Imagery. Movement imagery games require no physical movement but imagery of limb movements, mostly the hands, fingers, or feet. Players imagine movements to navigate, as in driving a virtual car (Krepki et al. 2007), or to make selections, as in playing pinball (Tangermann et al. 2009). While in mental state games the capability of BCI (i.e., detecting relaxation or concentration) is irreplaceable by another modality, movement imagery games can be played by more precise modalities. So, players who are not disabled may not enjoy movement imagery games if the interaction is not carefully designed. In this case, providing an intuitive interaction can immerse the players. To provide intuitive interaction, the mapping of imaginary movements to game commands should be coherent. For example, grasping an object at the left- or right-hand side can be matched to left- or right-hand imagery, while walking can be matched to foot imagery. These intuitive mappings can create the illusion that the game is recognizing player’s actions, even before they move due to the ERD which happens before the actual or imagined movement.

Movement imagery can be recognized quite quickly, without needing to average the signal (Ramoser et al. 2000; Tangermann et al. 2009). Therefore, movement imagery games are suitable for fast interaction. On the other hand, the number of commands in these games is limited to the number of distinguishable imaginary actions players can perform. Using other modalities in combination with BCI can increase the number of commands. However, the movements made to control other modalities, such as pushing a button or speaking, might contaminate the movement imagery signal with artifacts related to the other movements.

Evoked Response Generation. This class of games is dominated by SSVEP games, accompanied by rare examples of P300 games (e.g., Finke et al. 2009). SSVEP is a brain response to flickering light or images (Herrmann 2001). For example, when one observes a visual stimulus, say an image, that is constantly reappearing, say at a frequency of f, then the amplitude of the signals measured from the visual cortex are found to be enhanced at frequency f and its harmonics (2f, 3f, and so on). This way, if there is a single stimulus, then one can recognize whether someone is observing it. If there are multiple stimuli, then it is possible to recognize which of them someone is observing. Based on these possibilities, SSVEP games have been developed in two ways.

The first approach is to map the strength of SSVEP which is evoked by single stimulus to game actions. For example, a weak SSVEP can steer a virtual plane to the left, while a strong one to the right (Middendorf et al. 2000). Players can manipulate SSVEP strength in different ways. One way is to close and open the eyes to produce weak and strong SSVEP. But this would probably be too trivial to produce challenge in a game. Another way is to regulate SSVEP strength by the amount of attention paid. Research has shown that sustained attention can enhance SSVEP (di Russo and Spinelli 2002). That is, one can infer whether a person is simply exposed to a stimulus or if the person is actually paying attention to it. Sustained attention is an activity which can lead to a state of concentration (Mateer and Sohlberg 2001). This makes SSVEP suitable for concentration games, the advantages of which were discussed before.

The second approach, which is the more popular one, is to use multiple stimuli each of which is associated with a command. In almost all games built with this approach, BCI is used to select a direction. Players can select a direction to aim their gun in a first-person shooter game (Moore Jackson et al. 2009) or to steer their car in a racing game (Martinez et al. 2007). With this approach, a greater number of commands can be issued.

Evoked response generation is less suitable for fast games due to the signal-averaging process, which requires signals to accumulate for some time. But they are suitable for multimodality thanks to their high signal to noise ratio. The number of distinct commands in evoked potential games depends only on the number of stimuli. So, as long as the stimulation space is large enough to accommodate, (finitely) many stimuli can be presented to the player. On the other hand, a computer screen is a limited space so the number of stimuli that can be placed on the screen is also limited. Moreover, as their number increases, the frequency and position of the different stimuli come closer to each other. This makes paying attention difficult for the user as multiple stimuli could interfere with each other. Furthermore, a screen cluttered with attention-demanding flickering stimuli would clutter the visual channel and prevent the player from enjoying primary game elements, such as its visuals or the story line. Therefore, the stimuli necessary for evoked response games should seamlessly be integrated into the game environment. Stimulus properties, such as size, shape, or intensity, should suit the game visuals. They should make sense to the player within the game elements and the storyline. For example, an SSVEP stimulus can be integrated into the game as the wings of a virtual butterfly, which also flicker in reality (Legény et al. 2012).

Guidelines

The framework proposed in section “A Framework of BCI Games” offers BCI game developers a common basis to situate their games. Next, some guidelines will be provided based on the framework so that the developers can further make use of the framework.

Guidelines Regarding Playing Motivations

Regarding virtuosity, the BCI should not introduce challenge merely due to its technical shortcomings but rather in terms of the activities it demands the player to perform. Players should work toward finding the right activities to succeed in the game. They should feel that when BCI cannot understand them, they also have a role in this. Meanwhile, BCI performance should not be completely neglected. Players should experience uncertainty and curiosity, but they should also be able to predict, to some extent, the game response. Continuous recognition feedback is thus helpful. The challenge posed by the game should be dynamic. That is, “the level of challenge should increase as the player progresses through the game and increases their skill level” (Sweetser and Wyeth 2005). It has been shown that people can manipulate and learn to improve their voluntary mental actions as well as involuntary reactions as they keep interacting with a BCI that provides accurate feedback (Wolpaw et al. 2002). So, one can regard BCI control as a skill.

For fantasy fulfillment, capabilities of BCI that resemble fantastic abilities, such as telepathy or telekinesis, should be exploited where possible. Care should be taken not to make promises to provide such controversial abilities though. BCI should be used in situations where “it” is the best or the only modality, rather than replacing a better modality for satisfying the player’s motivations or needs.

Regarding sociality game designers should take into account that BCI changes the social bodily interaction; players used bodily interaction to communicate their thoughts and brain state to other players and spectators. Moreover, in order to achieve high level of control, players sometimes behave like paralyzed in order to prevent the contamination of the EEG signal with movement artifacts (O’Hara et al. 2011). But on the other hand, multiplayer BCI games with co-located BCI gamers can enhance the social experience due to collaboration and synchrony of brain states (Nijholt 2015).

As disclaimed before, the motivations provided to play BCI games do not constitute an absolute list, but an illustrative one. Moreover, games can certainly satisfy more than one motivation simultaneously. For example, it is not difficult to imagine a challenging multiplayer fantasy (BCI) game. Based on game-playing motivations, below are some recommendations for developing BCI games in general:

  • BCI should not introduce challenge merely in terms of its imperfect recognition performance but rather in terms of the activities the player performs to use it. Players should work toward finding the right mental activity to play the game. They should feel that when BCI cannot understand them, they also have a role in this.

  • BCI performance should not be completely neglected. Players should experience uncertainty and curiosity, but they should also be able to predict, to some extent, game response. Continuous recognition feedback is thus helpful.

  • The challenge that the BCI game poses should increase as the player skills increase. Players should be aware of the increasing challenge and their increasing skills through feedback.

  • BCI training should be done while the player is also training. Repetition is essential for training and also fun for the player if it is integrated well into the game.

  • BCI should be used in situations where “it” is the best or the only modality, rather than replacing a better modality for satisfying the player’s motivations or needs.

  • Capabilities of BCI that resemble fantastic abilities, such as telepathy or telekinesis, should be exploited where possible. Care should be taken not to make promises to provide such controversial abilities though.

  • BCI-based games change the social bodily interaction between players. Bodily interaction will be reduced in order to keep up control. On the other hand, multiplayer BCI games can increase brain-based social interaction due to required brain state synchronization or cooperation.

Guidelines Regarding Interaction Paradigms

Regarding the categorization of BCI games (mental state, movement imagery, and evoked potential games), there is no restriction for any category of game to fulfill a certain motivation (virtuosity, fantasy, or sociality). Thus, a game may satisfy more than one player motivation at the same time. But the claim is that a BCI game can exclusively be either an aBCI or a pBCI application. The latter is harder to achieve; therefore, the former has been the popular approach so far. Since any game requires voluntary control (Sweetser and Wyeth 2005), BCI games are intrinsically aBCI applications. Still, it is possible to design a pBCI game in which the involuntary player state influences the game. But as soon as the player realizes the relation, they will start manipulating their state to gain an advantage in the game. Thus, the game will turn into an aBCI application. So the key point in designing pBCI games is to manipulate the game with respect to player state in such a way that the player does not become aware of it.

  • Mental state regulation games are more suitable for slow-paced interaction since a player psychological state does not change instantly. It takes time until a player concentrates to do something or to get rid of frustration. Evoked response games are also more suitable for slow-paced interaction, but this is because of the cumulative duration of stimulation and data collection. Movement imagery games may be used for fast-paced interaction since the neurological correlates of movement imagery are instant (they even precede the imagery; Shibasaki and Hallett 2006) and corresponding data is not averaged during analysis.

  • Mental state games are less suitable for games that require large numbers of distinct commands, because the number and intensity of the psychological states that BCI can infer are limited. Similarly, in movement imagery games, the number of limbs that BCI is capable of differentiating is limited. On the other hand, evoked potential games depend only on the number of stimuli. As long as the stimulation space is large enough to accommodate, (finitely) many stimuli can be presented to the player.

  • BCI game activities should be intuitive to keep the player in the flow. For example, steering a car to the left/right would better be matched to left-/right-hand imagery rather than concentration/relaxedness.

  • The stimuli necessary for evoked response games should seamlessly be integrated into the game environment. They should make sense to the player within the game elements and the storyline. Stimulus properties, such as size, shape, or intensity, should suit the game visuals.

  • Just as in any computer game, the long-term effects of BCI game activities should be carefully considered. Especially in mental state games, positively affective activities should be preferred. Otherwise, long durations of play may change brain functioning in an unwanted direction.

The Complete Picture

In this section, the relation between the two descriptors of the framework will be discussed. Specifically, the emphasis will be on the questions: which interaction paradigms can satisfy which player motivations and in which ways? Where possible, existing or hypothetical BCI games supporting our discussion will be mentioned.

As any game can do, a BCI game can satisfy more than one set of player motivations (the first descriptor of the framework) at the same time. It might not satisfy any player motivations, for example, if it is just an experimental game. This is illustrated by the set diagrams in Fig. 2. Similarly, a BCI game can make use of multiple interaction paradigms (the second descriptor of the framework). The font stylings of the games in Fig. 2 indicate the different interaction paradigms and their combinations. For example, in the game Bacteria Hunt (Mühl et al. 2010), the players chase fleeing bacteria by controlling an ameba. It is a mental state regulation game because when the players are relaxed, the bacteria flee more slowly. It is also an evoked response game because when the ameba catches a bacterium, an SSVEP stimulus (a circle) appears on the screen, and the players concentrate on this circle to eat the bacterium. Using multiple interaction paradigms is a practice that has been mostly considered in assistive BCI applications to improve the performance, and such applications are called hybrid BCIs (Pfurtscheller et al. 2010). While there is no restriction on using any interaction paradigm to satisfy any player motivation, there might be preferred and non-preferred matchings.

Fig. 2
figure 2

Diagram showing the set of player motivations and example BCI games. Italic, underlined, and bold fonts represent mental state regulation, movement imagery, and evoked response generation games, respectively

Full size image

A BCI game can offer a positively affective challenge if the game can hide BCI recognition errors under the player tasks. In the game alpha World of Warcraft (aWoW) (Plass-Oude Bos et al. 2010), the players play a druid that transforms from a bear to a humanoid when the players are relaxed and back again when they are stressed. So, it is a mental state regulation game. The level of relaxedness is determined according to the alpha band power. But the players always think of the high-level tasks of getting relaxed or stressed to transform. There might be times when players cannot transform desirably in the game. Especially for players who are familiar with the game World of Warcraft or similar games, this is not an unexpected situation. These players would consider that it should take some effort and time for a druid to transform. Therefore, they would keep trying to find the right strategy to transform. The same principle may be applied to movement imaginary games. For example, in a game (let us call it Lift Stone for the sake of easy referencing) the players might try to lift a stone by imagining arm movements. Players who are familiar with the phenomenon of telekinesis would find this mapping (imagining arm movement and lifting) an intuitive one and consider that it should take some effort to lift something from far. For evoked potential games, due to the explicit stimuli, offering meaningful challenges is more difficult though not impossible. For example, in a game (let us call it Kill Monster) players might try to kill a monster by concentrating on its heart which flickers like an SSVEP stimulus. People know that, though it does not flicker, the heart beats and it is the organ of vitality. Furthermore, killing a big monster should require some effort. In contrast, for example, in the Bacteria Hunt game, the SSVEP stimulus is not related to game elements (the bacteria), and there is no motivation to keep looking at a flickering circle to eat a bacterium.

To provide fantasy, a BCI game should incorporate additional, inner state information from the players to improve their sense of presence in the virtual world. In the game World of Warcraft, players play the role of a druid. So, the idea is that the players should put themselves in the place of their avatar. The original game translates player actions (e.g., pressing the key W) into game actions (e.g., moving the avatar forward), but it cannot go beyond that. In aWoW, they are represented not only by their actions but also by their psychological states. This way, they can feel more present in the game world. Movement imaginary games cannot convey psychological state information, but they can represent covert player intentions. For example, in the hypothetical Lift Stone game mentioned in the previous paragraph, the players might feel as if the game is understanding their intention without having them move. Going even further, through the readiness potential (Shibasaki and Hallett 2006), the BCI game can actually recognize player intentions “before” the players move or imagine moving their arms.

In a BCI game, sociality might emerge from the explicit social behaviors (e.g., vocalizations, gestures) of co-players while they are competing against or cooperating with each other. In this sense, any interaction paradigm is suitable for use. For example, in the game Brainball (Hjelm 2003), in which co-players regulate their mental states to roll a ball on the table toward each other, players might, for instance, talk to tease each other or generate facial expressions to manifest the difficulty they are experiencing. Sociality might also be formed through communication of implicit player states. For this sort of sociality, mental state regulation is the natural paradigm of choice. For example, in the game Brainwave Drawing (Sobell and Trivich 1989), co-players try to synchronize their mental states in terms of their brain signals in different frequency bands.

Conclusion

In this chapter a framework for BCI games is presented, formed partly by research in the BCI community and partly by research in the game community. The framework lets BCI game developers situate their game ideas among the other BCI games. This way, the developers can take into account the benefits and drawbacks of exploiting particular player motivations and interaction paradigms. As a result, one can have more BCI games which are both enjoyable and reliable.

Within the framework, two descriptors are used to represent a BCI game. One descriptor specifies which player motivation(s) the BCI game can satisfy. Three examples of player motivations are challenge, fantasy, and sociality. The ways BCI games can satisfy each of these motivations are discussed and some guidelines are provided. The other descriptor specifies which interaction paradigm(s) the game is built upon. Three types of interaction paradigms are described: mental state regulation, movement imagery, and evoked response generation. The constructing of these categories is based on the literature on BCI and game research. The interaction paradigm(s) should be chosen to provide intuitive control. For example, steering a car to the left/right would better be matched to left-/right-hand imagery rather than concentration/relaxedness.

But there may be many more player motivations that a BCI game can fulfill or interaction paradigms it can make use of. So, obviously, developers should not feel restricted by the defined categorizations while developing their BCI games. On the contrary, they should investigate the alternative categories. Moreover, the categories for both descriptors are not mutually exclusive. So, a BCI game can satisfy more than one player motivation, or it can accommodate multiple paradigms for control (Mühl et al. 2010).

Last, but not the least, the importance of empirical user experience evaluation should be stressed. While heuristics and recommendations can help to build better BCI games, empirical studies can yield practical information on game characteristics that the players like and dislike.