Keywords

1 Introduction

Games provide engaging and enjoyable activities, but understanding game usability has had priority over understanding game enjoyment, [1]. According to Boyle et al. [2], if we focus our critical view on the theory focused on motivation, communication and media, we can highlight two relevant topics to structure our knowledge about engagements in digital games, pleasure of playing on one side and the motivation on the other side.

Csíkszentmihályi [3] developed the flow theory to describe the rewarding, subjective, and emotional state of pleasure that appears when user is “absorbed” in leisure activities as something valuable. The idea of flow is based on the ideal combination between skill possessed by the user and the challenge present in a particular activity [3]. For the same author, in addition to the two aspects mentioned above, the experience must be intrinsically rewarding, immersive, with a high level of concentration, a sense of control over the activity, provide direct and immediate feedback and must have clear goals.

In addition to notion of flow, it is also necessary to take into account the human motivation. In this sense, Deci and Ryan [4] developed the theory of self-determination. This notion of self-determination is based on the idea that human behavior is determined by general human needs such as competence, autonomy and relatedness. Competence refers to the need to participate in activities that make us feel capable and effective, since autonomy refers to the need to experience freedom in the activities we choose, while the relatedness refers to the need to feel a connection with others people.

The subjective experience, the physiological responses, the motivation, the use, the market and the impact of digital games, are factors to be taken into account in the structuring of our knowledge about the issues inherent in digital games engagement [2].

In general, the term ‘engagement’ can be defined as a generic game involvement. But the definition can be more deep, other relevant terms include immersion, presence, flow, psychological absorption, and dissociation. These can be conceptualized as representing a level of ever-deeper engagement in game-playing [5].

The conversion of the feeling of satisfaction, immersion and other aspects resulting from the use of digital games for leisure, educational purposes or in training in something measurable, is an essential condition not only for validation but also for acceptance of research work.

Studies of subjective experience in games have compared pleasure, presence, immersion, and excitement. Weibel et al. [6] found that when playing against a human opponent, there was an increase in the sense of presence, flow, and pleasure compared to a computer opponent. The participation of the body does not always lead to an increase of the presence felling. The same author compared the presence and emotions of the participants in two different game conditions, on one side, the participants interacted through the movement of their body and on the other used the game controller. They found that body involvement had no significant impact on the sense of presence, emotional valence values either positive or negative, or emotion/excitement dimension compared to the use of game controller.

User engagement is an important subject to evaluate the quality of a digital game. Normally, the evaluation of user engagement can be done in two ways: using indirect methodologies, like questionnaires and interviews after the user interacted with the digital game and/or through direct methodologies, related with measurements of physiologic parameters like: heart rate, galvanic resistance of the skin or electromyography of the face muscles. In the first approach, the data collected are subjective and dependent of the user memory. In the second approach, the tools used are expensive and intrusive and can affect normal interaction with the experience based in digital game.

Using galvanic resistance of the skin measurements to determine state of arousal, Bersak et al. [7] developed ‘Relax-to-Win’, a therapeutic game where the player’s level of relaxation controlled the speed of a racing dragon. Galvanic resistance of the skin equipment works by testing the conductivity of the skin. The higher the player’s state of arousal, the more they sweat and the greater the skin conduction. Unfortunately the electrical resistance of the skin will also change if the player tightens a muscle, or perspires heavily [8] and this can be totally inappropriate for games style that requiring quick fingered dexterity.

Nowadays, face-reading technology is a promising new tool in measuring emotional reactions in a non-intrusive way. Several software packages for face-reading analysis have been introduced in the market [9,10,11,12], most of them dedicated to develop studies in the neuromarketing. Those software solutions use a standard camera to capture facial expressions when the participants visualize the contexts. Almost all software solutions provide two category of emotion metrics: dimensions of emotion and discrete emotions. Dimensions of emotion are used to characterize the emotional response while discrete emotions are used to describe the specific emotional states [13].

The dimensions of emotion measures includes:

  • Valence – A measure of the nature of the user experience with the content (can be positive or negative).

  • Attention – A measure of the user attention, using the orientation of the face to assess if they are looking directly at the screen or if they are distracted while viewing content.

  • Expressiveness – A measure of how emotionally engaging content is, estimate by accumulating the frequency and intensity of the discrete emotions.

The discrete emotion measures includes:

  • Smile – When user displaying a natural, positive smile.

  • Concentration – When user frowning that is not induced by a dislike response and thus more likely the result of focus.

  • Surprise – When User showing a face of surprise, indicated by raised eyebrows.

  • Dislike – When user showing expressions of dislike or even disgust(nose wrinkles, frowns and grimaces).

1.1 Goals

Explore the potentialities of a face-reading tool to measure the emotional participants reactions, when they play a digital game. We will be concentrated in the following aspects: sensitivity of the tool to measure the emotional reactions at the different moments of the game, and the relation with the emotions reactions and the usability problems of the game.

2 Methodology

2.1 Participants

Seven university students participated in this preliminary study (two males and five females). The average age of participants was 23 years old. One participant indicated that are not a gamer, two indicated that they play regularly and four play casually.

2.2 Game Apparatus

Participants played a computer car game that we build in Unity® game engine (version 5.4), Fig. 1. This is an arcade car racing game type, which the user controls a car through the traditional game controller, making the race in only one lap. The car game was divided into four parts: Initial logo; Intro animation; Main Menu (game mode select menu and car select menu) and In-game (Circuit and pop-up in the end). To capture the participants emotion reactions, we integrate into the game project one script produced by the ErgoVR Laboratory, using the Affdex SDK for Unity® of Affectiva®. With this script, the computer camera is turned on and in some predefined triggers, detects the participants’ facial expressions to detect: attention, anger, contempt, disgust, fear, joy, sadness, surprise, valence and engagement (Fig. 2).

Fig. 1.
figure 1

Images of the game.

Full size image
Fig. 2.
figure 2

Sequence of game moments.

Full size image

The game was displayed in a ASUS® notebook computer, with Intel® Core™ i7 processors and NVIDIA® GeForce® GTX 850 M graphics with screens up to full HD 1080p.

In the first game moment, a collider is triggered when initial logo appear, to capture the player emotional reaction during 8 s. In a second moment, the participants watch an animation of the game entrance with 15 s and the emotions reactions were measured. In third moment, the trigger is on when participants enter in the car selection menu with a duration time equal to 10 s. The last moment is the in-game zone and we put 3 triggers, one at the beginning of the circuit (5 s), another at an midpoint (20 s) and one at the end of the race that triggers the final race pop-up (5 s). All this trigger turn on the notebook computer camera for each moment with duration time as mentioned above and capture the participants emotion. All info is converted in a log file with time, trigger name and emotions types with values between 0.00 and 99.99 (only valence parameter have positive and negative values).

2.3 Procedures

All participants provided written informed consent and all game experience were made in the same place and with same conditions. The experience was made in a room with a notebook computer, with the presence of one person that observes but not interferes in the experience (only interferes if some unexpected event happen). Few moments before the game experience, each participant receives the essential info to play the game, as the game type and the controller buttons that they need to know. Each participant took approximately 5 min to complete the game experience. After finish the game experience, the participant can’t tell the next participants what has happened in their experience to not affect the next experience.

3 Results and Discussion

Will present and discuss the reach results in order to give answers to our main objective, explore the potentialities of a face-reading software related with it sensitivity to measure the emotional reactions at the different moments of the game and the relation with the emotions reactions and the usability problems of the game.

Figure 3 shows the emotional reactions of a male participant (engagement, joy, surprise, valence) and the attention, in the five games moments. We can verify an evolution of the participant’s emotional reactions at the different moments of the game. The values presented correspond to the average for each emotional parameter. At the beginning of his gaming experience (initial logo), he show few variations of emotional states, due to the fact that it was an early logo and there were no significant variation of events or interactions. Also, the attention level is low, only 80%, showing some distraction, which is comprehensible at the beginning of the game. In the second moment of game (intro animation), we began to observe a slight rise in the parameters such as engagement, joy and valence, which may have been caused by the appearance of an animated sequence of events susceptible to causing some variation of the emotional states. At third moment (car select menu), the values ​​continued to rise, here the participant could already interact with elements of game like visualizing and selecting the desired vehicle for the race. In the fourth game moment (in-game start), the values ​​were the lowest of all game experience, this may be due to the lack of initial interaction because in these 5 initial seconds the participant could not drive, waiting until the countdown is finished and only after that 5 s the race started and he can move on. In the fifth moment (in-game midpoint), it was where the values were higher, being due to the interaction provided by the game experience and the events presented, that can change according to the participant and its ability.

Fig. 3.
figure 3

Evaluation of the sensitivity of the tool in the different game moments. Male participant.

Full size image

Observing and analyzing the data collected from one of the female participants, Fig. 4, a possible reason for this result is that she comes from an earlier situation where she was interacting with a virtual reality system. Also it was a new experience for her, because she was not a gamer. Unlike the previous case, she showed high emotional instabilities, which in this case may not just be related to the game. In the ingame start and midpoint moments, the emotional reactions of her have the same pattern as the previous participant.

Fig. 4.
figure 4

Evaluation of the sensitivity of the tool in the different game moments. Female participant.

Full size image

There will be a need a rigorous control of participants’ previous emotional states, his own expressiveness and their previous experiences related to digital games. One solution to this problem, is to create a first trigger at the beginning of the interaction with a neutral situation or, where the participant can disconnect from previous situations.

Figure 5 shows an example of the evolution of emotional reactions at a game level, in this case, the second game moment (initial animation). At the beginning of this animation, the camera is positioned in the back of a car with a focus on the license plate with the icon of the game, and then the camera zooms out and revealing that it was not another screen with a logo, but it was a vehicle and as the camera moved away it also revealed the city. All the participants saw this entry animation, at this moment the participant can only visualize the animation, they can’t interacting with the elements of game, there are no variations of the events according to the user and their capacity or game skill, however the results differ according to each participant, but some participants present similarity in the emotional parameters, being that at the beginning, the first and second participant, male and female respectively, had an initial activation and soon after the values lowered.

Fig. 5.
figure 5

Emotional variation of the first and sixth participant.

Full size image

The need to calibrate the placement of the triggers synchronized with the collection of images (screen recorder), would be a more efficient way to synchronize and identify these moments with the emotional reactions. This would give us the chance to check which event motivated a particular emotional reaction with precision and without the need to be someone monitoring the experience in the same place where the participant is playing. One way to interpret the differences in the emotional reactions is to ask the user why these reactions occur, in order to develop a explanatory models that justify the cause-and-effect connection.

Figure 6 shows the emotional reactions on car select menu. This is the place that participant start to interact with the game and elements present in it, such as vehicles, option buttons among other information. Comparing the data collection of first and second participant (male and female), we can see that results differ from each other. The first participant did not report significant problems in this select menu, with high values of engagement and joy. However, the second participant presented significant values and oscillations of the contempt emotional reaction variable. In this game moment, the second participant after see the vehicle and choosing the first to start playing, could not move to the next moment because he didn’t find the button to move on to the next game moment. At this moment the person that who observes present in the room when reporting the inability of the participant to progress to the next game moment, intervened showing the user where he should click to proceed. This UX problem interferes with the user’s emotional state and can make him feel a little bit frustrated with the game experience.

Fig. 6.
figure 6

Car select menu.

Full size image

4 Conclusion

This paper explores the potentialities of a face-reading tool to measure the emotional participants reactions, when they play a digital game. We studied the following aspects: sensitivity of the tool to measure the emotional reactions at the different moments of the game, and the relation with the emotions reactions and the usability problems of the game.

The face-reading tool demonstrated great potential not only in the evaluation of emotional reactions at different moments of the game but also in the detection of some usability problems as verified in this preliminary study. The recording of an emotional pattern in the different moments of the game gives us the possibility to have an overview and to develop or increase solutions for each moment of game according to the values presented and the need for improvement of each of these moments. In this context, our exploratory results revealed the need to classify the emotional reaction patterns associated to the game moments and player behaviors. These data will allow the development of reference patterns that will allow a better interpretation of emotional reactions in various moments of the game. This information is also important to develop strategies to changing the events of the game in real-time, depending on the need to activate or not the emotional reactions. Using the face-reading tool for usability purposes may be a field to take into account in future projects, because it has shown that behind a emotional state may be hidden a usability problem. In the next stage of research work, more participants will be needed to formulate a precise response, as well to check if there is a solid behavioral pattern. Also in next stage of research work, comparing values not with one but with two or three types of games will add value to the results.

For more accurate data collection, it will be necessary to take into account that each user expresses himself in different ways and some may be more expressive while others may not. To overcome this situation, it will be necessary to create a resting emotional state that will act as a reference point.

An additional method of data collection as questionnaire or brain computer interface (BCI) can also be necessary to more precise results.

This tool for the demonstrated potentialities can be an advantage in the evaluation and development of games, but also in the user-centered Design process.