Multimedia Games User Experience Data Collection: An Approach for Non-experts Researchers

Abstract

This chapter aims to present game development for researchers who wish analyze game user experience by data collection, in their research, however find difficulties for application development. The chapter is divided into two parts: the first part presents game design theory highlighting definitions and examples about data collection involving guidelines and usability. The second part presents a game development approach using the Construct 2 as a tool for game development aiming to store multimedia resources data (audio, videos and photos) and variables (quantitative and qualitative) generated during users interaction.

1 Introduction

Game development refers to an area of knowledge that embraces several lines of studies in Computing, Education, Design, Simulation, Entertainment, among others. Because it involves various possibilities of integration of contents and advanced interaction modes, many researches can benefit from its mechanisms to attend specific demands of scientific methodology procedure, especially data collection.

It is believed that scientific research can obtain many advantages from the use of games [5], but this scenario was, for a long time, difficult to access, due to various obstacles that made the immersion of games in the field of scientific researches timid, including exorbitant development costs and inaccurate graphical representations that limited the simulation fidelity. The advances in game engines (for example, Cry-Engine,¹ Unity,² and Unreal Engine³) have greatly reduced the problems associated with its adoption. Therefore, it is time to seriously consider the utility of serious games for the research area [5].

In the face of the countless domains of knowledge that approach the game as a theme and attending to the segment that holds the entertainment as a background, emerge the name serious games. In general terms, the expression “serious games” means that games of this type have an explicit educational motivation, carefully thought, and are not intended to be played exclusively for enjoyment or entertainment purposes. However, this interpretation does not mean that the serious games are not, or should not be, funny; on the contrary, the idea is that the serious games possess creative and pleasing elements as their components. The serious games are projected to have an impact on the target audience, which is beyond pure entertainment [12, 17, 29]. One of the most important areas of application is in the field of education, due to its recognized potential, taking into account the continuous necessity for improvement of techniques and update of educational resources [11, 38]. Another more recent initiative for use of serious games is its application with the elderly audience. This public represents a community of potential users who can benefit greatly from digital games [34], as they represent a tool that contributes to the quality of life during the aging process, alleviating the decline of some aspects resulting from this process, such as motor, perceptual, cognitive, and psychosocial aspects [3].

In this context, according to [37], one of the most promising areas for games applications aimed at the elderly public is the neuropsychological assessment. The games can revolutionize the cognitive assessment of the elderly in clinical settings, allowing for more frequent, accessible, and pleased assessments [35]. In a study of different moments and clinical situations of the elderly, studies [13] reveal that serious games have been very important to maintain and develop the cognitive and social skills of the elderly. In the specialized literature, studies point out that the serious games for the elderly may be an option to improve the deterioration of cognitive functions [36], the maintenance of self-image, motor functions [39], and social life (affection and solitude) [24].

In the face of the relevant potential of research involving games, this chapter adds concepts about the use of games; in particular, it is presented the data collection procedure adopting games and practical examples of development.

2 Terms and Definitions

There are many terms and definitions used in research for game development and application. Therefore, in this chapter we make a clipping, considering the concepts of the area of games most related to the activity of data collection in the research area, both in the production of complete games or in the use of gamification elements, explained later, to contribute in the engagement of the games.

Player engagement is directly associated with the elements of construction of a game, as well as the enriching of the user experience. In the view of the diversity of user’s abilities (both limitations and intensified capacities), it is essential to use guidelines that contribute to the inclusion of a greater number of specificities, contributing with their usability and accessibility.

2.1 Games and Data Collection

The use of games in the research area aims to enrich the end objective of applied researches, especially in the context of data collection, in the obtainment of scientific evidences that promotes confirmation of questioning under investigation. Thus, the use of games by researchers is not only associated with the planning and development of games as finished or complete products, but also in the use of gamification elements in order to offer support to the tasks required during the activities developed in the research.

Gamification is understood as the use of mechanics, ideas, and aesthetic of games to engage people, motivate actions, promote learning, and solve problems [25]. Therefore, when performing gamification, its use the elements and design techniques of games in situations outside the game environment, in order to obtain greater user participation and involvement [31].

In the context of computer aid to researchers, the concept behind the term “game” can be summarized, as defined by [32]: it refers to a system in which players get mixed up in an artificial rule-defined conflict and that has a quantifiable result.

In general terms, the development of games are based on certain elements that can characterize and even generate the user engagement, such as the system, the players, abstraction, challenge, rules, interactivity, feedback, quantifiable results, emotional reactions, and history [25]. These elements defined below should be considered in the ambit of any type of games.

• System: set of interconnected elements that occur in the context of the game;

• Players: people interacting with the game or with other people;

• Abstraction: refers to a mental image of reality to define the context of the game;

• Challenge: the game incorporates activities and objectives that instigate players to achieve goals and results at different levels of difficulty;

• Rules: contain the fluidity structure of the game, setting the environment for a proper gameplay;

• Interactivity: it is the process of relationship of the player with the content, with the game, or with other players;

• Feedback: return of the system to the actions performed by the players;

• Quantifiable results: the game must clearly reproduce, to the user, the concept of gain or loss, from the interaction with the player;

• Emotional reactions: The game should provide feelings of pleasure or frustration, depending on the outcome of the player’s activities;

• History: The use of the narrative of a story contextualizes the player to the scenario, introducing a meaning to the game.

In particular, in the context of data collection, games propitiate that innovative forms of application of its elements can be used in procedures to collect data in an unbiased and judicious way, according to the rigor prescribed in the scientific investigations. From the perspective of research methods in general, data collection is considered one of the most important stages of a research [6, 27, 28, 30], whether qualitative or quantitative. Among the main tools used during data collection, there are:

• 1. Observation: when the researchers without any mediation directly notice the facts. It must be planned, registered, verified, and checked for validity and accuracy. Among the disadvantages, the behavior of the observed ones can be changed due to the presence of the researcher.

• 2. Interview: requires formulation of questions to the investigated, being a social interaction. It makes it possible to obtain data referring to a spectrum of information. Among the advantages, it provides clarification and capture of interviewee’s expressions (body, gestures, voice…). Among the disadvantages, it can be influenced by the interviewer’s personal opinions, as well as involve costs for training and application of interviews.

• 3. Questionnaire: set of written questions presented to the interviewees, it allows that a large number of respondents can participate/collaborate. Among the disadvantages, an exclusion of those who cannot read and write, as well as a requirement of differentiated treatment for the visually impaired people. For example, obstruct the knowledge of circumstances of the interviewee when he replied to the questionnaire.

• 4. Documents: are sources in “paper” (archives, statistical records, diaries, biographies, journals, magazines, reports, memoirs letters, newsletters…) that store different data, which enable the acquisition of knowledge represented therein.

Among the disadvantages: it requires meticulous rigor in obtaining and interpreting the assembled material. It is worth mentioning that these instruments can be used in a complementary way, according to the type of research. In any type of survey (data collection) with the target audience of a research, the utmost care is required when collecting information. Among care, in the context of the theme presented in this chapter, game elements must be properly designed, as discussed in the following section.

2.2 Games and User Experience

Among the definitions to the expression “User experience (UX)”, many researchers converge, considering that UX embraces all aspects of user end interaction with a certain system, service, or product [14, 19].

To [14], a relevant aspect to be considered in the game development, related to the user experience, is that it can be boring when very easy, and frustrating when very hard. The majority of single-player games allow to the player just a gross adjustment in the level of difficult, usually making the choice between the modes of the games possible: easy, medium, hard, or expert [22]. This approach is static in the face of the player interaction with the game and may present a divergence between the player’s skills level and the general difficulty of the game.

Few commercial game developers have implemented dynamic adjust systems for their games, among these, it can be cited God Hand^Ⓡ (Clover Studio and Capcom, 2006), in which during the game a gauge regulates the intelligence and strength of opponents. This gauge increases whenever the player succeeds in dodging enemy attacks or striking opponents, decreasing as the player is hit. A great reward is offered to the players who beat opponents that, according to the gauge, are very difficult to their skill level.

However, in the commercial games, in the field of serious games, no initiative was identified in this sense. Not so distant from the adaptability of games to the different skills levels, the user experience may also be related to interface issues. These issues are relatively more complex for people (potential players) who present some decline in their cognitive or motor functions, encouraging research into the design of games intended for universal use, to everybody.

It is important to ponder over a perspective of developing traditional software, which is not a game, considering the user experience. The development with UX for traditional software aims to the user removes the problems. On the other hand, in the game development, the user experience refers it to offer “problems”/challenges to the user. In both cases, the users’ required reasoning and their cognitive capacity must be examined.

In general, the work of a developer who concern with the user experience of traditional software is forwarded to create an excellent experience to the user when using the product with a purpose. Thus, this developer does his best to make the interface clear (so the user needs to think as little as possible), suggestive (so the user knows what possibilities of interaction he has), flexible (that adapts as the user develops skills), and rich in information and feedback (so the user knows more about something or situations that bring problems).

It is about driving users to build a mental model that will help them do their job. Classic examples: the steering wheel of a car or the metaphor of the entire work area. The steering wheel is hiding from the user the fact that there is a rather complex mechanism between the wheel and the tires. Instead, it is trying to get the user to think “turn the wheel, turn the car”. This is an intentional illusion. The work of game developers, usually, is aimed at creating a great experience to the player to play a game. Therefore, they do their best to create the game:

• Challenging: the game is often required to make the user think, carefully, using memory and reasoning capacity.

• Exploration: it is usually required that the user think that there is always more possibilities to be discovered.

• Climbing: for players to learn to play better, while they play.

• Surprising: for the players to be captivated by the spectacle that the game presents. With this relation of excitement, there is even an invitation to error, because it is often required that the users learn through their mistakes.

Finally, while the user experience in traditional software aims to clearly conceal all the intricacy of the product, in the UX games the aim is clearly to teach how to deal with the complexity. Still, accessibility in games is requisite of important quality, and in the next section, it is described.

2.3 Games and Accessibility

Despite the growing interest in using digital games in different areas, many people, including the elderly ones, are often deprived of its use, whether by some decline, visual, motor, auditory, or cognitive restriction [1, 15].

In order to produce an objective reference to game developers, and in order to include as many people as possible, considering the various needs presented by people with disabilities, a collaborative effort among producers, specialists, and researchers was the accessibility guidelines for games [9].

The specialists and end users should not only use the guidelines during the game development, but also in the continuous evaluation of the application. Given the variety of approaches available to the accessibility assessment for digital games, the use of more than one evaluation method is important to obtain better quality results [10].

In order to consider the limitations of individuals who may be affected by the lack of accessibility, it is generally used the classification defined by the World Health Organization (WHO) in International Classification of Impairments, Disabilities, and Handicaps (ICIDH). It covers the following commitments: visual, which is the consequence of a certain degree of loss of vision; auditory, which refers to the partial or total loss of the ability to hear, by one or both ears; motor, which is the loss or limitation of the function of muscle control, movement, or mobility limitation; and cognitive, which represents a mental or psychological illness, from a retardation developed during infancy to Alzheimer’s or senility, as a result of the aging process.

As the popularity of technology increases, efforts are intensified to understand those affected by these commitments, as well as making technology accessible to the greatest number of people. A result of this guidance is a worldwide reference and consists of the Web Accessibility Initiative (WAI), which developed the Web Content Accessibility Guidelines (WCAG) document, developed from the World Wide Web Consortium (W3C) process [20]. The WCAG guidelines guide the production of content (texts, images, forms, sounds) for the Web that is accessible to people with different types of disabilities.

With the reality of the increasing popularity of digital games, the focus of work and research to provide accessibility to this type of interactive systems should also increase, but efforts are still limited in this direction [42]. There are neither official guidelines, standards, or global initiatives, comparable to WCAG in the field of digital games, nor related governmental or legislative actions. However, there are two interventions in order to obtain this set of guidelines: one published by Special Interest Group (SIG), of the Independent Game Developers Association (IGDA), which proposed 19 accessibility guidelines in 2004. Updated in 2010, it was obtained from an experiment with 20 games, prioritizing visually impaired users, and another one from the organization Norwegian MediaLT, which also in 2004 published a set of 34 accessibility guidelines.

More recently, in 2012, in order to produce an objective reference to digital game developers, and to include as many people as possible, considering the different needs presented by people with disabilities, it was carried out a collaborative effort between producers, specialists, and academics, emerging the accessibility guidelines for games.

Parallel to the achievement of accessibility guidelines for digital games, formalized internationally, it is evident the progress of game quality, from the creation of a structured process of game design, called Unified Design (DU) [16]. The DU includes as one of its steps, the assessment of technical accessibility, which can verify the use based on the guidelines, but also resort to the participation of experts and end users.

3 An Approach for Non-experts Researchers

According to [33], it is increasingly important to explore new forms of interaction, in particular, for collecting data from user experiences. According to [40], the games development and the gamification use allow reaching a significant portion of users, especially younger users, since direct contact with games can occur more frequently in their daily lives. The adoption of gamification elements in case studies, not necessarily related to the games area, can provide new perspectives of data collection, new experiences to the users, and new methods to conduct academic research.

However, there is a need for an approach that satisfies the following requirements: (a) offers several possibilities for supporting the development of games, handling of gamification elements intuitively, (b) that is easy to use even for researchers with no previous experience with the area of games and especially (c) enable the design of case studies, as well as the collection and storage of variables for future analysis.

Considering the previously mentioned requirements on the data collection-oriented games context for researchers who do not have experience in game development, the authors have developed an approach that will be presented in detail in this chapter. The approach aims to assist researchers in the development of their own experiments, allowing the adoption of game elements in their research. The approach is represented in Fig. 9.1.

Fig. 9.1

Approach use example to assist researchers in the development of their experiments involving games and data collection

• 1. Experiment Design: refers to the planning of the purpose of the experiment and what is desired to be obtained with such experiment should be performed after the game development in the initial stage and will be the guide on how the game should be developed. In order to assist the researcher in this design of his experiment, the theoretical concepts about games and user experience will be presented in Sect. 9.3.1.

• 2. Data Collect: occurs during user interaction with the game, which can be designed for multiplatforms (Web, mobile, and desktop). Section 9.3.2 presents the multimedia data collection guidelines and describes the data collection regarding quantitative and qualitative variables. This subsection describes also a case study to exemplify a data collection performed through games using cognitive tests as research base.

• 3. Data Storage: one of the most important parts of game-related research, it is the interaction with users is the way in which the variables will be stored, these topics will be the discussion in Sect. 9.3.3. Such planning guarantees that the data can be analyzed and manipulated in several ways aiming the gain of knowledge related to the objective that the experiment set itself to achieve. Correctly stored data can be later shared and contribute to advancement in the state of the art.

• 4. Data Analysis: after the data collected have been stored in a database, queries to the data can be made by the researcher aiming to knowledge increase. Using statistical methods, several modes of analysis can be performed, on Sect. 9.3.4 will be presented examples of analysis of data collected through games and what results could provide.

Next in this session, each part of the proposed approach will be discussed individually, presenting state-of-the-art examples as well as the practical application of the approach through the Construct 2 tool.

3.1 Experiments Design

According to [26], the most important activity in Experiments Design is to decide which variables the research needs to collect to reach the established goal. According to the author, thinking about which variables can be obtained during the user interaction and the design of the experiment helps convert generic questions (e.g. Will the user like my game?) to focused and testable research questions (e.g. Can a task get done faster by comparing two interfaces?).

In order to better understand the experiments design process, let us analyze one famous game under the research perspective, the Flappy Bird game. Launched in the year 2013, this game was developed for mobile devices, became very popular mainly because it is very difficult, in the opinion of players, who felt challenged to earn higher scores. Because of the great popularity of this game, several researchers have used this case for better understanding the user experience. As an example of this research’s, [23] published one papers named “Exploring Game Space Using Survival Analysis” was conducted with the follow goals:

• To help designers explore game space, like layout, scenarios, and sprites.

• Better understand the relationship between design and player experience.

From the defined focus on the listed objectives, the next step of the research was to delimit the analysis by establishing a research question, in the context of the objectives, which should be discussed and answered. “How can parameters, without changing the rules, affect the game difficulty?”. One of the ways found to answer this research question was to conduct a case study containing the following two stages:

• Stage 1—Questionnaire: focus on collecting independent variables: gender, age, game playing experience, and exposure to Flappy Bird games and its clones.

• Stage 2—Ability Measurement: focus on collecting dependent variables: precision, reaction time, and actions per second.

Once the research question is established and what experimental variables will be used in the data collection process, the next step is the development of the game.. Using the case study presented earlier, below we present the approach (Fig. 9.1) to replicate data collection using the Construct 2 tool. To reply the game and apply the approach, a generic Flappy Bird game model that accompanies the free version of the tool will be used. Available as a free Construct 2 template, the development screen for this game is shown in Fig. 9.2, containing the elements needed to develop the data collection mode in the variables.

Fig. 9.2

Flapping bird game template in Construct 2

The first phase of data collection, from the example cited above, used the following independent variables: gender, age, experience as a player, and exposure to the game. It is very common that in case studies, a questionnaire containing questions about users profile and opinions will be applied at some point. To collect the same variables in the game development tool, a new layout must be added to the game, in which will be inserted the fields in which the player must answer the questionnaire.

An advantage of using forms to collect data while playing games is that you can use your own information to change the interaction experience. If a user says he or she is experienced in a certain subject, a possibility (depending on what one wants to collect) lies in the difficulty level that can be adjusted for each user, and thus, at the end of the experiments, the collected data may reflect most loyal, the real user experience.

The second stage of data collection, from the example mentioned above, used the dependent variables: precision, reaction time, and actions per second. For these variables, quantitative data must be properly collected and clearly presented to researcher for further analysis.

To collect precision variable, users were instructed to simply click a button whenever a constantly moving line was horizontally aligned with a target on the screen. The test was repeated 20 times at three different speeds. To verify the possibility of the tool replicating the same data collection, the Flappy Bird template on Construct 2 has been modified as shown in Fig. 9.3.

Fig. 9.3

Graphics elements created to replicate precision variable collection

In Fig. 9.3, the (a) item shows the position of the red target added along with a black line that performs horizontal movements, and the (b) item shows the setting used to assign the line movement behavior. This item is important as it allows for easy line speed change by only indicating a value in the Period field. The (c) item shows the position of the button to be pressed (in the work of which is not specified which name is assigned to the button). These graphics have been added to the game interface using the Construct 2 tool without the need for programming skills, and can be easily repositioned and dragged using the mouse (drag-and-drop), allowing focus on data collection and not in programming these elements by some specific language or method. To verify that the precision rule has been respected, a program has been added to the example as shown in Fig. 9.4.

Fig. 9.4

Conditional rule for collect the “Precision” variable on game

A condition has been defined on line 22 of the figure, so that a variable called variable_targetAligned will only be added to a value when the button (buttonTarget) is clicked and the line is between the values X ≥ 190 and X ≤ 210; these values represent the alignment of the target and line in the game interface. This range of values on the X axis can be set for greater flexibility of line alignment with the target. Thus, at the end of the interaction, it is possible to know precisely how many times the user was able to perform the condition of pressing the button in line alignment with the target.

This condition can also be reset during user interaction. For example, on line 23, you can increase line speed by creating the following condition “when variable_targetAligned is greater than 5, the movement period will be 8” as shown in Fig. 9.5.

Fig. 9.5

Conditional rule to collect the “Movement Period”

To measure reaction time, players were instructed to press the button as quickly as they could see the horizontal line on the right side of the game window. The average delay, which allowed the reaction time to be calculated, was obtained using the moment the line was positioned to the right of the window and the moment the user pressed the button. To collect this variable, a schedule has been added, as shown in Fig. 9.6.

Fig. 9.6

Conditional rule to verify the “click” event and the horizontal line position

In Fig. 9.6, the line 24 contains the rule that only adds one value to the variable “variable_Button_lineRight” and captures the time (seconds) in the variable “variable_Button_lineRight_Time” when the following condition is met: “When the button “buttonTarget” is clicked and the position of the line “line” is greater than or equal to 300” (X axis value ≥ 30 indicates if the line is to the right of the game window). In the line 25 rule, the moment (time in seconds) that the line is to the right of the window (X ≥ 300) is added to the variable “System_lineRight_Time” by subtracting the two time variables. It is possible to calculate the average delay as objectified in the design of the “Reaction Time” collection.

To collect the data referring variable Actions per Second, users were instructed to keep the button pressed for 10 s. On [23] paper was not specified when users should perform this task. To replicate this collection, the following schedule was added to the example on line 26: “If the buttonTarget button is pressed for 10 s, add a value to the variable ‘variable_touchButton”’ as shown in Fig. 9.7.

Fig. 9.7

Conditional rule to verify the user button touch time

The Construct 2 and other nonprogramming tools have alternative ways of touch-related actions, allowing the researcher, who is not familiar with game development, different alternatives to collect data by performing tasks. By analyzing the [23] work and using the Construct 2 tool, it was possible to understand an example of state-of-the-art experiment design and its alignment with the approach proposed in this chapter. Thus, researchers who do not have prior knowledge of game development have an approach to start their research or experiments to collect user experience data.

3.2 Data Collection

As presented before, there are many aspects that can be studied in experimental scientific works, as well as their cataloged variables, which can vary widely depending on the test and the objective when performing data collection procedures inside game context. For this reason, sometimes, the unfamiliarity with tools, techniques, and languages used in game development, as well as ignorance of gamification-related elements, may discourage investigators involved in experimental investigations from using alternative forms of data collection in their case studies, for example.

There are currently many tools and resources available related to the game development area. Aiming to help those researchers who do not have expertise with game programming or development, below are some examples of tools that enable the design of games without requiring technical knowledge.

• GDevelop: is an open-source, cross-platform game creator designed to be used by everyone—no programming skills required. Site: https://gdevelop-app.com/

• Construct 2: is a powerful ground breaking HTML5 game creator designed specifically for 2D games. It allows anyone to build games—no coding required. Site: https://www.scirra.com/construct2/.

• RPG Maker: allows you to customize every aspect of your game with an easy-to-use interface, making it perfect for beginners yet powerful enough for experts. Site: http://www.rpgmakerweb.com/. GameMaker Studio 2: Making games development accessible to everyone means taking away the barriers to getting started. Site: https://www.yoyogames.com/gamemaker.

• Godot: completely free and open-source under the very permissive MIT license. Site: https://godotengine.org/.

• BuildBox: the power to design, build, and launch 3D and 2D mobile games without coding. Site: https://www.buildbox.com/.

• GameSalad: is the revolutionary game development platform that allows anyone to create the game of their dreams with a sophisticated visual programming interface. Site: https://gamesalad.com/.

To exemplify the approach, this session will present and discuss an example of multimedia game development for data collection. For this example, the Construct 2 tool was selected because it does not require programming language knowledge, is intuitive and easy to use, and allows the user to collect multimedia data. More information about the use of the tool can be found at [8] and [2].

One of the main features of the Construct 2 tool is not to impose the knowledge of any programming technique or language, making it an ideal tool for researchers with no previous experience in game development. Its programming occurs through events that tell the system what the relationship and behavior that all elements in the game should/can do, as well as the interaction with variables and controls. With this tool, it is also possible to export the game to multiple platforms, such as Web, mobile, and desktop, in addition to communicating with database for variable storage. Figure 9.8 shows an example of logic event rules shown in the Construct 2 tool; it is possible to find this same logic structure in other game development tools.

Fig. 9.8

Conditional rule interface required for game development in Construct 2

Due the focus of this chapter is provide ways for user experience possibilities oriented data collection through an approach, will not be discussed basic elements as the download and installation tools. Thus, a set of examples will be presented below, which, from an approach to assist researchers in the development of their own experiments, enables the adoption of game elements.

3.2.1 Speech Recognition Data Audio Collection

In Sect. 9.2.3, accessibility guidelines are described that should also be used to improve the user experience. Among them, using as an example the “Speech” skill, a W3C [20] intermediate category guideline recommends: “speech recognition should occur on the basis of individual words from a small vocabulary (e.g., ‘yes,’ ‘not,’ ‘open’) instead of long sentences or multi-syllable words.” In order to collect data related to the “Speech” skill, the following events can be programmed in the Construct 2 tool as shown in Fig. 9.9.

Fig. 9.9

Conditional rule to speech recognition

After the game project receives the object named “UserMedia,” the events listed in Fig. 9.9 can be created and handled. In line 1, the “UserMedia is recognizing speech” event defines that the microphone should be accessed whenever the game is run in the browser and provides user feedback which exemplifies the text “Currently recognizing speech …”; the text content can be changed or replaced by an image, for example. Between lines 2 and 4 events are defined the exception in case the microphone or the browser does not support multimedia access.

Between lines 5 and 7 is defined the main speech recognition event; there is a condition to click a button to start or pause recognition. In line 7, the “Request speech recognition (Continued mode, Interim results)” setting defines the language used for recognition, and the “Continuous mode” setting complies with the aforementioned guideline of recognizing individual words (For recognition of whole sentences the setting “Single phrase mode” should be used).

3.2.2 Video Screen Recorder Data Collection

Capturing the screen during gameplay can be a valuable resource when it comes to analyzing the user experience. According to [26], screen record is a creative approach to collecting data, especially when there is interaction with some software. One problem related to screen record use, as a data collection feature, is the software availability in this segment. In some cases, specialized software required license for use or because the game frame rate does not perform the correct screen record.

The Construct 2 tool allows to record the screen during entire gameplay or record special moments related to user actions and, as output, generate a video that can be exported in several formats. Figure 9.10 shows the logic rules used for this purpose.

Fig. 9.10

Conditional rule to video screen recorder

In line 1 of Fig. 9.10, the browser compatibility test is defined and the feedback generated in the example is given on interface by text display, but can be changed. Line 2 tells the system that when the game starts (“On start of layout”) and the browser supports screen capture, the record button will be active. In line 3, the event that starts recording is configured, which is conditioned at the click of a button. In line 4, the action of interrupting the recording by means of a button is foreseen, and in line 5, the immediate download of the result of the capture in video format that can have its format (AVI, MP4, etc.) is changed.

Another feature on Construct 2 tool that can be used to collect more accurate data is that it allows data screen recording during a specific moment of interaction on gameplay, using user action as a condition. In Fig. 9.11, for example, recording only occurs when the user gets 20 points and will stop when the user reaches 30 points.

Fig. 9.11

Conditional rule example to video screen recorder according user interaction

In line 3 of Fig. 9.11 is setting the system condition that will only start recording when the value of the variable “userPoints _variable” is equal to 20 and line 4 indicates the end of record occurs when the same variable reaches the value 30. Programming such as this permits greater flexibility and case studies can be performed more precisely to collect expected data defined in design of the experiment.

3.2.3 User Photos Data Collection

During user’s game interaction, the Construct 2 tool also offers the ability to capture photos before, during, and after the game is running. User video recording in the currently available version does not yet offer such support. Figure 9.12 shows an example of the programming required for webcam access.

Fig. 9.12

Conditional rule to user photos data collection

For photo capture, the following logic rules defined in the Construct 2 tool, shown in Fig. 9.12, show the main definitions to be made:

• 1. Line 1: Checks if one webcam is connected and properly installed when the game is loaded.

• 2. Lines 2 and 3: Allow listing and selection of detected webcams.

• 3. Line 4: Using a button allows the previously selected webcam to be started; also displays the detected webcam resolution. With this logic rule it is already possible to view in-game webcam footage in real time.

• 4. Line 5: If the webcam is recognized, the buttons that allow you to take a picture are activated, and stop using the webcam.

• 5. Line 6: Displays the status if the selected webcam is not working.

• 6. Line 7: Required for the system to display the capture content at all times.

• 7. Line 8: Provides an action in the event that the button that stops recording is triggered.

• 8. Line 9: One action if the browser does not have multimedia support.

• 9. Line 10: Allows the capture of the user’s photo using one button. It also allows you to configure image quality and store it in memory.

• 10. Line 11: Allows to download the previously captured photo.

It is possible also to take user’s photo under conditions associated with an action. This possibility enables the capture of a moment (or sequence of moments) when the user performs determination action or any condition is reached. In Fig. 9.13, for example, the user photo is taken when the variable “userPoinst_variable” is equal to the value 10.

Fig. 9.13

Conditional rule to user photos data collection according user interaction

3.3 Data Storage

A very important step in research involving user experience data collection is developing strategies for data storage generated during the interaction. Such strategies should ensure the correct analysis in search of knowledge and allow research reproducibility through data sharing. The data should therefore be stored in such a way as to allow easy manipulation, subsequent consultation, and reproducibility of analyses.

In this section we describe an example of data collection in real time and storage inside one online database following the approach described in Fig. 9.1, developed using the Construct 2 tool. The possibility of data storage in an online database aims to ensure all user interactivity data is secured, accessible, and be analyzed from anywhere. Section 9.2.3 discussed the relationship between accessibility guidelines and game aims including as many people as possible, considering the diverse needs presented by people with disabilities. The guidelines should not only be used during the game development process, but also in the ongoing application evaluation by experts and end users. Given the variety of approaches to assessing accessibility for digital games, the use of more than one assessment method is important to achieve higher quality results [10].

According to [9] the guidelines for games can be divided into three types: basic, intermediate, and advanced. These categories include differences in the number of people benefited, the impact provided to users, and the cost of implementation. The guidelines are also grouped into subcategories related to the different skills: motor, cognitive, visual, auditory, and speech. Using the “Motor” skill guideline as an example, it is recommended: “make interactive elements that require stationary precision (e.g., cursor/touch controlled by menu options).” Using this guideline as a base, Fig. 9.14 shows a game interface in which users must drag the apples to the basket with the mouse, allowing them to collect two variables: the “amount” of apples and the exact “time” in that the apple was put in the basket.

Fig. 9.14

Game interface example for collecting Motor skill data

The game demonstrated in Fig. 9.14 has been developed using the Construct 2 web tool and can be accessed locally or hosted on a web server. To store the data variables “amount” of apples in the basket and “time” in which the apple is collected an online server was necessary, following the LAMP⁴ scheme, a code in PHP language to perform access and insertion in the database.

In Listing 9.1 is presented the logical rule to create the table called “apples” that will receive the user interaction data. In line 3, the “score” variable will receive the amount of apples placed in the basket and in line 4, the “time” variable will receive the times (seconds since the beginning of the gameplay) of each apple was placed in the basket.

Listing 9.1 SQL code used for database table development

In order to store variables inside the database, a code as shown in Listing 9.2 in the PHP language was hosted on the web server. The game programming makes a request using the POST method to the PHP source code. In line 2, a header was added to allow the game to access the database, from line 4 to 7, the database connection variables were created and in line 9, the variable $dblink makes the connection with the database using the previous PHP variables. Finally, lines 11 and 12 contain the PHP variables “score” and “time” that will receive the values sent by the game via the POST method. Line 13 executes the query to insert data into the database, and in line 15, the connection is terminated for security reasons. This PHP source code exemplifies only a simple way to insert data into the database, other structures in the database area can be used for better data storage results.

Listing 9.2 PHP code used for insert values in game database

Figure 9.15 shows a logical rule for sending data to an external database. To make this send process, the Construct 2 tool uses AJAX⁵ that must be added beforehand in game development.

Fig. 9.15

Conditional rule to external database storage

In Fig. 9.15 line 1, one condition has been created and triggered every time when a collision occurs between the “basket” and “apple” sprites; the “apple” sprite disappears (apple Destroy function) from the game interface so that the user understands that the apple has been placed in the basket. The value 1 should also be added up to the current value of the variable “variable_Basket.” The most important part of the programming still occurs on line 1 which makes AJAX call as follows: variable_Url & “savescores.php?score=” & variable_Basket & “&time=” & variable_Time, where:

• variable_Url: a global variable that contains the online server address (“http://www.youraddress.com.br” or IP address can be used). An important detail, at the end of the address assigned to this variable, “/” must be included so that concatenation using the symbol “&” can be performed.

• “savescores.php?score=”: this is the continuation of the address that will be concatenated with the server address global variable. In it should be placed the name of the file containing the PHP code hosted on the server. In this example, the web page file has the name “savescores.php,” as the request will use the POST method. The continuation of the request sends the score and time variable value will be received in the database.

• variable_Basket: this variable has the value of which apple was placed in the basket. Since researcher desired to know the time when the apple is placed in the basket, every time an apple is placed, a new database entry is made. In this way, it will be possible know how much time has passed between the user placing apple number 1 and apple number 2 in the basket.

• “&time=” & variable_Time: the second value sent to the bank will be the variable “time” which, in this example, is concatenated with the variable “variable_Time.” This variable gets the exact time an apple is placed in the basket.

• (method “POST”, tag “PostScore”): configures which request method the AJAX call should perform and assigns a tag to this call. This feature allows to verify if the call was made successfully.

The line 2 as shown in Fig. 9.15 tells the game system that at all times (“Every Tick”) the value of the _Basket variable must be updated in the interface. A text box named Score has been created on the basket in the game interface to inform the user the quantity of apples inside the basket.

In line 3 the system updates the text box on the interface, labeled “Clock,” every 1.0 s with the current time since the game started. By default, the tool works with milliseconds and needs to be converted to minutes and seconds, using the expression: zeropad(int (time / 60 % 60), 1) & “:” & zeropad (int (time % 60), 2). Also in this line, the time value is added to the variable “variable _Time” every 1.0 s the value is displayed in the interface and the value that will be added to the database is always the same.

In line 2 as shown in Fig. 9.15, the system checks if the AJAX call named “PostScore” was executed successfully (it is also possible to assign actions if the call is not executed successfully). The action assigned in the example adds the value 1 to the variable “variable_Save” and displays this value in the interface through a text box called “save,” this variable was created to inform that the data is being successfully saved to the database.

In Fig. 9.16, the result of the stored data query is presented. The first apple was placed in the basket at 3 s and the third apple was placed at 30 s. Thus, it is possible to verify the speed that each user, using their motor skills, is interacting with the game. As a possibility of collecting new data, it would be feasible to also control the speed at which new apples appear on the screen, check the order in which users place apples in the basket, and change the position of the basket, for example.

Fig. 9.16

Query in Apples table returning stored values

Through the game example described here, using accessibility motor skills concepts and the Construct 2 development tool, it was possible test one example to store the data generated during user interaction on online database follow the approach described in Fig. 9.1. There are other ways to store data using the tool, but among those currently available, this mode proved to be efficient for further analysis.

3.4 Data Analysis

The case study described here exemplifies a data collection performed through games. In this example, the data collection performs an important role in research and it was made as a part of an investigation about the conversion of cognitive tests batteries for the digital medium. The test app was developed in the form of a game. The digital cognitive test was submitted to an experiment with users, based on usability heuristics [7]. Heuristics are processes that seek to find problems that recurrently occur, and in general, the possible solutions are recognized.

Due to the end user of the cognitive test is the elderly public, accessibility issues have been considered since its planning. An example was the choice of the color palette, which was made respecting its appreciation by colorblind individuals, more specifically in the three possible situations⁶: Tritanopia, Protanopia, and Deuteranopia.

The tool Color Oracle⁷ was used to inquire the palette in the three situations and then the colors that in one or more cases were very close to each other were eliminated, becoming almost indistinguishable. Even so, caution is still needed when designing the game’s assets. Feedback and objectives cannot rely exclusively on color to differentiate objects. Sounds and shapes should also be used.

In the battery of cognitive tests, eight aspects related to the intellectual requirements of the users were treated, namely: (Test 01)—Visual Perception, Incidental Memory and Immediate Memory, (Test 02)—Praxis, (Test 03)—Abstraction, (Test 04)—Digits Extension Test (Direct), (Test 05)—Digit Extension Test (Inverse), (Test 06)—Attention, (Test 07)—Verbal Fluency Test, and (Test 08)—Facial Recognition. From the definition of these aspects, a correlation was sought between them, in order to identify/verify the level of reach that the portability for the digital medium would allow, without there being a significant change in the context.

The data collections in the Case Study were carried out having a set of cataloged variables, referring to the aspects related to the cognitive loads required of the users. Then, cataloged variables are listed, which exemplify how the experiment was planned with users using the game (playing).

• 1. (Test 01)—Visual Perception, Incidental Memory, and Immediate Memory:

  • (a) Number of objects: it is the number of objects that show up in the screen simultaneously. This variable can have a direct impact on the relative performance of the same player. For example, an interaction with fewer objects on the screen may have better results.

  • (b) Amount of Shapes: it is the number of different shapes that show up on the screen. This variable can have a direct impact on the relative performance of the same player. For example, a test where triangles and squares appear may have fewer results than one in which only squares appear.

  • (c) Elements of Confusion: it is the number of objects that will be inserted to create confusion, relevant only in the previous version of the test, in which the player needed to mark which objects were different from the memorized matrix. This variable can have a direct impact on the relative performance of the same player. It is expected that in a scenario with more elements of confusion the player makes more mistakes.

  • (d) Score: player’s score from 0 to 100. This data only stores how many of the spaces were filled correctly, ignoring spaces that were filled unnecessarily. To circumvent the possibility of the player filling the entire array the number of objects that can be placed was limited to the same number as the variable “Amount of Shapes.” This data is important to compare the matrix inserted by the player and the original matrix, evaluating if the player actually memorized the original matrix.

  • (e) Total Time: it is the sum of the time since the player started the test until completing it. This may be indicative of difficulty in interacting with the tablet screen, difficulty in understanding the test as well as motor limitations or distractions.

  • (f) Clicks:

    • Clicked form: it is the data that stores how the screen was touched by the user, relevant only in cases with more than one available way. This may indicate difficulty in interacting with the tablet screen, difficulty in understanding the test as well as motor limitations.

    • Color: it is the data that stores the color of the touched object, relevant only in cases where there will be more than one object color. This data may indicate difficulty in distinguishing colors or difficulty in understanding the test.

    • Time: it is the time between the current and previous clicks. This data may indicate difficulties in interacting with the tablet screen, difficulty in understanding the test as well as motor limitations.

    • Click type: it is the data that marks if the player touched a button, an object of interest or none of the previous two, thus setting a miss click. This may indicate difficulties in interacting with the tablet screen, very small objects, and motor limitations.

    • Accuracy: it is the data that marks whether the touch made was a hit or an error, within the test hit and error settings. This data may indicate difficulties in interacting with the tablet screen, difficulty in understanding the test as well as motor limitations.

• 2. (Test 02)—Praxis

  • (a) Score: player’s score from 0 to 100, marking how many objects were fitted in correctly. This data stores the evaluation of the connections two by two, with a small margin of error for each connection, dividing the maximum score by the number of total connections that will be verified. This may indicate difficulties in interacting with the tablet screen, difficulty in understanding the test as well as motor limitations.

  • (b) Total time: it is the sum of time since the player started the test until completing it. This may be indicative of difficulty in interacting with the tablet screen, difficulty in understanding the test as well as motor limitations or distractions.

  • (c) Clicks:

    • Shape: it is the shape touched by the user. This data may indicate that the player has more difficulty interacting with a certain piece of the Tangram, whether its size or format.

    • Color: it is the color of the shape played by the player; it is important because there is more than one copy of some shapes, but different colors and/or rotations. This data may indicate that the player has difficulty in distinguishing certain colors.

    • Time: it is the time between the current and previous clicks. This may indicate difficulties in interacting with the tablet screen, difficulty in understanding the test as well as motor limitations.

    • Click type: this data indicates whether the player touched a button, an object of interest, or none of the previous two, setting a miss click then. This may indicate difficulties in interacting with the tablet screen, very small objects, and motor limitations.

• 3. (Test 03)—Abstraction

  • (a) Score: player’s score from 0 to 100, marking how many interactions of the abstraction test were properly answered. This may indicate difficulty in interacting with the tablet screen, difficulty in understanding of the test as well as motor limitations.

  • (b) Total time: it is the sum of time since the player started the test until completing it. This may be indicative of difficulty in interacting with the tablet screen, difficulty in understanding the test as well as motor limitations or distractions.

  • (c) Clicks:

    • Time: it is the time between the current and previous clicks. This may indicate difficulties in interacting with the tablet screen, difficulty in understanding the test as well as motor limitations.

    • Click type: this data indicates whether the player touched a button, an object of interest, or none of the previous two, setting a miss click then. This may indicate difficulties in interacting with the tablet screen, very small objects, and motor limitations.

    • Accuracy: it is the data that marks whether the touch made was a hit or an error, within the test hit and error settings. This data may indicate difficulties in interacting with the tablet screen, difficulty in understanding the test as well as motor limitations.

• 1. Sequence Size: it is the variable that stores how long the sequence on the screen is. This variable can have a direct impact on the relative performance of the same player. For example, longer sequences can cause more confusion by repositioning the various numbers on the screen.

• 2. Total errors: it is the data that stores how many times the player touched a number that was not next in the sequence. This data may indicate difficulties in interacting with the tablet screen, difficulty in understanding the test as well as motor limitations.

• 3. Total time: it is the sum of the time since the player started the test until completing it. This may be indicative of difficulty in interacting with the tablet screen, difficulty in understanding the test as well as motor limitations or distractions.

• 4. Clicks:

  • (a) Pressed Number: it is the data that stores which digit was pressed. This data may indicate difficulty in understanding the test and confusion with the numbers shown on the screen.

  • (b) Time: it is the time between the current and previous clicks. This data may indicate difficulties in interacting with the tablet screen, difficulty in understanding the test as well as motor limitations.

  • (c) Accuracy: it is the data that marks whether the touch made was a hit or an error, within the test hit and error settings. This data may indicate difficulties in interacting with the tablet screen, difficulty in understanding the test, distraction as well as motor limitations.

For each of the tests, the data set represented by the cited variables was captured during the interaction process with the elderly. These measures were used in the convergence analysis phase, with the tests validated according to [7] guidelines. The main objective in the use of these variables was to represent the result of each test and to allow the computational analysis, without the human perception.

3.5 Others Data Collection Possibilities

As technology advances new forms of interaction can be offered to users creating other experiences for new applications or exploring new possibilities. For example, the 2006 published work [18] proposes virtual environment interaction to reduce time and unify the exploration and analysis of qualitative and quantitative volumetric data. The study case described in this work that the user should use a 3D glasses, a flat screen, and a pen control to interact with the projected 3D objects. Although there were not many hardware resources at the time, researchers were looking for ways to provide users with new experiences for collecting and interacting with data. As mentioned earlier, currently new possibilities can be used for data collection during user interaction with new experiences in multimedia games, one of these possibilities is the use of Extended Reality (XR).

XR is a new terminology recently adopted in both market and state-of-the-art applications. According to [21], XR encompasses a wide spectrum of the reality continuum including virtual reality (VR), augmented reality (AR), and mixed reality (MR). In simple words, XR comprehends the three immersive technologies (VR, AR, and MR) on the same “umbrella” terminology aiming to unify technology and people. Because it offers new possibilities in game development and data collection, we selected the Virtual Reality area to better understand the user experience data collection potential.

As an example of the use of state-of-the-art VR games, the paper [4] proposes a virtual reality (VR) interactive game to offer an experience themed around racially motivated police brutality. The users comes face to face with the characters of the story, filmed with live action, and interacts in the space with them directly using gaze interaction and voice recognition aimed at.

To achieve the goal of exploring the emotional impact of VR space versus traditional film, during user interaction head movement data is collected (gaze interaction) to indicate options in the game’s UI and also the user’s own voice for action selection. Another example of user data collection generated by physical interactions with games in virtual reality is discussed in the work of [41] which aims a qualitative study with seven people, so that we could demonstrate how our visualization could represent the individual differences in the nature and level of physical activity for each game. For the data collection of use the paper describes Microsoft Kinect V2 use which allowed data collection about what specific part of the body players used during each VR game and a head-mounted display for Virtual Reality game viewing and heart-rate data collection for each participant. For data comparison users were asked to rest for 2–5 min to get a resting heart-rate and after each game, there was a 2–10 min break, to collect recovery heart-rate. With this data it was possible to create body maps to illustrate how the body maps provide a compact representation that can enable people to know what interactions and muscle groups each game involves, how much exertion they can expect, and how to make a set of games in a session a valuable part of their broader physical activity.

In order to investigate the possibilities that the potential use of Virtual Reality technology and gesture-based interaction may provide, we conducted an exploratory research using the technologies mentioned above and the approach proposed in Fig. 9.1, below our used methodology:

Experiment Design: identify the technical feasibility required for user data collection through the use of Virtual Reality and Gesture-Based Interaction technologies. The following hardware and software resources were used to conduct the exploratory research:

• Lenovo Mirage Solo: for Virtual Reality tests, the Lenovo Mirage Solo⁸ has been selected. This head-mounted display allows the mobile app installation for the Android Operating System. This device does not require external sensors for use; another interesting feature of this headset is the light weight, which helps usability of users.

• Senso Glove: for interaction through gestures, the Senso Glove⁹ was selected. Similar to a regular glove, it allows gesture capture made by the user’s hands for interaction in the virtual environment.

• Software: for a prototype development Unity engine was selected because it possesses several features for Virtual Reality development and Senso Glove support. The C# language was used with Unity and as a development SDK we used the Senso Glove SDK¹⁰ which allows the raw data collect generated from the users hand gestures. Lastly, the Google Daydream SDK indicated by the manufacturer for development was used for import Lenovo Mirage Solo capture movement features.

Figure 9.17 shows a user example testing the Lenovo Mirage VR headset and Senso Glove interaction.

Fig. 9.17

Gesture-based interaction applied to Virtual Reality elements interaction for data collection exploratory research

Data Collection and Storage

For technical feasibility tests, the research of a Virtual Reality application was started. Firstly, an application was developed in VR through Unity that can be seen in Fig. 9.18.

Fig. 9.18

User making gestures through a glove to interact with a virtual environment

In the application on Fig. 9.18, the users should simulate assembling and disassembling hardware components of a hardware server using their own hands. The participants were was not properly informed about how they should make the procedures hold the server pieces in front of them; this intentional lack of information was adopted in order to verify the strategies used by users while learning a new means of gesture interaction.

After some initial observations, some cases made it impossible for gestural interaction to occur satisfactorily because was not possible directly communication between the glove and the headset due driver incompatibility problems. Because of these technical restrictions, it was necessary to use one laptop to mediate the communication between user generated data, using Senso Glove, and Lenovo Mirage Solo headset; for tests Lenovo ThinkPad T460 laptop was selected but for study replicability it is possible if user has another laptop with the same hardware specifications. This new hardware arrangement (Glove → Laptop → Headset) caused some delay on communication impacting the user experience.

Data Analysis

Through this simple exploratory research using gloves and VR headset, with some difficulty, it was possible to capture raw data from the sensors present in Senso Glove. For future research, such data could verify the correct positioning of the hand of the users in certain virtual games that require hand gestures precision.

4 Conclusions

This chapter covered the topic of game development for researchers in areas related to experiments that consider user experience. In particular, the games adoption in scientific research is discussed to assist data collection and case studies. This chapter serves a wide range of researchers, even those who encounter difficulties or are unfamiliar with game element development. The use of games in research presents itself as a topic of great relevance for professionals and researchers involved with user experience, especially for those who want alternative ways of acquiring knowledge and conducting research through experiments and case studies.

The authors experience has shown that game development can be both a challenge and an innovation for research in various fields of science. Researchers have the knowledge to develop experiments containing data collection from user interaction. However, their lack of familiarity with the tools, techniques, and languages used in game development can often discourage them from adopting innovative strategies in existing experiments or in devising new ways to gain knowledge through experience analysis of the user involving gamification elements.

Finally, in this chapter, we present a game development approach using the Construct 2 tool that allows several possibilities to support game development and, mainly, enables the experiment design of case studies, data collection, and storage for future analysis. For better use of the approach, an overview of the gaming area was presented aiming at data collection, containing references about the user experience, considering the topics of usability, accessibility, and related studies examples.