Keywords

Introduction

Game-based learning consists of challenge; response; and feedback; which are three key elements of any learning-game design; known as the magic cycle of playful learning. Game design features (i.e.; motivational elements; game mechanics; visual aesthetics; narrative; and background sounds) provide the learning experience (Plass et al., 2015). Learning dynamics are based on the quality of game design features which are common either for games or within the encapsulating gamification process; that is; a less structured playful activity using a part of game elements such as points; badges; and leaderboards (Deterding et al., 2011). A sub-genre of digital games is the serious games (SGs).

The earliest and widely used definition states that SGs are deliberately educational; the goal of engaging users for entertainment purposes is absent (Abt, 1970). SGs’ flexibility allows them to be used in many educational scenarios and domains (Feng et al., 2018). Also; many researchers have acknowledged their instructional value; the relevant literature reports; in most cases; positive learning outcomes (Connolly et al., 2012; de Freitas, 2018; Erhel & Jamet, 2019). However; the evaluation process of the functional components of SGs remains rather unclear (Alonso-Fernández et al., 2018; Zhonggen, 2019). Many supported the view that we lack a well-grounded methodology for measuring their effectiveness (Serrano-Laguna et al., 2018) and that past studies have not solved this problem (Shi & Shih, 2015). Several reasons are responsible for the absence of a rational solution. The field of SGs is fragmented across diverse disciplines (de Freitas & Ketelhut, 2014). An additional difficulty is the existence of different game genres; which; in most cases; have few in common. Thus; the results can be generalized only to SGs of the same genre (Ravyse et al., 2017). Research trying to take into consideration many salient factors that render SGs effective is rather uncommon (Ravyse et al., 2017). Finally; there is no common consensus on how some features are defined or what sub-features are incorporated in a factor. Many researchers used different terms for describing the same factor or used different evaluation methods for examining it (Fokides & Atsikpasi, 2018). Thus; the problem is not so much the lack of assessment methods as other researchers suggested (e.g.; Serrano-Laguna et al., 2018); but issues in these methods per se.

Educators; policy-makers; and software designers have to be reassured that SGs are effective enough to be used in teaching (Westera, 2019). In this respect; solid evaluation methods are needed; able to overcome the abovementioned problems. Toward this end; certain steps have to be taken with the first one being to listen carefully to what the users have to say. What is more; attention to their views has to be paid without making any a priori assumptions on how or what shapes their experiences when playing SGs; so as to avoid the biases and weaknesses of previous research. This was exactly the study’s objective. As it will be scrutinized in the sections to follow; by using a questionnaire consisting of ten open-ended questions; it tried to examine the users’ experience in SGs (both playing and learning) in an effort to determine which factors are important; which are not; and how they are related to each other.

Factors Commonly Used in Serious Games’ Assessment

Given SGs’ complexity and the fusion of leisure and “serious” purposes; established evaluation methods may fall short; more suitable methods need to be employed (de Freitas & Ketelhut, 2014). Not only pedagogical aspects need to be considered; but equally influential variables are features such as gameplay; game mechanics; aesthetics; and narrative (Faizan et al., 2019). For some; engagement and motivation were the most significant factors (e.g.; Huang et al., 2010). Others examined narration as a contributing factor (Khan & Webster, 2017). Winn (2009) focused on learning (in terms of content and pedagogy); storytelling (such as narrative; character; and settings); gameplay/mechanics; and interface. Enjoyment; usability; and learning effectiveness were the most commonly used evaluation criteria when measuring both the game’s quality and effectiveness (Steiner et al., 2015). Others focused on immersion; interaction; gameplay; feedback; challenge; scenario; fun; and learning-game integration (e.g.; Faizan et al., 2019). In a comprehensive literature review; Calderón and Ruiz (2015) identified 18 features for assessing serious games (e.g.; learning outcomes; understandability; game design and aesthetics; user’s satisfaction; usability-ease of use-playability-learnability; usefulness; motivation; educational aspects; engagement; user’s attitudes-emotions; efficacy; social impact; enjoyment; and interface). After reviewing the relevant literature; ten factors were identified; commonly used for measuring SGs’ impact on the users’ experience:

  • Motivation. The foremost reason for using SGs in an educational context is their appeal (e.g.; motivation; fun) (Westera, 2019). The user is willing to invest effort (and time) in playing because the activity; by itself; is rewarding and not because he/she is expecting some extrinsic rewards. The assumption is based on the close relationship SGs have with commercial games and the high entertainment value the latter have. The positive influence of SGs on intrinsic motivation was also emphasized (e.g.; Dreimane & Upenieks, 2020; Kaimara & Deliyannis, 2019).

  • Realism-interactions. Realism stipulates how closely real life is replicated within a game. While realism certainly has visual and audial aspects; it is not limited to these. Psychological dimensions are also included (Ravyse et al., 2017). Another factor to consider is interaction modalities. That is because interactions enhance the sense of realism (Mortara et al., 2014). Therefore; in this study; realism and interactions were treated as a single factor.

  • Presence-immersion. These subjective experiences suffer from definitional problems; though they are quite similar; they highlight different facets of what the players feel during playing (Fokides & Atsikpasi, 2018). Presence describes the psychological state in which one perceives the virtual objects as being real (Ivory & Kalyanaraman, 2007). Immersion is the sense of “being” in the application/game. Immersion is a manifold construct; conceptualized as challenge-based; sensory-based; and imagination-based (Ermi & Mäyrä, 2005). Nevertheless; it can be argued that immersion is a more suitable construct; given that it can explain a broader range of subjective experiences (Jennett et al., 2008). Given the above; this study used “immersion” as an umbrella term; encapsulating presence.

  • Playability-usability. Playability can be viewed as the experiences a player has when interacting with a game (Voida & Greenberg, 2012). A subset of playability is usability; a term describing how easily a player can learn how to control a game (Pinelle et al., 2008). The terms usability and playability are used interchangeably in many circumstances (Sánchez et al., 2012). Therefore; in this study; the term “playability” was used for both playability and usability.

  • Enjoyment. The enjoyment one feels when playing games is related to a range of attributes such as satisfaction and motivation (Boyle et al., 2012). Enjoyment is used in most evaluation frameworks; and many studies reported enjoyment as a contributing factor in the effectiveness of digital educational games (e.g.; Connolly et al., 2012; Kaimara et al., 2020; Steiner et al., 2015).

  • Feedback. Feedback gives players the sense of progress (Cheng et al., 2015). Therefore; the role of feedback’s mechanism is to inform players of the results of their actions/activities; to allow them to reflect on these results; and to reconsider their strategies. As a result; self-directed learning is fostered leading to positive learning outcomes and knowledge retention (Sušnik et al., 2018).

  • Narration. The narrative portrays the game’s events; introduces the game’s fictional context (Charsky, 2010); and keeps players tied up to the game (Couceiro et al., 2013). In the context of SGs; the role of the narrative is to provide declarative knowledge for players (Kiili, 2005).

  • Interface. The interface’s role is to assist and guide players through the game. It is an important aspect in educational games; and; as such; designing a friendly interface requires consideration and attention to details (Laamarti et al., 2014).

  • Learning goals. Well-designed; tough; but achievable goals motivate players while providing an engaging and pleasurable experience (Shi & Shih, 2015). In SGs; goals are not limited to gaming. There are also learning goals that have to be reached. Regardless of the goals’ nature; SGs are goal-directed through clearly defined and measurable achievements (Bellotti et al., 2013).

  • Learning outcomes. All the above factors were used for assessing learning; which is the ultimate goal of SGs and the most well-studied factor (Faizan et al., 2019). The learning outcomes’ assessment can be based on educational objectives’ taxonomies; while psychomotor; cognitive; and affective domains can outline the learner capabilities (Gilbert & Gale, 2007).

What became evident from the above literature review is that different factors for SGs’ assessment were used; different genres of SGs were studied; and the learning subjects/settings were also dissimilar. What is more; it seems that researchers have not reached an agreement on the definition of many factors; others are ill-defined; and; in some cases; their boundaries are supple; given that they may incorporate other factors as well (e.g.; presence and immersion; playability and usability). Thus; there might be a significant problem in quantitative studies which utilized scales (with close-ended questions). Trying to capture elusive factors using just a few items in a scale leaves room for misinterpretations. Even more importantly; participants are asked to answer questions that might not even be relevant to how they view a given factor. On the other hand; qualitative studies give enough freedom to users to express themselves; thus achieving an in-depth understanding of their views (i.e.; how they define the factors and how they think they interact); but they suffer from limited sample sizes.

Consequently; researchers; in order to surpass the abovementioned limitations; are in need of a different methodological approach for examining the users’ experience (both playing and learning) when playing SGs. On the one hand; this method should allow researchers to draw conclusions based on robust sample sizes. On the other hand; the method should give participants the chance to freely express their views. As it will be further elaborated in the following section; this was exactly what the study at hand tried to achieve.

Method

As already mentioned; the study’s objective was to examine the users’ experience (both playing and learning) and to determine how different factors are related to each other. On the basis of the arguments presented in the preceding section; it was decided to focus on the ten most commonly used factors in SGs’ evaluation. A descriptive research method was followed (Bernard & Bernard, 2012); using a survey tool consisting of ten open-ended items. As the raw data were qualitative in nature (open-ended questions); they were thematically coded and then they were quantified. By following this method; a large sample size was achieved; while; at the same time; participants freely expressed their views and feelings.

Research Questions

One general research question traversed the whole study; which may be expressed as “How do users believe that the aforementioned factors interplay with each other and shape their experience when playing SGs?” This general research question was then broken into ten specific ones (one for each factor analyzed in the preceding section); as presented in Table 9.1.

Table 9.1 The research questions
Full size table

Participants and Duration of the Project

Students enrolled at the Department of Primary Education (University of the Aegean) and the Department of Audio and Visual Arts (Ionian University) were recruited; as both groups are potential users of the SGs employed in this study (presented in the “Materials” section). Besides being potential SGs users; students from both departments attend a number of courses related to the development of educational software (educational games and SGs included). Thus; they were aware of the main principles behind the use and design of SGs. An invitation was posted to the Facebook groups these two departments maintain; addressed to students interested to participate. Students were also informed that they will be asked to play an SG (or two if they were interested in doing so) and complete a short questionnaire. An outline of both games was also provided (e.g.; learning content; games’ scenario; and genre); so as to avoid unengaged participants. Participating students were also informed that the study was conducted on a voluntary basis and that personal data from each game session was going to be recorded (i.e.; the computer’s IP address and the session’s duration). Furthermore; instructions were provided on how to install the games and log in to them. The total number of recruited students was 384.

Materials

An issue the study had to resolve; prior to the beginning of the research process; was what SGs to select. Then again; one has to be reminded that the study’s objective was not to examine/evaluate specific SGs; the objective was to record users’ views (either good or bad) and through this to examine specific factors’ interconnections. In this respect; the game’s quality and genre were irrelevant. What was important was to select SGs in which the ten factors/features discussed in the preceding section were present, so as users to be able to comment on them. Following this line of thinking; two games developed by Triseum (https://triseum.com) were chosen as the study’s material. Although they differ quite a lot; both are typical SGs; addressed to young adults (university students). Moreover; both are well received by their intended audience and awarded on several occasions.

The first one; called “ARTé Mecenas;” is a turn-based 2D game; supporting courses related to arts’ history. Users assume the role of the head of the Medici family during the tumultuous Italian Renaissance. They have to balance relationships with powerful states; the Catholic Church; and merchant fractions; as they struggle for financial dominance. At the same time; users try to play an essential role in the creation of famous artworks and monuments of the Renaissance. Players’ decisions affect the welfare of the Medici Bank and ultimately the course of art history. While playing; the actual course material is presented (e.g.; details for actual artworks; buildings; and historical facts). The game’s objective is to enable students to appreciate the interconnectedness of economy and art (e.g.; through art patronage). The second game; called “Variant: Limits” is a 3D game attempting to connect mathematics and gameplay; empowering deeper engagement with the content; while making the learning experience more fun. The game’s goal is students to appreciate the notion of curriculum-based calculus concepts. The calculus topics covered are (a) finite limits (e.g.; one-sided limits); (b) continuity (e.g.; intermediate value theorem and continuity at a point); and (c) infinite limits (horizontal and vertical asymptotes). Users explore a vast virtual world (a fictitious planet) and manipulate objects for opening and passing through gates within it; using calculus principles and theories. The objective is users to successfully understand increasingly complex calculus concepts and to help the game’s main character to save the planet by reaching her final destination.

Instrument

A questionnaire available online was used which consisted of ten open-ended questions. Each research question had a corresponding item in this questionnaire (Table 9.2). All items urged the participants to make suggestions that would improve a specific game factor. The rationale behind this setting was that these suggestions might reveal other factors that may have an effect on the factors in question. Answering all the questions was not mandatory as it was possible that some participants might not be able to come up with a suggestion or might not be willing to provide a response. On the other hand; they were asked to be as specific and as analytic as possible in their responses. The questionnaire was open for submissions for the whole duration of the project.

Table 9.2 The open-ended questions
Full size table

Procedure and Data Processing

As already mentioned; the participants were asked to play either (or both) of the two games. The only condition was that they had to play them for a minimum of 2 hours and/or complete at least two levels. As both games included an introductory/tutoring level; for familiarizing the players with the interface/controls; time spent by playing this level did not count as playing the game(s) per se. After confirming that a participant actually played the game(s) (by examining the log files); he/she was provided with the questionnaire’s link.

Given that the research questions were epistemological in nature; meaning that they were related to knowing and understanding the phenomena of interest; and given that participants responded to open-ended questions; a thematic coding analysis was considered more appropriate (Saldaña, 2015). This method involves the identification of text passages linked by a common theme; the indexing of these passages into categories; and the establishment of thematic ideas (Gibbs, 2007). There was no need to transcribe verbatim the participants’ responses as these were already in a digital form. Ten documents were created (one for each question) and the corresponding replies were copy-pasted to them. Two individuals with expertise in SGs acted as coders and ATLAS.ti was used for extracting/labeling the codes and themes. The coders’ reliability was assessed (a) in a pilot test in which a randomly selected quarter of the responses was used and (b) formally during the coding of the full dataset. Cohen’s kappa coefficient was used for determining the raters’ consistency; and it was found to be very good [κ = 0.910; p < 0.001; 95% CI (0.903; 0.917)] (Landis & Koch, 1977). During the coding of the full sample; all responses were viewed once; for identifying the main ideas. A second round followed; having as an objective to label these ideas with codes. This process was repeated twice for reducing the redundancy of the codes and themes.

The next stage was to obtain quantitative data. The most common strategy for quantifying the qualitative data in a single comprehensive dataset was followed; that of counting the number of times a qualitative code or theme occurred (Driscoll et al., 2007). The results of this process are presented in the following section.

Results

The total number of responses was 3863. Following data screening; 1118 were excluded; leaving 2745 valid ones; coming from 384 participants who played 239 times the 2D game and 189 times the 3D game. The excluded responses were either (a) too general (e.g.; “the game was not motivating;” “everything was ok”) or (b) irrelevant and unresponsive (e.g.; “I don’t play games;” “I don’t know”). All in all; eight themes were identified; and the number of codes in each ranged from 7 to 22.

Table 9.3 presents the results of the coding procedure regarding what might improve the games’ sense of immersion. Evidently; the games’ audiovisual features (N = 112) and realism (N = 79) were considered important for improving immersion. Quite interestingly; feedback (N = 61) and the quality of the learning material (N = 40) were also important factors in making a game more immersive. The games’ realism can enhance enjoyment (N = 148); as well as the audiovisual features (N = 98) and the quality of the learning material (N = 86) (Table 9.4).

Table 9.3 Immersion
Full size table
Table 9.4 Enjoyment
Full size table

According to the participants’ responses; learning effectiveness; besides being shaped by features related to the quality of the learning material (N = 113); can be influenced by feedback’s quality (N = 115) and; far less; by the clarity of the learning goals (N = 34) (Table 9.5). Audiovisual features (N = 215); together with features that enhance realism per se (N = 91); can improve the games’ realism. No other factor seems to have played an important role (Table 9.6).

Table 9.5 Learning effectiveness
Full size table
Table 9.6 Realism
Full size table

Features that improve narration (N = 98); the games’ feedback (N = 54); and the quality of the learning material (N = 53) were the prominent ones affecting the quality of narration/storyline (Table 9.7). The clarity of the learning goals was almost equally affected by feedback’s features (N = 82) and the quality of the learning material (N = 75); closely followed by learning goals’ features per se (N = 65) (Table 9.8).

Table 9.7 Narration/storyline
Full size table
Table 9.8 Learning goals
Full size table

Only features related to feedback itself can improve this factor (N = 116) (Table 9.9). On the other hand; feedback’s features greatly affected playability (N = 115); while the interface’s quality was far less important (N = 51) (Table 9.10).

Table 9.9 Feedback
Full size table
Table 9.10 Playability
Full size table

For improving the interface; the participants suggested changes in feedback’s features (N = 76) and playability (N = 60) (Table 9.11). Finally; the most influential factor regarding motivation to learn was the quality of the learning material (N = 196). Indeed; the participants indicated a multitude of features directly connected to this factor (Table 9.12). Realism was also a factor; but its impact seems to be far less important (N = 44).

Table 9.11 Interface
Full size table
Table 9.12 Motivation
Full size table

Table 9.13 and Figs. 9.1 and 9.2 summarize the number of responses in each factor. Given that the 2D game was played 239 times; while the 3D game was played 189 times; the results of the latter were multiplied by 1.265; in order for the responses on both games to be comparable. On the basis of the participants’ number of responses in each question; the following were observed (less than 30 responses in a factor were considered insignificant):

Table 9.13 Factors’ matrix for both games
Full size table
Fig. 9.1
figure 1

Factors’ interactions in the 2D game

Full size image
Fig. 9.2
figure 2

Factors’ interactions in the 3D game (Effects with less than 30 responses were omitted)

Full size image

  • Although it was not included as a question in the questionnaire; a new factor emerged; that of audiovisual features. Moreover; in both games; participants suggested that improvements in this factor will improve the games’ realism (N = 102/143); enjoyment (N = 47/65); and immersion (N = 68/56).

  • In both games; participants suggested that features related to realism can improve enjoyment (N = 102/58) as well as immersion (N = 47/40).

  • The quality of the learning material seems to be a very influential factor in both the 2D and the 3D game; it strongly affected motivation (N = 118/102); learning effectiveness (N = 77/46); and enjoyment (N = 51/44). A minor difference was noted between the two games concerning this factor; as in the 2D game it also affected the learning goals’ clarity (N = 58); while in the 3D game it affected the quality of narration/storyline (N = 34).

  • Feedback also proved to be an influential factor. That is because; in both games; features related to this factor can improve the interface’s quality (N = 40/46); playability (N = 72/54); and learning effectiveness (N = 40/95). Two differences between the two games were noted concerning this factor. In the 2D game; changes in feedback can influence narration’s quality (N = 40); while in the 3D game; they can affect immersion (N = 47) and the learning goals’ clarity (N = 77).

  • It seems that playability and the interface’s quality have interchangeable roles in the two games. In the 2D game; the latter affected the former (N = 38); while in the 3D game; the former affected the latter (N = 46).

  • Quite interestingly; enhancements in enjoyment; immersion; narration; motivation; and learning goals’ clarity will not have an impact on any other factor.

Discussion

For examining the users’ experience when playing SGs and for revealing how factors essential for determining this experience interact; the study’s participants played two SGs and recorded their views by answering a short questionnaire. It has to be noted that the games were fundamentally different. Even so; remarkable similarities between the two games were noted. Indeed; the data analysis established the dominant role of two factors in both games; that of leaning material’s and feedback’s quality. Besides; according to participants’ responses; improvements in these factors will greatly improve the SGs’ learning effectiveness (N = 123/135). In this respect; the findings of this study are in line with previous research which established the significant effect the learning content (e.g.; Mortara et al., 2014) and feedback (e.g.; Alonso-Fernández et al., 2018; Cheng et al., 2015; Ravyse et al., 2017) have on the learning outcomes.

Moreover; the quality of the learning material had an overwhelming impact on motivation in both games (N = 118/102). This finding is interesting as there are only a few references in the literature signifying such a connection. Then again; it is not irrational. If the learning material is boring or hard to understand; learners will lose their interest and will not be motivated to continue studying (or playing an SG). For example; Habgood and Ainsworth (2011) advised that the learning content and game mechanics have to be well integrated in order for the game to be more motivating. In the same line of thought; Gunter et al. (2008) added that if the learning content does not fit well in the game situation; the motivation for learning is not enhanced at all. What is also very interesting is that participants connected their views for the quality of the learning material with their sense of enjoyment (N = 51/44). The relevant literature suggested either that such connections do not exist or that the path has the opposite direction. For example; Connolly et al. (2012) suggested that the games’ fun and enjoyment increase the players’ interest for the subject matter and not the other way around. This finding suggests that if users consider the learning material not well presented and difficult; not only their motivation to learn will be negatively affected but also their sense of enjoyment.

As for feedback; the results indicated that in addition to learning effectiveness (N = 40/95); it also had an impact on playability (N = 72/54); interface’s quality (N = 40/46); goals’ clarity (only in the 3D game) (N = 77); narration’s quality (only in the 2D game) (N = 40); and one’s sense of immersion (only in the 2D game) (N = 47). Feedback’s role in SGs was mostly related to the learning outcomes (Sušnik et al., 2018). Few suggested that feedback might have an effect on other factors as well. The study’s findings imply that feedback might have a more important role than previously suggested. For example; Prensky (2007) stated that feedback and learning goals are closely related; players can monitor their progress to a goal through the game’s feedback (e.g.; through score changes and through changes in the game world per se).

According to participants’ responses; audiovisual features had an impressive effect on realism (N = 102/143); while both had a strong impact on enjoyment (N = 47/65 and N = 102/58; respectively) and on immersion (N = 68/56 and N = 40/47; respectively). These findings further support the findings of other studies. For instance; Hunicke et al. (2004) in their Mechanics; Dynamics; and Aesthetics Framework considered aesthetics as the component that encapsulated the games’ fun element. Huang et al. (2010) viewed advanced graphics (i.e.; realism) and audiovisual effects as features that can make a game more attractive. Ivory and Kalyanaraman (2007) found that high realism had a significant impact on presence; involvement; and arousal; while Nacke et al. (2010) noted that sound and music affected immersion.

Although the data analysis brought to light interesting factors’ interactions; more intriguing was the absence of some connections. This is probably the study’s most significant finding; yet the most puzzling one. To start with; realism and audiovisual features did not have an impact on the games’ learning effectiveness. Contrary to this; research has demonstrated that the level of realism had an impact on the learning outcomes (e.g.; Ravyse et al., 2017). On the basis of the study’s results; it can be supported that there is no positive correlation between fidelity levels and knowledge transfer (Vogel et al., 2006). This finding may also serve as an indicator that participants did not consider two of SGs most prominent gaming features as being important for their learning experience when playing them.

Immersion; enjoyment; and motivation did not emerge as themes from the data analysis. In addition; they seem to be at the receiving end of factors’ relationships. The same applied for playability and the interface’s adequacy as they affected only each other. Thus; there are many missing links suggested by the relevant literature. One such is the link between enjoyment and learning effectiveness (Connolly et al., 2012). This result suggests that what is learned when playing SGs is not attributable to the game’s enjoyment but to other more decisive factors; such as instruction; support; and explicit learning tasks (Iten & Petko, 2016). Another missing link is between motivation and learning effectiveness. Although research suggested that a strong link between these two factors exists (Westera, 2019); a meta-analysis has concluded that the motivational appeal of serious games is not that much more different than other instructional methods (Wouters et al., 2013). Moreover; others suggested that a delicate balance has to be achieved (e.g.; learning vs playing and freedom vs control) in order to develop really engaging SGs. In this respect; the study’s findings might have reflected a problematic integration of the above; which; in turn; resulted in the SGs lack of motivational appeal (Wouters et al., 2011).

A number of studies concluded that engagement and immersion; in addition to mediated effects; had a direct positive impact on learning (e.g.; Abrantes & Gouveia, 2012). Contrary to that; Hamari et al. (2016) found that although engagement in the game had a positive effect on learning; immersion did not. The findings of the present study are in support of the latter with some reservations; as immersion is an elusive and ill-defined factor. Finally; the results did not link narration with learning effectiveness. Then again; it is not that clear whether the narrative fosters learning given that some studies reported positive (Cordova & Lepper, 1996); contradictory; or even negative results (McQuiggan et al., 2008).

On the basis of the study’s findings and their subsequent discussion; it can be concluded that the games were viewed as a form of digital learning material rather than as educational/serious games; the participants knew that they were actually studying a digitally presented subject matter and not playing a game. In support of this argument are the observed as well as the missing factors’ interactions. It has to be reminded that the only factors linked to learning effectiveness were feedback and quality of the learning material. Both factors are related to the “serious” or “learning” aspects of SGs. Learning effectiveness was not found to be influenced by SGs’ “gaming” aspects (i.e.; immersion; playability; enjoyment; audiovisual features; and realism).

Implications for Research and Practice

Though research regarding SGs has been building up gradually over the past years; it has resulted in a fragmented and; up to a point; in inconsistent literature. Several factors contributed; SGs are cross-disciplinary in nature; key SGs’ features are defined differently and used in different contexts; and multi-methodological approaches are used for their assessment (de Freitas, 2018). The lack of common consensus on how to measure SGs’ effectiveness; as well as on how to measure the users’ views for these applications; suggests that we have to rethink the suitability of the assessment tools used in this kind of research and develop more robust ones. There are a number of steps that have to be followed in order to achieve this; the first one being to give voice to SGs users; after all; they are the ones at the receiving end of the line and the ultimate judges of their effectiveness; pros; and cons. The present study suggested that open-ended questions can be used for recording the users’ views. Although these questions were limited in number and many more could have been included; interesting results emerged. Given that; it is recommended that future studies can also utilize open-ended questions in order to gain a deeper understanding of participants’ views and attitudes toward SGs. The SGs’ industry can also benefit in a similar way. For example; developers can focus on certain features of interest and compare versions of the same SG and determine how the latest version compared to the previous one.

What is more; the study’s findings suggested that users were not “deluded” by the SGs gaming features; they were aware that they were using a piece of educational software and not a game. This finding confirms; almost word-for-word; Michael’s and Chen’s (2005) definition for SGs; that of being games not having entertainment; enjoyment; and fun as their primary objective. Yet; it has significant implications for researches and SGs’ developers alike; as it raises some straightforward questions such as: “What is the added value of SGs; if users already know that their purpose is to teach something?” and “Where is the balancing point between learning and gaming in an SG?” It goes beyond the scope of the present study to give answers to these questions; it is up to the developers to decide whether they want to add more gaming features or not and up to the researches to examine the impact of such decisions. What it can be suggested is that our views for SGs are far from being consolidated; much more research is needed in a domain characterized by blurred boundaries which also relies on very diverse perspectives and approaches.

Limitations and Future Research

Although the study’s results were thought-provoking; there are limitations that should be acknowledged but also provide several avenues for future research. The sample size; although more than adequate; could have been larger and more diverse; students from other areas of study could have been recruited. Therefore; reservations do arise regarding the generalizability of the results. The participants were asked to play the SGs for at least 2 hours. One might argue that this was a rather limited length of time and might raise concerns whether this was enough for players/participants to develop a comprehensive view for the SGs. Only two SGs were examined. On the other hand; SGs cover a wide range of genres and learning domains. It is possible that different factors’ interactions might have emerged if other SGs were used.

Future research will help to identify similarities (or differences) with the findings of this study. In addition; the target group can encompass students from other disciplines or even individuals of all ages; so as to examine if and how different age groups and individuals from different scientific backgrounds view SGs. Moreover; a larger variety of SGs can be examined in order to further refine the differences. Other research tools can also be utilized; observations and interviews will allow an in-depth understanding of how subjective and objective SGs’ features interact. Finally; the study’s findings can provide quite a lot of ideas for the development of a more comprehensive scale for assessing SGs. Indeed; this is a path worth exploring; as there is still the need for establishing evaluation criteria and tools for assessing the various dimensions of SGs.

Conclusion

Despite the above limitations; the study provided an idea of players’ views; feelings; and attitudes toward SGs; not indirectly through a scale (which is the norm) but directly; by asking for their thoughts and judgments. What is more; the study examined ten factors that was theorized to be important; while the bulk of the existing literature focused on a much smaller number of factors. Thus; the study’s contribution to the relevant literature is that it (a) utilized a method that is not commonly used; (b) examined a substantial number of factors that have an impact on one’s learning/gaming experience when using SGs; (c) quantified the results; which; in turn; revealed interesting factors’ interactions; and (d) indicated that users probably view SGs as form of digital learning material rather than as games. In conclusion; the study’s findings might prove useful to researchers in understanding the factors’ interactions responsible for shaping one’s learning experience when playing SGs.