Games are very compelling (ESA 2012) and students often spend hours engaged in a single game (Rideout 2011; Rideout et al. 2010; Gutnick et al. 2011). Humans learn by playing, and one of the best ways to learn is through failure in low-stress situations, which can be introduced in an academic environment (Toprac 2018). Games have been widely explored as a tool to enhance learning (Civic Enterprises 2006); however research on gamification has focused primarily on preschool and elementary-age children in particular, and university-level studies related to academic success in STEM areas are few. We agree with the suggestion of Nobel Laureate Carl Wieman (2014) that educational research could be employed to compare different active learning methods. In this chapter we describe a study to assess the effects of a computer-based gaming environment developed by the first author, called LABMATTER. The game was used as a supplement to enhance students’ knowledge base in first-semester general chemistry and was compared to another active learning control group and a group of students who experienced only traditional pedagogy without a High-Impact Practice (HIP) intervention.

Theoretical Prospectus

Research into game-based approaches for adults is far less common than games developed for younger players. Educational games and the concept of play are very difficult to define as they depend on several transient, contradictory, and subject-dependent qualities. Play is often described as the opposite of serious work. The boundaries are often bent, broken, or blurred during play, and, as such, it can be challenging to differentiate between work and play.

The definitions of what establishes game and play are broad. Play is perceived as being beneficial in the promotion and development of imagination, creativity, and spontaneous learning. These same characteristics are often necessary for a student’s growth in any educational setting. The utility of approaches that involve gaming or play in education may result from tactile activity with physical and emotional sensations. Creative skills are best refined using a range of approaches so that students can investigate issues from different perspectives. Several researchers suggest that the fundamental motivation for all games is to learn, and play is a safe way to learn, because games are effective supplements to traditional classroom pedagogies at all levels (Clark et al. 2010; Clark and Sheridan 2010). Educational methods based on games are often founded in a constructivist theory of learning. One requirement of constructivism is that students are challenged; specifically they are called to engage prior knowledge with the knowledge intended to be learned. New knowledge is constructed from interactions of people with materials, not simply transmitted (Williamson 2008). Challenges enable each student to devise his or her own knowledge and may also require alternative teaching approaches to engage and motivate the novice.

Recent efforts have been made in using games to encourage construction of scientific knowledge (Clark et al. 2010). Four fundamental constructs within the unified learning model (ULM) address the conditions required for meaningful learning; these include prior knowledge, engagement, working memory allocation, and motivation (Shell et al. 2010) (Fig. 27.1). Working memory, where connections are made, is governed by motivation and meaningful engagement and is the site of assimilation of new material. Accordingly, the storage of learned information is affected by prior knowledge and determines one’s ease, speed, and efficiency of information processing.

Fig. 27.1
A Venn diagram shows shared responsibility of prior knowledge with the subject matter, expert, novice, relevance, motivation, meaningful engagement, and competency.

Aspects of the unified learning model (ULM)

Full size image

In the context of digital games, specifically first-person role-playing games, scaffolding is a particularly useful component for supporting individuals of varying ability levels. In educational games, scaffolding typically takes the form of leveling and hints or clues. Computer games most often include a sequence of game levels that increase in difficulty as the player becomes increasingly more efficient at each level and advances in the game. Educational learning increases by elevating the cognitive demands during game progression, rather than simply the use of psychomotor skills (eye-hand coordination and manual dexterity). Accordingly, beginning levels provide more scaffolding for students’ learning, which is gradually alleviated as the player completes each game level. The player’s selection of a wrong answer also provides an opportunity for additional support or tutorials. When students answer incorrectly, the game can provide additional support in the form of hints or clues for the approaching problem. These hints can offer more help with each subsequent problem, until the student can solve problems beyond the novice level. Clues prompt an accelerated retrieval of sensory data to which humans already receive a tremendous amount of sensory input at any given moment. Sensory allotments simply collect pieces of data into an output that is transferred to the appropriate parts of the brain.

Development

Gamification using an escape room methodology and activities is in its infancy in formal educational settings at the university level. (In an “escape room” game, people are trapped inside a space for a specific amount of time and need to solve a number of puzzles or problems to get out.) A computer game’s “interface” is the system of hardware (e.g., joystick, mouse, touchscreen) and software applications (e.g., move, scroll, or drag) that are used to access and interact with the game. The functions associated with the interface can serve to either facilitate or impede the game. As the student completes each mission or level, the student acquires scientific skills (e.g., conversions, balancing equations, solving stoichiometry problems, etc.). In this regard, by using gamification techniques, students solve problems for themselves by carrying out “missions” that require them to apply reason, sensation, and then reflection.

LABMATTER (Jenkins 2017) was developed using Unity 3D which is an open-source game engine that comes with certain standard assets to help an individual engage in game development (www.unity3d.com). Programming involves using C# (i.e., C-Sharp) and JavaScript. The game engine enables the user to develop a game without having to program every single detail. The purchased standard assets used in this study can be found at the Unity Asset store that includes the environment, characters (https://www.mixamo.com/#/), models, and some of the game mechanics. An algorithm code (programmed by employing Lua) was used to constantly change the chemical compound and the “stem” numbers allowing the student to play the game and get new numbers each time. This feature enables the learner to get multiple practice attempts, therefore helping to reinforce the content learned in lecture.

The LABMATTER game was launched using HTML5 in combination with the developer-owned website using Blue-host (server provider) and WordPress (website development carrier). Hypertext Markup Language (HTML) was used in combination with plugins to operate the game. Modifying the browser cache was used to promote online play of the game. Talentlms (online lms = learning management system) (2018) enabled the user to login and to access the game by going to justmatter.talentlms.org. Mozilla Firefox (64-bit) and a high-speed Internet connection are required for desktop access.

Classroom Setting and Resources

The resources for each group are displayed in Fig. 27.2. MasteringChemistry® (Pearson 2018) is an online chemistry homework system available for students through the commercial publisher. (Student use of the homework system was not evaluated in this study.) The textbook chosen for this research pursued an “atoms-first” curriculum, and the lecture-based instruction was conducted at a large, southwestern R1 university. The assessments (exams 1, 2, 3, and 4) were given about every 4 weeks until the 12th week of the semester covering Chapters 1 through 10 in Chemistry: Structure and Properties (Tro 2017). OpenStax (2018) (Rice University, peer reviewed) is an open-book source that provides students with instant access, and students also had access to the structure-drawing software, MolView (Bergwerf 2018), that helps with drawing Lewis structures and molecular geometry. Figure 27.3 lists the chapters covered in each assessment period with the corresponding game level and chemistry topics addressed.

Fig. 27.2
A table shows intervention, homework, textbook, resource, open source, and assessments with group 1 experimental, group 2 traditional, and group 3 control.

Classroom resources

Full size image
Fig. 27.3
A table shows chapters from 1 to 10 categories with respect to assessment game level from 1 to 4 with their topics of chemistry named atomic structure and reactions.

Exam content and game level

Full size image

Methodology

The target population consisted of students enrolled (N = 133) in the spring semester of first-semester general chemistry (Chem I). Verbal instructions were given to the students in the classroom setting on how to play the game as well as how to access other needed resources. The student (or player) created an account at the Talentlms website where they were given a list of random numbers that decided which students would play the game and which would be given access to the online videos (time-on-task equivalent control). The LABMATTER game and online videos were provided to the students in the first week of class where individuals subsequently signed into their specified group (Fig. 27.2): group 1 represented the game-level students, while group 3 was the corresponding online video activity group. Non-compliant students by default became the unintended group for comparison purposes subsequently referred to as the traditional group or group 2.

All students were given an open-ended, calculator-free pretest at the beginning of the semester to assess their “numerical automaticity” called the MUST (Math-Up Skills Test) (Albaladejo et al. 2018) that was scored based on the number of questions correct out of a maximum of 20 possible questions. This 15-min quiz has been shown to be highly reliable and strongly correlated to Chem I outcomes. The MUST was used in a statewide study that included the present university.

Multiple-choice exam data were collected from Scantrons®. The criteria for average success are the following: MUST scores of 6.95/20 or 34.75% and above and course average of 70.0% and above. The assessments required students to synthesize, analyze, and make judgments using higher-order thinking skills with measurable levels. The time spent on the game and an online activity was recorded using Talentlms® software.

Demographic information was collected to determine the baseline characteristics of the class using Talentlms and to collect students’ attitudes toward learning by using an e-system instrument called the System Usability Scale (SUS) (Brooke 1996). The SUS was administered after the first exam and at the end of the semester following the fourth exam to confirm the students’ attitudes toward using active learning methods. All students in the class (N = 133) had access to the same course material, but 10 failed to complete the demographic information, and therefore a smaller population (n = 123) was used to describe the characteristics of the students available as reported in Table 27.1. Data from all 133 students were used to evaluate the active learning interventions. Table 27.1 displays data pertaining to the students’ gender, MUST scores, classification, ethnicity, employment, and status as first-generation students.

Table 27.1 Demographics (n = 123)
Full size table

Demographic data indicate that males (n = 51 or 41.5%) outperformed females on the MUST quiz given to all students at the start of the semester suggesting that males enter Chem I with greater arithmetic automaticity than females. The majority (69.1%) were classified as lower-level (freshmen + sophomores) students, and more students self-reported as white (W), non-Hispanic than the listed minorities (i.e., Hispanic (H), black (B), and other (O)). The majority of the students were not employed (61.8%), but for those who were employed, more worked off-campus than on (38/46 = 82.6%). Over 25% (31/123) reported that no parent or guardian had obtained a degree, thereby establishing their status as first-generation students.

Several analyses of demographic variables and measures of prior knowledge support each of the following conclusions: (a) lower-division males entered the course with stronger automaticity skills than corresponding females; (b) approximately the same proportion of students in each treatment group benefited from a high school chemistry course based on AP/IB, Pre-AP, and regular curricula; and (c) those enrolled in groups 1 (game) and 3 (online videos) had significantly stronger mathematical skills than those enrolled in group 2 (traditional).

Research Questions

The semester exams were used to determine the effect and significance of the game, online activity, and the traditional treatment. The dependent variables were the four midterm exam scores, while the student group (i.e., active learning engagement vs traditional) was the independent variable.

The research questions addressed are:

  • Q1: What differences exist in the course averages of students who used LABMATTER versus those who used the online videos and those who only experienced a traditional curriculum approach without a noted High-Impact Practice (HIP)?

  • Q2: What differences exist in students’ performance on each of the four semester exams between treatment groups?

  • Q3: How do students perceive the efficacy of LABMATTER as a tool for enhancing problem-solving in general chemistry?

Statistical Analyses

The IBM®Statistical Packages for Social Sciences®Statistics Version 24 (SPSS®) software was used to carry out Pearson correlation, one-way analysis of variance (ANOVA), and post hoc (Tukey) analysis. The Pearson correlation is able to identify grouping of variables that have a statistical difference. A Pearson correlation with a two-tailed test was performed at alpha level of 0.05 to compare the exam scores of the experimental group (game), the traditional group, and the control group (online activity).

For evaluation of students’ perceptions (i.e., effectiveness, efficiency, and satisfaction), the System Usability Scale (SUS) produced a single number to represent the overall measure and usability of the system studied. A SUS score of 90 is interpreted as excellent, 80 is good, and anything below 70 is unacceptable (Brooke 1996).

Results

Research Question #1

  • Q1: What differences exist in the course averages of students who used LABMATTER versus those who used the online videos and those who only experienced a traditional curriculum approach without a noted High-Impact Practice (HIP)?

Tables 27.2 and 27.3 summarize the mean total scores of midterm examinations by treatment group and disaggregated by gender. Students enrolled in the game treatment (group 1) and the video treatment (group 3) outperformed those in the traditional treatment (group 2). However, no differences were found between those in the game and video treatments.

Table 27.2 Average MUST scores for each group
Full size table
Table 27.3 Performance by gender (n = 133)
Full size table

Disaggregation of the exam scores by gender revealed that both males and females in the game and video treatments outperformed males and females in the traditional treatment, but no differences were found between those enrolled in the game and video treatments.

Research Question #2

  • Q2: What differences exist in students’ performance on each of the four semester exams between treatment groups?

Exam 1 covered Chapters 1–4 in Atoms First. As demonstrated in Table 27.4, there is a clear distinction between the groups. Comparisons between groups 1 and 2 and between groups 2 and 3 reveal a statistically significant difference (p < 0.05). Groups 1 and 3 display no statistically significant difference (p = 0.841). Post hoc analyses were conducted on exams 1, 2, 3, and 4, but only those from exams 1 and 2 are reported below.

Table 27.4 Exam 1 comparison between groups
Full size table

A Pearson correlation of 0.179 indicated a relatively weak relationship between tests and time in the game and video treatments using two-tailed analysis. However, there was a statistically significant difference between each group investigating test versus time with an alpha level of 0.040 indicating that the more time spent engaged, the higher the test average.

On the second exam, covering Chapters 4–5 in Atoms First, there was also a statistically significant difference (p < 0.05) between student groups. A post hoc (Tukey) analysis was conducted on the study’s three groups yielding a statistically significant difference of 0.003 (Table 27.5). Groups 1 and 3 demonstrated no statistically significant difference resulting in a of value of 0.997. However, as with the first midterm exam, comparisons of groups 1 and 2 and groups 2 and 3 revealed statistically significant differences.

Table 27.5 Exam 2 comparison between groups
Full size table

Pearson correlation was conducted using a test versus time analysis. A quantitative value of 0.288 was found between test and time (in the game and video treatments). This result implies that there is a highly positive correlation between study time and achievement scores. The significance of the relationship between study time and achievement within groups 1 and 3 are promising indicators that positively affect students.

The third exam covered Chapters 6–8 in Atoms First, and the fourth exam covered the remaining two chapters, 9–10. Like the first two exam units, there was a statistically significant difference (p < 0.05) between the active learning groups (1 and 3) and group 2. However, the Pearson correlations for exams 3 and 4 produced no statistically significant relationships (p < 0.05) and very low positive correlations (0.093 and 0.025) between the exams and the time engaged.

Research Question #3

  • Q3: How do students perceive the efficacy of LABMATTER as a tool for enhancing problem-solving in general chemistry?

Research question 3 analyzed student’s attitude toward LABMATTER using the SUS. There are ten items on the SUS to which the students responded using a Likert positive/negative scale to produce a score revealing how students felt about using the game. The students perceived the game as a useful medium for learning chemistry with an average SUS score of 70.3 out of a maximum score of 100. This score was deemed acceptable according to published suggestions (Brooke 1996). As the game becomes more sophisticated and programming glitches improve, so will students’ attitudes about playing this game.

Course Outcomes

As a general addendum to this study, Table 27.6 reports grades for successful students (grades of A, B, or C) and those deemed unsuccessful (grades of D, F, or W). The overall grades for group 1 compared favorably to students in group 3 (63.9% vs 57.7% successful) and both groups outperformed students in group 2 (33.3% successful).

Table 27.6 Course grades by group
Full size table

Summary

The results of this study suggest that LABMATTER offers students an opportunity to practice problem-solving skills relevant to introductory college chemistry in a more inviting medium, thereby, if only slightly, increasing students’ chances of success over online academic videos in widespread use and published by commercial sources. Interactive games for students can provide stimulating, rewarding, and even exciting venues that significantly improve retention of scientific material, maximize engagement, and improve learning. Wieman (2014) observed that “active learning methods achieve better education outcomes” (p. 8319). When students are engaged and motivated, classroom performance improves, and active learning is one of the best ways to accomplish meaningful engagement.