Keywords

Student Engagement in Postgraduate Education: Using Game Theory to Improve Results

I never teach my pupils. I only attempt to provide the conditions in which they can learn. – attributed to Albert Einstein

Student engagement is an important aspect of both the quality of the program that is delivered to students, as well as student retention within the program (Angelopulo 2013; Dill and Beerkens 2012; Kurantowicz and Nizinska 2013; Nelson et al. 2011). All educational institutions, of whatever level, are concerned with student engagement. This concern has led to the development of a range of measures and rewards to encourage teaching staff and those responsible for providing the learning environment to improve student engagement. These incentives have been accompanied by a formidable set of rules and procedures; however, progress has not always been satisfactory. An important reason for this lack of success has been the “gaming” that has taken place in the classroom, both by students as well as by teachers. There is therefore a need to reexamine the problem of student engagement from the perspective of the theory of games.

Traditionally, engagement in higher education has been perceived as more of a responsibility of the student rather than the teacher (McMahon and Portelli 2004). While students in primary education faced disciplinary action for not performing to the expected standard of the class, students in higher education were largely left to their own resources with regard to engagement, with the lecturer providing the required materials to master the subject and the student expected to take responsibility for learning (Zyngier 2008). This set of beliefs is well illustrated by the quotation at the beginning of this chapter.

Literature Review of Student Engagement in Higher Education: What We Already Know

Kuh et al. (2007) summarize the usual meaning of the term “student engagement” when they state that:

Student engagement represents two critical features. The first is the amount of time and effort students put into their studies and other educationally purposeful activities. The second component of student engagement is how the institution deploys its resources and organizes the curriculum, other learning opportunities, and support services to induce students to participate in activities that lead to the experiences and desired outcomes such as persistence, satisfaction, learning, and graduation. (Kuh et al. 2007, p. 44)

Starting in the 1950s, student engagement came to be seen as a joint responsibility of the individual academic, the University, and the student. The reasons for the lack of engagement began to be investigated, and methods to remediate any problems began to be suggested. At the same time, an era of accountability for universities began to emerge – with some referring to this as an age of “centralization, bureaucratization, and commodification” of universities (Tuchman 2009). Regardless of whether this change was seen as desirable or not, student engagement began to be monitored, measured, and eventually managed – a trend that continues to the present day.

While student engagement is a rather broad term with a range of possible interpretations, according to the different perspectives from which it is viewed, this chapter focuses on the interaction between student evaluation of teaching and pedagogical strategies aimed at engaging students that are particularly relevant for MBA and similar masters-level degree programs offered in off-campus and/or blended learning modes. It is important to explore this interaction because firstly, it helps to explain how pedagogical strategies tend to “adapt” to meet different student expectations under different learning modes and, secondly, it offers an insight into the vexed issue of the gap that emerges between the quality of learning achieved and the student evaluation of teaching. Many of the problems associated with student evaluation of teaching have already been addressed at length in the extant literature (Leckey and Neill 2001; McGettrick 2005; Schuck et al. 2008; Valsan and Sproule 2008). We, for the first time, explore the interactions between student evaluation of teaching and pedagogical strategies through the methodological lens of “game theory.” Game theory is an apt conceptual tool for this purpose as it offers a formally established model of interaction between utility-maximizing agents that has been applied successfully in a wide variety of disciplines ranging from international diplomacy to evolutionary biology. For the benefit of the general reader, we present a brief introduction to the basics of game theory in a subsequent section of this chapter, prior to applying this tool as a methodological lens in the specific context of our problem.

Measuring and Assessing Student Engagement and the Use of Student Evaluation of Teaching (SET) Measures

Student engagement is inherently difficult to assess. An earlier summary of this literature can be found in Quaye and Harper (1970). The most popular indicators of engagement have been based on directly asking students about their level of engagement, often in the form of a questionnaire. This has been done both at the level of the individual institution as well as at the national level.

Measuring Student Engagement at the National Level

The use of public funds is normally subject to scrutiny; this applies equally to the funding of higher education where there are continuing attempts to measure student engagement. There are usually two major objectives of such programs: accessing the changes in engagement at a national level and comparing the level of engagement between various teaching bodies so that successful program can be propagated. The National Survey of Student Engagement (NSSE) provides indicators of student engagement across universities in the USA and Canada. The survey covers colleges and universities that span all eight of the Carnegie classifications (www.camegieclassifications.iu.edu). The survey is based on a consideration of four major themes that relate to the environment that students experience in higher education. These themes are academic challenge, learning with peers, experiences with faculty, and campus environment. The associated indicators have been tested over a multiyear period and are considered to provide good-quality information about each aspect of engagement. Table 1 describes these themes and their respective indicators.

Table 1 NSSE themes and indicators for student engagement
Full size table

Similar work has also been undertaken in the UK, where the Higher Education Funding Council for England (hefce) publishes the National Student Survey on an annual basis (http://www.hefce.ac.uk/lt/nss/). The hefce reports increasing level of satisfaction among students in 2015, especially with regard to assessment and feedback. Many other countries produce similar reports. In Australia the Department of Education and Training has recently re-badged these reporting activities as Quality Indicators for Learning and Teaching (QILT) (http://www.qilt.edu.au) which includes the “Student Experience Survey.”

Overall these measurements display not only the improvement in student engagement over time, but more importantly, the desire to continue to improve this important aspect of education.

Measuring Student Engagement at the Institutional Level

In formulating a new approach to remediate student engagement, institutions of higher education asked their students directly about their attitudes toward the various teachers, the course material, the manner in which the material was presented to them, and so forth in order to capture data on student evaluation of teaching (hereafter SET). While there is some controversy in the use of SET scores (Felton et al. 2004), studies have shown a strong positive link between student performances and SET scores. For example, Stehle et al. (2012) evaluated SET with three measures: (i) overall instructor quality, (ii) overall course quality, and (iii) student’s subjective learning. They found that the mean value of examination scores (as a measure of students’ performances) bears a significantly high positive correlation with the mean SET score. These results seem to suggest that a higher mean SET score implies better teaching and engagement, which is subsequently reflected in the students’ academic performance.

Institutions of higher education almost universally use SET data in the performance evaluation of teachers in terms of tenure, promotion, and salary rise decisions (Chen and Hoshower 2003). Attempts to improve student engagement by using SET have met with varying degrees of success (Culver 2010; Ituma 2011; Oliver et al. 2008). However, because of the implications for career prospects, SET can make individual teachers behave strategically while choosing the level of intensity or difficulty at which each class is taught. In other words, teachers, as rational, individual utility-maximizing agents, could, as a matter of personal career strategy, “modulate” the type of classroom practice so as to maximize the chance of high teaching evaluation scores, which may or may not improve the quality of learning.

Thus one of the important problems that arise in this context is that the quality of teaching and the SET can both potentially be the subject of gaming, driven by a perception of conflicting interests, which in turn can lead to outcomes that are unintended at best and perverse at worst (Haskell 1998; Johnson 2010; Schuck et al. 2008; Zabaleta 2007). It is important to note that we are talking about two mutually exclusive entities (the lecturer and the student), both of whom are expected to operate within some preestablished rules of conduct to achieve the overt objective of effective knowledge dissemination. However, there may very well be underlying latent objectives that are unique to each entity, which can ultimately decide the outcome of their interaction. Game theory offers a formal model to study the different ways in which two or more utility-maximizing agents can interact in a way that affect not only themselves but all other parties in the game. Therefore, it is an apt choice of methodological lens in the context of our problem. A brief introduction to the fundamentals of game theory is presented in the next section.

A Gentle (i.e., Non-mathematical) Introduction to Game Theory

Game theory is concerned with the incentives, possible actions (and reactions) of participants, and the information that is available to each party. The theory provides a formal method that is useful in examining interactions that occur where cooperation or conflict exists. The use of game theory in the education literature has largely been confined to education policy (Law and Pan 2009; Niklasson 1996). This chapter extends that work into the classroom. While game theory can be viewed as a branch of applied mathematics, the usefulness of game theory in the academic/student dyad requires no formal knowledge of the mathematics of game theory (nor are any formal proofs presented, or required, to follow the arguments presented). Rather the discussion focuses on how educational policy on engagement can be formulated to meet the challenges faced in the classroom and to ensure that both students and academics are able to benefit from such policies.

Student Engagement from the Perspective of Game Theory

In a “game” each player has a choice of strategies (possible courses of action). The players choose the strategy that will result in the greatest payoff for themselves, given the information available to the player at the time the choice is made. To understand the actions of each player, it is necessary to consider the strategies, payoffs, and information available to each player. Once the game has been analyzed, it may be possible to change the outcome of a particular game by influencing these variables. Applying game theory in the context of higher education yields many promising results. This section of the chapter introduces the major elements that are required to use game theory and illustrates each element by placing the game in a classroom setting. The following sections then develop the results of the games that are specifically concerned with student engagement and that are typically played out in the classroom.

The elements required for a game to exist are as follows.

The Players

In the simplest of games, there are only two players (e.g., the most well-known of all games consists of two people) were asked to decide who will confess to a crime in order to receive a reduced sentence, usually called the “prisoners’ dilemma” (Poundstone 1970). In the classroom the simplest games would comprise the teacher and a representative student, which is the same as stating that all students are exactly similar in terms of the game. In more complicated games, it is possible to include multiple groups of students. The decision to expand the game to include subgroups of students would be based on how, and why, each subgroup of students reacts in a different manner to the actions of the teacher, and how the outcomes of one subgroup affect other subgroups in the class. The additional complexity of having more than one group of students might be easily outweighed if the groups are considerable different in character. The manner in which outcomes are different in three different groups of students (on-campus, off-campus, and blended) is described later in this chapter.

Strategies Available to the Players

If only one course of action (strategy) is available to a player, then that player’s actions are completely determined, and they can no longer be conceived of as a “player” in the game. Thus if there are only two players (teacher, student) and one player has only one possible course of action, the game is reduced to a simple deterministic outcome. This is the basis under which much of the analysis of student engagement has thus far been conducted. The possible strategies available to both teachers and students are thus worthy of study if game theory is to be usefully employed. Let us consider these in turn.

University teachers in higher education may adopt a set of strategies, many of which will affect the relationship with students and enhance (or degrade) student engagement. Consider the following strategies that a teacher might adopt:

  • Setting the standard of work to be mastered in order to pass the course

  • Constructing the syllabus to serve a particular objective (research, application, etc.)

  • Requiring independent learning by the students

  • Minimizing contact with students in order to achieve other personal objectives (research publications, etc.)

  • Maximizing SET scores and promotion opportunities

Many of these strategies might seem contradictory in that they may have different rewards for the academic (see the following section) and may have different effects on students. In particular, the strategy chosen by the academic may have varying effects on the level of student engagement. At this point in the analysis, it is not clear which combination of strategies the teacher will choose. Both the payoffs from the expected outcome and the reaction of students to the chosen strategy need to be considered; this is the exact purpose for which game theory was developed.

Students too have numerous strategies (choices) available to them, some of which are:

  • The amount of time and/or effort that they will devote to the course of study

  • The schedule of when effort will be expended (e.g., throughout the course, mostly toward the time of the examination, etc.)

  • The volume of outside reading or independent work to be done (or, in contrast, the amount of “spoon-feeding” expected from the teacher)

  • To focus only on those parts of the course that result in grading outcomes

  • Cutting corners/cheating

  • Rewarding or punishing lecturers by means of artificial SET scores

The Costs and Benefits Available to Each Player (the Payoffs)

The incentive for each player may well be idiosyncratic. Hence it is not possible, or desirable, to assign numerical values to these. Since the objective of the analysis is often to change behavior in some manner, usually all that is required is to have knowledge of the marginal effects – either from observation or the theory of behavior. Such information is almost always available.

Knowledge Available to Each Player

Finally, it is important to understand what knowledge of the game each player possesses when forming his or her strategy. In the games that take place in the classroom, it may not be unrealistic to assume that all players have knowledge of the game itself and the possible strategies that the other parties can choose – this situation is generally referred to as “common knowledge” in the game theoretic literature. In what follows we will therefore consider that all players are aware of their own possible strategies, the possible strategies of the other players, and the payoff of each strategy. It is possible to consider more complex situations, but they add little additional insight in the present context.

Redesigning the Classroom for Better Student Engagement

The appeal of game theory as a form of analysis is that it allows games to be redesigned to achieve better social returns. Many situations do not require detailed knowledge of the mathematical formulations of games in order to understand how to construct policy to change the existing outcome; these situations may be thought of as simple games, where the choice of action by one party is invariant to the actions of the other party (technically these are known as “dominant strategies” in the literature). Where a dominant strategy exists, changing the outcome may be as simple as changing the costs or benefits to one or more parties. Even in these situations, game theory is useful because it can validate our formulation of the problem and can be adapted if the structure of the game changes in the future.

In the following section, three different situations that affect student engagement and where there is not a dominant strategy are analyzed. The lack of a dominant strategy means that each player’s choice of action affects the other player(s), and a choice must be made without knowing what the other player(s) will do. Games of this type are called “normal form games,” but the analysis of the outcome of such games is not necessarily convoluted (although at times it may be necessary to seek expert advice!). In particular, this chapter focuses on the problem of student engagement in three different scenarios: the type of class instruction (on-campus, off-campus, and blended learning), ethical problems in the use of SET, and the assessment of students’ progress.

On-Campus Classes, Off-Campus Classes, and Blended Learning

Many lecturers in higher education who have taught the same class over a number of years have experienced unexpected differences between cohorts of students. This issue becomes particularly prominent when a cohort is noticed to have a significant percentage of students who view themselves as “misfits” and may even be regretting their decision to enroll in a postgraduate program of study and are therefore typically at risk of underperformance and lack of an engaging experience (Morrison et al. 2003). Noticed differences in cohort composition may necessitate changes in teaching practices as a particular class progresses over the semester. When students are divided into separate on-campus and off-campus groups, such differentiation can lead to the differences in cohorts becoming even more pronounced. The relevant literature is replete with empirical analysis of the comparative effectiveness of these alternative modes of teaching in relation to students’ performances (Bourne 1998; Graham 2006; Morrison et al. 2003). However, it is not yet clear when (or why) each of these modes of teaching may perform better than the other. Confounding matters even more, some studies have found no significant difference to exist between the modes of delivery in relation to students’ performances (Tiantong and Arreeraad 2013).

It is possible to use a game theoretic approach to analyze the effect of alternative modes of teaching (on-campus, off-campus, and blended modes) on student engagement and ultimately on student evaluation of teaching – however, an additional element has to be introduced into the game setting. In addition to the teacher and students (players), the various strategies, and the payoffs for each player, information asymmetry may exist. Information asymmetry implies that there is less than perfect communication between the teacher and the students as to the possible strategies available and payoffs of the other player (thus removing the assumption of common knowledge introduced earlier). Furthermore, given the relative costs of communication, the information asymmetry is assumed to be greater in the off-campus mode than for blended learning and least in the on-campus class.

As an example, consider that there is an asymmetry of information about the level of comprehension that the lecturer expects to students to attain in order to successfully complete the unit of study. The lecturer will not be immediately aware of students’ willingness to devote effort to learning but will make adjustments over time (Peacock 2001). Such adjustments will depend on the teacher’s possible strategies and payoffs and may include, for example, incentivizing students (changing the payoffs) to encourage students to work harder. As a consequence, in the presence of information asymmetry, over time an on-campus class will lead to a better match between teaching intensity and the effort that students are prepared to devote to learning. The off-campus class would then adjust at a slower rate, if at all, particularly so if it is composed of mostly mature-age students as is typically the case in postgraduate programs that are offered via distance and/or blended modes.

As desirable as an ultimate convergence of expectations of widely varying student cohorts and teaching intensity might be from a social perspective, there is nothing in the setting of this game to ensure that this is what will invariably occur. The teacher might in fact be more motivated to lower standards (i.e., teaching intensity) than to increase the payoff for students to work harder – particularly so for mature-age, postgraduate students who have other more pressing priorities apart from studies. Alternatively, the lecturer might not receive the signal that requested less work from the off-campus class and paradoxically deliver the course at a higher standard than he or she might otherwise have done! In fact, it is possible to show the somewhat counterintuitive result that by “matching” the learning effort demanded by students, the lecturer will (under specific circumstances) produce less than optimal results but that this outcome is by no means always the case.

The value of this example is to demonstrate that simplistic rules to regulate teaching of postgraduate classes in particular with different student characteristics (and hence expectations) are often impossible to formulate and an effort to do so may cause serious harm to both students and teachers. Simply insisting on an increase in effort on the part of students to match preconceived standards is not always optimal. By considering the problem in a game theoretic setting and by forcing an analysis of ALL the factors at play, administrators have a greater chance of delivering a practical outcome that is the most optimal for all involved and will stay optimal under most scenarios.

Grade Inflation, Ethics, and Student Engagement

Although the SET is a valuable source of information that is useful in the management of an academic department, it is subject to gaming by both students and academics alike (Valsan and Sproule 2008). The resulting ethical issues have been difficult for the academy and its members to address, and there is little published research in this area. However, the studies that have been published describe an attempt to quantify the problem. Reis and Klotz (2011) point to the seriousness of the problem: However, anecdotal evidence supports the notion for a potential loss of academic integrity amongst academic teaching staff especially when soft marking and grade inflation are employed as techniques to improve SET as part of performance reviews. Reis and Klotz examine the dilemma from the perspective of nursing education, by using an ethical principles framework. This approach to ethics is commonly used in health-care education and involves examining a dilemma in terms of the effect each proposed solution has on a set of ethical principles. As an example of such a framework (Beauchamp and Childress 2001) would suggest the following aspects be examined:

  • Respect for autonomy

  • Beneficence

  • Non-maleficence

  • Justice

In considering that SET may contribute to grade inflation, Reis and Klotz (2011) suggest that a conscientious educator of professionals faces “an unpalatable choice” between:

  1. 1.

    Refusing to compromise teaching standards by indulging in grade inflation or other tactics, and accepting the consequence of receiving relatively low SET scores that can impact negatively upon their career

  2. 2.

    Maximizing their teaching evaluations by compromising their professional standards to the extent of watering down course content, lowering academic course requirements, or implementing grade inflation, thereby compromising the long-term professional development of students and helping to qualify nurses who possibly are not fit to practice

These are the strategies that the educator may choose between. In terms of an ethical principles framework, the dilemma is resolved by examining each action in terms of the relevant set of ethical considerations. Reis and Klotz suggest using the items discussed by Beauchamp and Childress (as discussed above) although other frameworks might be more appropriate.

In terms of game theory, by setting up the game so that undesirable outcomes are no longer attractive to the player, such outcomes may be more easily avoided. The trick is to find the correct way to set up the game. The result, which is not unexpected, is to use the SET as a voluntary instrument for teachers to increase the quality of their teaching rather than as an instrument to judge their performance in the classroom, particularly in the context of postgraduate programs that are offered via multiple modes of delivery. The fact that the same conclusion is derived whichever approach of analysis is used, demonstrates the overall robustness of this conclusion.

There appears to be no literature that discusses the use of SET by students in terms of game theory. However Clayson et al. (2006) are able to demonstrate the strong relationship between students’ expectations of high grades with higher SETs, stating that “After 10 weeks of exposure to an instructor, students continued to change their evaluations systematically with changes in their expected grades,” thus demonstrating a significant implied threat by students, i.e., a strong signal to teachers. Controversially Clayson et al. conclude that “The results of our study strongly suggest that instructors can ‘buy’ evaluations with grades” (p. 64) which leads to a consideration of the strategy that will be chosen by teachers.

Using the lens of game theory, all of the following are solutions that might be suitable for administrators. Each solution changes the outcome, usually be changing the payoffs:

  • Replacing SET with third-party measures of student learning (Armstrong 1998)

  • Less reliance on SET for promotion and other professional advancement

  • Some form of statistical norming of the SET results that results in a reasonable estimate of the effect on students (Greenwald and Gillmore 1997)

  • Frequent measurement of SET in a given class to remove any timing biases and to reduce manipulation

Assessment and Student Engagement

Assessment continues to be both a requirement in higher education (for without it certification carries little meaning) as well as one of the major sources of discontent among students (Clouder 2012). The National Student Survey (NSS) in the UK shows general dissatisfaction with assessment and feedback by students. Nevertheless, there are ways in which the discontent of students may be ameliorated. Applying the principles of game theory described above and viewing the game from the students’ perspective, it is possible to change the game in the following ways.

Payoffs can be changed by reducing risk to the student, for example, by having multiple assessments over the course of the semester, with timeous and useful feedback on improving performance throughout the course. An additional measure might be to circulate past assessments (or constructing pro forma assessments where courses are not yet well established). For example, a renowned finance academic posts the following message on his blog site for students:

Past quizzes and exams: I have included just about every quiz I have ever given in my corporate finance classes below. The solutions are also available. Enjoy!!!! (Source: http://pages.stern.nyu.edu/~adamodar/, papers from 1986 to 2014 are available)

Player strategies can be enhanced by allowing students some form of control over the timing of feedback on teaching and the release of final results. An example that one of the authors experienced as a student was being allowed to not take a final examination provided a sufficiently high grade-point average was achieved over the semester. Being aware of his performance throughout the semester (and thus being confident that a final examination will not impact his overall result given that the cut-off grade-point was achieved) offered a “degree of dissociation” between performance and evaluation of teaching that may not have been achieved if he had to compulsorily sit the final examination.

Conclusion

Game theory is no longer an esoteric field of research and has proven to be useful in many practical applications. The classroom often presents an environment in which the players are able to make strategic decisions, and in order to understand and manage the outcomes of such a process game theory is invaluable. Fortunately, it is not necessary to formulate many of these situations in purely mathematical terms for useful conclusion to be drawn and for policy to be formulated. This chapter illustrates how greater student engagement, especially in MBA and other similar postgraduate study programs that are typically offered in flexible and blended learning modes, may be achieved by using game theory. Other classroom conundrums that are particularly relevant for postgraduate education (e.g., peer assessment of group assignments) can also be ameliorated by in-depth analytical exploration via the methodological lens of game theory, and based on the findings from such explorations, the management of higher education may be further enhanced, but we leave this issue for future research.