Keywords

This chapter examines the importance of decisions about how we measure contract cheating frequency for attempts to reduce the opportunity for this behaviour. After outlining the range of approaches that have been taken to measure this academic integrity issue so far, we provide a summary of the various imperfect ways that criminologists have been measuring crime (a related type of deviant behaviour). We discuss the relevance of the lessons learned from criminology and emphasise the importance of triangulating multiple approaches to measuring contract cheating moving forward, to assist the development and evaluation of detection and prevention strategies.

The Variety of Measurement Approaches Influence Contract Cheating Estimates

The definition that is used across contract cheating research has a direct impact on the research findings. For example, one major variable issue is whether payment is a necessary component of a contract cheating transaction: ‘commercial’ contract cheating (e.g., Newton, 2018, estimated a historic average of 3.5% of students) versus broader definitions involving sharing and help (e.g., Bretag et al., 2019, estimated 15.3% of students bought/traded/sold notes and 27.2% shared completed assignments). Estimates are also affected by the scope of behaviours that are included in the ‘contract cheating’ category label. For example, Bretag et al. (2019) examined a spectrum of seven outsourcing behaviours, ranging from buying/trading notes and sharing assignments to paying a third party to take an exam, with the latter having an estimated prevalence of 0.2%. While there is no ‘right’ approach to resolving these issues, and there are good reasons for looking at this problem from varying perspectives, all definitional decisions have a direct influence on any attempts to quantify contract cheating prevalence and incidence.

On a less overt level, once a definition has been settled on, the way that contract cheating is measured also has a substantial impact on the prevalence and incidence estimates that are produced. Moving beyond definitions, the remainder of the first part of this chapter will examine the various estimates that have been produced across the main methodological categories used to date. In broad terms, we examine the role of survey estimates, demand for contract cheating services, what we know about people getting caught for this form of academic misconduct, and other approaches to gaining insight into how frequent this issue is.

What Do ‘Offenders’ Say? Self-Report Surveys

To date, the literature on contract cheating has relied heavily on self-report methods involving surveys of ‘offenders’, which also typically produce the highest prevalence estimates of this behaviour (Curtis et al., 2021). Curtis and Clare (2017) aggregated data on ‘commercial’ contract cheating from five studies and found a prevalence rate of 3.5% of students engage in contract cheating, with 62.5% doing so on more than one occasion (incidence estimates). They note, however, that this estimate was skewed by the data from one of the studies (see Zafarghandi et al., 2012) which had a rate of engagement of 7.9%. By removing this data, prevalence dropped to 2.1%. Producing comparable estimates, Newton (2018) completed a systematic review of the commercial contract cheating literature from 1978 to 2016 and found a prevalence of 3.5%. However, Newton (2018) also argued that engagement in contract cheating is on the rise, with prevalence rates of over 20% in research almost entirely from 2009 onwards. Highlighting the importance of measurement methodology, it is unclear whether higher rates of reported engagement in contract cheating are a true reflection of an increase in contract cheating behaviour or whether they reflect variations in the methodology of the research being done (e.g., shifts in attitudes towards self-reporting, more encompassing definitions, and better and more varied methodologies of research).

To demonstrate the importance of these methodological factors, we highlight the Bretag et al. (2019) survey that incorporated a broad examination of contract cheating frequency. As explained above, Bretag et al. (2019) counted ‘sharing’ behaviours (e.g., providing an assignment for any reason or buying/selling notes) in addition to ‘cheating’ behaviours (e.g., students obtaining a completed assignment to submit as their own). They included within their definition of ‘cheating’ behaviours incidences where the student obtained an assignment, but no financial transaction was involved. Looking across these different behaviours, the prevalence rate of students who reported obtaining an assignment to submit was 2.2%, whereas 27.2% of students reported providing an assignment for any reason (Bretag et al., 2019). Bretag et al. (2019) noted that students who engaged in cheating behaviours were more likely to also engage in sharing behaviours and were twice as likely to provide another student with a copy of an assignment. Bretag et al. (2019) found that 37% of students in the cheating group had obtained an assignment, 68.5% of whom had submitted the work as their own. Again, when examining the incidence of obtaining an assignment to submit, 79.4% did so once or twice, while 20.6% did so three-or-more times (Bretag et al., 2019). This survey also found that 13.3% of the cheating group exchanged money to obtain an assignment, but it is unclear how often this translated into submitting the purchased work for assessment (Bretag et al., 2019).

To further complicate the interpretation of survey results, research by Curtis et al. (2021) incorporated an innovative method of estimation of contract cheating prevalence by incentivising truth-telling, using a Bayesian Truth Serum methodology as per John et al. (2012). This method triangulated prevalence estimates that participants produced with respect to peer prevalence, peer admission, and personal admission (relating to their lifetime and during the most recent year). Using this method, Curtis et al. (2021) produced estimates that were three to four times higher than those derived from admission rates in other self-report studies (as discussed above): 7.9% of students having ever bought and submitted assignments from commercial contract cheating sites and 11.4% having ever submitted work from commercial file-sharing sites. These findings clearly emphasise the importance of methodology when estimating the frequency of this problem behaviour.

A range of other methodological factors also influence frequency estimates. First, the country of origin of research participants impacts measurements, with Australian work indicating 2.2% of students submitted assignments completed by a third party (Bretag et al., 2019) compared to Czech work estimating a prevalence of 7.6% (Foltýnek & Králíková, 2018). Second, the prevalence time periods of interest matter. Most of these surveys have focused on prevalence (‘have you ever?’ questions) compared to incidence (‘how often have you?’). For example, Curtis et al. (2021) estimated a lifetime contract cheating prevalence of 1.8% versus a one-year estimate of 0.7%. Finally, there is a clear indication that students at English-speaking universities whose first language is not English are more likely to engage in contract cheating, relative to native English speakers (Bretag et al., 2019; Curtis et al., 2021). This means that, in addition to how contract cheating is defined, survey estimates will be influenced by who chooses to respond, where the research is conducted, how far back participants are asked to report (lifetime vs previous year), and whether respondents are incentivised to tell the truth about their past behaviour.

Can We Measure the Demand for Cheating Services?

Looking to alternative approaches, this section considers work that has attempted to measure the demand for contract cheating services, as an alternative measure of the extent of this problem. To this end, Amigud and Lancaster (2020) examined how social media, specifically Twitter, facilitates contract cheating through enabling cheat-curious students and contract cheating services to find each other. Amigud and Lancaster (2020) analysed 1579 tweets and demonstrated that at least some of the demand for contract cheating is publicly available. Looking at this issue from a different perspective, Bretag et al. (2019) estimated the attrition between procurement and submission of purchased assignments, finding that only 68.5% of students reported submitting an assignment they obtained from a third party. Finally, others have reviewed public data from providers of contract cheating services to explore how these services advertise and operate (e.g., Lancaster, 2019).

Administrative Data: Who Gets Caught Contract Cheating?

Although the detection rates for contract cheating are imperfect and likely to be low, another way to estimate the size of this issue is through administrative data relating to who gets caught for contract cheating. The 2014 MyMaster scandal in Australia highlighted the issue of contract cheating at 16 universities through a single site (Visentin, 2015), which had targeted Australian-based international students with contract cheating services focused on writing assignments and completing online tests (McNeilage & Visentin, 2014). The journalists exposed 700 receipts of payment to the contract cheating provider, with the purchasers coming from several Australian universities and across a range of courses. As a fall-out, one university indicated that 24 students (across 51 units) received a failing grade for courses completed in 2014 with another university indicating 43 students (who had logged 128 requests) had been subject to disciplinary hearings relating to the use of this service. In a different context, Baird and Clare (2017) also incorporated measures of detection (and whistleblowing) when evaluating the effectiveness of a targeted contract cheating prevention intervention focused on a business capstone unit.

Another way students may be caught engaging in contract cheating is if the contract cheating service reports them. Yorke et al. (2020) examined students’ willingness to engage in contract cheating when presented with the risk of being blackmailed by the service. Of their sample of 587 students, 14 were willing to cheat when not faced with a risk of being blackmailed (scenario 1). However, when presented with scenario 2, which included a risk of blackmail, only 7 of the 14 students were still willing to cheat. The remaining respondents were not willing to cheat in either scenario.

Has Anyone Tried Anything Else?

Looking beyond surveys, service demand, and administrative data about who gets caught, other relevant studies have used different methods to estimate aspects of the contract cheating problem. For example, Clare et al. (2017) examined differences between students’ performances on supervised and unsupervised assessment items within units, to identify rule-based, unusually big differences suggesting that students did much better than expected when they were not supervised. This study found unusual patterns in 2.1% of the marks examined (a frequency remarkably like prevalence estimates for contract cheating from several of the surveys discussed, above). Although this was not a confirmation of contract cheating engagement, it was a useful demonstration of the potential value of analysing existing administrative data to expose non-random, unusual patterns indicative of a student doing much better on assessments that were not supervised.

Using a different approach, Rigby et al. (2015) measured students’ hypothetical willingness to engage in contract cheating (i.e., buy an essay), based on the cost of the essay, the risk of being caught, the potential penalty if caught, and the grade they would receive on the purchased essay. Students were presented with eight scenarios, where they could choose to buy an essay from one of three options, based on the variables listed above, or to buy none. Rigby et al. (2015) also measured students’ risk aversion with a gambling task. Rigby et al. (2015) found that 7 students, from a sample of 90, were willing to cheat in all 8 scenarios presented to them, whilst 50% of their sample were unwilling in any circumstance to hypothetically purchase an essay. Willingness to purchase an assignment was influenced by students’ risk aversion and whether English was their first language, with those who were less risk averse and had English as an additional language more likely to choose to buy an essay.

A further study that demonstrates the significance of methodology and the fallibility of self-report is the work of Kolb et al. (2015), who conducted interviews to ask students about cheating opportunities, with a specific focus on why they do not cheat. Kolb et al. (2015) emphasised cheating as a ‘conscious deception’ and conducted interviews at the start and end of a seminar. They found that 5.9% of their sample reported having engaged in a cheating behaviour in their first interview, but only 2.9% of the sample reported cheating during their second interview. The reasons for these differences were not fully explored by Kolb et al. (2015), but it is possible they relate to the time period issue discussed above, with follow-up interviews conducted at the end of semester and a research emphasis on ‘recent’ scenarios where students may have had the temptation to cheat.

Clearly, therefore, how much contract cheating we think is occurring is influenced by (a) the definition, (b) where and when the sample is taken from, (c) the estimation method that is used, (d) the attrition between procuring work and submitting work, and (e) the contextual ‘risk’ involved with the actual (and hypothetical) situation. Moving on, we demonstrate the similarity of the importance of these methodological factors for measurement of a different type of deviance: the measurement of crime.

How Does Criminology Measure Deviance?

At a high level, ‘crime’ is a form of deviance that shares a lot of common measurement issues with attempts to quantify contract cheating. As with the various measures discussed above, there is no ‘right’ way to measure crime. This section describes the main methods that have been used, along with their respective strengths and limitations. We demonstrate how the approaches to measuring contract cheating can map cleanly into the same categories used to quantify crime: catching offenders, resource use, crime that happens but does not come to the attention of authorities (the ‘dark figure’), and non-crime proxies that are used to quantify problems. The absence of a correct measure and the development of related, flawed measures has required criminology to adopt a triangulation approach to measurement, whereby the best-available data is considered in parallel to give insight into the prevalence and incidence of crime.

Catching Offenders: The Role of Administrative Data

One of the original ways to measure crime depended on counting the things the criminal justice system knew about: what gets reported to police, what police record, who gets apprehended, what happens in court, and who gets sentenced. Police recorded crime statistics were first published in the UK from the mid-nineteenth century, the US from 1930, and Australia from 1964 (Morgan & Clare, 2021). There are strengths associated with these measures, in that they capture a lot of detail about the records that are made (offender/victim information, context information about where and when things occur) and they are embedded in a legislative and policy framework. However, these measures are imperfect. Different policing jurisdictions have different laws (and different interpretation of laws); there can be longitudinal variations within agencies that mean recording practices change (making crime appear to go up or down, when really nothing is ‘different’), and police have discretion that influences the recording of reported events (how and if reported events are entered into police records—for a detailed discussion on ‘attrition’ see Tilley & Burrows, 2005). It is also the case that not all crime events are reported to police (estimated to be 42% overall, Flatley et al., 2010); there is wide variation in the rate at which specific types of crime are reported (with the most highly reported crimes, such as burglary and car theft, strongly influenced by insurance requirements, e.g., Australian Bureau of Statistics, 2021), and that event seriousness, individual victim characteristics, and victim-offender characteristics influence the likelihood of reporting (see Tarling & Morris, 2010, for a discussion of these factors).

Accessing the Dark Figure of Crime Through Victim/Offender Surveys

To address these (and other) measurement limitations with official crime statistics, commencing in the 1960s, criminologists started using surveys to tap into the ‘dark figure of crime’ (see Morgan & Clare, 2021, for a discussion). Most surveys have focused on victimisation, randomly selecting a representative sample of the population, and using common language (‘hit’ instead of ‘assault’) to ask people about a specific period of time (i.e., the last 12 months) to expose crime that never comes to the attention of the criminal justice system. Self-report offending surveys have also been undertaken (although less frequently and systematically, e.g., Budd et al., 2005) to uncover more about the prevalence and incidence of offending behaviour, irrespective of whether it has come to the attention of authorities. Operating in a similar way to victimisation surveys, random samples of the population are asked questions about things they have done that would constitute crime. A common finding across these exercises is the non-randomness of these patterns, with very small subsets of victims and offenders accounting for a very large amount of the crime that is captured by the surveys (e.g., Hales et al., 2009). Strengths of this approach to measurement include (a) results are independent of issues relating to reporting, recording, and discretion, (b) the information is taken directly from the victim/offender source, (c) there is limited influence of politics and managerial pressures from within the justice system, and (d) they can reveal meaningful longitudinal patterns that can demonstrate changes in victimisation and offending (Morgan & Clare, 2021).

Despite these strengths, just as with the administrative approaches to measuring crime, there are also limitations with surveys. The estimates that are produced are influenced by methodological decisions including questionnaire length, the order of questions, how the survey is conducted (e.g., in-person vs online), and the time period in question (12 months vs 5 years vs lifetime). Individual respondents can also forget, confabulate, and/or lie, resulting in verifiable inconsistencies between survey-based accounts and police records (e.g., Averdijk & Elffers, 2012), or respondents can choose to under-report their own criminal behaviour (Bernasco et al., 2020).

Indirect and Novel Measures into Specific Crime Issues

Another window into the volume of crime is provided by police calls for service. In addition to demonstrating variations in demand for police services over time, these data, which are collected based on police activity but not influenced by discretion and recording decisions, can give meaningful insight into temporal and geographic crime patterns. Recent examples have used this measure to look at time/space service overlap from police, fire, and ambulance workload (Clare et al., 2019), and to monitor the impact of COVID-19 lockdowns on crime and disorder during the early months of the pandemic (Ashby, 2021). It is important to note that calls for service do not equal crime, as police attend a lot of non-crime calls, so this measure has the potential to over-count crime-related activity.

It is also worth briefly considering some alternative approaches to measuring aspects of crime. Drug test data is frequently used to monitor trends in drink/drug driving (e.g., Midgette et al., 2021) or prevalence of drug use in offender populations (e.g., Doherty & Sullivan, 2020). Emergency room data and hospital admissions provide another window into certain types of violent crime, such as intimate partner violence, alcohol-related violence, or violence against vulnerable groups in society (e.g., Macdonald et al., 2005). The common themes across these approaches are that they focus on relatively specific types of crime and provide a non-random estimate of the prevalence and incidence of the crime they relate to. Finally, social media is emerging as an alternative way to measure crime, with studies demonstrating the relevance and utility of Twitter for monitoring low-level crime and disorder in micro-geographic areas (Williams et al., 2016). Facebook is also being used to help monitor cybercrime victimisation (Aliyu et al., 2020).

Lessons for Measuring Contract Cheating: Triangulation Is the Key

As with approaches to measuring contract cheating, there is no single, correct way to measure crime. In both cases, ‘problem’ estimates are influenced by the way the data are collected and the context within which collection occurs. To emphasise the commonalities of the approaches adopted in these two contexts, Table 2.1 uses the varying focuses on administrative data, surveys, resource use, and indirect/novel measures to align the major measurement approaches used so far for contract cheating and those developed over a much longer period in criminology focused on measuring crime. The consistency is useful, as it can be used as a platform to encourage contract cheating researchers to embrace the imperfect nature of measurement in this area. Rather than seeking to find the single, right measure of how much contract cheating is occurring, adopting a triangulation approach to measuring the issue moving forward will be the most useful for assisting the development and evaluation of detection and prevention strategies.

Table 2.1 Comparing and classifying the various approaches to measuring crime and contract cheating
Full size table

As discussed above, surveys are limited as they are influenced by factors such as who is asked, what time period is covered, how the behaviour is defined, prevalence and incidence, and memory errors of respondents. Resource use is also imperfect because engaging with a provider does not mean people submit the work and there are grey areas around file sharing. Furthermore, the utility of administrative data is also limited, influenced by who is caught, variations in policy and practice over time and across institutions, and the threshold of proof involved (suspected vs proved). Triangulating these imperfect estimates whenever possible will help give the best representation of the current state of the problem. In a crime context, an increase in police recorded crime could represent an increase in real crime or it could reflect an increased willingness to report crime events to police. Without surveys against which to compare victimisation (with the closest potential survey represented in the work by Harper et al., 2019), it is very difficult to know which of these was driving the increase in official statistics. For the same reasons, a triangulation approach could help address the concerns raised from Newton’s work as to whether the recent increase in prevalence estimates represents an actual increase in contract cheating, a shift in methodology/measurement, changes to definitions, a combination of these factors, or something else entirely. Furthermore, relying on multiple measurement strategies may well mean researchers and policy makers are staying alert to emerging problems (such as cyber fraud, in a crime context, which traditional victimisation surveys and police records do not capture well). This type of issue is of particular concern when it comes to the dark figure of contract cheating and the impact of COVID-19, as we did not know what the prevalence and incidence of this misconduct were before the pandemic, but we can reasonably assume it will have increased as a result of the rapid changes to assessment structures and opportunities.

In conclusion, we urge contract cheating researchers to be cognisant of the measurement issues in this research area, learn from the developments in a related, fuzzy measurement space provided by criminological research, and commit to increased use of mixed-methods and data triangulation in contract cheating research. As has been seen within criminal justice research, this will assist the development and evaluation of contract cheating detection and prevention strategies.