Introduction

Recidivism, and its numerous operational measures, is perhaps the most commonly used variable in the criminological and penological literature. Recidivism – “reversion of an individual to criminal behavior” (Maltz, 1984, p. 1) – has served as the determinate role in the understanding of how criminal and delinquent behavior correlate with social factors (such as early childhood trauma, caregiver attachment, gang affiliation, and community social structure (Benda & Toombs, 2002; Boxer et al., 2017; Caudill, 2010; Dooley et al., 2014; Hipp et al., 2010; Trulson et al., 2012; Vitopoulous et al., 2019; Wolff et al., 2020)), individual characteristics (i.e., substance abuse, serious mental illness,, developmental factors (Baglivio et al., 2017; Cloyes et al., 2010; DeLisi et al., 2015; Langevin & Curnoe, 2011; Ozkan, 2016; Zettler et al., 2015)), and the continuity of criminality(Cochran et al., 2014a, b; Huebner & Berg, 2011; Trulson et al., 2011, 2016). Recidivism has been used to assess the utility of criminal sanctions, correctional supervision strategies, and correctional programs (Baay et al., 2012Mears et al., 2012; Bales & Piquero, 2012; Caudill & Trulson, 2016; Cochran et al., 2014a, b; Evans et al., 2014; Gainey et al., 2000; Krebs et al., 2009; Lanza-Kaduce et al., 1999; Lowenkamp et al., 2009; Peters et al., 2015; Steiner et al., 2012; Trulson et al., 2011; White et al., 2012; Young et al., 2013). Recidivism also has contributed greatly to the exploration and development of risk / needs assessment instruments (Andrews et al., 2006; Jones et al., 2010; Lowenkamp et al., 2006), including understanding the nuances of such offender characteristics as gender (Baglivio, 2009; Scott et al., 2016), race (Skeem & Lowenkamp, 2016) and offense specialization (Coid et al., 2015; Rajlic & Gretton, 2010). Collectively, recidivism is one of the most influential measures in criminology and penology.

Despite the popularity of recidivism as a standard benchmark, operationalization of recidivism as a variable is inconsistent and the consequences of using differing metrics of it remain unknown in the juvenile justice system. Commonly, studies using official measures of juvenile recidivism have operationalized it through rearrest (see, for example, Boxer et al., 2017; DeLisi et al., 2015; White et al., 2012), reconviction (Baglivio et al., 2017; Cochran et al., 2014a, b), and reincarceration for a parole violation (Benda & Toombs, 2002). Some studies have used multiple measure of recidivism (Peters et al., 2015; Wolff et al., 2020; Young et al., 2013) and, of the studies using multiple measures of recidivism, there is evidence of disparity between these measures (Wolff et al.). Yet the origin and magnitude of these disparities remains unknown. While evidence from samples of adult parolees suggests disparity between outcomes based on recidivism measurement (Hoffman & Stone-Meierhoefer, 1980; Ostermann et al., 2015), those findings have limited applicability beyond those samples. First, there are collateral social consequences of being imprisoned and these consequences are different than the consequences of being placed on community supervision in lieu of imprisonment (Decker et al., 2015; Kolstad et al., 1995; Turanovic et al., 2012). Given that the majority of individuals under correctional supervision are supervised in the community, it is important to understand how manipulation of the recidivism measure affects the outcomes of individuals sentenced to community supervision. Second and more importantly, the lack of focus on the consequences of recidivism operationalization in the juvenile justice system is concerning. The American juvenile justice system is distinctly less legalistic than the adult criminal justice system and through the parens patriae philosophy the juvenile justice system is geared to balance more toward the needs of the juvenile. These limitations in the literature and the uniqueness of the juvenile justice system’s orientation leaves a void in what is known about the consequences of recidivism operationalization in juvenile justice.

The purpose of this study is to close the gap on what is known about the consequences of operationalizing recidivism in juvenile justice. Court processing data were collected on 10,830 juveniles under court supervision in a large southern state and, through event history analysis, the risk of three measures of recidivism – offense date, referral date, and adjudication date – were calculated. These hazards were then compared statistically to assess whether different operational definitions of recidivism produced distinctly different hazards of recidivism and to observe the consequences of different recidivism operational definitions on correlates of recidivism. This study extends what is known about the effects of various operational definitions of recidivism on the risk of recidivism and disparity among co-variates of recidivism.

Literature Review

Measuring Recidivism

Scholars interested in the operationalization of recidivism have identified the continuum nature of Type I –false positives (being categorized as a recidivist when the individual desisted from criminality) – and Type II – false negatives (being classified as a non-recidivist when the individual persisted in criminality) – errors in measuring recidivism (Blumstein & Larson, 1971; Maltz, 1984) and theorized the impact of the court processing on higher standard measures of recidivism among adults after incarceration (Ostermann et al., 2015). Blumstein and Larson theorized that decisions made later in the criminal justice system process (e.g., conviction) would decrease Type I errors and increase Type II errors, and Maltz suggested an ordered categorization of various measures of recidivism (that is, arrest requires a lower standard of proof compared to the beyond a reasonable doubt standard of conviction). Looking to the empirical literature on adult recidivism measures, scholars have reported a potential delayed recidivism phenomenon among samples of formerly incarcerated adults when using conviction as the recidivism metric (Ostermann et al., 2015) and suggested that arrest and conviction measures, while on similar trajectories, are non-proportional (i.e., non-convergence) when measured over time for adult offenders (Hoffman & Stone-Meierhoefer, 1980). McGovern et al., (2009) further suggested that the use of different recidivism proxies influenced the correlation between an individual’s ethnicity and recidivism for formerly incarcerated adults.

Maltz (1984) also proposed a more complex compromise in measuring recidivism that accounts from the timing of the event under measurement and the due process of a court decision, but conceded that such a sophisticated data collection approach could prove to be unreasonable. Maltz’s concerns, however, revolved around two aspects of measuring recidivism: identifying the most valid metric – the measure most closely associated with individual behavior – and identifying the most reliable metric – the measure most widely available across jurisdictions.

Measuring Recidivism in Juvenile Justice

While scholars have explored the dilemma of measuring recidivism in the criminal justice system, less attention has been given to the impact of operational definitions of recidivism in the juvenile justice system. Studies in juvenile justice relying on recidivism as a dependent variable generally use one of three official measures: a new referral (or rearrest), a new adjudication, or reincarceration (see, for example, of rearrest: Boxer et al., 2017; Cochran et al., 2014a, b; DeLisi et al., 2015; White et al., 2012; a new adjudication: Baglivio et al., 2017), and reincarceration for a parole violation: Benda & Toombs, 2002).Footnote 1

While there are parallels between the adult criminal justice system and the juvenile justice system, there is reason to believe that measuring recidivism may be an exception. Because the juvenile justice system is based in the idea that the individual is not yet fully matured and acts in a non-adversarial posture, it may be that the disparity explored in the adult system behaves differently in the juvenile justice system. For example, Blumstein and Larson’s (1971) due process concerns may be less relevant in the juvenile justice system because of the court’s posture. Exploring this phenomenon may help to understand the relationship between front-end measures of recidivism (e.g., referral) and deeper-end recidivism measures (e.g., adjudication).

Evidence of disparity in how recidivism is measured in the juvenile justice system has come by way of scholars expressing methodological concerns around how over-policing in certain communities may influence outcomes and tangential findings from studies using multiple operational definitions of recidivism. For example, Vitopoulos et al. (2019) justified their use of adjudication (conviction) for a new offense as the measure of recidivism because “it is believed that conviction, rather than arrest data, more accurately accounts for youth belonging to marginalized communities having a greater likelihood of police involvement are arrest without subsequent conviction” (p. 356). Vitopoulos et al.’s (2019) concern, similar to that of McGovern et al. (2009) in the adult literature, extends the Type I error concerns expressed by previous scholars (Blumstein & Larson, 1971; Maltz, 1984) into the juvenile justice system.

In a similar vein, there is preliminary evidence of variance in recidivism estimates among juvenile samples when multiple operational definitions of recidivism are used among the same juveniles. For example, Wolff et al. (2020) recently found that verified gang members were more likely than non-gang affiliated individuals to be rearrested (odds = 1.589, p < 0.01) and more likely to be readjudicated (odds = 1.425, p < 0.01). Their work explored the “sex-specific effects of gang membership and behavioral disorders on juvenile recidivism,” but using two measures of recidivism with one sample highlights the issue at hand (p. 879). Although their focus was on the consequences of gang affiliation and behavioral abnormalities, and not on the effects of using multiple operational definitions of recidivism, their findings provide preliminary evidence that measuring recidivism may very well matter in outcomes. Wolff et al.’s work demonstrated that verified gang membership was correlated statistically with recidivism, but it also demonstrated that operational definitions of recidivism can produce varying outcomes. It is unknown if these variances were statistically relevant and if there was a disparate effect of the operational definition of recidivism across sub-groups of individuals under juvenile court supervision.

Summary

There is concern in the juvenile justice literature about the operationalization of recidivism (Vitopoulos et al., 2019) and there is preliminary evidence to suggest a disparity between operational definitions of recidivism (Wolff et al., 2020). The consequences of using less restrictive recidivism proxies has the potential to increase Type I errors, while the consequences of using more restrictive recidivism proxies means that a recidivist remains unaccounted for – Type II error (Maltz, 1984). Assessing the impact of different operationalizations of recidivism, then, seems especially critical as recidivism is so central to the study of juvenile crime and the juvenile justice system. This study seeks to close some of the gap in understanding the consequences of operational definitions of recidivism in the juvenile justice system.

Current Study

This study tested two hypotheses focusing on the effects of recidivism criterion selection on time to failure among individuals under juvenile court supervision. Previous literature has established that selection of criterion and exposure time have the potential to artificially distort recidivism outcomes (Type I and Type II errors) among adult offenders (Ostermann et al., 2015), yet there remains ambiguity in the juvenile justice system on: (a) how the criterion selection affects recidivism risk and (b) how the criterion selection affects the covariates of recidivism. To address these gaps in what is known about operationalizing recidivism in juvenile justice, we set out to test the following hypotheses:

  • H1: The use of different operational definitions of recidivism produces disparate recidivism risks among individuals under juvenile court supervision.

  • H2: Recidivism covariates are inconsistent across multiple operational definitions of recidivism among individuals under juvenile court supervision.

Methods

Data

Data for this study originated from a large southern state’s juvenile justice system. The state collected and archived individual level process data for every juvenile arrested or referred to county juvenile probation departments for any status, misdemeanor, or felony offense. The age of jurisdiction was 10 to 17 for individuals responsible for a felony offense. Data were collected between 2001 and 2007 and included social background, criminal history, and case processing information.

Sample

Included in this study were all juveniles with complete records referred to the juvenile probation for a felony offense between 2001 and 2004. The cutoff of 2004 for inclusion in this study was selected to provide each case at least two years of follow-up time between the referral date and the final recidivism data collection date. Additionally, only those juveniles with at least two years between their age at the base referral and the age of majority (18 years old) were included.Footnote 2 This provided each case with two years of recidivism exposure captured in the state’s records. The full sample included 10,830 juveniles referred for at least one felony offense that were adjudicated and placed on formal probation to be served in the community.Footnote 3 Approximately 63% (n = 6,844) of these 10,830 juveniles under court supervision for a felony offense were first time referrals.

Variables

Covariates

Table 1 describes the sample through several recidivism covariates. The sample consisted of a racially diverse group, with individuals categorized as: 2,291 (21%) African American, 4,744 (44%) Hispanic, 3,724 (34%) white, and 71 (1.0%) “other” race. The average age at the base referral was 13.85 (min = 10; max = 15) and approximately 89% (n = 9,660) were males. Approximately seven percent (n = 802) of the sample were confirmed gang affiliates.Footnote 4 Just fewer than one-half (47%; n = 5,098) of the sample lived with a single parent and approximately 22% (n = 2,383) resided with both parents, with the balance (31%) reporting a wide range of living arrangements.

Table 1 Sample Descriptive Statistics
Full size table

Several measures of abuse and psychological complications were included in this study. Specific to experienced abuses among this sample of individuals under felony juvenile court supervision, approximately four percent (n = 398) reported being sexually abused, approximately four percent (n = 458) reported experiencing physical abuse, and approximately four percent (n = 411) reported experiencing emotional abuse.Footnote 5 The Massachusetts Youth Screening Instrument – Version 2 (MAYSI-2) was used to collect information on seven psychological domains (alcohol / drug use, angry / irritability, depression / anxiety, somatic complaints, suicide ideation, thought disturbances, and traumatic experiences) for individuals (see, for a comprehensive review, Grisso et al., 2012). The full sample included the following mean averages for each domain: alcohol / drug use = 1.15 (scale 0–8), angry / irritability = 2.90 (scale 0–9), depression / anxiety = 1.15 (scale 0–8), somatic complaints = 1.96 (scale 0–6), suicide ideation = 0.59 (scale 0–5), thought disturbances = 0.39 (scale 0–5), and traumatic experiences = 1.22 (scale 0–5).

Also included to describe the sample and to predict recidivism were several criminal history and current offense metrics given the correlation between these factors and case disposition (see, for example, Caudill et al., 2013). Specific to criminal history, individuals included in this sample had, on average, approximately 0.71 (min = 0; max = 15) previous offense referrals, with approximately 63% (n = 6,844) of the sample being first time referrals. Focused on the base referral, the mean average number of offenses included in the referral was 1.32 (min = 1; max = 21), the offense level included 4,478 (41%) State Jail Felony offenses, 1,571 (15%) Felony 3 offenses, 3,842 (35%) Felony 2 offenses, and 939 (09%) Felony 1 offenses (the most severe). Of those felonies, 27 percent (n = 2,943) were categorized as violent crimes, 45 percent (n = 4,926) were categorized as property crimes, and the balance (27%; n = 2,961) of the offenses categorized as other crimes. Approximately two-thirds (67%) of the sample was detained in association with their felony referral and the average time served in detention was 9.66 days (min = 0; max = 406).

Recidivism Criteria

To test the aforementioned hypotheses and develop a better quantitative understanding of how various operational definitions of recidivism affect the conceptual notion of recidivism in juvenile justice, this study used three unique measures of recidivism: offense date, referral date, and adjudication date. These operational definitions represent three unique measures of recidivism in that each has a specific date representing when the event occurred. Of this cohort, approximately 64% had a recorded new offense date, approximately 63% were referred for a new offense, and approximately 24% were adjudicated for a new offense.

Time to Offense

Time to offense represented the time between the date an individual was placed on court-ordered supervision (probation) and the date of a new offense. Table 5 (Appendix 1) is a life table for recidivism using the offense date criterion. As demonstrated, the sample started with 10,830 at risk individuals during the first two weeks following being placed on community supervision. By the final interval, 52–53 included the final two weeks of the two-year observation period, approximately 64 percent of the sample had recidivated, leaving approximately 36 percent of the sample as non-recidivists. The greatest risk of recidivism occurred during interval 1–2, representing weeks three and four, with 478 juveniles recidivating during this interval.

Time to Referral

Time to referral for a new offense represented the time between the date a juvenile was placed on community supervision and the date the juvenile’s new case was formalized (i.e., the process where the juvenile is notified of the referral for a new offense and fingerprinted for that offense) either through entry into a detention facility or through formalization of the case at the juvenile probation department. In the state from which this sample originated, law enforcement officers were generally instructed to execute an arrest for all felony-level offenses and for only those misdemeanor offenses which occurred within their presence. These processing paths presented some variation in the time between the offense date and the referral date as demonstrated by comparing the results presented in Table 5 (Appendix 1) (offense date) with the referral date life table presented in Table 6 (Appendix 1).Footnote 6 Similar to the results of the offense date life table (Table 7), the sample for the referral date life table contained 10,830 at risk subjects. The second interval (1–2) resulted in the greatest number of recidivists (n = 432). The final observation period (52–53) represents two years of recidivism exposure time and, as demonstrated, approximately 37% of the sample avoided referral formalization for a new offense during those two years.

Time to Adjudication

The final recidivism proxy, time to adjudication, represented the time between the date a juvenile was placed on community supervision and the date a subsequent referral for any new offense was disposed of by the juvenile court. The follow-up period for all three measures of recidivism was two years after their base referral disposition date that resulted in the juvenile’s community supervision, so any new recidivism outcome that occurred beyond two years post-base referral disposition was considered a non-recidivism / non-failure event. As with the other measures of recidivism, observations of adjudications started with a sample of 10,830 subjects and after two years, approximately 76% of the sample avoided being adjudicated. Unlike the previous two recidivism variables, however, the risk of recidivism was depressed during the early intervals when recidivism was operationalized as adjudication. As demonstrated in Table 7 (Appendix 1), Interval 23–24 contained the highest risk for recidivism when measured as adjudication, with the greatest number of recidivists (n = 76).

Analysis Protocol

Survival analysis served as the foundational statistical modeling to address the hypotheses. Survival analysis, most commonly associated with Cox regression (Cox, 1972, 1975), is superior to binary statistical models (e.g., chi-2 and logistic regression models) in that survival models include not only if an event occurred, but also when the event occurred. In modeling recidivism, event history analysis models the risk (hazard) of recidivism over the exposure period (incorporation of time to failure = recidivism), while, at the same time, avoiding the overly restrictive normal distribution assumption of linear modeling. Although most prominent, the Cox regression model also introduces the proportional hazards assumption that requires failure hazards be proportional over time (Box-Steffensmeier & Jones, 2004; Singer & Willett, 2003). To account for the time-varying covariates in the current data, we incorporated the Royston-Parmar survival model instead of overfitting the Cox model to accommodate co-variates that violated the proportional hazards assumption (Royston, 2001; Royston & Lambert, 2011; Royston & Parmar, 2002). Model fit statistics (BIC) were used to identify the best-fit model for all three recidivism criteria. A “hazard” model with six degrees of freedom was the best fitting model for analyses.

To address the first hypothesis—the use of multiple operational definitions of recidivism produces disparate recidivism risk – Royston-Parmar models were performed for each recidivism criteria and the resulting recidivism hazards were compared using paired t-tests. The second hypothesis—the coefficients of recidivism predictors are inconsistent across multiple operational definitions of recidivism – was tested by comparing each co-variate’s hazards produced in the Royston-Parmar models to one another by calculating the z-scores (offense versus referral, offense versus adjudication, and referral versus adjudication).

Findings

Table 2 presents the results of three Royston-Parmer event history models. Included in these models were 10,830 observations (Log likelihood (offense) = -16,239.178; (referral) = -15,748.609; (adjudication) = -9,815.285) and the models’ flexibility was able to accommodate three time-varying co-variates (age at referral, number of days in detention, and the subject being of Hispanic descent). As demonstrated in the table, nine of the 20 co-variates were significant predictors of recidivism across all three proxies (offense, referral, and adjudication). Those juveniles with significantly greater risks of recidivism were also those that reported being African-American, were older at the time of their base referral, were male, were gang affiliated, reported being physically abused, reported elevated drug and/or alcohol abuse, had a more extensive criminal history, and were adjudicated for a property offense for the base referral. Conversely, juveniles with a depressed risk of recidivism were those individuals that lived with both parents and those juveniles that reported elevated somatic complaints levels. Six co-variates shifted in significance across the three recidivism models. The relevance of these co-variates to recidivism shifted for those juveniles reporting being Hispanic, residing in a single-parent household, reporting higher levels of anger / irritability, and reporting higher levels of depression / anxiety. Additionally, significance levels varied for the level of the offense associated with the base referral and for the number of days in detention for the base referral.

Table 2 Recidivism Models: New Offense Date, New Referral Date, and New Adjudication Date
Full size table

While the results of the Royston-Parmar model (presented in Table 2) provided estimates for the relationships between the predictor variables and the three operational definitions of recidivism, it did not test the differences in recidivism risk across the three proxies. Table 3 compares the sample recidivism risk across all three recidivism measures (offense, referral, and adjudication) through three paired t-tests. All three comparisons were significant (p < 0.001), with the risk of recidivism being greatest (mean = 0.012) when it was measured as offense and lowest (mean = 0.003) when measured as adjudication. The results suggest that the recidivism hazard for each outcome (offense, referral, and adjudication) were statistically unique when compared to the other two recidivism measures. Figure 1 represents these dynamic, yet distinct recidivism measures over time. Recidivism hazard for a new offense and a new referral followed similar trajectories peaking just before five weeks, while the recidivism hazard for a new adjudication was depressed and peaked in a plateau just after 20 weeks.

Table 3 Paired T-Test Comparison of Two-Year Recidivism Hazards
Full size table
Fig. 1
figure 1

Recidivism Hazard: New Offense Date, New Referral Date, and New Adjudication Date

Full size image

The second interest of this study was in how different operational definitions of recidivism influence the correlations of predictor variables. Table 4 represents the results of three paired t-test statistics comparing predictor coefficients of the three Royston-Parmar models: (a) offense compared to referral, (b) offense compared to adjudication, and (c) referral compared to adjudication. While the co-variates between offense date and referral date were statistically similar, seven co-variates produced statistical inconsistent coefficients across the offense date / adjudication date and across the referral date / adjudication date comparisons. Specifically, the coefficients were statistically different (p < 0.05) for the following variables: Hispanic, emotional abuse, single-parent home, depression / anxiety, suicide ideation, criminal history, and days in detention associated with the base offense. Looking back on the event history analyses reported in Table 2, juveniles reporting higher levels of depression / anxiety were at a greater hazard of recidivism when recidivism was measured as adjudication (coef. = 0.029; p = 0.040) compared to when recidivism was operationalized as a new offense (coef. = -0.011; p = 0.266) or a new referral (coef. = -0.011; p = 0.284). While criminal history was a significant predictor of all three operational definitions of recidivism, those juveniles with more previous referrals were statistically less likely to be adjudicated (coef. = 0.074; p = 0.000) compared to the elevated hazards of a new offense (coef. = 0.138; p = 0.000) and a new referral (coef. = 0.141; p = 0.000). Base referral days in detention was statistically associated with increased hazards of being adjudicated (coef. = 0.004; p = 0.000), but not for a new offense (coef. = -0.001; p = 0.441) or a new referral (coef. = -0.001; p = 0.435).

Table 4 Co-Variate Comparisons over Recidivism Operationalizations
Full size table

Discussion

The goal of this study was to understand to a greater degree the consequences of using different operational definitions of recidivism given its relevance to a wide range of topics in the juvenile justice literature. Our findings extend into the juvenile justice system what has been seen in the adult criminal justice system and what has been suspected in the juvenile justice system: using different operational definitions of recidivism produces significantly different estimates of recidivism timing. There are important findings demonstrated in Table 3 and visualized in Fig. 1 that add to the juvenile recidivism literature. First, the use of any one operational definition of recidivism (based on the offense date, the referral date, or the adjudication date) produced distinctly and statistically significant different estimates of recidivism hazards. Second, using different operational definitions of recidivism shifts the relevance of several predictors of recidivism. The hazards of recidivism shifted significantly when operationalized as adjudication (compared to being operationalized as a new offense and operationalized as a new referral) for juveniles reporting being Hispanic, experiencing emotional abuse, residing in a single-parent household, reporting feelings of depression / anxiety, reporting suicide ideation, having more sophisticated criminal histories, and spending more time in detention (see Table 4).

These findings build on what is known about juvenile recidivism. Typically, recidivism has been operationalized as either re-arrest, re-conviction, or re-incarceration. However, extant literature on the subject suggests that each of these measures have positive measurement attributes (e.g., using arrest is closer in time to the offense than conviction) and negative measurement attributes (e.g., using arrest has a lower due process threshold than conviction) (Maltz, 1984). From the adult criminal justice system, Ostermann et al., (2015, p. 786) found that the operational definition of recidivism had a non-nominal influence on whether parole was successful: “Our findings illustrate that the determination of the effectiveness of parole supervision is a function of both the influences of parole supervision and the manner in which recidivism and court processing time are specified in the evaluation process.” Included in this issue is the consequences of Type I errors – counting a non-recidivist as a recidivist– and Type II errors – failure to count a recidivist as a recidivist (Willbach, 1942). Our findings build on what prior scholarship has been unable to do by testing statistically the discriminate effects of various measures of recidivism. Further, our findings go a step further and identify co-variates that had statistically significant variations in coefficients across the operational definitions of recidivism.

This study contributes to the juvenile justice literature by testing two hypotheses: (H1) The use of multiple measures of recidivism produces disparate recidivism risks among individuals under juvenile court supervision and (H2) recidivism covariates are inconsistent across multiple operational definitions of recidivism among individuals under juvenile court supervision. We used survival analysis to establish the hazards of recidivism over time given the concerns surrounding court processing time and then compared the calculated hazards across three recidivism measures – offense date, referral date, and adjudication date – to test the first hypothesis. We then compared the coefficients of each variable from the survival analyses to test the second hypothesis.

Table 3 reports the test of the first hypothesis – the use of various operational definitions of recidivism produces disparate recidivism risks among individuals under juvenile court supervision – and the results reject the null hypothesis. The results suggest that the hazards of recidivism are significantly different (p < 0.001) for all three operational definitions of recidivism. The hazards of recidivism associated with the offense date were significantly greater than the hazards of recidivism associated with referral (mean = 0.013 compared to mean = 0.012, respectively). Following suit, the hazards of recidivism associated with referral were significantly greater than the hazards of adjudication (mean = 0.012 compared to mean = 0.003, respectively). These results suggest that as the standard of proof increases, the hazard of being classified as a juvenile recidivist decreases significantly. Figure 1 represents these disparities and the uniqueness of adjudication compared to the other two measures of recidivism. While all three operational definitions of recidivism are statistically unique, the offense date and referral date follow a similar trajectory. This is not the case for adjudication. Recidivism operationalized as adjudication lags the other two recidivism measures and does not peak until somewhere around week 20 after the exposure date. These findings are especially relevant to the Type II errors in measuring recidivism (Willbach, 1942). The finding that court processing among this sample of juvenile offenders lasted, on average, five months is consequential and suggests that using a dispositional measure (i.e., adjudication or any other higher standard, see Maltz, 1984) would truncate the recidivism exposure time. Further, adjudication was generally depressed across the period under observation compared to offense and referral operational definitions.

Table 4 reports the results of the test for the second hypothesis – recidivism predictors are inconsistent across operational definitions of recidivism among juveniles – by comparing statistically (by calculating the z-scores across models) the coefficients and standard errors of the predictor variables resulting from survival analysis models. Our findings suggests that several co-variates produced statistically different hazards of recidivism when recidivism was operationalized differently. The results suggest that juveniles that reported being Hispanic, experiencing emotional abuse, residing in a single-parent home, experiencing depression / anxiety, and having suicide ideations experienced statistically different hazards of recidivism across the operational definitions of recidivism. Additionally, the findings suggest that criminal history and days in detention associated with the base offense have statistically different correlations with the various measures of recidivism.

Aligned with previous concerns regarding disparity of racial correlates across operational definitions of recidivism among adult offenders (McGovern et al., 2009), juveniles reporting being Hispanic were significantly more likely to be categorized as a recidivist if either the offense date or referral date were used as the recidivism proxy compared to when the adjudicated date is used as the recidivism proxy.

Further, evidence suggests that other characteristics, such as emotional abuse, residing in a single-parent household, depression / anxiety, suicide ideation, criminal history, and base referral detention days, vary across the measures of recidivism and may have an undesired effect on measuring recidivism. While not all of these co-variates were significant predictors of recidivism in all three event history analysis models, the hazards of recidivism for these covariates shifted significantly when comparing models. For some of these co-variates, such as depression / anxiety, criminal history, and base referral days in detention, comparing the three operational definitions of recidivism resulted in shifts in significance in the recidivism models. As demonstrated in Table 2, juveniles presenting depression / anxiety were no more likely to have a new offense or a new referral, but the hazards of recidivism was significantly greater for adjudication among this group. The same phenomenon occurs when considering suicide ideation, except these juveniles were significantly less likely to experience adjudication. Those juveniles with more sophisticated criminal histories were at elevated risk for recidivism across all models, but the risk of adjudication was significantly lower for those with more extensive criminal histories than it was for a new offense or for a new referral. The number of days in detention for the base referral, while non-significant in predicting the new offense or new referral, increased the likelihood of being classified as a recidivist when predicting a new adjudication. Collectively, these findings suggest that there is a group of known correlates of recidivism in juvenile justice that behave differently depending on the operational definition of recidivism.

Implications

The results of this study have several implications for policy and practice within the juvenile justice system. First, the disparity between adjudication and the two other recidivism measures (shown in Fig. 1) is striking and suggests that adjudication has limited utility as a measure of recidivism within the juvenile justice system. While relying on adjudication may reduce the potential for Type I errors as there is a higher standard of legal proof compared to referral (i.e., beyond a reasonable doubt compared to probable cause, respectively), it simultaneously increases the Type II errors as the court process time artificially depresses the hazard of recidivism during the first year of exposure time. This phenomenon has been observed among a sample of adult offenders(Ostermann et al., 2015 and suspected in the juvenile justice system (Vitopoulos et al., 2019; Wolff et al., 2020). Our findings suggest that recidivism risks vary across operational definitions, particularly when adjudication is compared to earlier stages on the juvenile justice process.

The lack of proportionality during the first five months of recidivism exposure between adjudication and referral not only questions the utility of adjudication as a measure of recidivism, it suggests that Type II errors in recidivism are dynamic over time for juvenile offenders. Figure 1 demonstrates that the difference between arrest and conviction was greatest during the initial recidivism exposure time and then the hazard of conviction began to mirror the hazard of arrest at approximately five months after exposure. This lagged effect questions the validity of a continuum relationship between Type I and Type II errors attributed solely to the due process error. This lagged recidivism effect suggests the due process error is not the only cause of depressed recidivism hazards when using a deeper decision point for juvenile offenders. Instead, it appears the juvenile court workgroup’s influence also involves delayed measurement of recidivism.

Our findings suggest that the offense date or the referral date for a new offense are more appropriate measures of recidivism than the adjudication date in juvenile justice. The results presented here suggest that referral date is statistically different than the official offense date; however, the referral date hazard mimicked the offense date hazard over time (see Fig. 1). In essence, the referral date recidivism hazard estimates are proportional to the offense date recidivism hazard estimates, while the adjudication date hazard estimates are non-proportional to the offense date recidivism hazard estimates. This lagged effect of adjudication hazard estimates suggests that it would be more appropriate to use referral date as the recidivism proxy than the adjudication date if the offense date is unavailable. In other words, examination of recidivism within a prescribed time frame should include an official measure that has a date as close to the offense date as possible to reduce artificially inflating the time to recidivism. Further, adjudication hazard was significantly depressed across the period under observation compared to the other measures of recidivism. Any incorporations of these recommendations should also consider the potential negative measurement attributes of referral date, such as a lower standard of culpability.

While informing the decision on how to operationalize recidivism in juvenile justice is important, the findings suggesting disparate consequences of using different operational definitions of recidivism. While many of the predictor variable coefficients remained steady in their correlation (either positive or negative) across all three recidivism measures, some of these co-variates’ coefficients shifted and some of these shifts have very meaningful implications. Given that the hazards of recidivism shifted significantly for juveniles reporting being Hispanic, emotionally abused, in a single-parent household, depressed / anxious, suicidal, more criminally sophisticated, and having a history of detention; it is important that these co-variates be considered when developing recidivism studies and program evaluation strategies.

The results suggesting a disparity of recidivism hazard when measuring adjudication (compared to offense and referral) may be indicative of the juvenile justice system’s unique purpose and the parens patriae philosophy. While being propelled further into any justice system may appear unfavorable, it may be the case that the juvenile justice system has a tendency to consider extra-legal factors in pursuit of justice. Caudill et al. (2013) found that extra-legal factors were more sporadic in influence than legal factors; however, their work did not include measures of emotional wellbeing. Our findings suggest that the juvenile justice system responds to emotional needs – juveniles experiencing depression / anxiety were significantly more likely to be adjudicated and juveniles reporting suicide ideation were significantly less likely to be adjudicated – is an indication that the juvenile justice system may take a more wholistic approach. These findings further support and expand on the uniqueness of the juvenile justice system.

Beyond methodological and theoretical value, these findings have political and funding implications. This is especially true in the current evidence-based landscape. Federal and state funding organizations are increasingly interested in program effectiveness and recidivism reduction is often the programmatic goal. How the evaluation defines recidivism can have, as demonstrated here, significant effects on the recidivism results. This is of particular concern given the findings suggesting a disparity in the hazards of recidivism for co-variates when comparing adjudication to the other operational definitions of recidivism. The same holds true for any program that aims to reduce recidivism, funded by an external organization or funded by tax revenues. Defining recidivism for juvenile justice programs without a clear understanding of the different interpretations associated with various operationalizations could have immediate public safety consequences and long-term financial consequences if underperforming programs are continually supported due to faulty operationalization.

Limitations

The findings presented here expand what is known about the importance of deliberate development of recidivism measures for juvenile justice, but there are limitations. First, although the study sample was large and diverse, the sample originated from only one state. Second, our study used official records to create the variables, including the three measures of recidivism. Of the known issues with official records data, the most relevant of these is the reliability of the offense date variable. The date of offense variable had the greatest likelihood of inaccuracy because of the potential for estimating. In property crime cases, for example, a non-nominal amount is reported with an estimated most likely date of the crime as the exact date is unknown. The other two operational definitions of recidivism – referral date and adjudication date – have potential for inaccuracies, resulting from such things as entry error. That said, we have no reason to believe (based on our evaluation of the data) that these variables were somehow compromised or unreliable.

Third, we were unable to observe the juvenile courtroom workgroup’s influence on the court process. While the findings suggest the adjudication is delayed by the court process, we were unable to estimate the influence of these factors. While this does not impact the lagged effects, having information on the juvenile courtroom workgroup (e.g., privately retained defense counsel versus public defender, see for example, Brady and Peck (2021)) may have informed some of the variation in the time between referral and adjudication. Ultimately, though, having this information would be additive and not diminish the differences between recidivism hazards across operational definitions of recidivism.

Conculsions

Over the course of the last century, criminologists and penologists have implemented recidivism as the dependent variable of interest and, in some academic circles, scholars have theorized on the consequences of using various official measures of recidivism. These scholars have warned of the consequences of haphazard operational definitions of recidivism (Blumstein & Larson, 1971; Maltz, 1984; Ostermann et al., 2015; Willbach, 1942), but much of this attention has been on adult offenders in the criminal justice system and testing of these assumptions has remained elusive. We set out to apply this theoretical framework to the juvenile justice system and test these assumptions with an eye toward informing the decision on how best to operationalize recidivism from official measures for juvenile offenders.

This study produced useful and policy-relevant findings for juvenile justice. Our findings support the notion that using various operational definitions of recidivism produces unique recidivism risks for juvenile offenders. Offense date, referral date, and adjudication date are unique measures and, as demonstrated here, selection of the most appropriate recidivism proxy can have an influence on the Type I and Type II errors. Our findings also support previous concerns about operational definitions of recidivism and the relationship of predictor variables (McGovern et al., 2009). In some ways, the results suggesting a disparity of recidivism hazard when measuring adjudication (compared to offense and referral) may be indicative of the juvenile justice system and the parens patriae philosophy. For example, the findings suggesting that a juvenile experiencing depression / anxiety is no more or no less likely to have a new offense or a new referral, but is significantly more likely to be adjudicated could be an indicator that the juvenile justice system attempts to enhance the quality of life of the juveniles. Adjudicating the juvenile would permit the juvenile justice system to have more influence on the near future of the juvenile’s environment. Ultimately, how recidivism is operationalized matters when it involves overall risk of continued criminality and when concerned about the relationship between covariates and continued criminality.

While our findings fill important gaps in the literature on measuring recidivism within juvenile justice, there remain areas of research that would further clarify these issues. First, it would be informative for future research to better understand the consequences of mix matching standards of proof and dates. For example, Maltz (1984) suggested that the most appropriate official measure of recidivism is the combination of the arrest date with the standard of proof of an affirmative action by the prosecutor. This would require mix matching the arrest date and prosecutorial decision and then comparing that recidivism proxy to the other official measures. This approach may also help to narrow the gap around disparity of co-variates influence on recidivism measures. Studies focused on these issues would help to further close the gap on what is known about recidivism operationalization.

The value of the findings presented here to public policy are just as obvious as their value to the scholarly environment. The past decade has seen a merging of the scientific community and public policy to create an era of evidence-based practices. Evidence-based practices are those programs, actions, and approaches that are designed to reduce criminality and supported by scholarly evidence. The prominence of evidence-based practices in juvenile justice is testament to a belief in rehabilitation, a desire to be as efficient and efficacious as possible in program delivery, and consideration of the juvenile’s best interests. Understanding the limitations of measures of recidivism further permits those responsible for public policy and program evaluation to refine legislation and correctional programs to maximize juvenile justice sanctions and rehabilitation efforts and, ultimately, preserve public safety.