Despite the dramatic decreases in violent crime in the United States over the past 40 years, violent crime and firearm offense rates rose across the nation during the last decade (FBI 2017; Rosenfeld 2016). Specifically, violent crime in the US rose 6% between 2015 and 2016 (Friedman et al. 2017), while gun crimesFootnote 1 rose 17%, and homicides increased a stunning 23% from 2014 to 2016 (Asher 2018; FBI 2017). These increases translated to thousands of additional deaths and injuries nationwide. Given the devastating toll that these crimes take on victims and communities, violence and gun crime are a major concern for law enforcement, policymakers, and the public. Consequently, policing agencies have redoubled their efforts to reduce gun crime and violence in their communities.

Evidence-based violent crime prevention efforts have taken varied forms, ranging from place-based proactive police patrols targeting high-risk crime hot spots, as in the Kansas City Gun Experiment (Cohen and Ludwig 2003; Sherman et al. 1989; Sherman and Rogan 1995; Weisburd et al. 2004; see also McGarrell et al. 2001) to person-based deterrence strategies, such as Boston’s Operation Ceasefire, where law enforcement action, increased sanctions, and social services were directed towards specific high-risk offenders (Braga 2008; Braga et al. 2002; Braga et al. 2001; Braga and Weisburd 2012).

In a similar vein, Project Safe Neighborhoods (PSN), developed in 2001 by the US Department of Justice (DOJ), is an evidence-based initiative aimed at reducing violence and gun crime through the use of proactive policing, enhanced enforcement and sentences for repeat offenders, and a collaborative effort of police, prosecutors, probation/parole officers, community leaders, judges, and academics to identify, deter, and/or prosecute the offenders most responsible for the increased gun crime and violence (Dalton 2002; McGarrell et al. 2009; Papachristos et al. 2007). PSN is unique in that it draws upon several deterrence-based crime prevention strategies (e.g., focused deterrence, “pulling levers,” the Strategic Approaches to Community Safety Initiative (SACSI), Project Exile) to focus law enforcement and criminal justice resources on deterring and/or incapacitating the “individuals, social networks, locations, and contexts thought to be driving the violent crime problem” (McGarrell et al. 2010, p. 167). As studies consistently indicate that a small segment of people (and places) are responsible for the vast majority of crime, policing strategies that focus on preventing crime among prolific offenders, such as PSN, tend to have the greatest impact on crime reduction (Braga et al. 2001; Farrington 1995; Uchida and Swatt 2013).

While evaluations of PSN initiatives indicate these strategies are generally associated with moderate positive effects on violent crime reduction (McGarrell et al. 2010), questions remain about the magnitude of effects across PSN programs, which vary in context and application, and the duration of effects over an extended period of time. Specifically, most PSN programs have relied upon subjective or sweeping approaches to identifying the offenders that will receive the intervention, leading to heterogeneity in treatment effects, and potential mis-estimations of the impact of PSN on crime. This study builds on prior research by evaluating the impact of a new PSN program using an objective assessment to target offenders with a multi-faceted array of evidence-based risk factors on violence and gun crimes rates over an extended period using a quasi-experimental design. The research questions in this study are: (1) Does a new PSN initiative aimed at deterring and incapacitating objectively identified prolific violent and gun offenders associate with a decrease in violence and gun crimes rates in Tampa, and (2) what are the long-term effects of this PSN program on violence and gun crime?

Project Safe Neighborhoods Initiatives: Deterrence in Theory and Practice

In recent years, PSN initiatives have received considerable attention, as the approach has been associated with significant decreases in violence and gun crime across multiple evaluative studies (Decker et al. 2007; Hipple, Corsaro and McGarrell 2010; Hipple et al. 2007a, 2007b; McDevitt et al. 2007; McGarrell et al. 2007; McGarrell et al. 2010; McGarrell et al. 2006; see, however, Barnes et al. 2010). Underlying the PSN initiative are concepts from classical deterrence theory and behavioral economics, which suggest that “crime can be prevented when the costs of committing the crime are perceived by the offender to outweigh the benefits” (Braga et al. 2018: 210; Apel and Nagin 2011; McGarrell et al. 2010; Zimring and Hawkins 1973).

PSN strategies focus primarily on increasing the certainty and severity of punishments that result from committing crime (Papachristos et al. 2007), as these tend to be most strongly and negatively associated with future offending (Lochner 2007; Loughran et al. 2016; Matsueda et al. 2006; Nagin 1998; Paternoster 1987; Pogarsky et al. 2017; Pratt et al. 2006; Weisburd et al. 2008). Experimental research has indicated that “advertising” broad changes in policy relating to offenders’ certainty and severity of legal sanctions can directly impact their perceptions of risk, which then decrease the likelihood of future offending (Loughran 2019; Pickett et al. 2016). Both deterrence and incapacitation are used in PSN through the “advertising” and increased use of federal prosecution for illegal gun possession and qualifying violent federal offenses, more severe sanctions for these offenses, and more proactive policing efforts to increase the certainty of arrest of prolific offenders (McGarrell et al. 2010).

However, unlike general deterrence strategies, which aim to raise perceptions of risk of crime and its consequences for the general population, PSN aims to deter the specific individuals who have offended in the past and are at highest risk of re-offending (Cook 1980). Central to the PSN model is the proactive identification and prioritization of people that are the greatest contributors to violence and gun crime in the community (McGarrell et al. 2010). As such, PSN initiatives focus on prolific violent offenders and/or gangs, as these small groups are responsible for committing the vast majority of violent crimesFootnote 2 (Braga et al. 2001). Successful deterrence (or if necessary, incapacitation) of these individuals, by altering their behavior through changes in severity and certainty of punishment, can lead to the greatest crime reductions (Pogarsky 2002).

To date, PSN strategies have relied on a single-item criterion (e.g., gang member, repeat violent offender; Boyle et al. 2010; McGarrell et al. 2006; Papachristos et al. 2007; Tita et al. 2004) or subjective determinations (e.g., gang audits, consultations; Braga 2008; Braga et al. 2008; Braga et al. 2014; Corsaro and Engel 2015; Papachristos and Kirk 2015; Engel et al. 2010; Fox et al. 2015; Sierra-Arevalo et al. 2015) to identify the individuals to prioritize with their efforts. This study relies upon a new assessment composed of multi-faceted evidence-based criteria to objectively identify prolific violent offenders.

After identifying those at highest risk of being prolific offenders, PSN aims to disincentivize future criminal behavior by increasing the perceived certainty of apprehension and severity of criminal justice sanctions for those who re-offend (Bynum and Decker 2006). It should be noted that PSN efforts are not intended to lead to mass arrests and severe sanctions for all identified prolific offenders. Instead, it aims to alter the perception of certainty and severity of punishment, as these are more influential than the true likelihood of arrest and increased punishment when deterring offending (Geerken and Gove 1974; Jaynes and Loughran 2020; Loughran 2019). Indeed, research on decision-making tipping points indicates that even among prolific offenders, increasing the perceived certainty of apprehension among potential offenders up to 30% to 40% was sufficient to result in a reduction in offending (Loughran et al. 2012). Coupled with Nagin et al.’s (2015) assertion that police play a sentinel role in the messaging of deterrence elements such as certainty and severity, the value of PSN’s efforts to “update” prolific offenders’ risk perceptions, by active advertising of the program and potential follow-through on the certainty and severity of sanctioning, becomes clear (Anwar and Loughran 2011; Paternoster 2010; Wilson et al. 2017).

To do this, interventions implemented under the PSN model use a collaborative task force of law enforcement, prosecutors, criminal justice officials, community members, and academics to alter perceptions and calculus of risk among high-risk prolific offenders using strategies such as enhanced enforcement, stricter sanctions, proactive crime prevention, and the widespread advertising and evaluation of these policies (McGarrell et al. 2009; Papachristos et al. 2007). Together, these efforts aim to alter perceptions of apprehension certainty and punishment severity among those likely committing the most amount of crime, and result in the greatest amount of crime reduction.

Impact of PSN initiatives on violence and gun crime

PSN and related deterrence-based strategies have shown largely positive effects on crime and violence (Land 2015). For instance, an evaluation of PSN’s forebearer, Project Exile (Raphael and Ludwig 2003) found that the 40% drop in homicide rates in Richmond, Virginia during the program was significantly greater than that of comparable large US cities during the same time frame (Rosenfeld et al. 2005). Papachristos et al. (2007) found that the PSN initiative in Chicago, using federal prosecutions for prior felons illegally possessing guns, stricter sentences for targeted offenders who recidivate, reducing available firearms using proactive policing efforts, and marketing deterrence messages resulted in a 35% reduction in crime rates for the treatment neighborhoods over the 2-year period.

Moreover, by reducing the strain of “blanket policing” in communities most impacted by gun violence, gangs, and prolific offenders, through focusing police attention on the most high-risk repeat offenders and preventing violence and gun crime in these areas, PSN strategies have notable benefits for both crime reduction and police-community relations (Brunson 2015; Circo et al. 2019; McGarrell et al. 2006; Meares et al. 2009). This has led to the adoption of PSN as the U.S. Department of Justice’s archetypical model for local, state, and federal law enforcement agencies to combat violence and gun crime over the past decade (Braga et al. 2008; McDevitt et al. 2007).

Limitations to PSN initiatives and evaluations

Still, there are healthy concerns about the variability in magnitude and duration of desired effects stemming from the PSN model. For instance, Zimring noted that there has been a lack of evaluations of applied deterrence programs such as PSN, and that Operation Ceasefire is “more of a theory” rather than “a proven strategy” (Seabrook 2009, p. 39). In an analysis of policing programs with interventions directed at high-risk offenders, Weisburd and Eck (2004, p. 53) stated that the evidence for overall effectiveness at the time was “weak.”

Another issue to consider is the heterogeneity in treatment effects across evaluations, given the considerable variability and subjectivity in the processes used to identify prolific offenders subject to intervention in PSN initiatives. For instance, many PSN and deterrence programs identified individuals to receive the treatment based upon “crime incident reviews” and “gang audits,” which are subjective evaluations of future offending risk based upon police reports, street intelligence, and/or criminal associations via social networks (Braga 2008; Braga et al. 2014; Corsaro and Engel 2015; Engel et al. 2010; Fox et al. 2015; Papachristos and Kirk 2015; Sierra-Arevalo et al. 2015). Similarly, in Lowell, MA, the intervention targeted gangs and “impact players” believed to be most prone to engage in violence based upon police reports and intelligence (Braga et al. 2008).

Other PSN and deterrence-based programs identified prolific offenders with less subjective but broader criteria, such as targeting all those currently on probation or parole with histories of gun carrying/use involved in the drug trade and/or gang members (McGarrell et al. 2006), all gang members with warrants and/or probation or parole violations (Tita et al. 2004), all gang-involved individuals located in a 2-mi2 “Ceasefire Zone” (Boyle et al. 2010), and all offenders with a history of gun violence and gang participation recently assigned to probation or parole (Papachristos et al. 2007). In short, the method that PSN and related programs use to identify those considered to be prolific offenders appears to vary greatly, which likely impacts the effectiveness of these programs. Indeed, effect sizes for these types of programs ranged from d = 0.18 for the PSN initiative in Chicago (Papachristos et al. 2007) to 1.19 for the focused deterrence program in Lowell, MA (Braga et al. 2008).

There are also questions regarding the generalizability of PSN initiatives, as the majority of evaluations were in major US cities such as Boston (Braga et al. 2001; Braga et al. 2014; Piehl et al. 2003), Chicago (Papachristos et al. 2007), Indianapolis (McGarrell et al. 2006), and Los Angeles (Tita et al. 2003; Uchida and Swatt 2013). The validity of some findings have also been challenged, as less sophisticated analytical techniques were used in early evaluations, and re-analysis of the data with more advanced methods has occasionally led to differing results (see Levitt 2003; Ludwig 2005; Rosenfeld et al. 2005).

Finally, important questions have been raised regarding the duration of effects for crime reduction programs, such as PSN, over extended periods of time (Braga et al. 2014; Wellford et al. 2005). For instance, Nagin (1998) stressed the importance of assessing the long-term effects of deterrence-based programs, which may lose their effect over time. Braga et al. (2014, p. 136) similarly noted that there is “a growing body of literature suggesting that it is very difficult in practice to sustain these initiatives over an extended time period.” Indeed, a recent evaluation of the Kansas City No Violence Alliance (NoVA) program, a deterrence-based approach, found considerable heterogeneity in the program effects throughout the 3-year (2014–2016) follow-up period (Fox and Novak 2018). While a sizable reduction in homicides and gun crime was observed in the first year post-implementation, the effects diminished during the second year, and in the third year the homicide rates returned to pre-implementation levels and gun crime actually exceeded the original rate (Fox and Novak 2018).

Although PSN programs have generally been successful in reducing violence and gun crime, there are many questions regarding the best approach to identify who will receive the deterrence-based treatment, the generalizability of effects across locations, and whether PSN initiatives maintain effects over extended periods of time. Given the growing popularity of PSN, and the billions of dollars the U.S. Department of Justice has allocated to support these programs across the nation (McGarrell et al. 2010), it is increasingly important to assess the magnitude, treatment heterogeneity, and duration of PSN program effects on violence and firearms offenses.

Identifying prolific violent offenders using an evidence-based risk assessment

Like most of the USA, the city of Tampa (population: 360,0000) experienced a rapid uptick in violent crime between 2014 and 2015, including a major spike in homicides and firearms offenses (Behrman 2015). By 2015, Tampa had experienced its highest murder rate since the violent era of the “cocaine cowboys” in the 1980s (Marrero 2018). This led the Tampa Police Department (TPD) to develop a new strategy to reduce violence and gun crime in their jurisdiction. TPD’s strategy combined the PSN practices of increased enforcement and sanctions to deter prolific offenders with a new evidence-based risk assessment to more effectively identify prolific violent and firearm offenders that will be subject to the PSN intervention.

This risk assessment, called the Violent Impact Player List (VIP List), is based largely upon the scoring criteria used to identify chronic offenders in the Los Angeles LASER program (Uchida and Swatt 2013) and in line with recommendations on use of chronic offender lists to effectively reduce crime (Bynum and Decker 2006). The VIP List draws upon evidence-based risk factors weightedFootnote 3 to reflect the behaviors most predictive of future violence and firearm offending and was designed to help law enforcement effectively identify the “serial trigger pullers” at highest risk of committing additional gun crime and violence. Those who qualify as top “scorers” on the VIP ListFootnote 4, based upon the sum of the points for the behaviors in the scoring criteria, have the highest statistical likelihood of committing subsequent violence and gun crime (Uchida and Swatt 2013, pp. 292–293). Only those scoring above a 25-point thresholdFootnote 5, indicating the presence of multiple risk factors, were labeled as “VIPs” using the objective and stringent criteria. This limited group was subject to the PSN strategy aimed at deterring or incapacitating them from future offending through the use of enhanced enforcement and legal sanctions, proactive investigations, and messaging the increased certainty and severity of punishment.

Of the thousands of violent and gun offenders in Tampa (over 2000 violent and firearms offenses occur annually, Marrero 2018), less than 50 qualify for the VIP List at any given time. In sum, the following evidence-based risk factors were used to calculate offending risk in the VIP List (a complete discussion of these items is provided in Appendix 1):

  1. 1.

    Prior firearm offense/arrest—5 points if present

  2. 2.

    Violent criminal history—5 points if present

  3. 3.

    Gang member/affiliation within past 5 years—5 points if present

  4. 4.

    Probation or release from prison within past 3 years—5 points if present

  5. 5.

    Suspect in a shooting—5 points if present

    • Associate of suspect in shooting—3 points if present

    • Victim in shooting—1 point if present

  6. 6.

    Felony nonviolent arrests within past 2 years—1 point each

The VIP List has three primary advantages over prior methods used to identify prolific offenders in PSN and related deterrence-based violent crime prevention programs. First, all risk factors in the VIP List capture behaviors known to increase the risk of violent and firearm offending (Scott 2017). Alternative options used to identify prolific offenders in PSN programs rely on broader or more subjective criteria, such as all individuals in a gang or who committed a prior violent gun crime (which is overly broad) or relying on “audits” and “referrals” from officers or detectives (which is subjective). This will likely reduce perceptions of bias based upon group identity, particularly as gang membership alone is not enough to qualify for the treatment using the VIP List, and allows for a more strategic focus on prolific offenders even among violent offenders and gang members (Scott 2017). Similarly, as only behavioral factors within the offenders’ control were used in the scoring of the VIP List, all those with high enough scores to warrant police attention have “put themselves on the list.”

Finally, the VIP List was designed to identify those currently at highest risk of committing violence and gun crime, and therefore relies on only recent criminal behavior to predict future offending (Kurlychek et al. 2006). The benefits of this approach are that the VIP List identifies those who have a high volume of risk factors in recent years (versus over the entire criminal history), and it provides offenders an opportunity to remove him/herself from the VIP List entirely in a maximum of 5 years. This ensures that those who have desisted from offending and/or left a gang are not subject to police attention, and that valuable police resources are not spent on individuals at a lower risk of re-offending at the current time. Certain criminological risk factors such as impulsivity, delinquent peers, and community disorganization were not included in the VIP List, as police officers do not have routine access to these measures. However, the VIP List is an objective scoring system of evidence-based risk factors identifiable from police data, thereby increasing the validity and utility of the method.

In short, this new approach was developed to help law enforcement objectively and accurately identify on those at highest risk of being prolific violent offenders. Ideally, this will increase the effectiveness of PSN, as criminal justice resources are used to efficiently target those most likely to be repeat offenders based upon the presence of multiple risk factors, not subjective assessments or the presence of a single risk factor alone. Consequently, this model should be more effective in identifying those at highest risk of being prolific offenders, decrease law enforcement’s “net” in the community, and increase both the magnitude and sustainability of the crime prevention effects when using this evidence-based PSN approach.

Deterring VIPs using applied behavioral economics and interagency collaborations

All those identified as a high-risk of prolific violent and/or gun offender on the VIP List are subject to specific deterrence efforts using the PSN approach. Specifically, TPD’s Violent Crime Bureau and partnering federal agencies aimed to increase the certainty of arrest by allocating resources to collaborative proactive investigations and enhanced enforcement for serious crimes currently committed by VIPs (Scott 2017) and advertising this change in policy through the news media (see Marrero 2018) and officer communications with the VIPs. For instance, in 2016, TPD and the FBI conducted a proactive and resource-intensive investigation of a VIP engaged in a large-scale credit card fraud scheme that victimized dozens of people across the nation. This enforcement effort resulted in a federal conviction and a lengthy prison sentence and was unlikely to have transpired prior to the implementation of the PSN strategy. Follow-through on enhanced enforcement also assists with messaging and increases perceptions of certainty of arrest for other offenders in the VIPs’ networks.

Additionally, TPD worked closely with the U.S. Attorney’s Office (USAO), State Attorney’s Office (SAO), and Public Defender’s Office in Tampa to increase the severity and certainty of punishment when VIPs chose to re-offend. While judges are ultimately responsible for sentencing, TPD and the USAO increased sanction severity by opting to charge VIPs with federal crimes, which often result in longer sentences than state charges. For instance, the USAO agreed to prosecute all prior felon VIPs arrested for possessing a firearm, which corresponds to a sentence of 4 to 6 yearsFootnote 6. Similarly, the USAO and SAO are proactively assisting TPD with VIP cases to obtain higher quality of evidence, and ensure a greater likelihood of sanctions. Additional details on the PSN initiative and its implementation are described in Appendix 2.

In short, by developing policy and engaging in collaborations aimed at deterring and/or incapacitating those responsible for committing the largest proportion of violence and gun crime and using an objective and selective tool to identify those at highest risk of being prolific offenders, TPD successfully implemented a new PSN approach aimed at reducing violence and gun crime rates in their jurisdiction. This study evaluates the impact of this new PSN approach on violence and gun crime rates in Tampa, and the duration of the effects that are observed.

Current study

This study examines the magnitude and duration of effects of a new PSN initiative over an extended follow-up period in a mid-sized US city using a quasi-experimental design. Specifically, we compare gun crime and violence in Tampa, Florida to that of similarly situated local agencies over a 6-year (3 pre-implementation, 3 post-implementation) period. Based upon prior research on PSN programs, we posit two hypotheses for this study:

Hypothesis 1: The PSN initiative will be associated with a moderate and negative effect on violence and gun crime rates in Tampa in the post-test period.

Hypothesis 2: Effects of the PSN initiative will be highest in the first post-test period and will decrease in magnitude in the subsequent post-test periods.

By conducting this study, we are able to measure the magnitude and duration of effects of this new PSN initiative on violence and gun crime in order to shed new light on several unanswered questions in the policing, deterrence, and crime prevention literature.

Method

The randomized controlled trial (RCT) is the gold standard in evaluations, as random assignment is required to establish causality between a treatment and effect (Lösel 2008). However, RCTs are often difficult to implement given the ethical and practical challenges associated with randomization in policing (Braga et al. 2018) and generating a large enough sample when analyzing data at the agency level, such as the present study. Nonrandomized experiments are therefore also used to test for treatment effects, and may yield strong causal inferences depending upon the design. The nonequivalent groups design is the strongest of the nonrandomized experiments, as it includes all design features of the RCT except randomization (i.e., pre- and post-test measures, treatment, and control groups) and may also include additional covariates to improve equivalency between the treatment and control groups. While causality is not conclusively established, the additional design features are used to eliminate many confounds and closely mimic randomization (Fox and Farrington 2015).

This study utilizes a quasi-experimental design with multiple pre- and post-test measures of violent crime and gun crime rates, plus use of multiple control groups, covariates, and statistical methods to increase equivalency between the experimental conditions. This design qualifies as a level 3 design on the Maryland Scientific Methods Scale, as it assesses outcomes before and after the experimental treatment using multiple control groups selected for their similarity to the treatment group (Sherman et al. 1997). A level 3 design is sufficient to develop valid conclusions on treatment effects (Farrington et al. 2002).

Treatment and Control Groups

The unit of analysis in this study is the police department and associated jurisdiction, with treatment and control groups composed of the following law enforcement agencies: Tampa Police Department (TPD), Hillsborough County Sheriff’s Office, Polk County Sheriff’s Office, St. Petersburg Police Department, Ft. Myers Police Department, and the Orlando Police Department. These agencies were selected based upon their similarity on several factors, including geographic proximity (all in central Florida), agency location (touristic, coastal), population size, total police officers, total index crimes, violent crimes, and gun crimes in their jurisdictions, and total violent crime arrests each year. In this study, TPD served as the treatment (= 1) and the remaining five agencies were designated as control conditions (= 0).

Pre- and post-test measures

The total violent crime rate and gun crime rate in each agency’s jurisdiction before and after the PSN initiative was implemented were collected semi-annually (i.e., every 6 months) from the Florida Department of Law Enforcement (FDLE) and the US Bureau of Alcohol, Tobacco, Firearms and Explosives (ATF). Violent crimes include murder, robbery, and aggravated assaults. Gun crimes include all violent offenses committed with the use of a firearm.

The pre-test period covers the 3 years between 1 January 2013 and 31 December 2015 (= 1). The start date of the PSN intervention was 1 January 2016, and the follow-up period includes the 3 years through 31 December 2018 (post-test = 2). To address the second research question, changes in treatment effects within the post-test period are further analyzed in 1-year increments (2016 = 1, 2017 = 2, 2018 = 3).

All crime and covariate data were collected semi-annually for the participating agencies, in accordance to ATF and FDLE reporting guidelines, transformed into per-capita rates to facilitate uniform comparisons across agencies and approximate a linear combination of agencies without extrapolation (Bartos and Kubrin 2018). There are 72 total data points in the study with 36 points in both the pre- and post-test periods (i.e., 2 data points annually, for 3 years, across 6 total agencies).

No significant differences in the violent crime rate, gun crime rate, index crime rate, population size, and officer to citizen ratio in the pre-test period exist between the treatment and control agencies. Only a significant difference in violent crime arrest rates was found (p < .01, see Table 1), suggesting a strong level of equivalence between the conditions in the pre-test period (n = 36). Moreover, the agencies were purposively selected for features related to culture and location, which are difficult to measure using administrative data. Post-test data also indicate high equivalency across conditions (n = 36).

Table 1 Features of agencies in treatment and control conditions
Full size table

Causal identification analyses

To address the first research question and evaluate if the experimental treatment, the PSN initiative, led to a reduction in violence and gun crime, the synthetic control method (SCM) is used (Abadie and Gardeazabal 2003). The SCM approach to causal identification is increasingly utilized in aggregate-level evaluation research, such as assessing the effects of decarceration (Bartos and Kubrin 2018), right-to-carry laws (Donohue et al. 2019), legalized prostitution (Cunningham and Shah 2018), and state cigarette taxes (Abadie et al. 2010) on macro-level outcomes, as this method allows researchers to address the common problem of approximating what would have occurred in the experimental condition had it not received the treatment (i.e., the counterfactual).

SCM is an extension of the “difference-in-differences” (DiD) model, which estimates the effect of a treatment as the change in the distance between two time series for an outcome occurs post-intervention. However, DiD models assume that the treated and untreated groups follow “parallel trends” pre-intervention, but this may not be the case when examining large and diverse units such as states, regions, or agency jurisdictions (Bartos and Kubrin 2018). Consequently, SCM is designed to relax the assumption of parallel trends (i.e., equivalency) in the treatment and control groups in the pre-intervention period, in order to conduct aggregate level evaluations when perfect equivalency is difficult to establish. Through this process, SCM substantially reduces the risk of spurious effects by minimizing unaccounted for variation among nonequivalent groups when randomization is not possible (Abadie et al. 2010).

To fulfill this assumption, SCM synthesizes a control group with characteristics highly similar to the treatment group by weighting nontreated groups (i.e., the “donor pool”) across a series of control variables, in a process somewhat similar to propensity score matching. In short, some degree of initial nonequivalency between the treatment and donor pool is assumed for SCM; otherwise, the method would not be needed. In this case, the donor pool is populated with the five control groups that did not receive the treatment, using the aforementioned control measures as weights in the SCM to create the synthetic counterfactual (i.e., “Counterfactual Tampa”). This process allows SCM to create a “better comparison unit than any individual unit available that exists” (Bartos and Kubrin 2018, p. 700).

To estimate the effects of PSN on the violent crime and firearms offenses, the post-test outcomes for the treatment group (Tampa) and constructed synthetic control (Counterfactual Tampa) are compared, and “any difference between the two time series can be interpreted as the causal effect” (Bartos and Kubrin 2018, p. 701). The treatment and synthetic control groups are also evaluated in terms of the root mean squared prediction error (RMSPE) term, where any potential post-test gap that emerges beyond the range of the pre-test RMSPE indicates a significant and meaningful effect size.

To address the second research question regarding the duration of PSN treatment effects on violence and gun crime, a series of conditional zero-truncated negative binomial regressions are conducted (Long 1997). As no participating agencies had a time period where zero violent and/or gun crime took place, and the data are over-dispersed, the negative binomial models are zero-truncated to account for the lack of zeros in the count data. To measure the treatment’s effect on violent and gun crime rates by condition in each of the post-test periods, conditional models are estimated using all previously noted controls and a lagged measure of the dependent variable to account for the effect of the level of violence and gun crime in the previous period (k-1) on the subsequent period (k). These models, which parse out treatment effects in separate testing periods, assess the PSN treatment’s effect on violent crime and firearms offenses for each year in the post-test period. A z-test is then utilized to compare the magnitude of effects across the intervention periods (Paternoster et al. 1998). To do this, the z-test compares the unstandardized coefficients of a single variable across conditional models to determine whether there is a statistical difference among the observed effects (Gibson, Walker, Jennings, & Miller, 2010). The formula for the z-test is:

$$ z=\frac{b_{1-}{b}_2}{\sqrt{\mathrm{SE}{b}_1^2+\mathrm{SE}{b}_2^2}} $$

where b1 and SEb1 represent the coefficient and standard error from the first conditional regression model, while b2 and SEb2 correspond to the unstandardized coefficient and standard error from the second model. A z-test result of 1.96 or more, in absolute value, suggests that the observed coefficients significantly vary across the conditional models (Paternoster et al. 1998).

Finally, an estimate of the PSN treatment’s effect size is calculated for comparability to other PSN interventions. As this is a four-group (pre-post/control-treatment, PPC) experimental design, Monte Carlo simulations show that the most valid calculation for effect size is using the pooled pre-test standard deviation to weight the differences of the pre-post-test means for the outcome across the treatment versus control conditions (Morris 2008). The formula for PPC d is:

$$ {d}_{ppc2}={c}_p\left[\frac{\left({M}_{\mathrm{post},\mathrm{T}}-{M}_{\mathrm{pre},\mathrm{T}}\right)-\left({M}_{\mathrm{post},\mathrm{C}}-{M}_{\mathrm{pre},\mathrm{C}}\right)}{{\mathrm{SD}}_{\mathrm{pooled}}}\right] $$

where Mpre, T and Mpost, T represent the pre- and post-test means for the treatment group, while Mpre, C and Mpost, C represent the pre- and post-test means on the outcomes for the control group, respectively. The pooled pre-test standard deviation is calculated using the formula:

$$ {\mathrm{SD}}_{\mathrm{pooled}}=\sqrt{\left[\frac{\left({n}_T-1\right){\mathrm{SD}}_{\mathrm{pre},\mathrm{T}}^2+\left({n}_C-1\right){\mathrm{SD}}_{\mathrm{pre},\mathrm{C}}^2}{n_{\mathrm{T}}+{n}_{\mathrm{C}}-2}\right]} $$

where nT refers to the sample size of the treatment condition, and nC is the sample size of the control group. Finally, SDpre, T and SDpre, C represent the standard deviations for the mean values on the outcome measure in the pre-test for the treatment and control conditions, respectively. Finally, the bias correction can be calculated using:

$$ {c}_p=1-\frac{3}{4\left({n}_{\mathrm{T}}+{n}_{\mathrm{C}}-2\right)-1} $$

Results

PSN effects on violent and gun crime rates

To evaluate whether the PSN program had an effect on violence and gun crime rates in Tampa after the experimental treatment was implemented, as noted by the first research question, the SCM approach is utilized. Figure 1 presents the violent crime and gun crime rates for treatment condition (navy line) compared to the control conditions in the donor pool (gray lines), plotted out for the pre- and post-test periods (before and after the red vertical line, respectively). As can be observed, the plot line for the treatment group falls centrally among the control conditions in both models, further supporting that the control groups are close comparison donors for the development of a synthetic control group in the SCM. However, as the treatment and control conditions are nonequivalent in nature, the synthetic control groups are developed to more accurately estimate what would have happened to gun and violent crime rates in the treatment group, had the PSN treatment not been implemented.

Fig. 1
figure 1

Treatment and donor agencies in synthetic control model estimating the effect of PSN on violence and gun crime rates

Full size image

Synthetic control groups are statistically constructed in the SCM model using a convex of weighted values for each participating agency’s pre-test violent crime or gun crime rate, index crime rate, violent crime arrest rate, and officer to citizen ratio using the “synth” routine for Stata 13.1. Next, graphs of the crime rates for the treated group (Tampa) and SCM estimated synthetic control group (Counterfactual Tampa) are presented for the pre- and post-test periods. Any divergence in crime outcomes for the treatment and synthetic control group in the post-test period is then used to estimate the impact of PSN initiative.

As shown in Fig. 2, there was a notable divergence in the violent and gun crime rates for the treatment and synthetic control conditions in the post-test period. Specifically, results of the SCM indicate that the PSN program appears to have a sizable effect on violent and gun crime rates in the treatment condition, compared to what would have happened based upon this synthetic statistical model. Specifically, the post-test gap that emerged for the violent crime SCM (RMSPE = 0.14) is substantially larger than the model’s pre-test RMSPE (= 0.02). Similarly, the gap between the treatment and synthetic control groups for gun crime in the post-test period is double (RMSPE = 0.08) the gap in the pre-test period (RMSPE = 0.04). This indicates that the PSN initiative had an impact on these offenses beyond the range of matching error, and is significantly distinguished from zero when compared to what statistically should have occurred in the synthetic counterfactual condition.

Fig. 2
figure 2

Estimate effect of PSN on violent and gun crime rates in treatment and synthetic control conditions

Full size image

Duration of PSN effects on violent and gun crime rates over follow-up period

To address the second research question regarding the duration of the effects of the new PSN initiative over time, conditional zero-truncated negative binomial regressions were used to estimate the treatment effect on violent crime and gun crime rates in the pre- and post-test periods (2016, 2017, 2018), while controlling for index crime rate, violent crime arrest rate, officer to citizen ratio, and lagged violent or gun crime rates (see Table 2). Tests for multicollinearity indicated that all measures are independent (VIFs = 1.42–3.32). All models were statistically significant (p < .001).

Table 2 Zero-truncated negative binomial regression models evaluating the PSN treatment effect on violent and gun crime rates across intervention periods
Full size table

No significant difference between the treatment and control groups were observed in the inferential models for violent (b = −0.04) or gun (b = −0.07) crime rates in the pre-test period. The incidence rate ratios (IRRs) are used to illustrate the relative hazard of the treatment condition experiencing a higher count of cases in the outcome measures compared to the control groups. In the pre-test period, the IRRs are approximately 1.0, suggesting that there is little difference in risk of the outcome for the treatment vs. control conditions, as the incidence of violence and gun crime for each group in this period is about the same (IRRs = 0.96 and 0.93, respectively.)

In the first year after the PSN program was implemented (2016, post-test 1), the PSN treatment was associated with a large and statistically significant reduction in violent (b = −1.23, p < .001) and gun (b = −0.64, p< .01) crime rates in the treatment vs. control conditions. The IRRs suggest that the treatment condition had about a 70% lower risk of violent crime and 50% lower risk of gun crime than the control condition, after holding all other measures in the model constant (IRRs = 0.29 and 0.52, respectively). The z-test comparing the effect sizes from the pre-test to first post-test period show a strong and statistically significant difference in effects for both violent (z = 8.56, p < .001) and gun (z = 2.04, p < .05) crime rates.

In the second year of the follow-up period (2017, post-test 2), a significant reduction in the violent (b = −1.13, p < .001) and gun (b = −1.60, p < .001) crime rate for the treatment vs. control agencies is observed. Specifically, the treatment group had a 68% lower risk of violent crime and 80% lower incidence rate of gun crime, as compared to the control condition (IRRs = 0.32 and 0.20, respectively). The z-tests suggest the change in PSN effects for between the first and second post-test periods were significant for gun crime (z = 2.20, p < .05), but while sizable, not significantly different in magnitude for violent crime rates during these periods (z = −0.27).

Finally, during the third year of the intervention (2018, post-test 3), the use of the PSN program again showed a significantly lower rate of violent crime (b = −0.48, p < .01) but was not statistically significant for the violent gun crime rate (b = −0.50) for the treatment versus control groups, when controlling for all other factors. The third year effects did not significantly differ from the previous intervention period for violent crime (z = −1.52), but a significant reversal of effects for gun crime occurred between the second to third post-test periods (z = −2.06, p < .05).

Overall, the weighted effect sizes associated with the PSN program for violent crime and gun crime were d = −0.16 and −0.22, respectively (see Morris 2008), with a raw reduction of 24.4% in violent crime and 24.0% in gun crime rates in the pre- to post-test period in Tampa. Together, these findings indicate that the PSN treatment maintained desired effects at least 3 years post-test, although the magnitude of the effects vary (and potentially wane) over time, largely in support of hypothesis 2. Discussion of these findings occurs in the next section.

Discussion

The goal of this study was to determine the magnitude and duration of effects of a new PSN program designed to decrease violence and gun crimes in Tampa, Florida through the use of targeted deterrence and enforcement efforts for a select group of prolific offenders identified objectively using a multi-facted evidence-based approach. This evaluation was accomplished by using a quasi-experimental design with pre- and post-test measures of crime for the treatment (TPD) and control groups. Analyses were also conducted to evaluate the prolonged impact of the PSN initiative on violence and gun crime in Tampa over a 3-year post-intervention period.

Results of this study largely support both of the hypotheses regarding the magnitude and duration of PSN program effects, and suggest that this approach is associated with reductions in violence and gun crime rates over time using multiple analytical approaches. Specifically, treatment effects were found to be robust across all analyses, providing ample support that the PSN program was associated with significant reductions in violence and gun crime in the treatment jurisdiction. The results also indicate that the new PSN initiative was particularly effective for reducing firearms offenses. These findings raise three larger points for discussion.

First, it should be noted that the effect sizes in this study are consistent with those seen in similar deterrence-based policing initiatives. For instance, in a meta-analysis of 24 focused deterrence initiatives, Braga et al. (2018) found an average effect size of d = 0.38. Gang interventions showed the highest average effects (d = 0.66) followed by interventions among high-risk individuals (d = 0.23) and drug market initiatives (d = 0.09) (Braga et al. 2018). The effect sizes in this study (violent crime d = −0.16; gun crime d = -0.22) are similar to the average effects seen for these deterrence-based interventions for high-risk individuals, suggesting that use of the VIP List to identify prolific offenders may have a notable effect on violence and gun crime reduction, even several years after the initial program implementation. However, more evaluations of the PSN approach and VIP List should be conducted to determine the reliability of the present study’s results.

The findings support our second hypothesis that effects would be strongest during the first post-test period and decrease in magnitude in the later post-test periods. Specifically, the treatment effects of the PSN initiative on violent crime were largest in the first year, maintained through the second year, and then waned in the third year. While there was not a linear decrease in effects as predicted, and the effects of the PSN program remained significant and in the intended direction across all 3 years of the intervention, the slight decrease in effects in the third year provides partial support for hypothesis 2. For gun crime, positive effects were found immediately after implementation, and in contrast to our second hypothesis, the effects increased even further during the second year of implementation. However, the effect on both violence and gun crime decreased in strength by year 3 post-implementation.

While these findings are positive, it is even more notable that during the second post-test period, a serial killer in TPD’s jurisdiction (an extremely rare and tragic occurrence) committed four homicides using a gun over a 51 day time span. Although these murders accounted for less than 2% of the total gun crimes in Tampa during second post-test period, it also signifies the limitations of the PSN initiative, in that random gun violence is unlikely to be prevented through these efforts.

A second point of discussion is the potential presence of diffusion of crime control benefits from the treatment group to neighboring control group agencies (which did not use the PSN program). According to Clarke and Weisburd (1994), diffusion of crime control benefits refers the spread of the beneficial factors of an intervention beyond the places that are directly targeted. In the case of PSN programs, if an individual or group targeted by PSN desists from offending due to deterrence or incapacitation, it would reduce the crimes they commit in the treatment agency jurisdiction, and potentially reduce the crimes they commit in neighboring control group jurisdictions as well. It is also possible that members of the control group who witnessed the effects on peers subject to the PSN treatment may also experience a diffused deterrent effect. Braga et al. (2014) found this type of diffusion of crime control benefits in the second Operation Ceasefire study, and referred to it as “spillover deterrence” from the treatment to control groups. Spatial spillover deterrence effects, more akin to those in the present study, were found in related interventions among gangs in Los Angeles and Drug Market Initiative program in Nashville (Braga et al. 2018). The possible diffusion of crime control benefits from the current PSN program to control group agencies, who had a slight decrease in crime during the post-test period without the use of any known intervention, could indicate that this phenomenon may occur across the spectrum of PSN and related deterrence-based program types.

While certain control group agencies are in close geographic proximity to the treatment group (e.g., Hillsborough County Sheriff’s Office, St. Petersburg Police Department), others are up to a 2-hour drive away. This limits the spatial spillover deterrence effects that may have occurred, as offenders are unlikely to commit crimes in areas so far apart. Furthermore, other explanations for the decrease in violent crime in the control groups may exist, such as regression to the mean and external pressures on the agencies to also reduce violence and gun crimes using other strategies. Most notably, spatial spillover deterrence effects, if they did occur, appear to have only reduced the incidence of violent crimes in the control groups, as the number of gun crimes occurring in the control agencies still increased during the post-test periods, while they decreased during that time for the treatment agency (see donor crime rate trends in Fig. 1).

The third major point of discussion relates to the practical implications associated with the use of the PSN approach. A primary concern is whether TPD used the PSN program to “mass arrest” their way to lower crime rates, and related concerns regarding impact on the criminal justice system, the community, and perceptions of the department. To test this, an examination of the total arrests by TPD in the pre- to post-test periods was conducted. These results indicate that TPD reduced their total arrests from a pre-test mean of 17,271 down to 9491 in the final post-test period. Such a dramatic drop in total arrests suggests that TPD did not “mass arrest” individuals to reduce crime. Rather, it appears that TPD targeted a small group of people at risk of committing the highest number of violent and firearms crimes, and then successfully deterred or incapacitated them from committing future offenses.

Limitations and future research

Overall, this study is unique in that it evaluated the impact of a new high-risk individual-based PSN program implemented in a city without federal grant funding, and measured the duration of the effects over an extended (3-year) follow-up period, as deterrence scholars have specifically called upon researchers to do (Braga et al. 2014; Nagin 1998).

That said, like any other empirical evaluation, this study has its limitations. First, this was not a randomized experiment, opening the possibility that variables other than the PSN program could contribute to the observed reductions in violence and gun crimes seen in the treatment group over time (Braga et al. 2014; Corsaro et al. 2012; Papachristos et al. 2007). For instance, while efforts were made to collect data to properly match the treatment and control agencies on relevant measures, certain latent constructs such as organizational culture, command staff leadership style, resource availability, and others could still differ among the groups, and therefore have an effect on the results. However, as the treatment and the control agencies all experienced similar violent and gun crime rates at the start of this study and are similar on most other constructs relevant to these crime outcomes, and Tampa witnessed sizable and statistically significant drops in these incidents after implementing the PSN program, the potential for spuriousness in the findings is decreased. Nevertheless, it is essential that the first RCT evaluation of a PSN program take place, in order to truly eliminate the impact of spuriousness and determine the causal effects of PSN initiatives on violence and firearms offenses.

Another limitation stems from the fact that this study utilized official police incident data, which has potential issues such as the under-reporting of offenses and police decisions not to charge for certain crimes (Black 1970). However, given the severity and nature of the crimes analyzed in this study, these threats to the data are minimized. Moreover, while official data have flaws, they are one of the best resources for gaining meaningful insights on crime trends (Cook and Laub 2002; Schneider and Wiersema 1990) and the evaluation of crime reduction programs (see, e.g., Cohen and Ludwig 2003; McGarrell et al. 2001; Sherman and Rogan 1995).

Nevertheless, additional research should be continued in this area, particularly using a rigorous RCT design, so we may have an even more definitive answer regarding the causal effects of deterrence/applied behavioral economics on crime. Furthermore, future studies should examine the effectiveness of PSN programs that address other types of criminal activities (e.g., illicit drug sales, online child predatory offenses, and property crimes), to determine if this program has a similar effect with nonviolent offenders, and evaluated in rural and smaller urban areas to determine if PSN approaches are effective across diverse settings (Berg and DeLisi 2005). Researchers may also want to examine whether the addition of other empirically tested criterion could strengthen the VIP List’s predictive validity, and how other criminal justice technologies such as NIBIN, ShotSpotter, and Real-Time Crime Centers can effectively work in combination the current PSN approach.

Conclusion

This study examined the magnitude of effects associated with a new PSN strategy on violence and gun crimes using a quasi-experimental design, and whether the effects were sustained over an extended period of time. Results suggest that the PSN program was associated with an overall reduction in violent crimes (d = −0.16) and gun crimes (d = −0.22) over the entire post-test period. However, the effects for violent crime diminished over the 3-year follow-up, while the effects for gun crime remained relatively constant. This reduction in crime is estimated to have prevented more than 250 citizens from becoming victims of violence and gun crime in Tampa. This PSN approach should be evaluated by future researchers, and used as a foundation to develop more effective evidence-based strategies to prevent violence and devastating gun crimes across the nation.