The effects of the Summer All Out Foot Patrol Initiative in New York City: a difference-in-differences approach

Abstract

Objectives

The New York City Police Department’s “Summer All Out” (SAO) initiative was a 90-day, presence-based foot patrol program in a subset of the city’s patrol jurisdictions.

Methods

We assessed the effectiveness of SAO initiative in reducing crime and gun violence using a difference-in-differences (DiD) approach.

Results

Results indicate the SAO initiative was only associated with significant reductions in specific property offenses, not violent crime rates. Foot patrols did not have a strong, isolating impact on violent street crime in 2014 or 2015. Deployments on foot across expansive geographies also have a weak, negligible influence on open-air shootings.

Conclusions

The findings suggest saturating jurisdictions with high-visibility foot patrols has little influence on street-level offending, with no anticipatory or persistent effects. Police departments should exercise caution in deploying foot patrols over large patrol jurisdictions.

Introduction

For decades, police departments have deployed high-visibility foot patrols (Ratcliffe et al. 2011; Rosenbaum and Lurigio 1994). There are several potential goals for such tactics: interpersonal reassurance of the general public, deterrence of would-be offenders, and a broader emphasis on standing as an emblem of public safety (Piza and O’Hara 2014; Ratcliffe et al. 2011; Wakefield 2007). Foot patrols have often been characterized by practitioners and laypersons as a “proactive, non-threatening, community-oriented approach” (Wakefield 2007: 343). Yet, empirical evidence linking the influx of uniformed foot patrols to measurable reductions in crime and disorder is mixed and often weak (Ratcliffe et al. 2011). Still, foot patrol is often presumed by strategic planners in law enforcement agencies to be a useful application of police resources (Cowell and Kringen 2016).

In the spring of 2014, gun violence escalated in several jurisdictions throughout the city of New York. In response, the New York City Police Department (NYPD) instituted its first iteration of the “Summer All Out” (SAO) initiative, a 90-day intervention that deployed approximately 300 additional uniformed officers to several precincts and police service areas (PSAs) during the summer months. High-visibility, saturation foot patrols comprised the bulk of their assignments during the intervention phase. These extra officers were drawn from administrative positions—located at police headquarters and other off-site locations—far removed from participating jurisdictions. Upon conclusion of the initiative, members returned to their original assignments.

The SAO initiative began shortly after the second appointment of William J. Bratton as Police Commissioner. Though crime has declined substantially across the USA since the mid-1990s, these declines were particularly steep in New York City (Baumer and Wolff 2014; Zimring 2012; but cf., Krivo 2014), and Bratton had presided over much of this decline during his first tenure as Commissioner. Yet, prior to 2014, under Raymond W. Kelly, the NYPD had been broadly criticized for its overuse of formal legal sanctions (Eterno and Silverman 2012), which intensified further following Floyd, et al. v. City of New York, et al. (2013), where a federal judge found the NYPD’s widespread practice of “stop-and-frisk” constituted a policy of indirect racial profiling. Newly elected Mayor Bill de Blasio and newly appointed Commissioner Bratton sought to strike a tempered tone in policing inner-city communities. Thus, when reported gun violence pierced through many New York City neighborhoods in the spring of 2014, they sought a less intrusive approach than the classic police crackdown—the SAO initiative.

Police officials publicly announced the SAO initiative, giving explicit details to the mainstream media of the intervention’s duration and the precise patrol jurisdictions where SAO officers would police on foot. The initiative was heralded as crucial to the city’s ongoing violence-reduction strategy (see, e.g., Macedo et al. 2015). In the initial phase of the initiative, NYPD executives seized on the opportunity to report initial crime reductions—comparing monthly counts of reported crimes in a single intervention month with that same month in the previous year. Police officials continued to trumpet its seeming success after the program concluded that year, and it was reimplemented in successive years under the assumption it was a critical preemptive crime-prevention strategy.

At the same time, large-scale interventions of this kind—where many participating personnel are temporarily reassigned to special projects—can incur significant monetary costs. We estimate the SAO initiative costs the NYPD a minimum of $2 million in overtime funding each year. Please note: this total only includes the compensatory requirement for SAO officers to travel to and from their temporary assignments. It does not include costs associated with program design and officer retraining.¹

Unfortunately, we are unaware of any internal police reports or peer-reviewed scientific research evaluating the impact of the SAO initiative on crime and violence. Further, though some early research (Eck and Maguire 2006; Kelling et al. 1974; Manning 1977) suggested increases in the number of police on random patrol failed to have general deterrent effects, Nagin’s (2013: 235) review of current research suggests there is a consistent and substantial general deterrent effect of increased police presence on serious crime (see also Nagin et al. 2015). Thus, the present study aims to evaluate the effects of the SAO initiative on reported crimes and shooting incidents in 2014 and 2015. From a theoretical standpoint, empirically evaluating the efficacy of saturation foot patrols provides evidence of the crime-suppression benefits of officers serving a “sentinel-based” function (Nagin 2013; Nagin et al. 2015).

Policing and general deterrence

Police interventions aimed at general deterrence, such as the SAO initiative, rest on two crucial hypotheses. First, would-be offenders should logically weigh the costs and benefits of their actions before committing crime (Beccaria 1963[1764]; Becker 1968; Zimring and Hawkins 1973). Second, the criminal justice system has the capacity to alter the cost/benefit calculations of would-be offenders via the threat of formal punishments (Nagin 2013; Nagin et al. 2015), particularly the likelihood of apprehension (Apel 2013; Pratt et al. 2006). This in turn can cause individuals to change their behavior, deterring them from future criminal actions (Cook 1980). Crucially, policy interventions cannot directly manipulate laypeople’s subjective perceptions of sanction risk (Nagin 2013; Pickett and Roche 2016). Instead, policy intervention can influence the objective risk of apprehension in certain areas and times (e.g., by putting police in field), with the hope that individuals’ subjective perceptions will update accordingly. Deterrence is thus a theory of information and its transmission (Sherman 1990).

The SAO initiative was largely an exercise of police presence alone (see Kubrin et al. 2010; Ratcliffe et al. 2011). Unlike multifaceted “focused deterrence” programs (see Braga et al. 2018), there were no efforts to inform or incapacitate serious or chronic offenders (e.g., Braga et al. 2019; Corsaro and Engel 2015). Instead, sharp surges in the number of uniformed police visible to the public were intended to lead to increases in the objective risk of being apprehended. In theory, the high visibility of foot patrol assignments would cause potential offenders to notice this change and, in turn, cause them to update their subjective perceptions of the risk of apprehension and punishment. This would, in turn, lead to decreases in criminal activity (see Nixon and Barnes 2018; Pogarsky and Loughran 2016).

To do this, the SAO initiative relied on police officers to fulfill a “sentinel” role, using their presence to act as “capable guardians” of public spaces (see Cohen and Felson 1979; Eck and Weisburd 1995; Felson 1994, 1995) and thus discourage potential offenders from taking criminal action in the first place (Nagin 2013; Nagin et al. 2015). Some scholars have argued for a formal theory and typology of the effects police may have on criminal activity (Cullen and Pratt 2016). Nevertheless, others have found it sufficient to argue the police are especially suited to a sentinel role, particularly in areas where criminal opportunities are abundant and there are few private or informal capable guardians (Nagin et al. 2015; Marvell and Moody 1996). Indeed, the striking iconography of police officers (e.g., uniforms, badges, tactical equipment), and the direct risk they pose to would-be offenders as formal agents of social control, may have a far greater deterrent effect than any non-police agents or groups (Telep and Hibdon 2018: 19; but, cf., Kleck and Barnes 2014).

Police staffing

Several informal and formal evaluations of the effects of precinct-level staffing surpluses on crime have been conducted over the last 60 years. One of the earliest occurred in New York City in 1954.² This NYPD program nearly doubled personnel assigned to the 25th Precinct in East Harlem for a period of 4 months (Wilson 2013). In 1966, a study sought to quantify the impact of a 40% increase in police manpower on precinct crime rates over an 8-month period in New York City’s 20th Precinct while staffing in other precincts remained constant. Though not ideal field experiments, these two programs suggested augmenting manpower in a specific precinct might yield short-term crime control benefits (Press 1971). In particular, the 1966 study found appreciable net reductions in average weekly crime—most notably among street offenses—between low- and high-manpower periods, over and above changes in control precincts that did not experience a staffing increase (Press 1971).

As crime rates sharply increased during the late 1960s and early 1970s, the Kansas City Preventive Patrol Experiment called into question the efficacy of random motorized patrols as a viable deterrence strategy (Kelling et al. 1974; but, cf., Sherman 2013). Following this, two experimental evaluations suggested changes in foot patrol did not yield appreciable crime control benefits (Bowers and Hirsch 1987; Kelling et al. 1981). The first, as part of New Jersey’s Safe and Clean Neighborhood Act, involved the City of Newark implementing a broad foot patrol experiment (Kelling et al. 1981). The study assessed the impact of varying the dose of foot patrols across twelve experimental beats. Results indicate crime rates, as indicated by both police data and victimization surveys, were unaffected across the treatment beats. Although consistent with the public relations focus of neighborhood foot patrols, a later analysis found reductions in community members’ fear of crime (Pate 1986).

The second, in 1983, focused on the Boston Police Department’s implementation of a program that shifted officers away from random motorized response to foot patrols (Bowers and Hirsch 1987). These foot patrols were not found to be associated with reductions in citizen calls for service or reported crimes. Still, other studies have found modest support for foot patrols as a viable deterrence strategy. Beginning in 1978, the police department of Flint, Michigan, co-produced a grassroots foot patrol program with community stakeholders (Trojanowicz and Baldwin 1982). Net reductions in reported offenses were observed in ten of the 14 experimental beats.³ Additional benefits were observed with respect to calls for service, which fell by nearly half between these two time periods. Nevertheless, it should be noted many older evaluations were limited by issues of both measurement and statistics, such as the selection of inappropriate treatment areas (Ratcliffe et al. 2011; Sherman and Weisburd 1995).

Fortunately, there have also been more recent foot patrol interventions in major American cities. A randomized controlled trial of police foot patrols in Philadelphia circa summer 2009 found a significant reduction in violent crime after 12 weeks (Ratcliffe et al. 2011). Assessments of Newark’s 2008–2009 Project Impact initiative (Piza 2018; Piza and O’Hara 2014), which combined both proactive “enforcement actions” and more sentinel-like “guardian actions” of foot patrols, also yielded generally positive results. Piza and O’Hara’s (2014) results indicate total street violence was reduced. Further, incidents of murder, shootings, and non-domestic aggravated assault also decreased within the experimental area, with little evidence of displacement to other areas. Piza (2018) later found guardian actions in particular, such as business and bus checks as well as citizen contacts, conducted by foot patrol officers were associated with a decreased likelihood of robberies and overall violence.

Over time, the geographic distribution of crime has become a primary focus of researchers and practitioners seeking to understand the relationship between place-based strategies and crime (see Weisburd 2015; Telep and Weisburd 2018). Rather than deploying officers randomly or widely on foot or motorized vehicles, these recent policing strategies concentrated their efforts on specific places responsible for a large proportion of crime (see Braga et al. 2018). Deterrence at micro-places, or “hot spots,” is now well-established in the criminological literature (Braga and Bond 2008; Novak et al. 2016; Sherman et al. 1989). Much of this is the result of successful researcher-practitioner co-production, allowing for more robust experimental evaluations of place-based policing strategies. For instance, recent reviews of a large body of quasi-experimental and experimental studies of hot spots policing interventions suggest these strategies can provide statistically significant, albeit somewhat modest, reductions in crime (Braga et al. 2012, 2014; National Research Council 2004). Moreover, it seems unlikely these crime reductions were attributable only to displacement to adjacent areas (see Weisburd et al. 2006).

In the hot spots literature, the size of treatment areas and the tactics employed at those areas can vary significantly (cf., Sherman and Weisburd 1995; Chainey and Ratcliffe 2005). Still, while hot spots studies constitute a crucial adjacent literature with relevance to focused deterrence policing strategies, the SAO initiative, which the current study assesses, is not itself a hot spots policing intervention. First, while prospective hot spot areas do vary in size, the SAO’s treatment units span entire New York City precincts and are likely too large to fall under the classic conceptualization of a hot spot (Chainey and Ratcliffe 2005; Sherman and Weisburd 1995). Second, the SAO initiative was only a surge of sworn officers. It did not involve elements of problem-oriented policing, and thus bears little resemblance to hot spot interventions that analyze local problems to craft particular responses within problem areas (e.g., Braga et al. 2019; Telep and Hibdon 2018). Thus, the SAO initiative is better considered a more traditional foot patrol surge, an “unfocused community policing strategy with only weak-to-moderate evidence of effectiveness in reducing fear of crime” (Ratcliffe et al. 2011: 797; see also, National Research Council 2004). Nevertheless, some reviews of research suggest even unfocused policing activity can reduce criminal activity (Kubrin et al. 2010; Sampson and Cohen 1988; Wilson and Boland 1978). The current study assesses whether this holds for the SAO initiative.

Evaluation design

Study setting

Our study setting is the City of New York, the most populous city in the USA, home to approximately 8.5 million residents, more than a third of which are foreign-born. The city spans a geographic area of 302 mi² and encompasses five boroughs, which vary significantly in size and demographic composition. As a unique metropolitan environment with a heterogeneous population, local law enforcement must adapt to a multifaceted collection of demands voiced by community residents.

The New York City Police Department

The New York City Police Department (NYPD) is the largest municipal police department in the country, employing approximately 36,000 full-time sworn officers—more than twice as many officers as the second largest police department in the USA. Their members perform a wide variety of law enforcement functions related to public safety and security, counterterrorism, and emergency management. Our evaluation research is concerned principally with the NYPD’s Patrol Services Bureau (PSB), its largest operational bureau. Presently, PSB is divided into eight patrol boroughs, which are further subdivided into 77 police precincts. Though not perfectly precise, demarcated precinct borders comport reasonably well with the boundaries of established neighborhoods. The “precinct” jurisdiction is the primary aerial unit of observation.

The “Summer All Out” initiative

The present evaluation research is focused on the 2014 and 2015 iterations of the “Summer All Out” (SAO) initiative. Police officials designed the initiatives with the express purpose of augmenting the visible display of uniformed officers on foot within a subset of precinct jurisdictions. On average, the geographic area of a typical precinct jurisdiction is 3.8 mi². Residential precinct population varies widely, but a typical precinct ranges from 70,000 to 150,000 residents. Each participating SAO precinct received a contingent of approximately 20 SAO officers, on average.⁴ This would net a typical SAO jurisdiction with four to five additional officers on visible foot patrol per square mile during deployment hours, which is the equivalent to the addition of two uniformed officers per 10,000 precinct residents.⁵ Precinct staffing remained constant during the intervention phase; the SAO surplus was intended to supplement the precinct’s current patrol strength. Please note: when set against the background of typical American law enforcement operations, the SAO deployment was colossal. The NYPD’s subset of “surplus” foot patrol reinforcements, which represents only 1% of their total patrol contingent, exceeds the entire headcount of most law enforcement agencies in the USA, where three-quarters of all local police departments employ a total of thirty full-time sworn officers or less (Reaves 2015).

Precinct commanders deployed personnel on foot and were given latitude regarding deployment zones and scheduling.⁶ The magnitude of mobilized guardianship onto public street segments was not trivial, even if dispersed across rather large aerial units. Most precinct commanders deployed SAO officers on two semi-overlapping shifts to ensure coverage during the late evening and early morning hours. In general, NYPD patrol duty scheduling is organized into three 8-hour shifts; each shift is comprised of three squads, only two of which are present for duty on a given shift. Two “squads” are comprised of, on average, 26 to 32 officers. Thus, the average SAO contingent was a near 70% increase in patrol strength during peak crime hours.

SAO officers were instructed to maintain their visibility in shared public spaces and to exercise their “professional presence” whenever necessary. Members were not encouraged to initiate formal interactions with members of the public. Citywide enforcement contacts (e.g., arrests, summonses, Terry stops) declined monotonically throughout our observation period. In particular, “stop-and-frisk” reporting decreased three-quarters in 2014 compared with the previous year. Disengagement persisted again in 2015, where stop activity was half that of its previous year’s output.⁷ Reductions were particularly pronounced in participating SAO jurisdictions—a trend that persisted during the intervention phase. Among the jurisdictions selected in 2014, stop reporting was 12% lower during the intervention phase when compared with the previous 90-day period. A similar pattern emerged in 2015, where the average stop output of a typical SAO jurisdiction during intervention months was one-fourth of that produced in the previous 90-day period. The decline in Terry stops, even among the city’s high-crime jurisdictions was curious. In the years preceding the SAO initiative, NYPD officers heavily invoked this legally sanctioned practice to routinely intervene with suspected criminals in public domains.⁸

Further, SAO officers’ sustained presence on foot in relative isolation rendered them mostly unavailable to field calls for police service (cf., Wood et al. 2014). It is tactically unsafe to respond to emergency 9-1-1 service requests, unassisted, while on foot. Internal policy indicates the NYPD typically deploys “two-officer” motorized police vehicles to perform patrol-related duties. It is rare to observe uniformed police resources responding to service requests unaided.

It is also worth highlighting that many SAO officers were previously assigned to relatively coveted administrative positions. A large proportion of the selected officer contingent were seasoned members and perhaps less likely to exhibit a crime-fighting orientation. Prior research has demonstrated that a penchant toward crime-fighting begins to wane after 5 years of service (Brown 1988). Moreover, walking a police beat has been widely regarded as more difficult than standard vehicular patrol and more traditionally viewed as form of punishment (Wilson and Kelling 1982). Once the intervention concluded, the mainstream media received chronicles of discontentment among the senior rank-and-file; displeased with their reassignment to foot patrol, some SAO officers attempted to evade deployment by feigning injury or filing for early retirement.⁹

In light of our understanding of the SAO blueprint, the present paper offers an exclusive assessment of the swift intensification of visible guardianship in a subset of sanctioned patrol jurisdictions. The SAO model’s strict sentinel-focus is most consistent with a “watchman” style of policing, whereby the principal function of the officer is the maintenance of order (Wilson 1968). As noted earlier, the timing of the initiative coincided with unprecedented levels of formal disengagement with the community.

The first iteration of the SAO initiative began on July 7, 2014, and concluded on October 4, 2014. Nearly 300 uniformed officers were reassigned from administrative positions and apportioned to ten participating precincts for the 90-day intervention phase.¹⁰ At the conclusion of the SAO initiative, the additional officers were pulled back and returned to their previous assignments. Except for the 113th Precinct in Queens, all precincts were located in Brooklyn or the Bronx.

The second iteration of SAO initiative commenced on June 6, 2015, and concluded on September 5, 2015. Again, the department reassigned nearly 300 uniformed officers to ten participating precincts for the 90-day intervention phase. With the exception of the 120th Precinct in Staten Island, all were again located in Brooklyn or the Bronx. Figure 1 shows the precincts that received the intervention in 2014 and 2015, seven of which received the intervention in both years.

Fig. 1

Selected precincts for the “Summer All Out” (SAO) Initiative on crime and violence

Analytic framework

Reviews of weekly CompStat reports suggest department executives chose precincts for the SAO initiative on the basis of their high count of total reported shooting incidents. To minimize potential selection bias associated with program implementation, models were estimated using a difference-in-differences (DiD) framework. Under this specification, changes in monthly crime rates and monthly shooting counts for precincts subjected to the intervention were compared with the changes in non-participating precincts where the intervention was absent. Precinct jurisdictions selected to implement foot patrols for the 90-day treatment phase serve as our treatment group.

While SAO precincts were typically densely populated territories with disproportionately high crime rates, these treated precincts were not the only areas experiencing a surge in violence during the summer months of 2014 and 2015. Other areas of the city, most particularly the northern sections of Manhattan and southern sections of Queens also exhibited elevated rates of crime and a high concentration of gun violence, but were never subjected to the program.¹¹ Other non-participating jurisdictions with comparable crime and disorder concerns were also located in geographically proximate regions of the city, such as within the same borough. Thus, we designated all non-program precincts as our control group.¹²

One of the principal identifying assumptions of the DiD estimation strategy is the visual exhibition of parallel group trends prior to treatment (Blundell and MaCurdy 1999; Meyer 1995). Put differently, the design assumes trends in outcomes between treatment and control groups would move in tandem in the absence of program exposure (Gertler et al. 2016). Trend equivalence before a policy change is often implicitly assumed, yet the visual observation of any parallelism, or divergence, in trend is amenable to explicit empirical testing should serial observations of crime outcomes exist prior to treatment.¹³ Most of New York City’s major crime indices, in general, exhibit low volatility over time across treatment and control jurisdictions.¹⁴

Figure 2 depicts monthly trends in New York City’s violent and property crime indices, separately, by year.¹⁵ We juxtaposed SAO adopters with all citywide jurisdictions in the months surrounding the exposure period. Shaded regions highlight the 90-day intervention phase in both years. As indicated in Fig. 1, a discrete subset of jurisdictions were treated across years, though many previously treated precincts received the intervention more than once. Each trend line represents the unique evolution SAO and non-SAO adopters in each treatment year.

Fig. 2

Aggregate violent and property crime trends, by SAO Adoption Year

Comparing the unconditional evolution of mean per capita crime trends between SAO and non-SAO jurisdictions aids the assessment of parallel trends before the foot patrol surge. Pre-treatment crime trends exhibit a visual stability before the onset of SAO guardianship; month-to-month volatility is relatively low pre-intervention. Program exposure appears to induce a transitory deviation from this common pre-treatment trend. Surprisingly, average violent crime rates in program precincts deviate upward in the 2 months post-implementation of the 2015 SAO initiative.

Data

Crime data were obtained from the NYPD’s crime data warehouse via their open data portal. The full dataset consists of the population of felony and misdemeanor crime complaints (n = 2,134,067) between 2012 and 2016. We also secured, separately, the population of shooting incidents (n = 5,783) for the same period.¹⁶ A “shooting incident” as defined by the NYPD is an adversarial exchange resulting in at least one person being wounded by a fired bullet. These incidents are not merged with the felony and misdemeanor data in order to calculate a traditional crime rate.

Note: shooting incidents are crimes in and of themselves, representing acts of felonious assault or manslaughter, though they may be a consequence of the commission of other violent crimes. Thus, in order to avoid double counting, we model shooting incidents separately as a Poisson process. Shooting incidents may also be a reliable barometer of the violence occurring within a jurisdiction. These events are difficult to underreport should victims require hospital treatment.¹⁷ In addition, a substantial proportion of shooting incidents occur in visible public spaces. Our data suggests shooting incidents are not randomly distributed about the city. Their incidence is typically clustered in neighborhoods already fraught with crime.

We proceeded by winnowing the available pool of offenses to only crimes likely to be influenced by the widespread distribution of foot patrols in public spaces. First, we kept only those offenses where the local precinct was listed as the investigative authority. Second, we restricted attention to “street” offenses. One-third of either felonious or misdemeanor crimes investigated by the local precinct occur on visible street segments immediately navigable by uniformed patrolmen. In half of all “street” crimes, officers also specified a “Location Type” for the reported offense. Of those, 9 out of 10 reported street-level crimes occurred either “in front of” or “opposite of” the incident location. Our selection criteria reduced the pool of offenses to 698,222 crimes available for analysis. Official crime reports were aggregated by month and normalized by the residential precinct population size.

The shooting data is similarly restricted to those incidents occurring in public domains. Gun violence within city limits generally materializes in shared, communal public spaces where residents can freely travel. Of all precinct shootings observed over the 5-year period, 7 out of 10 occurred in open-air settings. Our selection criteria reduced the pool of incidents to 4012 shootings available for analysis.¹⁸ Aggregated events thus represent the total monthly count of open-air shooting incidents reported by the police.

Models for crime indices

The NYPD monitors their major crime indices by comparing jurisdictions with themselves in the previous year. Put differently, for any precinct and year, the epoch under evaluation (e.g., 28-day cycle) is typically matched with its counter-epoch in the preceding year. Impact evaluations in this context can thus be achieved using month-to-month comparisons of crime counts (e.g., comparing violent crime in July 2013 with that in July 2014). In essence, precincts serve as controls for themselves.

The shortcomings of this evaluation approach are apparent. Index crime, in general, has been trending downward since the mid-1990s.¹⁹ If counterfactuals are not temporally aligned, then average differences in index crime between jurisdictions exposed and unexposed to treatments could be due to longstanding maturational trends in New York City’s major crime indices. Thus, any net reduction in index crime counts could be due to secular declines in crime over the last two decades.

To overcome these concerns, we used a DiD approach to estimate the impact of the SAO initiative on logged monthly crime rates.²⁰ The effects of SAO exposure on crime rates were estimates via least squares. Model 1 takes the form of the classical DiD specification:

$$ \log \left({y}_{pt}\right)=\alpha +\gamma {Treat}_p+\lambda {Post}_t+\delta \left({Treat}_p\times {Post}_t\right)+\theta {X}_{pt}+{\varepsilon}_{pt} $$

where the left-hand side denotes the logged per capita crime rate for precincts p in months t. The variable Treat_p is a group-level indicator equal to unity if precinct p was selected into the treatment group, 0 otherwise. Standardized treatment months are represented by the time dummy Post_t, indexing post-treatment months in both treatment and control groups.

In the 2014 SAO initiative, post-treatment months were July, August, and September; only the 6 months pre-treatment were compared with the 3 months post-treatment. In the 2015 iteration, post-treatment months were June, July, and August; accordingly, we only compared the 5 months pre-treatment with the 3 months post-treatment. Months beyond conclusion of each iteration were ignored. Interacting the group and time indicators gives us an estimate of δ, the treatment effect for precincts p that received the surplus manpower during treatment months t. The variable X denotes precinct-level controls.²¹

Model 1 isolates the impact of the SAO initiative on crime by estimating separate DiD models in only those years when the intervention was in effect; this may be insufficient for two reasons. First, we assume the initiative has an immediate and constant impact on crime with no further effects; this assumption may be impractical. Instead, program effects may grow and/or decay in treatment withdrawal periods. Second, we assume precinct selection was independent of pre-existing crime trends. When subsets of patrol jurisdictions receive interventions on the basis of a transitory spike in violence, then earlier models could potentially overstate effects should crime revert to pre-program levels in the post-period. To overcome these concerns, we estimate a second equation that explicitly models the movement of crime before and after each 90-day treatment window. Pooling years 2012 to 2016 together accounts for all iterations of the SAO initiative in one equation. The 5-year panel results in over 60 “month-year” observations. Model 2 is a “generalized” DiD equation adapted from Bertrand et al. (2004) and is estimated via least squares:

$$ \log \left({y}_{p,t}\right)={\varphi}_p+{\omega}_t+\sum \limits_{j=-2}^1{\delta}_j SA{O}_{p,t+j}+{\varepsilon}_{pt} $$

where φ_p and ω_t are fixed effects for precincts and months, respectively. Estimation of the right-hand side fixed effects was achieved using a series of P − 1 indicators for precincts and T − 1 indicators for months. Treating precinct- and month-specific effects as parameters to be estimated is algebraically equivalent to estimation in deviations from means (Angrist and Pischke 2009). The “generalized” DiD model is a two-way fixed effects estimator and is amendable to settings with intermittent program exposure over time. Precinct fixed effects control for unobserved time-invariant precinct-level heterogeneity; month fixed effects account for common temporal shocks, absent the SAO intervention, affecting all precincts. Such a model systematically accounts for any omitted cofounders that are fixed over time and across precincts.

The treatment variable SAO_pt represents our interaction term from earlier (i.e., Treat_p × Post_t). Since the post-period is not standardized across years and a different subset of jurisdictions were treated in each iteration of the program, then our policy indicator must be instantiated manually to account for this interaction. Thus, SAO_pt is a treatment dummy equal to unity if a precinct was in the treatment group (i.e., either 2014 or 2015—or both) and in a post-treatment period, 0 otherwise. SAO_pt does not demarcate a specific treatment group; rather, it is a discrete indicator for program precincts during program months. Such a policy indicator can thus switch on and off depending on the intervention phase and offers a more flexible modeling strategy than the classical DiD model. Two iterations of the SAO initiative were instituted and removed over the 60-month observation period. Merging all years together allows us to account for more of the systematic variation in crime outcomes over time.

Note: this model is not restricted to assessing static treatment effects. In fact, the right-hand side summation parsimoniously specifies the estimation of static and time-varying treatment effects. Each δ is subscripted to index the j-th additive treatment effect, such that −2 ≤ j ≤ 1, inclusive of 0. We estimate one lead and two lags of the SAO policy variable. Notice, as we increment along this summation interval, we adjust the full time configuration by j. Our lead parameter (i.e., δ₁) estimates an anticipatory effect (see Smith et al. 2002); lagged parameters (i.e., δ_−2,, δ₋₁) estimate persistent effects. Our static parameter δ (i.e., δ₀) is still our causal estimand of interest and captures the immediate effect of treatment.

DiD assumptions would imply that δ_j = 0 ∀ j > 0. Consistent with a Granger causality test, future SAO_pt should not predict y_pt (Granger 1969). Put differently, consequences should not precede causes (Angrist and Pischke 2009). In our setting, however, we may observe general deterrent effects preceding program exposure. To illustrate, macro-level changes in police presence are often determined at the top of the NYPD administration, and large personnel deployments typically accompany of a formal announcement of the initiative’s purpose. Press releases surfaced approximately 1 month prior to the commencement of the SAO initiative. Substituting SAO_{p, t + 1} into the full model will help capture any anticipatory behavior within program precincts in the month immediately before SAO adoption.

Assessing possible anticipatory effects is of substantive interest when program implementation involves a time gap between its announcement and ultimate effective date (Wing et al. 2018). Preemptive crime-suppression benefits may be a consequence of the early reporting of each initiative’s start date.²² However, a “zero-bounded” coefficient on a leading treatment variable has its methodological benefits. Failing to observe emerging ex ante changes in our outcome before the onset of the intervention supports claims of common trends before the surge in street-level guardianship. Moreover, estimates of δ_j, j ≤ 0 may persist or deteriorate with the passing of the initiative. Our goal of lagging the SAO treatment dummy is to investigate potential lasting effects once the initiative concludes. The first and second lag captures any “carry-over” effects in the months beyond program termination.

Our decision to express potential outcomes as a function of time-varying treatments, e.g., y_pt(SAO_{p, t + 1}, SAO_pt, …, SAO_{p, t − 2}) was not arbitrary. Applied researchers typically substitute “future” or “past” versions of a discrete treatment variable into their specification.²³ Still, our approach may be insufficient to identify the dynamics of treatment throughout each transient SAO exposure phase. Later, we will fully explore the relevant time dependencies by interacting a year-specific treatment indicator with period (i.e., month) dummies specific to SAO and non-SAO jurisdictions.

Robust variance-covariance estimators were used for all models employing a DiD estimation strategy. One-way “precinct-level” cluster robust standard errors were estimated for all classical DiD models with two groups and two discrete time periods.²⁴ However, in the more general DiD setting, we double cluster on precincts and months (Cameron et al. 2011; Petersen 2008; Thompson 2011). Double clustering along the cross-sectional and longitudinal dimension accounts for within-cluster dependence and is amenable to settings with geographic-based spatial correlation.

Models for shooting incidents

The NYPD tracks shooting incidents separately from its major crime indices; this is partly due to their severity. We view shooting incidents as a crucial outcome of interest since it appears the NYPD administration used this metric as a criterion for precinct selection. The low cardinality of shooting events across time and space does not lend itself to estimation via least squares. Poisson regression models were used to estimate the effect of the SAO initiative on shooting outcomes, which we modeled separately from the NYPD’s major crime indices. The following two specifications proceed in a similar fashion and only apply to the shooting data. First, we model the immediate pre- and post-treatment months, separately, by year. We then follow with the “generalized” DiD approach which estimates all iterations of the SAO initiative in one equation.

Shooting outcomes show more volatility across time than the NYPD’s standard crime metrics. To proceed, we explicitly sought to model trend divergence in monthly shootings across treatment and control jurisdictions. Model 3 is an adaptation of the classical DiD specification adjusting for calendar effects in the months before the onset of each SAO initiative:

$$ \log \left({y}_{pt}\right)=\alpha +\gamma Trea{t}_p+\sum \limits_{\iota =-k}^{-1}{\eta}_{\iota }{Pre}_t^{\iota }+\lambda {Post}_t+\sum \limits_{\iota =-k}^{-1}{\rho}_{\iota}\left( Trea{t}_p\times {Pre}_t^{\iota}\right)+\delta \left( Trea{t}_p\times {Post}_t\right)+\theta {X}_{pt}+\log \left[\frac{n}{10,000}\right] $$

where the variable ${Pre}_t^{\iota }$ denotes a series of pre-intervention period dummies for each individual calendar month preceding program exposure (Ryan et al. 2015; St. Clair and Cook 2015). Each estimate of η_ι represents the individual additive effects of disjoint time periods relative to the intervention, such that −k ≤ t ≤−1. Each estimate of ρ_ι (i.e., ρ_−k, …, ρ₋₂, ρ₋₁) is obtained via the pairwise product of all pre-intervention period dummies with the main treatment indicator. The lower limit k varies depending on the intervention phase under evaluation. The interval is reflective of the k periods approaching treatment in a single program year, with ρ₀ serving as the baseline estimand.²⁵ The model excludes distant pre-event data that are more that k periods before the intervention.²⁶ Estimates of ρ_ι should be jointly indistinguishable from zero. Relative estimates close to the initial treatment may be plausibly bounded away from zero, in theory, and suggests possible anticipatory effects (Smith et al. 2002). However, in the absence of a theoretical framework for anticipation, any strong non-zero effects in the pre-treatment epoch would cast doubts on our assumption of trend equivalence and could be interpreted as selection bias (Lechner 2011).

Moreover, the natural logarithm of the residential precinct population size is incorporated into this specification as an offset rather than a covariate. The variable enters the model on the right-hand side but with its coefficient constrained to equal one. Algebraically maneuvering the exposure variable to the left-hand side and invoking the properties of logarithms results in an analysis of rates instead of counts. Though estimates are unlikely to vary drastically should the “absence” of shootings predominate across precincts and months, standardizing by “population-at-risk” nonetheless acknowledges the greater precision of rates in the face of larger population sizes (Osgood 2000).

Next, we follow with a separate fixed effects Poisson specification pooling all available years together. The 5-year panel results in over 60 “month-year” observations. Again, we explored the effects of all iterations of the SAO initiative on open-air shootings. Our time-varying treatment structure is the same as before with one lead and two lags of the main policy variable. As a final measure, we explored the sensitivity of the shooting data to longstanding precinct-specific time trends. New York City precincts have largely experienced a shared, secular decline in gun violence over the last 20 years. Much of this reduction in firearm-related violence predated the SAO initiative, affecting jurisdictions most beleaguered by gun violence. Acknowledging this slow-moving shooting trend, we fit Model 4 which takes the following form:

$$ \log \left({y}_{p,t}\right)={\varphi}_{0p}+{\varphi}_{1p}t+{\varphi}_{2p}{t}^2+{\omega}_t+\sum \limits_{j=-2}^1{\delta}_j SA{O}_{p,t+j}+\log \left[\frac{n}{10,000}\right] $$

where, as before, φ_0p denotes fixed effects for precincts. Here, φ_1pt and φ_2pt² denotes the interaction of each precinct-specific effect with a linear and quadratic time trend, respectively. Multiplying by these trends absorbs any differential pre-existing shooting trajectories between SAO and non-SAO jurisdictions. Note: each additional component of this Poisson equation exists as before, with the exception of the exposure variable.

Results

Crime indices

Table 1 reports estimates for 2014 and 2015, separately. Columns indicate outcomes for various composite and disaggregated crime indices. Model 1 only reports estimates of δ, the coefficients on the interaction terms. Results suggest the 2014 SAO initiative did not appreciably influence street-level crime rates. Its first iteration was not associated with a reduction in the violent crime rate, or more specifically, rates of assault and robbery. The 2014 SAO initiative was associated with a significant reduction in per capita property crime, though this decrease was small—a net average decrease of three felonious property crimes per 100,000 residents. Estimates suggest heterogeneity of effects, though drug and weapon-related crimes are primarily arrest-generating offenses. In addition, some evidence suggests the reporting of misdemeanor and petty offenses increased during the evaluation period.

Table 1 Effect of the “Summer All Out” SAO initiative on crime

DiD estimates for the 2015 SAO initiative were similarly weak and statistically insignificant. Mirroring program effects in the previous year, null findings were observed across all violent and property crime indices. In sum, results from year-specific DiD models suggest the SAO initiative did not have an appreciable, isolating impact on violent street crime during the summer months in any particular year.

Model 2 was fit to the full 5-year panel which investigates the effect of both iterations of the SAO initiative on several crime outcomes. To avoid extraneous output, dummies for precincts and months were omitted from tabular results. With the exception of property crime, we do not find evidence of an immediate effect of the SAO initiative on precinct-level street crime. Leading effects for drug and weapon-related offenses were significant and positive, which is not entirely surprising given the seizure of contraband accompanies many arrests. We interpret this effect with caution, as other pre-intervention arrest-generating activity may be responsible for any increase in crime reporting.²⁷ Violent crime, in particular, does not vary with program implementation. Disaggregated rates of robbery and assault are statistically indistinguishable from zero. We also did not find any evidence suggesting a lasting impact of the SAO initiative on violent street crime beyond program termination.

Burglary offenses were the only crimes where we did not restrict attention to “street” incidents.²⁸ Two-thirds of burglary reports for the period under study list “private house” or “apartment” (i.e., multiple-dwelling) as the premise type. Their associated location type is often listed as “inside of” or “rear of” these physical structures. High-visibility foot patrols should thwart, or arguably displace, the commission of particularly stealthy offense types such as burglary and motor vehicle theft.²⁹ In all models, results suggest the widespread distribution of uniformed officers on foot within precinct jurisdictions does not influence general rates of burglary offenses or street thefts of motor vehicles.

Shooting incidents

Table 2 reports results from Model 3 only, which explored the effects of the SAO initiative on open-air shootings in each year, separately. The estimate reported in Column 2 shows the 2014 SAO initiative produced a significant treatment effect. Exponentiation of our 2014 estimate for δ implies the 2014 initiative was associated with a 36% reduction in the expected rate of total reported shootings (i.e., e^−0.45 = .64). The 2015 coefficient also suggests negative program effects in its subsequent iteration, though the effect in this year is substantially weaker and less precisely estimated. However, adjusting for calendar effects in the periods before SAO adoption dampens any observed effects (see Column 3). The 2014 coefficient in Column 3 is less precisely estimated once we explicitly model the periods approaching program exposure. Similarly, the 2015 estimate in Column 3 is also less precise and substantially smaller in magnitude. Results from Model 3 suggest that SAO adoption was, to a certain degree, partially determined by gun violence in the months prior to its implementation.³⁰ Once we flexibly adjusted for shocks in the calendar months pre-dating the foot patrol surge, the effect of the SAO initiative on shootings was indistinguishable from zero. Column 3 also reports individual post-exposure period effects beneath the main interaction. We explore the dynamics of SAO exposure in greater detail in the next section.

Table 2 Year-specific Poisson estimates of the effect of SAO initiative on shooting incidents

Note: year-specific equations focus on the subset of months immediately before and after the onset of SAO guardianship. Table 3 reports results from Model 4 which assesses the effect of the intermittent implementation of all SAO initiatives on open-air shootings over the 60-month observation period. Poisson estimates were smaller in magnitude once we allow for the estimation of fixed effects for precincts and months. Under this specification, we did not find evidence of an immediate effect of the SAO initiative on shootings. Further, we did not find evidence of gun violence abating in anticipation of police officials’ promulgation of the SAO initiative’s effective dates, nor did we find evidence suggesting a residual impact of treatment in months beyond program termination.

Table 3 Poisson estimates of the SAO initiative’s effects on shooting incidents, 2012–2016

We also explicitly modeled cross-group diffusion via dummies for bordering jurisdictions never subjected to the SAO initiative. A positive coefficient suggests a spatial displacement of gun violence into neighboring jurisdictions. A negative coefficient suggests a diffusion of deterrence which carried over into non-SAO precincts. Lagging the variable by one period investigates any potential delayed effects in contiguous jurisdictions. Neither the neighbor dummy nor its lag produced a treatment effect. Effects were also small in magnitude.

Observed null effects for shooting incidents were robust to the inclusion of precinct-specific linear and quadratic time trends. Though computationally demanding as a modeling strategy, it buttresses claims of trend equivalence in years pre-dating the SAO initiative. Negligible differences in our point estimates demonstrate the insensitivity of results to alternative specifications.

Robustness tests

Is the effect of foot patrols on crime constant or dynamic?

The difference-in-differences (DiD) estimators used thus far mostly assume constant treatment effects during the intervention phase. Our policy variables might obscure important intertemporal effects in the periods before and after SAO adoption. While each intervention is transient (i.e., 90 days), our goal in this section is to assess transitory trends in the immediate months surrounding the surge in street-level guardianship. Prior research indicates treatment effects exhibit volatility in the early stages of an intervention, with effects decaying over time (Sherman 1990; see also Sorg et al. 2013).

Figures 3 and 4 report dynamic coefficients for all crime outcomes. Each coefficient is the interaction of a treatment indicator with individual month dummies specific to SAO and non-SAO jurisdictions. We plot coefficients associated with the two periods before and the four periods after SAO adoption. Vertical bands bounding each monthly estimate represent 95% confidence intervals. Unless otherwise noted, our standard errors were clustered at the precinct-level to address observational dependence within precincts.

Fig. 3

Dynamic treatment effects in 2014

Fig. 4

Dynamic treatment effects in 2015

Our strongest observed effects were for property offenses, which concentrated during the intervention phase. Though effects change over time, they do not persist—and most quickly rebound to the baseline average once the initiative ends. Though not shown graphically, we also extended this dynamic framework to assess a third leading treatment effect. We did not find evidence of emerging group trends in the periods before baseline that would explain any observed effects post-program exposure.

To summarize, we assessed the effect of the SAO initiative across 9 crime indices. Two iterations of the SAO initiative were instituted within our 5-year sampling window. Dynamic effects were assessed separately, by year. Months 1, 2, and 3 (i.e., t₋₁ − t₋₃) overlap with the 90-day exposure, while period 4 onward indexes all months beyond program termination. Of the 36 post-period effects in the 2014 iteration (three during the exposure phase and one after for each of the nine crime outcomes), only three coefficients were negatively bounded away from zero, all of which were specific to property offenses. Of the 36 post-period effects in the 2015 iteration, only seven were bounded away from zero. Six of our confidence bands were negatively bounded away from zero, most of which were for non-violent offenses, and one was positive. Evidence suggests reported street robberies increased in the final month of the 2015 iteration, though the effect was transient and did not persist.

We also find little evidence of anticipatory effects. Of the 18 coefficient leads reported in each year, only two were bounded away from zero. The second lead for burglary in 2015 was strong and positive. However, this divergent differential trend in the pre-treatment epoch immediately stabilizes and, consequently, does not persist into the post-period.³¹ Note: all violent crime leads were relatively stable before the onset of the intervention. The SAO initiative, while directed within areas with disproportionately high violent crime rates, had largely ignored other citywide jurisdictions with similarly urgent crime concerns. As an additional robustness check, we assessed violent crime leads going back 5 months before each intervention. All coefficients were bounded around zero.

What is the counterfactual world of open-air gun violence?

Police departments typically prioritize their resources in problem areas where the concerns voiced by community residents are often crime-specific. Firearm-related violence, in particular, is geographically clustered, exhibiting greater concentration within specific regions of the city. While each community will be subjected to varying levels of personal and property crime within a given month, not all will experience a gun discharge as a consequence of those incidents.

To test the robustness of our shooting analyses, we reran the basic two-way fixed effects Poisson specification (i.e., Model 4) using only the SAO policy dummy.³² We then compared estimates from the base specification with independent estimates of the main policy dummy using three alternative control groups: (1) all precincts from within the same patrol borough,³³ (2) all adjacent neighbors, and (3) SAO adopters only. The pairing of the treated with the untreated within geographically proximate settings has been used in applied work, albeit using smaller observational units (Branas et al. 2011; MacDonald et al. 2016). Others have also leveraged the adjacency of untreated jurisdictions as a nonequivalent but useful counterfactual, suggesting that bordering units serve as suitable proxies for “almost-treated” units (Cavalcanti et al. 2019). Moreover, restricting samples to “adopter-only” jurisdictions has also been used in applied settings, more so to limit model identification to variation in treatment timing (see Wolfers 2006). In our evaluation, treatment exposure is intermittent, with some jurisdictions moving into and out of a treated status across time. Only future or previous receivers of the initiative serve as controls. Table 4 reports estimates of the main policy dummy under each alternative. All substitutes failed to yield any immediate effects of the SAO initiative on open-air gun violence, suggesting our null findings are robust to alternative control groups.

Table 4 Poisson estimates of SAO initiative’s effects on shooting incidents, with alternative control groups, 2012–2016

We also investigated the possibility of treatment effect heterogeneity by geography. This is analogous to a methodological approach employed by Branas et al. (2011), which assessed the effects of vacant lot greening in Philadelphia. Treated lots were randomly matched to a nearby control lot within the same region, and the effects of the greening project on crime and health outcomes were explored by city section. Despite our attempts to isolate experimental settings, we did not find any region-specific effects associated with program implementation.

We also explored heterogeneous treatment effects by exploiting the timing of deployments (Green et al. 2014; Larsen et al. 2015). Not only was the SAO initiative a uniformed saturation of public space, but it also exhibited a temporal concentration on weekends and evenings. SAO officers were assigned steady patrol duty schedules; all “days off” were concentrated midweek (i.e., Monday–Thursday). Thus, SAO jurisdictions received a consistent saturation of foot patrols on weekends throughout the 90-day intervention phase. Our 5-year sampling window shows nearly half of all open-air shootings occurred on weekends. Moreover, deployment times were not concentrated during typical business hours. In fact, SAO guardianship was most pronounced between 4:00 P.M. and 4:00 A.M. Over the 5-year panel, nearly three-quarters of all open-air shootings occurred during non-business (i.e., evening) hours. Table 5 reports separate estimates of the SAO policy dummy under alternative temporal limits. Neither the daily nor the hourly restrictions resulted in significant program effects.

Table 5 Poisson estimates of SAO initiative’s effects on shootings, showing heterogeneity by day-of-week and hour-of-day, 2012–2016

We also assessed the dynamics of open-air gun violence separately from the NYPD’s standard crime metrics. A year-specific treatment indicator was interacted with month dummies specific to SAO and non-SAO jurisdictions. Month dummies were included for periods 1 and 2 before SAO exposure in each year and months 1, 2, and 3 after. This is our “effect window” (see Schmidheiny and Siegloch 2019). The baseline period (i.e., t₀) is the month before SAO adoption and is omitted as our reference period. Note: the months were binned beyond these time points. Month 3 before baseline is a dummy equal to unity in pre-treatment period 3 backward; likewise, month 4 after baseline is a dummy equal to unity in post-treatment period 4 forward.³⁴ Effects were assumed to be dynamic in between the endpoints, but constant beyond. Table 6 reports effects in the pre- and post-treatment epochs, separately, for each adoption year.

Table 6 Dynamic effects of SAO initiative on shooting incidents, by exposure year

Pre-period coefficients do not suggest anticipatory effects, though we acknowledge the period before baseline in 2015 is strong, and positive.³⁵ Negative effects emerge in the initial months after SAO adoption, though they do not persist. We do, however, observe a strong, lasting negative effect carrying over into the months beyond the 2014 iteration, but this effect vanishes once we allow for precinct-specific time trends.³⁶ Coefficients in Columns (1) through (3) in both years display a relatively consistent pattern.

Column (4) estimates dynamic treatment effects by comparing SAO precincts with all jurisdictions citywide—absent those precincts with a tangible border. The perception of offending risk in non-SAO jurisdictions sharing a territorial border poses a threat to our identification strategy. In other words, our “treatment” should not “spillover” into untreated jurisdictions (Duflo et al. 2007; Ryan et al. 2015). Estimates may be biased towards the null hypothesis should tightly bordered counterfactual jurisdictions become indirectly treated.³⁷ To account for this, our final model isolates experimental settings by excluding from our sample any non-SAO jurisdiction sharing at least one traversable border with a treated precinct. Coefficients increase in size but were imprecisely estimated using smaller subsamples. In general, null effects hold even after removing jurisdictions with a porous territorial border.

Does deterrence stop at precinct borders?

In the previous section, we discussed the possibility of a diffusion of deterrence beyond the periphery of treated jurisdictions. We argue that if shooting events did transcend precinct borders, then the shift, whether transitory or persistent, is likely local. To account this, we implemented a placebo treatment whereby SAO neighbors (those most likely to be affected by spillovers) are defined as the treated units. We then used all other citywide jurisdictions—excluding those originally selected to receive foot patrols—as the control group. The lead-lag structure was maintained. Coefficients associated with the placebo policy were negative, though indistinguishable from zero. We find no detectable impact on criminal offending or firearm-related violence among SAO neighbors. This refutes any suggestion that a diffusion of deterrence beyond the periphery of SAO jurisdictions biased our results towards zero. Tabular results associated with our placebo treatment procedure are available upon request.

As a final note, observed effects for shootings showed more volatility in the pre-treatment epoch and considerably more variation in specific regions (i.e., boroughs) of New York City. As such, we modeled the NYPD shooting data more rigorously than the crime metrics. Null findings observed for open-air gun violence survive an exhaustive set of alternative specifications and multiple qualitatively disparate counterfactual groupings.

Limitations

Our study is not without limitations. First, using reported crimes as a main outcome of interest may be problematic. On the one hand, a heightened foot patrol presence may cause residents to become resentful, and thus unwillingly to share information with the police or even report victimization (see Kochel 2011; Rosenbaum 2006; but, cf., Weisburd et al. 2011). On the other hand, this formal presence increases the frequency of contact between citizens and officers, perhaps resulting in a net increase in crimes reported to the police. The latter consequence may actually encourage residents to report victimization. While this may, paradoxically, improve the accuracy of crime reporting, it potentially dilutes the preventive effects of expanding the spatial reach of formal guardianship within communities.

Second, we are unable to control for other situational crime-prevention strategies occurring contemporaneously with the SAO initiative, such as Operation Impact,³⁸ the Neighborhood Coordination Officer (NCO) program, and the piloting of Operation Ceasefire in Brooklyn towards the end of 2014. Any suppression of violence associated with these programs potentially confounds the effects of the SAO initiative.

Third, we only model treatment dichotomously. A more sophisticated evaluation approach would be to model treatment in dosages. Acquiring a continuous measure of precinct staffing would allow us to explicitly model a dose treatment and identify how crime fluctuates at different staffing dosages (Mazeika 2014). Empirically modeling treatment in dosages has surfaced in applied work, and DiD methods are conducive to such measures (Abadie and Dermisi 2008; Acemoglu et al. 2004; Pedraja-Chaparro et al. 2016). Presently, it is unclear how fruitful of an endeavor it might be to address the nonuniform allocation of SAO officers. Most jurisdictions received a large baseline contingent of personnel, with only a few participating precincts receiving more than the average allotment. Moreover, proprietary concerns voiced by the NYPD rendered most staffing and personnel records unavailable for public consumption.

It is also worth noting that precinct geography complicates empirical assessments of dose effects in two seemingly interrelated ways. We cannot disentangle dosages of sentinel-based foot patrols from other concurrent resource deployments (e.g., motorized patrol response) within precinct jurisdictions. As well, large units of analysis obfuscate inferences drawn from the explicit modeling of law enforcement intensity, or lack thereof. Disaggregating our unit of analysis to smaller geographic locales (e.g., block-group level) would allow us to isolate smaller experimental segments within precincts to serve as viable control areas. We could not delineate the spatial reach of SAO foot patrols within precinct borders. Prior research suggests experimental evaluations of place-based enforcement strategies are often complicated by the units of analysis (Weisburd et al. 2008). Thus, the designation of large geographic regions as units of analysis (i.e., precincts) could potentially bias estimates of a program effect, with negative effects indicative of the collective law enforcement strategies aimed at reducing crime and disorder (Piza and O’Hara 2014).

Discussion

Traditionally, American policing has been characterized by an emphasis on randomized patrolling, rapid response to calls, and reactive investigations of criminal incidents (Sherman 2013). Yet, since the 1980s, the focus of managing police resources has shifted to “targeting, testing, and tracking” (Nagin et al. 2015: 77). Nevertheless, there is still substantial discussion of what impact the police can (Cullen and Pratt 2016) and should (Sweeten 2016) have on aggregate crime levels and which strategies work best to achieve those impacts (Braga et al. 2014; Braga and Weisburd 2012).

The present study suggests a weak, negligible influence of high-visibility, saturation foot patrols on aggregate rates of criminal offending. Using a nonequivalent control group and a difference-in-differences (DiD) experimental design, we tested the effect of the “Summer All Out” (SAO) initiative on reported crimes and shootings. Our results indicate aggregate rates of street crime do not vary with the deployment, or withdrawal, of SAO foot patrols. The 2014 SAO initiative was not associated with significant reductions in reported index crime or disaggregated rates of robbery and assault. Null effects were also observed in its subsequent iteration; all major crime indices in 2015 were similarly bounded around zero. The present evaluation also suggests foot patrol deployments did not appreciably reduce shootings at the street- level. Overall, the sharp dispersal of SAO officers within precinct jurisdictions appears to be ineffective at reducing violent street crime or open-air gun violence.

Due to the NYPD’s multifaceted support network, police executives have the ability to mobilize large numbers of surplus personnel to participate in parades, large-scale demonstrations, and other unplanned events. These mobilizations are typically brief, lasting only a few hours or several days. But the SAO initiative was different. The sharp dispersal of uniformed officers lasted for 90 days and was one of the largest and longest mobilizations to augment patrol operations in the NYPD’s history. Each SAO mobilization was large in absolute terms, in both personnel and geographic distribution. The diffuse dispersal of visible guardianship on foot across precinct jurisdictions may have diluted any potential crime-suppression benefits. Nevertheless, there is no indication that the NYPD is planning to cease SAO deployments in the future.

The present study may offer guidance for future research and policymaking. First, the NYPD should subject well-advertised, department-sponsored initiatives to empirical evaluation. Such evaluation is necessary to establish a program’s effectiveness. Internal evaluations of place-based enforcement strategies designed to suppress crime and disorder should be co-produced with local research institutions, allowing for the scientific advancement—and accumulation—of a growing body of evidence-based enforcement strategies. Reproducible results should then guide future policy and agency directives (see Risman 1980). We are not aware of any internal NYPD sanctioned research group that is proactively publishing peer-reviewed impact evaluations of department-sponsored crime initiatives.

Law enforcement agencies, in general, should have a vested interest in the systematic evaluation of their own self-initiated crime-prevention efforts to reduce violence. We showed earlier that repeated experimentation of the SAO initiative in communities across the city is costly. Weighing the financial strain of continued experimentation against other community policing efforts is critical moving forward, as budgetary coffers only swell to ensure the continued expansion of high-visibility policing.

Second, while visible policing itself should deter criminal behavior (Nagin et al. 2015), the content of officer behavior in specific places does matter (Cullen and Pratt 2016). Unfortunately, resource allocation strategies supporting the SAO initiative were opaque. If the SAO initiative continues, efforts should focus on process evaluation during the intervention period in order to get inside the “black box” of how these officers are allocated (Haberman and Stiver 2018; Wood et al. 2014). Department executives should make concerted efforts to adjust the content of officer behavior throughout the intervention phase. Moreover, strategic planners should employ entirely different situational crime-prevention strategies (e.g., problem-oriented policing) if a similar intervention is to be implemented in successive years. The 2014 and 2015 initiatives were designed with a similar blueprint. SAO officers served a basic sentinel function for an approximate 3-month intervention period. We recommend police departments instead experiment with a diverse range of community and intelligence-led policing initiatives.

Third, the widespread distribution of officers on foot across expansive geographies may have been too diffuse too affect offenders’ perception of offending risk (see Sorg et al. 2017). The SAO initiative was not a concentrated exposure of visible guardianship. Macro-dispersal of uniformed officers across densely populated regions may dilute the salience of the initiative. This would be consistent with the broader literature on community crime prevention, which is generally mixed, but with stronger evidence for interventions aimed at crime hot spots (Braga et al. 2014; Lum et al. 2011; Telep and Hibdon 2018). Hot spot size varies across studies (Sherman et al. 1989; Sherman and Weisburd 1995), but they are typically far smaller than the precinct jurisdictions chosen to participate in the SAO initiative (Weisburd 2015). The SAO initiative, and others like it, should consider interventions at smaller units of geography, both to allow for clearer assessments of their effectiveness and to allow for more efficient use of departmental resources (Telep and Hibdon 2018; see also Eck 2015; Kennedy 2016; Weisburd et al. 2014).

Fourth, the intensity or frequency by which SAO officers police their beats is potentially a function of their familiarity with their assigned jurisdictions. SAO officers were deployed on foot in relative isolation and with little knowledge of local crime conditions and the offending population. In addition, anecdotal evidence suggests SAO officers received insufficient time to adjust to their new environments and odd patrol duty schedules.³⁹ Some evidence even suggests SAO officers received minimal supervisory oversight.⁴⁰ Arguably, the selection of officers with less community awareness might offset any potential crime control gains.

Lastly, while randomized controlled trials (RCTs) have unparalleled power to establish causality, they may not always provide robust accounts of the sort of “counterfactual worlds” that practitioners and policymakers desire (see Nagin and Sampson 2019; Sampson 2010). Quasi-experimental studies, in particular the varied DiD approaches employed here, provides a useful addition to RCTs, as well as other post-hoc matching estimators used to identify treatment effects. Police officials benefit greatly from evidence about the causal nature of population-level interventions in policing, even when those exposures are confounded by design. As resource limitations direct law enforcement efforts to some jurisdictions and not others, DiD estimation strategies present crucial opportunities to evaluate the causal impact of those interventions.