1 Introduction

The adoption of “best practices” constitutes an important strategy for organizational improvement [1,2,3,4]. Best practices can be sought from high-performing outliers [5,6,7]. This is the first step in using the positive deviance process, a strategy that is gaining popularity in healthcare improvement [8,9,10,11,12,13,14,15,16].

The “positive deviance approach” to social/behavior change began in the area of childhood nutrition when it was noted that within communities with high levels of childhood malnutrition, some families had well-nourished children [17, 18]. These families, referred to as positive deviants, evidenced uncommon but successful behaviors or strategies that enabled them to find better solutions to a problem than their peers, despite facing similar challenges and having no apparent extra resources or knowledge. A similar approach, termed bright spotting, has been applied in primary care to the identification of patients with diabetes at high risk of complications who have managed to maintain their health [19].

The term positive deviance has been used in different ways in management improvement and research. The concept of positive deviance has also been used to describe behavior of individuals within organizations [20,21,22,23,24]. More recently, this concept has been applied to organizations themselves, especially in healthcare. For example in systematic review, Baxter et al. identified four key themes of positive deviance:

  1. 1.

    Positive deviants were defined as high performers

  2. 2.

    Positive deviants demonstrated different or uncommon behaviors

  3. 3.

    Positive deviance is a bottom-up approach

  4. 4.

    Positive deviant solutions are sustainable within current resource [8]

Although definitions of positive deviance vary widely, the phenomenon of positive deviance is often mentioned in the context of complexity, whether complexity science , complexity theory , or complexity thinking . For example, Lindberg and Clancy identified complexity science concepts and related then to the positive deviance process in reducing the occurrence of intravenous line infections in the intensive care unit [25]. These concepts included:

  • The number and nature of relationships

  • Difference and diversity in perspective and action

  • The extent to which power is centralized or shared

  • The degree to which variability and order exist

In being attuned to the number and nature of relationships, the positive deviance process seeks to involve all whose behavior affects the problem and relies on facilitators who are skilled at nurturing open conversation and encouraging participation in small group conversation [25]. Similarly, Spindler and Wagenheim described positive deviance dynamics as “typically a ‘grassroots’ self-organising unit, within teams, organisations, networks or society yet often outside the formal structure, that has a different, even contrarian, perspective of the social system. Such deviants can be considered holons that contain all the characteristics of the whole system to which they belong yet operate with an entrepreneurial independence. This autonomy and unique perspective allows them to observe the system’s blindness and realise different solutions (p. 647)” [26]. Thus, the positive deviance approach implies the necessity of thinking that the process unfolds in the context of a complex adaptive system, in short, complexity thinking. Further exploration of the philosophical (ontological and epistemological) underpinnings of complexity thinking is beyond the scope of this paper. Readers are referred to the work of Richardson and others [27, 28].

A structured approach to employing the positive deviance process includes the definition of the problem, the identification of positive deviant individuals or teams, and the discovery of the uncommon practices or behaviors that allow the positive deviants to be successful [29]. Addressing mortality in intensive care units following myocardial infarction provides an exemplar of this approach.

Bradley et al. noted that there was marked hospital variation in ICU mortality for myocardial infarction [9, 11]. Their first step involved identification of organizations that demonstrated exceptionally high performance in this area of interest—positive deviants. In this case, there was a well-established metric for the outcome. In addition, a criterion of consistent high performance was used. This was operationalized by analyzing data from two consecutive years. They then selected hospitals that ranked in the top and bottom 5% of performance during both years. In the original work on positive deviance in public health, the comparators were required to be those with similar access to resources. In contrast, Bradley et al. selected hospitals that were diverse in areas such as the volume of patients with acute myocardial infarction, teaching status, and socioeconomic status of patients. They were seeing to identify “information rich” participants that have certain characteristics, detailed knowledge, or direct experience relevant to the phenomenon of interest. Later studies examined what the positive deviants were doing that was different from other intensive care units, and those actions were shared with others.

Many hospitals have since adopted those practices with resulting improvement in outcomes. However, despite their good results, several questions are raised about extrapolating that approach to other issues. First, in this case there was a very well-established and uniform metric, something often lacking. In addition to there being a wide range of performance metrics , how the definition is operationalized can vary greatly. Both the length of time deemed necessary to demonstrate consistency and the procedures to identify high performance vary. In contrast to Bradley et al., others have examined performance during a single 1-year period based on 99 percentile confidence limits [30]. In addition, in the latter study, the comparator was limited to other hospitals in the same, albeit very large, organizational unit. In the original work on positive deviance in childhood nutrition, performance was assessed by simple observation rather than statistical analysis [18, 29]. The difference between malnourished and well-nourished children was obvious. This may or may not be the case with organizational positive deviance. There are no agreed upon methods to determine the criteria for high performance or to define an appropriate comparator. The identification of an outlier depends on the measures and statistical methods chosen, and different methods often do not agree [7, 31,32,33,34].

Diabetes , a common chronic condition, lends itself to this type of assessment because there are well-established performance measures [35]. Many of these measures have been based on assessment of glycemic control using hemoglobin A1c (HbA1c) levels and have focused primarily on rates of inadequate or undertreatment. Recently, more attention has been paid to the increasing problem of hypoglycemia which may be an unintended consequence of overly intensive glycemic control [36,37,38,39]. Overtreatment can result from setting HbA1c targets that are inappropriately low based on patients’ life expectancy, comorbid conditions, and other factors [40,41,42,43,44]. We previously developed a measure of overtreatment [30]. The objective of our study was to determine the effect of different factors on identification of high-performing outliers—positive deviants, especially consistency of performance over time, and use of different comparators. In addition, we included a reciprocal measure, one of undertreatment . Such undertreatment could be a consequence of efforts to reduce overtreatment or the combination of high rates of undertreatment and low rates of overtreatment could reflect lack of attention to diabetes care in general.

2 Methods

2.1 Research Design and Healthcare System Under Study

The design was serial cross-sectional, using yearly Veterans Health Administration (VHA) administrative data (2009–2013) [30]. VHA provides healthcare to eligible veterans of the Armed Services in >100 hospitals and their related clinics. During the period 2009–2013, VHA was organized into 21 regional networks (Veterans Integrated Service Networks or VISNs), each consisting of 3–10 facilities. The facilities vary by size, range of services offered locally, and other factors from which is calculated a complexity level. There are 18–32 facilities in each of the 5 complexity levels.

2.2 Outcome Measures

We used our previously developed measure for overtreatment and used an HbA1c threshold of 6.5%. Overtreatment rates were calculated at the facility level as the proportion of patients in the chosen population with HbA1c<6.5%. In brief, the population at high risk for hypoglycemia (hence overtreatment) included patients taking a diabetes drug known to have a relatively high frequency of hypoglycemia (insulin and/or sulfonylurea agents) plus having at least one of the following additional criteria: age 75 years or older, chronic kidney disease (defined as last serum creatinine measurement in a year greater than 2.0 mg/dL/176.8 mmol/L, or an ICD-9-CM diagnosis of cognitive impairment or dementia in ambulatory care) [30]. Undertreatment rates were calculated in the same patient population as the proportion of patients with an HbA1c greater than 9%. Rates were calculated for each year.

2.3 Outlier Status

Outlier status was assessed by a facility outlier value measure standardized within a year and for each of three comparator groups:

  1. 1.

    All VA hospitals

  2. 2.

    Hospitals within the same VISN

  3. 3.

    Hospitals within the same complexity level

The facility outlier value was calculated as:

$$\displaystyle \begin{aligned} \frac{\text{facility observed rate-mean rate in comparator population}}{\text{standard deviation of rates in comparator population}} \end{aligned} $$
(14.1)

3 Analyses

We calculated yearly rates of over- and undertretment as well as outlier status.

3.1 Results

Outlier values for our primary measure ranged from approximately − 1.5 to + 3.5 (Fig. 14.1). There was little effect of facility complexity and VISN on outlier identification. Use of different HbA1c thresholds results in different sets of outliers with only modest overlap (data not shown). Of the 14 facilities identified in the highest-performing decile for overtreatment in 2009, only 7 were consistent high performers in all the years from 2009 to 2013. While undertreatment in this population at high risk for hypoglycemia rose over time from a mean of 7.1% to 10.3%, consistent high-performing outliers for overtreatment had higher rates of undertreatment (HbA1c > 9%) in general. However, three of the high performers had overtreatment rates similar to VHA facilities as a whole; the remainder had higher rates at baseline that rose even more than facilities in general (Fig. 14.2).

Fig. 14.1
figure 1

Overtreatment outlier values over time by 2009 deciles using 6.5% threshold and all VA facilities as comparator. Asterisk, top performing decile; filled circle, remaining 90%

Full size image
Fig. 14.2
figure 2

Undertreatment rates in the population at high risk for hypoglycemia. Undertreatment rates in the population at high risk for hypoglycemia over time among facilities with lowest rates of overtreatment (dashed lines) and the average rate +95% Confidence interval among all VA facilities (solid line within shaded area). Seven facilities were identified as high performers using overtreatment metrics; high rates of undertreatment were observed in many and looking at the undertreatment rates over time and comparing them to the undertreatment rates of all VA facilities, several high performers with regard to overtreatment have consistently high rates of undertreatment

Full size image

4 Discussion

Statistical identification of positive deviants for diabetes overtreatment was dependent not only upon the specific measure used but also on time, i.e., high performance was consistent in only half of the facilities identified in the baseline year. Most importantly, our findings indicate the importance of including a balancing measure or countermeasure to identify unintended negative consequences. Thus, facilities which appeared to be similar based on one measure (equifinality) could be differentiated based on a balancing measure.

Equifinality and multifinality are each characteristic of open systems and represent opposite sides of the same coin. The principle of equifinality states that a given end state can be achieved by different paths or trajectories. In short, “all roads (may) lead to Rome,” though the routes vary. In contrast, the principle of multifinality states that different end states can be reached starting from the same initial conditions. In short, every journey, regardless of where it ends, begins with the (same) first step. These principles make process improvement possible. Open systems exhibit change, but their end states are not ordained in a linear fashion from their initial states [45]. Different healthcare systems starting from different initial conditions reach a similar state of quality care. Moreover, different configurations of factors (and their interactions) may result in similar end states, a phenomenon that provides the basis for the methodology of qualitative comparative analysis [46].

The limitations of our study notwithstanding (e.g., single condition which is multidimensional and involves patients, providers, and organizations; single healthcare system; our focus on identification of high-performing sites rather than the best practices themselves) our findings have implications for the positive deviance approach because they illustrate the danger of taking a very narrow approach to what constitutes equifinality. Although there are numerous methods for outlier detection and differences in both criteria and comparator, our study suggests that considerable thought needs to be given to this issue at the outset, before attempts are made to identify performance outliers. Specifically, the end state of low rates of overtreatment may be observed under both desirable and undesirable conditions. Ideally, low rates of overtreatment occur as a result of specific attention to patients at risk for hypoglycemia. Less ideally, a low rate of overtreatment may occur as an artifact of widespread undertreatment and a facility-wide tendency toward higher HbA1c levels. We term this phenomenon pseudo-equifinality. In other words, two roads may lead to Rome, but one may lead to the Pantheon and the other to the Cloaca Maxima (greatest sewer).