Persons with type-2 diabetes mellitus have a two- to fourfold increased risk of myocardial infarction and sudden death compared to persons without diabetes, and cardiovascular disease accounts for roughly half of all deaths in people with diabetes.1–5 Efficacious therapies for the treatment of patients with diabetes have long been available,6–10 and evidence from randomized controlled trials shows that intensive use of these therapies can reduce the burden of disease.9,11–16 However, less is known about how to translate these clinical research findings into real-world practice.17 A metaanalysis examining interventions to improve care for patients with diabetes found that studies rarely assessed the impact on patient outcomes.18

Translating proven therapies and interventions into routine practice and measuring change in population level health is not easy.19 The chronic care model (CCM) is a framework for managing chronic illness, which facilitates planning and coordination among providers while helping patients to play an informed role in managing their own care.20,21 The components of the CCM have been shown to be effective for improving certain process measures.18,22–24 Less is known about implementing the model as a whole and its impact on long-term cardiovascular disease risk factors.25

A collaborative intervention is a method used to help health care organizations apply continuous quality improvement techniques and affect organizational change.26 Collaborative interventions, also termed as Breakthrough Series, have been conducted for hundreds of teams addressing multiple clinical conditions.26,27 However, there have been few controlled trials evaluating the effectiveness of these interventions.23,25,27–29 In light of this, we conducted a multicenter evaluation of a collaborative intervention to implement the CCM for diabetes care. We ask: Is exposure to a CCM collaborative intervention associated with improved cardiovascular disease risk for patients with diabetes?

METHODS

Overview

We examine the impact of a CCM-based diabetes care collaborative on the 10-year risk of cardiovascular disease, defined as fatal and nonfatal myocardial infarction or sudden death, in patients with diabetes cared for at clinical sites where collaborative induced changes were implemented compared to patients at control sites within the same organization receiving the usual care.

Intervention

The intervention is a series of 3 learning sessions and a final meeting designed to help organizations implement the CCM for diabetes care. The first collaborative in 1999 was run by the Institute for Healthcare Improvement (IHI) and had participants from the Eastern, Western, and Southern regions of the United States, whereas the second in 2001 was limited to organizations from the State of Washington. Organizations in the first collaborative paid a flat fee of $12,000 to participate, whereas the second collaborative (run by Qualis, the Washington State Quality Improvement Organization and the Washington State Department of Health) cost $200 per attendee per session. Over the course of a year, participating organizations had an average of 8 members on a team with generally at least 1 physician participant. They were taught methods for making system level organizational change, the elements of the CCM, and evidence-based diabetes clinical care measures and therapies. Experts in diabetes disease management and quality improvement guided the teams to study, test, and implement improvements in essential care processes for diabetes in their intervention sites (Fig. 1). The 13 intervention sites made an average of 45 changes (SD = 18) during the study period. In delivery system redesign, most sites worked on enhancing care management roles, providing proactive follow-up, planning visits, and improving the visit system. All sites implemented patient education, and the majority also provided resources and tools to patients and worked with patients in care planning. Guideline implementation and provider education received attention from almost all sites. A more detailed description of the changes can be found elsewhere.30

Figure 1
figure 1

Chronic illness care collaborative intervention. Adapted from Cretin et al. 2004.31

Full size image

Study Design and Setting

Our study was a pre- and postevaluation of participants cared for at intervention and control sites. Representatives from 40 health care organizations volunteered to participate in one of the two CCM collaboratives. We contacted health care “organizations” with administrative oversight over multiple clinical “sites” for our evaluation. This enabled us to obtain control sites (other clinic or practice) of similar size, location, and financial type as the intervention site with similar patients from within the same organization. Of the original 40, 3 organizations dropped out of the collaborative and 20 were too small to provide within organization control sites, leaving 17 of whom 13 agreed to participate and furnish control sites. The participating organizations cared for patients in nonprofit clinics, a nonprofit health plan, for-profit physician groups, and veteran’s administration facilities. Expanded descriptions of the design, participating organizations, and selection of control groups can be found in the overall study’s technical appendix http://www.rand.org/publications/WR/WR269/, and a detailed review of the collaborative intervention and our evaluation methods can be found in Cretin et al.31

Outcome

We analyzed the impact of the intervention on change in the 10-year predicted risk of cardiovascular disease as determined by the United Kingdom Prospective Diabetes Study (UKPDS) risk engine (Isis Innovation Ltd 2001© Headington, Oxford, UK) for the study population as a whole and for high-risk and low-risk patients. The risk of getting heart disease in the next year derived from the UKPDS = 1 − [exp(−qd T)], where T is the duration of disease, d = 1.078 gives the increase in risk for each year of duration, and q is the product of terms of the form b i raised to the x i power.32 Each b i is taken from Table 3 of their publication and was estimated from UKPDS data; age at diagnosis in years, female, Afro-Caribbean ancestry, smoking HbA1c, systolic blood pressure, and log (total cholesterol/HDL). This formula is described in further detail in the technical appendix for this manuscript. We compared changes in the UKPDS risk score in the groups exposed to the intervention to changes in the control sample.

The control site is needed to adjust for local variation and secular trends. Because the control patients may differ in ways affecting reductions in risk factors, we adjust for characteristics that might affect the change in risk factors over the year. The two observation periods were the 11-month preintervention period and the 11-month postintervention observation period that followed a 3-month period beginning after the first collaborative meeting to allow the sites to begin making changes (Fig. 1). To calculate CVD risk in each period, we used the latest levels of the intermediate outcomes recorded in the medical record for HbA1c, cholesterol ratio, and the average of the 3 most recent for the blood pressures. The study protocol was approved by the Human Subjects Protection Committees from RAND, UCLA, and all participating oganizations.

Study Population

We used the diabetes registries from participating sites to identify patients. All patients with diabetes were approached to consent except for those at 2 clinical sites that had over 400 patients from which 200 patients were randomly selected. We identified 3,080 eligible patients. No data were collected on patients who declined consent (806). Patients were excluded if they were unable to consent because they died (12), were in a nursing home (9), did not speak English or Spanish (26), could not be found (198), or were too ill to consent (36). In addition, patients were excluded if the medical record could not be found (30) or if they said they did not have diabetes (28). This resulted in 1,935 consenting participants with charts available for review. (Fig. 2) The analysis of the combined cardiovascular risk score excluded patients missing more than 1 of the 6 modifiable component values (pre- and postcholesterol ratios, HbA1c, and systolic blood pressure) needed to calculate the UKPDS risk score. A full description of the exclusion criteria and calculation of the risk score can be found in the technical appendix of this manuscript.

Figure 2
figure 2

Patient selection.

Full size image

Data Collection

We obtained age, sex, cholesterol, HbA1c, blood pressure, medications, comorbid conditions, and additional values for the UKPDS risk score from the medical record. We also abstracted other chronic medical conditions needed to calculate a modified Charlson score.33 The modified Charlson score does not include HIV and moderate/severe liver disease (not collected in our sample) or diabetes. The comorbidity index used in regressions was based on the number of conditions where 0 = 0; 1–2 = 1; 3–4 = 2; and >5 = 3. We also abstracted treatment regimen (insulin, oral diabetes medications, or diet-controlled) from medical records during the preintervention period to calculate a measure of disease severity. After documentation of informed consent, charts were either reviewed at the site or deidentified charts were mailed to a central location. Trained abstractors entered data into a computerized tool, and completed records were sent to a central data repository. Lead abstractors initially reviewed a 10% subsample of the other abstractors’ record reviews to ensure quality and assess reliability. Kappa scores at the level of diabetes-related care measures ranged from 0.62 to 0.88.

Imputation

We multiply imputed missing values when one of four chart review values required for calculating changes in the UKPDS risk score was missing, specifically 44 pre- or postperiod HbA1c values (2%) and 395 pre- or postperiod total cholesterol or HDL values (17%). We also multiply imputed 163 smoking status indicators (14%). We used the Markov Chain Monte Carlo Method implemented in Proc MI in SAS, copyright (c) 2002–2003 by SAS Institute Inc., Cary, NC, USA. Estimated coefficients and variances from different imputations were combined using Proc MI analyzed in SAS. Only 26 systolic blood pressure values (<1%) were missing, so we imputed them with mean values.

Analyses

We performed bivariate tests of association between the baseline characteristics of intervention and control groups. We use a random intercept hierarchical regression model (Proc Mixed in SAS) to simultaneously adjust for clustering of observations within patients (pre and post) and patients within sites. Fixed effect patient level covariates are age, gender, comorbid conditions, and treatment regimen (ordered categorical indicator variables for diet-controlled, oral agents only, oral and insulin use, and insulin alone) to estimate the adjusted change in differences over time between participating and control site patients. The two observations for each patient were coded with indicators for being in the first or second collaborative, being in an intervention site, postperiod, and a postperiod by intervention site interaction term. The coefficient of the postperiod by intervention site interaction term is the estimated adjusted difference in change over time in UKPDS score for those in the intervention versus control groups.

RESULTS

Baseline characteristics were similar for intervention and control patients except that those in the intervention group were significantly more likely to be enrolled in an HMO (Table 1). Patients in the intervention group had higher preintervention mean HbA1c levels at 7.9% compared to the control group 7.7% (p = 0.02) (Table 1). There were no significant differences in mean systolic blood or baseline cholesterol ratios. The baseline 10-year cardiovascular disease risk for the intervention group at 30.7% was similar to the control group risk at 31.0% (p = 0.86).

Table 1 Baseline Characteristics*
Full size table

After the intervention, both the intervention and control groups had lower systolic blood pressures, total cholesterol to HDL ratios, and HbA1c values (Table 2). The intervention group had a reduction in 10-year cardiovascular disease risk from 30.7% in the preintervention period to 28.1% in the postperiod, whereas the control group’s risk was lowered from 31.0 to 30.2% (Table 2). The highest risk tercile in the intervention group had a reduction in 10-year cardiovascular disease risk from 52.7 to 45.7%, whereas the control group’s risk was lowered from 55.3 to 51.2% (Table 2).

Table 2 Unadjusted Difference in Differences for Outcomes
Full size table

Our adjusted analysis found a significantly greater risk reduction in the intervention group of −2.1% (95% CI −3.7%, −0.5%) (Table 3). In our adjusted analysis, the postintervention risk in the highest risk tercile was significantly lower by 4.1% (95% CI −7.1%, −1.0%) in the intervention group as compared to the control group (Table 3).

Table 3 Adjusted Risk Change Difference and Number Needed to Treat
Full size table

DISCUSSION

Our study shows that a collaborative intervention designed to help organizations implement the CCM for diabetes is associated with improved risk for cardiovascular disease predicted by the UKPDS risk score. These findings suggest that collaborative interventions may lead to reduced cardiovascular disease in patients with diabetes. Based on the results of recent efficacy trials for hypertension34,35 and cholesterol management,36 a stronger focus on cholesterol control and BP control in the CCM model could be associated with even bigger improvements in CVD risk. Recent observations of an independent association of HbA1c with cardiovascular disease risk also support a potential benefit in collaboratives reducing CVD risk.38

Within a 1-year postobservation period, implementation of the CCM model was associated with significantly greater reductions in lipids and HbA1c than what was observed in the control group. Although one should be cautious when applying NNT calculations to nonrandomized trials, we note that if the observed resulting risk reduction of 2.1% were realized, there would be one fewer myocardial infarction, fatal myocardial infarction, or episode of sudden death for every 48 patients with diabetes at an intervention site during the next 10 years. The risk reductions among the highest risk patients would correspond to one less CVD event for every 24 high-risk persons exposed to the intervention. The reduced cardiovascular disease risk of 2.1% in the overall sample and 4.1% in the highest risk tercile of persons with diabetes is comparable to the 3.2% reduction of major coronary events over 5 years from simvastatin use found in the Heart Protection Study’s diabetes population.36 However, the UKPDS risk engine was developed in the UK in the late 1990s and may not fully apply to US populations in the current time where there is a greater focus on pharmacological therapy directed at reducing CVD risk among persons with diabetes. As similar studies examine other CCM interventions, we will gain a greater understanding of which diseases and populations are best targeted for collaborative interventions.

The National Institutes of Health has called for more pragmatic research aimed at bridging the gap between clinical knowledge, effective practice, and improved health.38,39 Our methods show how a practical clinical trial can provide important information about the potential impact of organizational change on the health of patients. This broadens our understanding of how to reduce diabetes-related cardiovascular disease beyond the evidence from randomized controlled trials with highly restrictive protocols.11,16 According to the National Committee for Quality Assurance, patients with poor control of their HbA1c (>9.5) decreased nationally from 44.9% in 1999 to 42.5% in 2000.40 Such secular trends highlight the importance of having control sites in any evaluation of health interventions.

Other studies of the CCM have shown effectiveness in improving various process measures,18,22,24 but the clinical relevance of these measures is sometimes lost on providers.41,42 Our use of a measure of cardiovascular risk as a summary intermediate outcome provides useful information to decision makers and clinicians on how organizational change can affect the health of their practice. The UKPDS risk score was convenient for our evaluation, and such point scores can be useful for tracking progress and communicating cardiovascular risk to patients with diabetes, many of whom have inaccurate perceptions of their risk for cardiovascular disease.43 Clinicians and noncommercial organizations can download the program used to calculate patient risk, without charge, from the Oxford Center for Diabetes, Endocrinology, and Metabolism’s website http://www.dtu.ox.ac.uk/index.html?maidoc=/riskengine/download.html.

The few other controlled trials to test the effectiveness of the CCM have had mixed results.23,25,27–29 Horbar showed a benefit on neonatal intensive care, and Piatt’s trial showed improved intermediate outcomes in patients with diabetes, whereas Landon’s analysis of HIV care and Solberg’s study of preventive services showed little improvement in processes and outcomes of care for those exposed to the intervention.25,27–29 Our intervention was modest in many respects—teams spent 3 weekends learning concepts and methods to take back to their plans and practices, and then work them into their organizations over the year. However, by focusing on tangible processes that are known to improve outcomes and by tracking those outcomes, exposure to the intervention was associated with improvements.

Our study depended on the cooperation of a heterogeneous group of health plans and practices and therefore suffered from some inherent limitations. Four of the 17 eligible sites did not participate in our evaluation because of perceived administrative burdens such as acquisition of informed consent.44 Because the CCM intervention was not randomly allocated to sites within organizations, our results could suffer from selection bias if intervention sites were more motivated to make changes to improve care than the controls. To assess whether this was the case, the study conducted surveys of the staff participants in the study and surprisingly, we found that attitudes toward quality and overall assessments of staff attitudes were somewhat less favorable at intervention sites. Please refer to the technical appendix for the overall study on the ICICE website http://www.rand.org/publications/WR/WR269/ for a further detailed description of these survey results. That organizations as a whole volunteered to participate is not a problem for our comparison but does limit external validity.

Because our control sites came from the same organization as the intervention sites, they could have learned of and implemented changes. However, if there was contamination, then the real improvement in CVD risk reduction observed in the intervention sites would have been greater than what we have reported. Our results may not generalize to providers and facilities without the institutional or financial support for change. Although our study’s participating organizations had to use their own resources to carry out any changes, other studies of collaboratives have shown improvement in care among community health centers, which typically have fewer resources and more challenging populations than traditional practices.22 23 25 Our study’s patient population was roughly 90% non-Hispanic white and thus underrepresented the other 27% of patients with type-2 diabetes in the United States.45 Future studies are needed to test these methods in settings where more racially and ethnically representative patients with diabetes receive their care.

We have shown that the introduction of the CCM using a collaborative intervention for diabetes care is associated with a reduction in cardiovascular disease risk in patients with diabetes. Future testing and greater use of similar collaborative efforts may aid in reducing the burden of cardiovascular disease in patients with diabetes.