MEASUREMENT OF ADHERENCE TO CLINICAL PRACTICE GUIDELINES FOR OPIOID THERAPY FOR CHRONIC PAIN

As opioid prescribing has increased dramatically over the last 15 years [1], so has the population prevalence of opioid-related problems. Patients are often on multiple medications [2], and risky coprescribing with sedative hypnotics is alarmingly high in some settings [3]. Other problems include misuse, litigation against physicians, and overdose [4, 5]. Misuse can include a wide array of behaviors [6, 7], such as abuse, addiction, potentially harmful use patterns, and problematic nonuse, each of which has medical, social, and functional consequences for the users, their loved ones, and the public. Some misuse behaviors can lead to serious health consequences, ranging from poor management of pain to death. An additional concern is that data on the effectiveness of long-term opioid use are lacking (e.g., Ref. [8]).

Minimizing misuse is necessary for safe and effective use of prescription medications and has generally been defined at the level of the patient and described in terms of patient behaviors. Thus, a substantial research effort has been directed at identifying at-risk or misusing patients who may benefit from special clinical intervention (e.g., Refs. [9, 10]). While this patient focus is useful, provider and system factors also contribute to medication misuse and other adverse consequences to varying degrees, and these factors may be more amenable to quality improvement efforts within health care systems. Clinical practice guidelines (CPGs) provide recommendations regarding health care system and practitioner level practices that are thought to maximize safe and effective use and minimize misuse [9, 11].

In 2003, the Veterans Affairs (VA)/Department of Defense (DoD) published a CPG for management of chronic opioid therapy [11], which was revised in 2010 [12]. This revision is based in part on guidelines developed by Chou and colleagues [9]. This guideline identifies a few conditions or populations for which opioids are strictly contraindicated but focuses largely on processes of care to improve the safety and effectiveness of opioid therapy. As examples, the guideline encourages clinicians to proactively address side effects and drug combinations that may increase the risk of adverse effects, integrate nonopioid treatments, and use urine drug screens (UDSs) to discourage and detect medication misuse and diversion. Use of recommended care practices is considered essential for minimizing negative consequences of opioid prescribing without reversing gains made in improving pain management in clinical settings [9]. While a CPG is not enough to ensure change in clinical practice, this CPG provides a starting point for identifying gaps in the quality of prescribing, utilization, and monitoring of opioid therapy for pain management across health care settings.

It has been argued that effective quality improvement efforts require quality indicators which measure both process and outcome quality [13]. Moreover, to be successful, these measures must not require excessive documentation or additional assessment by clinicians. In order to initiate quality improvement in opioid therapy for chronic pain, it is important to assess existing practice patterns as well as current variation in use of recommended practices. A key step in increasing adherence to guideline-recommended practices is to systematically measure use of these practices and identify gaps in their current implementation.

As part of a quality improvement project, we developed metrics to assess the extent to which individual CPG recommendations are being followed at VA facilities across the country. Here, we describe development of a suite of process and outcome quality metrics that assess adherence to key recommendations specified in the 2010 VA/DoD CPG management of opioid therapy for chronic pain [12]. The practices were categorized into seven domains (see Table 1) that can be calculated from archived administrative data. These metrics used administrative data elements that are not unique to VA facilities and could be adapted for use in non-VA settings.

Table 1 Rationale and definition of each metric
Full size table

One of the primary goals of developing this suite of metrics is to examine adherence to the CPG at the facility level. This knowledge can then be used to shape system level behavior in opioid prescribing by identifying model facilities as well as poorer performing facilities, and then transporting and initiating interventions accordingly. It may also be useful for identifying areas in which guideline-recommended practices are not consistently followed in clinical practice overall. Additionally, these measures provide a tool to assess changes in clinical practice associated with quality improvement efforts.

Another goal is to develop procedures and precedent for creating a suite of quality metrics that parallel an entire written CPG, which could be adopted generally during their development to increase the utility of CPGs for guiding quality improvement. We are unaware of any other attempts in the literature to comprehensively consider and translate recommendations from a CPG into a panel of metrics to track guideline adherence, despite the significant number of guidelines and efforts to develop practice-specific process measures to evaluate use of targeted clinical practices. Thus, we believe a discussion of the methodology used to develop these metrics is extremely valuable. Specific findings from the metrics as applied to the VA will not be presented here, as presentation of such a large amount of data is beyond the scope of this paper and would obscure description of the methodology used to develop the metrics.

METHOD

Expert and project teams

A panel of experts in opioid therapy and VA pain management practice and policy (coauthors of this manuscript) were convened to guide development of individual metrics and metric definitions. This panel included: (1) the two co-chairs of the 2010 VA/DoD CPG workgroup; (2) the two National Program Directors for Pain Management in VA (one of whom was one of the cochairs of the CPG workgroup), (3) a clinical pharmacy specialist from the VA National Pharmacy Benefits Management Services, who has expertise in opioids and who was also a member of the CPG workgroup; (4) a primary care physician with expertise in opioid therapy and substance use disorders (SUDs); (5) the principal investigator who developed a clinical decision support system for opioid prescribing (ATHENA-opioid therapy) and who is also (6) the Director of the VA center charged with evaluating VA SUD treatment services; (7) a clinical psychologist with expertise in behavioral medicine treatment of pain; and (8) a doctorate level expert in patient safety practices. While members 5–8 were also part of the expert panel, they will be primarily referred to as the project team in the interest of clarity, as they were responsible for all facets of the development of the metrics (e.g., administrative duties). Team meetings with the expert team and project team via conference call occurred weekly early in the project and as needed thereafter.

Expert team members were requested to join the project because they are the leading experts in the VA on pain management and the new CPG, and they brought valuable expertise and perspectives to the project. The diverse professional backgrounds of the project team (e.g., behavioral medicine, program evaluation) provided a broad foundation for creating the metrics. However, any team such as this is inherently subject to the biases and opinions of its members. To address these potential biases, the expert team members were consulted frequently throughout the metric design process, and initial proposals from the project team were refined in response to their feedback.

Metric development process

In the first step of the metric development process, each member of the project team read the CPG, and members separately created lists of as many distinct recommendations and important elements of the CPG as possible. The diversity of the project team members’ backgrounds and knowledge of the field was beneficial to this process; although they identified overlapping themes, they were also likely to pull out distinct elements of the CPG relevant for treating patients. Taking the separately composed lists of important elements, the project team came together in several meetings and consolidated the separate lists into a single list of recommendations. Each recommendation was evaluated for its ability to capture the included clinical concepts in data available in existing VA administrative datasets.

In additional meetings the project team organized and reorganized this list of recommendations into clusters in discrete higher level domains with conceptual similarity. This allowed the project team to focus on higher level domains that captured important elements of the CPG even if the specific recommendations within that cluster fluctuated during the iterative process of allocating recommendations to domains. Due to the iterative nature of the allocation process, analogous to a thematic analysis of a text in a qualitative research study [14], specific recommendations were allowed to move between domains as the conceptualization of the domains evolved. Given the intention to use the developed metrics for quality improvement, recommendations were clustered with an eye toward grouping recommendations that might be addressed by similar interventions. This process led to a set of ten initial domains that emphasized important elements of how the CPG defined effective clinical care and could potentially be captured in administrative datasets.

These domains were then presented to the expert team (team members 1–4) for feedback in a series of conference calls. Team members were initially sent the domains and asked to evaluate each potential recommendation within the domains along several dimensions: (1) Clinical prioritization (high, moderate, low); (2) Strength of consensus on recommendation (good, fair, poor); (3) Prevalence of problems in this area (pervasive, common, occasional, infrequent); (4) Urgency of quality improvement in this area (high, moderate, low); and (5) Applicable populations of pain patients (e.g., all, acute, chronic, cancer, end of life). This step was important to familiarize team members with the initial structuring of domains and allowed them to have a clearer understanding of how the conference calls would proceed. Once they had time to review the ten domains across these dimensions, the project and expert team discussed them in a series of conference calls. Although expert team members were provided with nonnumerical categories to rate the dimensions, they made their own relevant comments. These comments proved useful, as they allowed us to more adeptly facilitate the discussion, moving forward only when we had reached consensus within each domain.

The expert team’s feedback led to significant changes to the specific proposals for measuring adherence within each domain. For example, in the domain containing recommendations related to “avoidance of sole reliance on opioids” the expert team pointed out that only about 30% of VA facilities have “pain clinics,” and thus while it was possible to capture their use by providers, such efforts would not likely lead to identification of the use of the diverse array of pain management interventions that are the target of the recommendation. Therefore, the project team sought alternative operationalizations of avoiding sole reliance on opioids rather than focusing only on capturing the use of pain clinics in available administrative data. The expert team agreed on a variety of alternative activities that could be considered avoiding sole reliance on opioids (e.g., physical therapy, weight loss programs). This part of the metric design process highlighted the importance of having a diverse team of experts interact with each other when evaluating the metric domains.

Concurrent with this evaluation of the initial ten domains, the project team began work on the concrete operationalizations of clinical concepts included in recommendations that the expert team agreed were important to capture. For example, in the domain about “Misuse Risk” the project team found diagnosis codes to identify patient populations considered at higher risk for misuse, and procedure, lab test, and clinic codes to indicate if patients were getting recommended treatments or monitoring to reduce misuse risk.

This process resulted in two of the initial ten domains being eliminated from further consideration, and their critical elements were incorporated into other domains. For example, we had originally created a “Referrals” domain in accordance with the guideline recommendation to refer complicated cases to appropriate specialty care. However, we were not able to obtain referral information from administrative data; we could only measure treatment receipt. Elements of the “Referral” domain were therefore shifted to other domains where possible. For example, patients with a recent SUD diagnosis should be referred to specialty SUD care, and the percentage of indicated patients who received specialty SUD treatment was shifted from the “Referrals” domain to the “Misuse risk” domain.

Each of the eight final domains now consisted of the overarching concept (e.g., order appropriate lab tests) and a critical concrete recommendation(s) within that domain (e.g., all patients should receive UDSs). A second round of conference calls with the expert team discussed the specific operationalizations of the elements in each domain. For example, the CPG specifies that patients should receive other pharmacotherapy (the clinical recommendation) to avoid sole reliance on opioids (the overarching domain). The expert team was asked to evaluate which specific medications were evidence-based alternative pharmacotherapies to opioids (e.g., identifying tricyclic antidepressants for pain).

As operationalizations were finalized by the project and expert teams, a team of programmers with extensive experience with VA administrative databases encoded the operationalizations and generated preliminary data on frequencies of included concepts and recent national performance on the draft metrics. The expert and project teams evaluated results for their face validity and refined operationalizations where they appeared problematic.

Definition of clinical concepts

Clinical concepts were defined based on the following data elements: (1) dates of clinical encounters; (2) diagnoses treated (per ICD-9 codes associated with encounters); (3) prescribed medications including fill date; (4) days supply; (5) formulation and daily dose; (6) labs ordered; and (7) procedures conducted (per CPT and HCPCS codes associated with encounters).

The work of the project team was informed by the process and results of an existing knowledge base constructed from the 2003 VA/DoD CPG for Management of Chronic Opioid Therapy [11] for use in the ATHENA-Opioid Therapy (ATHENA-OT) clinical decision support system [15, 16]. The knowledge base in a decision support system represents the defining and encoding of a CPG into actionable clinical practices. The accuracy and adherence of the ATHENA-OT knowledge base to the intent of the 2003 VA/DoD CPG has been previously validated through iterative testing by local clinical and content experts and three members of the committee who participated in development of the guideline [11, 16]. Existing definitions were reviewed by the project team to further ensure accuracy and to update domains where necessary based on recent improvements in the clinical evidence base and the 2010 VA/DoD CPG [12]. Specifically, ATHENA-OT included a limited list of medications that interact with opioids and a limited list of SUD diagnoses that were updated and reviewed by the expert team. Definitions not included in ATHENA-OT were drafted and reviewed by the expert team as part of the process described above.

Patient cohorts

Although the CPG focuses on opioid therapy for chronic pain, it was recognized that: (1) many of the recommended practices apply to all opioid prescribing (e.g., timely follow-up after opioid initiation); and (2) a substantial proportion of patients initiate opioid therapy for a new pain problem that becomes chronic only when it does not resolve. Thus, the expert and project teams decided to focus on clinical practices associated with all opioid prescribing that are relevant to all or most patients and to separate the population of patients who received an opioid prescription, based roughly on clinical perceptions of risk associated with certain types of opioids and duration of therapy. Specifically, the population the metrics were applied to include all patients with at least one outpatient opioid prescription for an oral or transdermal formulation from a VA pharmacy in the last 12 months, broken into four subgroups: Tier 1, prescriptions for long-acting opioid formulations; Tier 2, prescriptions for short-acting opioid formulations only, with greater than a 90 days supply over the prior 12 months; Tier 3, prescriptions for short-acting opioid formulations only, with a 90 days supply or less over the prior 12 months; Tier 4, prescriptions for Tramadol only. Of note, patients could be prescribed short-acting opioid formulations in addition to long-acting ones, or they may be prescribed more than one opioid in the same tier. Metrics are calculated at the facility level for each subgroup of patients.

In fiscal year (FY) 2010, the number of patients prescribed opioids in VA outpatient settings in each tier were the following: 95,059 patients in Tier 1; 399,024 patients in Tier 2; 535,522 patients in Tier 3; and 261,429 patients in Tier 4. For some metrics, we examined “new prescriptions,” defined as receipt of an opioid prescription with no opioid prescription within the previous 90 days. The total number of patients with new opioid prescriptions in FY2010 was 840,776.

Metric coding

Based on the definitions finalized with the expert panel, we wrote SAS code to calculate patient–level performance on each metric using encounter data in the VA National Patient Care Database and the VA Decision Support System laboratory and pharmacy files. Based on patient health care utilization patterns over the last 12 months, patients were assigned to the facility at which they received the majority of their health care, weighting individual outpatient encounters equivalent to residential and inpatient stays. Code was tested on data from fiscal year FY2009 and FY2010 administrative datasets.

RESULTS

The metric development process resulted in eight domains of clinical practice recommendations. These eight domains are: (1) side effect management, (2) dangerous drug interactions, (3) misuse risk, (4) appropriate follow-up, (5) avoidance of sole reliance on opioids, (6) safe and effective prescribing practices, (7) ordering of appropriate lab tests, and (8) conservative initiation dosing in frail patients. However, the eighth domain was eventually excluded because the expert and project teams were not confident that patients could be defined as medically frail based on available administrative data.

The final metrics and the rationale behind them are listed in Table 1. Some domains contain multiple metrics. Some proposed metrics that were theoretically codable were dropped because we could not determine whether the clinical action was related to opioid prescribing, as such clinical actions were common for other purposes. For example, in the “Side effects management” domain, clinical recommendations included prescribing antihistamines to manage opioid-related pruritis. Although we could identify antihistamine prescribing proximal to an opioid prescription, we could not tell whether this was for an opioid side effect or other more common indication.

Challenges to the operationalization of clinical concepts included lack of symptom data that might help distinguish clinical severity of disorders, and inconsistent use of administrative codes in clinical practice. For example, the CPG emphasized that as part of “Safe and effective prescribing practices,” patients should receive medication management and pharmacy reconciliation. While there are procedure codes that capture this activity and VA pharmacists regularly conduct medication reconciliation, VA pharmacists at most facilities did not code these activities prior to national guidance in FY2011 instructing them to do so.

In some cases, the project team identified multiple ways that recommendations could be appropriately followed or clinically coded to capture guideline-adherent management of opioid therapy. Because the expert team wanted to allow maximal clinical flexibility within the guideline recommendations, metrics generally include codes for all plausible guideline-adherent practices. Thus, the metrics were designed to be overly inclusive, allowing false positives in order to avoid false negatives. For example, in the “Appropriate follow-up” domain, the metric for follow-up of new opioid prescriptions gave credit for any clinical or phone encounter, rather than restricting follow-up appointments to specific clinic visits or provider types. Similarly, although there is clinical consensus that more than 4 g of acetaminophen per day is unacceptable, there is no clinical consensus on how much acetaminophen per day is appropriate. The metrics therefore identify the percentage of patients receiving more than 3 g of acetaminophen per day and more than 4 g. This design approach allows clinic managers to identify clearly nonadherent practices for quality improvement efforts. We expect that these metrics may be refined over time to more stringently identify guideline-adherent practices.

DISCUSSION

The development of metrics based on the 2010 VA/DoD CPG [12] is a critical step toward increasing the safety and effectiveness of opioid therapy for patients with chronic pain. These metrics allow facilities to determine areas they need to target to improve treatment for these patients. They also allow for tracking of the quality of opioid prescribing practices over time and for evaluation of quality improvement efforts. Our inclusion of non-VA-specific codes in metric operationalizations also means these metrics can be generalized to the private sector.

Currently, health policy experts and hospital administrators are lacking information to guide policy development and prioritization of implementation efforts to promote safe and effective opioid therapy. Although guidelines exist, these are based primarily on expert clinical consensus and encourage adoption of time- and resource-intensive practices (e.g., additional assessment and monitoring), which may be difficult to provide within current organizational structures and staffing patterns. These metrics will provide data to guide policy and operational decisions around opioid prescribing and pain management in facilities. For example, tracking adherence to these practices across facilities will identify areas in which there is substantial variation in clinical practice patterns, indicating areas where workable solutions exist but are not uniformly used. Such areas would likely be optimal targets for policy intervention that requires or stimulates uptake of good practices in place at some facilities. Tracking of these measures also allows examination of associations between use of these practices and adverse events (e.g., overdose mortality), resource utilization (e.g., unscheduled visits), and pain control (e.g., trajectories of pain scores over time) to identify practice patterns associated with the greatest impact on key health outcomes. This should help policy makers and hospital administrators prioritize implementation of practices with benefits in terms of safety and effectiveness of opioid prescribing.

This tool is also highly relevant to behavioral medicine professionals, as it provides measures of factors deemed critical to providing biopsychosocial care. The metrics developed allow for assessment of care coordination across settings, as well as medication adherence and receipt of psychologically oriented treatments. For example, if patients in one facility have high rates of encounters for psychologically oriented therapies, quality improvement teams might choose to examine the content of the psychotherapy encounters to determine if staff training in cognitive–behavioral therapy for chronic pain might improve the effectiveness of this existing local resource. We also suggest that this process of creating quality measures to assess adherence to CPG recommendations could be used to generate suites of metrics around other CPGs, including those where behavioral medicine professionals are core to effective treatment (e.g., Refs. [17, 18]).

Improving the safety and effectiveness of opioid therapy is currently a high priority for public health and clinical care. These metrics provide a needed tool for evaluating implementation of the recommendations of the recent CPG on management of chronic opioid therapy. Since beginning this metric development process, the VA program evaluators have decided to monitor these metrics on a quarterly basis to guide quality improvement in the management of chronic pain patients on opioid therapy.

Whereas monitoring adherence to guideline recommendations is a critical step toward quality improvement, once gaps in care have been identified, it is also necessary to determine which quality improvement strategies should be used and how they should be implemented. Guideline dissemination alone is unlikely to significantly affect practice, and innovative implementation efforts are needed to increase adoption of guideline recommendations and integration of recommendations into current care practices. It is not inherently straightforward for practitioners to interpret how guidance in a CPG translates into decisions they make in their daily interactions with patients. Multifaceted approaches that include the adaptation of psychological models for large-scale implementation of guidelines, including the use of behavioral reinforcers and motivational interviewing techniques, have been suggested [13, 19]. Continued work in identifying interventions that effectively improve clinical practices and in testing the impact of adherence to various clinical practice recommendations on patient outcomes is much needed. For example, others may want to develop interventions such as an informatics or panel management tool that could be used for tracking high-risk patients (e.g., patients with comorbid SUD diagnoses), looking at care management and treatment received for these patients. The clinical concepts defined in this metrics development process may be helpful for facilitating development of such informatics tools. Finally, it is important to note that although these metrics were designed to be considered at the facility level and include elements such as treatment provided in specialty clinics and other settings, it is possible that, in some cases, it may be useful to present these metrics by provider or clinic.

In summary, we have outlined a process for creating quality measures to assess adherence to recommendations of a CPG. These measures assess use of CPG recommendations across health care systems and will be used to guide quality improvement efforts by: (1) identifying effective model systems, (2) identifying gaps in care, (3) facilitating assessment of the impact of specific practice patterns on clinical outcomes, and (4) tracking the effectiveness of quality improvement interventions for changing clinician practice around opioid therapy. It may be beneficial to incorporate this process in future guideline development activities to facilitate quality improvement efforts based on CPGs.