ABSTRACT
BACKGROUND
Starting in 2015, the Center for Medicare and Medicaid Services (CMS) requires all Medicare providers to report quality measures through Physician Quality Reporting System (PQRS) or incur a 1.5 % financial penalty. Research indicates that physicians believe this reporting does not lead to high quality care; however, little research has examined what PQRS actually measures, which is reflective of the physicians and patient disease populations being assessed.
OBJECTIVES
(1) Identify the proportion of measures that apply to different medical specialties, types of quality measurement, and National Quality Strategy (NQS) priorities. (2) Identify how different specialties are required to measure quality and NQS priorities. (3) Compare the 2011 and 2015 measures.
DESIGN AND MAIN MEASURES
This was a categorical qualitative analysis of 2011 and 2015 PQRS measures. One hundred and ninety-eight and 254 individual measures, respectively, were analyzed by three domains: medical specialty measured, type of measure, and NQS priority category.
KEY RESULTS
Between 2011 and 2015, the type of measures changed significantly, with fewer processes (85.4 % vs. 66.5 %, p < 0.001) and more outcomes (12.6 % vs. 29.1 %, p < 0.001). The measures showed no significant specialty or NQS category differences. For subcategories within each specialty in 2015, differences in measure type were statistically significant: surgery had the highest percentage of outcomes (61.1 %) compared to 21.7 % of internal medicine and 5.9 % of obstetrics/gynecology. For NQS categories, internal medicine had the highest percentage of effective clinical care measures (68.5 %), compared to 22.2 % in surgery. Surgery had the highest percentage of patient safety (31.9 %) and communication and care coordination measures (27.8 %) compared with internal medicine (5.4 % and 6.5 %).
CONCLUSIONS
Our study shows that PQRS measures include many medical specialties and significantly more outcomes in recent years, particularly for surgery. PQRS still lacks sufficient measures for half of NQS priorities and sufficient outcomes to assess internal medicine and obstetrics/gynecology. CMS must continue to improve PQRS measures to better assess and encourage high-quality care for all Americans.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
INTRODUCTION
In 2007, the Centers for Medicare and Medicaid Services’ (CMS) introduced Physician Quality Reporting Initiative (PQRI), now known as Physician Quality and Reporting System (PQRS). The purpose of PQRS was to give providers “the opportunity to assess the quality of care they provide to their patients, helping to ensure that patients get the right care at the right time,” and also allow providers to “quantify how often they are meeting a particular quality metric.”1 Starting in 2011, the program provided incentives for reporting, but this year it began requiring all providers who treat Medicare patients to report quality measures or incur a 1.5 % penalty from standard reimbursement. Payment penalties are allotted based on whether or not the provider reported measures 2 years prior—so penalties in 2015 are based on reporting in 2013 and penalties in 2017 are based on reporting in 2015.1 This change represents a monumental step for the American healthcare system because it makes reporting mandatory by one of the largest payers in healthcare. It also represents a transition from a fee-for-service system to a pay-for-performance one, or what the federal government calls the value-based purchasing model—in other words, with PQRS tying performance measurement to Medicare reimbursement, CMS is making quality of care a national priority and a priority that likely will be present for a long time. Furthermore, PQRS’s impact will likely be even greater because its data will be used to implement value-based purchasing programs, and the commercial sector is likely to model these CMS payment reforms in the next few years.
It is important to note that CMS makes all decisions about the quality measures included in PQRS. CMS chooses the measures from different quality measure lists created by many different measure developers, including the National Committee for Quality Assurance (NCQA), American Medical Association’s Physician Consortium for Performance Improvement (AMA-PCPI), specialty societies (like American Board of Internal Medicine and Society of Thoracic Surgeons), Health Resources and Services Administration (HRSA), and CMS itself.1 CMS also chooses the number of measures and diversity of specialties to which measures apply, both of which have increased as PQRS’s mission has broadened since its inception.1 Thus, CMS is ultimately in control of the types of measures and specialties that PQRS assesses.
As a whole, quality measurement is a controversial topic, with many physicians and researchers expressing criticism of quality measurement programs. They believe that they create additional administrative costs and are thus expensive to participate in.2–7 They believe that they do not lead to quality improvement.4,8,9 Nearly 84 % of physicians do not believe that Medicare quality reporting programs enhance their ability to provide patients with high quality care. Over half of physicians are extremely concerned that measures do not apply to specialty care.9
However, there are no studies systematically analyzing PQRS measures themselves. Given the fact that Medicare is the largest healthcare payer and that PQRS affects thousands of physicians and millions of patients, understanding the type of measures that CMS has chosen to use in PQRS and how this selection has changed over time is essential, because this gives an understanding of the different measures that providers can then choose from to sufficiently report to PQRS. This information is important for physicians, policymakers, and patients to better understand how quality of care is currently being measured, where gaps in measurement exist, and thus, how measurement can be improved in the future. In this study, we analyzed the 2011 and 2015 PQRS measures to characterize the kinds of measures used, which is reflective of the physicians and patient disease populations being assessed.
METHODS
Data Source
We obtained the 2011 and 2015 PQRS measures using publicly available data from the CMS website.1 Data on this website include instructions for how providers can implement PQRS and measure lists for all years since 2011. Measure lists describe significant information about each measure, including a detailed measure description, the measure developer, and reporting options for each measure (claims, registry, electronic health record, etc.) as well as each measure’s National Quality Strategy (NQS) priority category, starting in 2013.1 The 2011 PQRS measure list is the earliest list available on the CMS website, with 2011 being the year during which incentives for reporting began. The 2015 PQRS measure list was the most current list at the time that this analysis was conducted.
Measure Categorization
We categorized measures according to specialty, type, and NQS category. For specialty, measures were categorized into five groups: internal medicine, pediatrics, obstetrics/gynecology, surgery, and other specialties. These categories were selected by the authors to be broad and comprehensive enough to give meaningful information about the physician specialties that PQRS measures. Specialties were designated as a subspecialty of internal medicine according to the American Board of Internal Medicine’s classifications.10 Measures were classified as other if they did not fit in the category of internal medicine, pediatrics, obstetrics/gynecology or surgery, and included measures for specialties like radiology, pathology, neurology, and psychiatry. Measures could apply to multiple specialties and were documented accordingly.
We also categorized the type of measure as structure, process, outcome or other according to the Donabedian model for assessing healthcare quality.11 Measures were already designated into these categories by CMS in the 2015 list. For this categorization, structure measures describe the characteristics of the health care facility, such as regulations about the physical plant or certifications, process measures describe the activities done in a health care facility like tests ordered or treatments prescribed, and outcome measure describe the results achieved after receiving treatment from a healthcare facility, such as having high blood pressure controlled to a certain level.
Finally, we analyzed measures according to the six NQS domains as designated by CMS: effective clinical care, patient safety, communication and care coordination, person- and caregiver-centered experience and outcomes, efficiency and cost reduction and community/population health.12 Since NQS categorization was not already provided by CMS, we categorized the 2011 list, designating measures with the same categorization as 2015 when they were the same both years. Table 1 shows ten example PQRS measures and how they were coded according to specialty, type, and NQS category.
The first author (B.F.) coded all of the measures and was the primary reviewer. The second author (T.B.) reviewed all of the coding. When there was disagreement about the measure’s coding designations, the two authors discussed the measure to come to a consensus.
Analysis
We calculated the percentage of measures in each category (specialty categories, Donabedian types, and NQS domains). We also examined the percentage of structure, process, and outcome measures and the percentage of each NQS category within each specialty category. We performed a longitudinal analysis comparing measure characteristics (specialty, Donabedian type, and NQS category) from 2011 to 2015 and used the Pearson Chi-squared test to test for significant changes in the characteristics from 2011 to 2015. For the 2015 list, we used the Pearson Chi-squared test to compare specialty differences in the measure type and NQS category. All analyses were performed using Stata software, version 13 (StataCorp).
RESULTS
The 2015 PQRS includes 254 performance measures with 146 (57.5 %) assessing internal medicine specialties, 60 (23.6 %) assessing surgery, 14 (5.5 %) assessing obstetrics/gynecology, 30 (11.8 %) assessing pediatrics, and 75 (29.5 %) assessing other specialties (Table 2). When we categorized measures by type, we found that ten (3.9 %) assess structures, 169 (66.5 %) assess processes, 74 (29.1 %) assess outcomes, and one (0.4 %) assesses other measures. For the NQS priorities, 138 (54.3 %) focus on effective clinical care, 34 (13.4 %) on patient safety, and 37 (14.6 %) on communication and care coordination. The three other NQS priorities (14 person- and caregiver-centered experiences and outcomes, 16 efficiency and cost reduction and 15 community/population health) together account for 17.7 % of measures (Table 2).
The 2011 PQRS includes 198 performance measures, where 126 (63.6 %) assess internal medicine, 36 (18.2 %) assess surgery, ten (5.1 %) assess obstetrics/gynecology, 23 (11.6 %) assess pediatrics, and 55 (27.8 %) assess other specialties. When categorized by measure type, four (2.0 %) assess structures, 169 (85.4 %) assess processes, 25 (12.6 %) assess outcomes and zero (0 %) assess other measures. For the NQS priorities, 129 (65.2 %) focus on effective clinical care, 18 (9.1 %) on patient safety, and 28 (14.1 %) on communication and care coordination. The three other NQS priorities (six person- and caregiver-centered experiences and outcomes, seven efficiency and cost reduction and ten community/population health) together account for 11.6 % of measures (Table 2).
Compared with 2011, the 2015 measures had no significant specialty differences, with the majority of measures applying to internal medicine (63.6 % vs. 57.5 %). The type of measures changed significantly between 2011 and 2015, with fewer process measures (85.4 % vs. 66.5 %, p < 0.001) and more outcome measures (12.6 % vs. 29.1 %, p < 0.001) in 2015 versus 2011. There was no significant change in NQS measure categories between 2011 and 2015 (Table 2). When we looked at subcategories of measure types and NQS categories within each specialty in 2015, we found statistically significant differences in the measure type: surgery had the highest percentage of outcomes measures (61.1 %), whereas only 21.7 % of internal medicine measures were outcomes, and obstetrics/gynecology measures had 5.9 % outcome measure with only one outcome measure (Table 3). For NQS categories, internal medicine had the highest percentage of effective clinical care measures (68.5 %), whereas only 22.2 % of surgery measures were for effective clinical care. Surgery had the highest percentage of patient safety (31.9 %) and communication and care coordination measures (27.8 %) compared with internal medicine (5.4 % and 6.5 %) (Table 3).
DISCUSSION
Our analysis demonstrates both progress and room for improvement in CMS’s PQRS performance measure selection. The 2015 measures represent progress because they assess quality for many specialties, particularly those that have been lacking measures in earlier PQRS lists or lists by other organizations.4,7,13,14 Approximately half of the measures assess surgery, pediatrics, obstetrics/gynecology or other specialties; the other half assess internal medicine. The specialty distribution is more even in the 2015 measure list, compared with that of 2011. Nonetheless, the fact that half of the measures are still considered applicable to internal medicine seems to support the common criticism that quality measurement efforts disproportionally assess primary care. A closer examination of the internal medicine measures demonstrates that they assess many internal medicine subspecialties. The 2015 list contains 56 more measures than 2011 and many additional measures for diseases like inflammatory bowel disease, rheumatoid arthritis, hepatitis, adult sinusitis, sleep apnea, and dementia.1,13 By including these different diseases, the PQRS list applies to many more specialties, even subspecialties within internal medicine, than lists used by private payers such as the Healthcare Effectiveness Data and Information Set (HEDIS).14
Notably, CMS included a sizeable amount of outcomes in the 2015 measure list. This is evident especially compared to 2011 and more specifically for surgery. This is significant because outcome measures often directly measure events with the clearest importance in clinical management and treatment. For example, patients and physicians alike are most interested in preserving quality of life, survival after serious conditions, avoiding hospital readmissions, and having positive medical experiences, in general.4,15
Despite this progress, our analysis demonstrates that CMS still needs to address some key aspects of quality with its PQRS measures. The current measures do not evenly address all NQS domains, focusing on clinical effectiveness with little emphasis on patient-centeredness, efficiency, and population health. This representation has remained relatively unchanged between the 2011 and 2015 lists. This is significant because the six NQS priorities appear to be CMS’s means of addressing the Institute of Medicine’s (IOM) six aims for the twenty-first century health care system as enumerated in the 2001 Crossing the Quality Chasm report.16 Many would argue that the US healthcare system still must adequately address these aims. Although different specialties may necessitate measurement predominantly by one NQS priority over another, there are certainly issues related to each priority that should be measured and that providers should strive to improve upon. CMS must continue to develop measures that address the three least represented NQS priorities for each of the major medical specialties. As an evolving program, with wide-reaching effects, PQRS has the potential to have a significant impact on better achieving all six NQS priorities more evenly.
The measures also lack outcomes for non-surgical specialties. Predominant use of process measures is problematic because studies show that only some processes correspond with meaningful patient outcomes.15,17
CMS must embrace its role as a leader in quality measure development and start designing outcome measures for PQRS specifically for the medical specialties that lack them. CMS also might consider collaborating specifically with additional medical specialty societies, in addition to its other measure developing collaborators, to advance measure development for medical subspecialties and non-internal medicine specialties. For internal medicine, CMS or its measure-developing collaborators might consider developing outcome measures around diagnostic quality and morbidity/mortality from chronic diseases. For obstetrics/gynecology, CMS might be really innovative, because other measure endorsers and developers like the National Quality Forum (NQF) and American Medical Association’s Physician Consortium for Performance Improvement (AMA-PCPI) either do not list obstetrics/gynecology outcomes or solely measure pregnancy care.18 Outcomes related to menopause management and hysterectomy, for example, might matter to many patients affected by PQRS (i.e., older adults). Although measure development will not be easy, CMS must use its broad relevance to many medical specialties through PQRS to be an innovator in the quality measurement field.
Undoubtedly, issues will arise transitioning to more quality measurement, particularly increased outcomes. They require more complicated risk adjustment, sufficient sample size, and carry the risk of surveillance bias.17 Another concerning issue would be incentivizing providers to avoid high-risk patients, a concern that 82 % of physicians believe could be a real consequence of performance measurement and reporting.19,20 As physicians become more rewarded for their performance, they may start to only treat patients who promise good outcomes, such as young healthy patients with the greatest access to care.21,22 This unintended consequence must be kept in mind when designing and implementing outcomes measures more broadly. It might require additional adjustments or incentives for physicians serving high-risk populations.
CMS must be heralded for its accomplishments in continuing to develop improved PQRS measures, while still being encouraged to fine-tune its approach to better measure quality. Moving forward, it will be important to understand whether increasing numbers and scope of measures have effects on physicians and patients. Nonetheless, in the next few years, due to its linkage of reimbursement and measure reporting, PQRS has great potential to provide an important impetus and opportunity for CMS to better meet national quality measurement needs and ultimately ensure long-term improved quality of care for all Americans.
REFERENCES
Center for Medicare and Medicaid Services. Physician Quality Reporting System. Available at: http://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/PQRS/index.html?redirect=/PQRS/. Accessed 23 Mar 2016.
Halladay JR, Sloane PD, Lefebvre A, Warburton SW, Wolf M. Should your practice participate in a quality-reporting program? Fam Pract Manag. 2011;18(1):9–14.
Sloane PD, Wroth T, Halladay J, Bray P, Spragens L, Stearns S, Sheryl Z. How eight primary care practices initiated and maintained quality monitoring and reporting. J Am Board Fam Med. 2011;24(4):360–9.
Meyer GS, Nelson EC, Pryor DB, James B, Swensen SJ, Kaplan GS, Weissberg JI, Bisognano M, Yates GR, Hunt GC. More quality measures versus measuring what matters: a call for balance and parsimony. BMJ Qual Saf. 2012;21(11):964–8.
Kessell E, Pegany V, Keolanui B, Fulton BD, Scheffler RM, Shortell SM. Review of medicare, medicaid, and commercial quality of care measures: considerations for assessing Accountable Care Organizations. J Health Polit Policy Law. 2015;40(4):761–96.
Berman B, Pracilio VP, Crawford A, Behm WR, Jacoby R, Nash DB, Goldfarb NI. Implementing the physician quality reporting system in an academic multispecialty group practice: lessons learned and policy implications. Am J Med Qual. 2013;28(6):464–71.
Panzer RJ, Gitomer RS, Greene WH, Webster PR, Landry KR, Riccobono CA. Increasing demands for quality measurement. JAMA. 2013;310(18):1971–80.
Federman AD, Keyhani S. Physicians’ participation in the Physicians’ Quality Reporting Initiative and their perceptions of its impact on quality of care. Health Policy. 2011;102(2–3):229–34.
Medical Group Management Association. Physician practice assessment: Medicare quality reporting programs. 2014. Available at: http://www.mgma.com/Libraries/Assets/Government%20Affairs/Issues/Federal%20Quality%20Reporting%20Programs/PPA-Results-Medicare-Quality-Reporting-Programs.pdf?ext=.pdf. Accessed 23 Mar 2016.
American Board of Internal Medicine. Board certification by specialty. Available at: http://www.abim.org/specialty/default.aspx. Accessed 23 Mar 2016.
Donabedian A. Evaluating the quality of medical care. Milbank Mem Fund Q. 1966; 44 (3): Suppl:166–206.
US Department of Health and Human Services. About the National Quality Strategy. Available at: http://www.ahrq.gov/workingforquality/about.htm#develnqs. Accessed 23 Mar 2016.
Center for Medicare and Medicaid Services. 2011 Physician Quality Reporting Measure List, Specifications Manual and Release Notes. Available at: http://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/PQRS/2011-Physician-Quality-Reporting-System-Items/CMS1254515.html. Accessed 23 Mar 2016.
National Coalition for Quality Assurance. HEDIS 2015. Available at: http://www.ncqa.org/HEDISQualityMeasurement/HEDISMeasures/HEDIS2015.aspx. Accessed 23 Mar 2016.
Chassin MR, Loeb JM, Schmaltz SP, Wachter RM. Accountability measures—using measurement to promote quality improvement. N Engl J Med. 2010;363(7):683–8.
Institute of Medicine. Crossing the quality chasm: a new health system for the twenty-first century. Washington: National Academies Press; 2001.
Berenson RA, Provonost PJ, Krumholz HM. Achieving the potential of health care performance measures. Princeton (NJ): Robert Wood Johnson Foundation; 2013.
Agency for Healthcare Research and Quality. National Quality Measures Clearinghouse: Measures by Organization. Available at: http://www.qualitymeasures.ahrq.gov/browse/by-organization.aspx. Accessed 23 Mar 2016.
Casalino LP, Alexander GC, Jin L, Konetzka RT. General internists’ views on pay-for-performance and public reporting of quality scores: a national survey. Health Aff. 2007;26:492–9.
Chien AT, Wroblewski K, Damberg C, Williams TR, Yanagihara D, Yakunina Y, Casalino LP. Do physician organizations located in lower socioeconomic status areas score lower on pay-for-performance measures? J Gen Intern Med. 2012;27(5):548–54.
Goldberg AL, Jacob A. Creating better doctors or merely finding better patients?. Family Medicine and Medical Science Research. 2013; 2(2).
Lynch HF. Discrimination at the doctor’s office. N Engl J Med. 2013;368:1668–70.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
The authors declare no conflicts of interest.
Disclaimers
None.
Funding
Dr. Bishop is supported by a National Institute On Aging Career Development Award (K23AG043499) and by funds provided as Nanette Laitman Clinical Scholar in Public Health at Weill Cornell Medicine. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Rights and permissions
About this article
Cite this article
Frankel, B.A., Bishop, T.F. A Cross-Sectional Assessment of the Quality of Physician Quality Reporting System Measures. J GEN INTERN MED 31, 840–845 (2016). https://doi.org/10.1007/s11606-016-3693-3
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11606-016-3693-3