INTRODUCTION

Approximately 15.3 million Americans rely on the individual market to purchase private health insurance.1, 2 In 2014, the Affordable Care Act’s (ACA) major coverage provisions created marketplaces where lower-income Americans receive subsidies to purchase private health plans. The ACA introduced “metal” levels of plan generosity, essential health benefits, protections for those with pre-existing conditions, and restrictions on underwriting premiums. The ACA also mandated the development of a quality rating system (QRS) for qualified individual market health plans.3

Health plan quality is multi-dimensional, encompassing measures of clinical quality processes and outcomes as well as measures of enrollees’ experiences with providers in plans’ networks. The provision of health plan quality information can yield value to multiple stakeholders. First, quality ratings can inform consumer decision-making, such as helping enrollees differentiate between high- and low-quality plan options. Second, quality information may be used by regulators to ensure minimum standards. Finally, quality ratings may motivate plans’ quality improvement activities.4

In 2017, the Centers for Medicare and Medicaid Services (CMS) implemented the QRS and Qualified Health Plan Enrollee Experience Survey to collect and report quality information for use by consumers and insurers. The QRS collects information on clinical quality, enrollee experience, plan efficiency, affordability, and management. Using these data, we examined variation in marketplace insurers’ plan quality performance and investigated how performance varies by organizational and market-related factors.

Prior research examining health plan quality performance has documented variation in overall performance in the commercial, Medicare Advantage, and Medicaid managed care (MMC) segments.5,6,7,8,9,10 Plan quality variation is a reflection of covered populations as well as organizational attributes that may reflect insurer priorities such as profits, market share, or quality reputation.11 Additionally, such attributes may correlate with insurers’ resources or capacity to engage in quality improvement.12, 13 Studies have investigated how performance varies by organizational attributes, including ownership status, size, strategic focus network, and product types (e.g., health maintenance organization (HMO), preferred provider organization (PPO)) as well as if an insurer is vertically integrated with a provider organization (provider-sponsored health plan (PSHP)), where a clinical entity (e.g., hospital, physician group, or health system) also controls the insurance organization, like Kaiser Permanente, Intermountain Healthcare, or Geisinger. Due to their operational, managerial, and financial alignment, PSHPs may be more adept at achieving high-quality performance. For example, PSHPs may more easily share data to identify enrollees’ care gaps or care transitions.

Market-related factors may also explain variation in health plan quality. Stronger insurance market competition may lead some insurers to invest in quality to compete more effectively for enrollees.18 However, more competition may also increase enrollees’ plan switching and reduce an individual insurer’s incentive to invest in quality improvement if its competitors may reap the benefits of such investments. Additionally, differences in population attributes may influence quality.13 For example, rural populations are generally less healthy than urban ones and thus may be associated with lower quality scores.19

To date, limited published research exists examining individual health plan quality. Recent works used the 2019 QRS data to examine variation in behavioral health and effectiveness of care measures, respectively.16, 17 These works found Medicaid Managed Care (MMC) organizations had lower quality ratings on 3 of 4 behavioral health measures and one effectiveness of care measure. Preferred provider organizations (PPOs) were also found to have lower quality relative to other plan types on two effectiveness of care measures. Non-profit insurers were associated with higher quality scores. Studies examining the Medicare Advantage market have reported higher quality among PSHPs versus non-PSHPs.10, 14, 15 Other researchers have found heterogeneous effects among PSHPs, whereby larger and non-profit organizations were associated with higher quality.9 Yet little is known about how PSHPs perform on quality ratings in the individual market.

METHODS

Overview

We analyzed the quality of products offered by health insurance marketplace insurers in 2019 and 2020. We examined variation in health plan quality measures across insurer products using descriptive statistics and multivariate linear regressions, with a focus on the relationship between insurer-product type and quality measures.

Data

Our main data source is health plan quality information from the 2019 and 2020 CMS Plan Quality Rating System (QRS) dataset. The ACA required the CMS to establish a quality rating system for ACA marketplace plans. Plans with more than 500 enrollees must submit data on QRS clinical measures and Qualified Health Plan (QHP) Enrollee Experience Survey responses as a condition of certification.20 The QRS measures come from the National Committee for Quality Assurance Health Effectiveness Information and Data Set and Pharmacy Quality Alliance, while the QHP enrollee survey is mostly based on the Consumer Assessment of Healthcare Provider and Systems surveys.

The QRS datasets were available from 2019 through 2020 and reflect insurer performance during the prior year. The data include 38 measures within the following domains at the insurer-product-state level: clinical quality management; enrollee experience; and plan efficiency, affordability, and management. The total number of insurer products was 565 during our study period. But, 172 insurer products were ineligible for data submission since insurer products were either too small or too new to the individual market, resulting in 393 insurer products in the QRS dataset.21

These quality measures are organized into a hierarchical structure (summary indicators, domains, and composites). Combinations of two or more individual measures are used to calculate a single composite score, and these, in turn, are combined to form a domain. Domain scores then are used to calculate three summary indicators for the global result, which is finally converted to a five-star rating system for each insurer product in each state.

Next, we merged on the Robert Wood Johnson Foundation (RWJF) Plan Participation Tracker Data to identify the population of ACA-compliant, individual market insurers operating in each state for 2019 and 2020, using a unique plan identifier for each qualified health plan approved by CMS. These data include information on each insurer’s operations across the United States and capture offered plans and insurer-product types.

We augmented these data with insurer-product organizational attributes using data from the National Association of Insurance Commissioners and RWJF plan tracker that identifies non-profit ownership, Blue Cross Blue Shield Association membership, and Medicaid focus. To construct the PSHP measure, we utilized multiple secondary information sources from the Alliance of Community Health Plans,22 the American Hospital Association,23 Baumgarten (2017),24 and Howard et al. (2018).25 In the few cases where there was lack of agreement in an insurer-product’s classification, we reviewed an insurer’s parent organization website to select the most appropriate classification. County population and rurality were derived from the Health Resources and Services Administration’s Area Health Resources File. We excluded plans with missing geographic information, and the resulting analytical sample includes 381 insurer products. The selection of our analytical sample is presented in Appendix Figure 2.

Outcomes

Quality outcomes were obtained from the QRS data. A rating, based on a five-star scale, is provided for the global rating and each component of the QRS hierarchy. Five stars represent the highest relative performance and one represents the lowest.

We started with analyses of insurer-products’ global rating and three summary indicators, including (1) clinical quality management; (2) member experience; and (3) plan efficiency, affordability, and management, providing a high-level evaluation for each insurer product. For the clinical quality management summary indicator, we then analyzed clinical effectiveness, patient safety, and prevention domains and their corresponding composite measure(s) within this category. A visualization of the QRS hierarchy is in Appendix Figure A1, and complete definitions of the outcome measures are in Appendix Table A1.26

Independent Variables

Our key explanatory variables included five indicators for insurer-product type. These binary indicators identify whether an insurer product (1) is part of a provider-sponsored health plan (PSHP); (2) has a Medicaid-managed care (MMC) focus; (3) has Blue Cross Blue Shield Association (BCBS) membership; (4) is a non-profit insurer; and (5) is a PPO. These indicators are not mutually exclusive.

We controlled for market-specific factors that may affect health plan quality scores. First, using Rural-Urban Continuum Codes,27 we created a rurality measure by calculating the proportion of an insurer’s geographic market that is comprised of rural counties. Second, we included a measure of marketplace competition in each insurer’s geographic market, a binary indicator for whether an insurer offers the lowest-premium silver plan in a geographic rating area-year. Marketplace enrollees are highly sensitive to premiums and thus tend to select the plan with the lowest premium in their chosen metal level. Whether an insurer offers the lowest-premium silver plan therefore indicates whether an insurer is likely to attract enrollees through premium competition. We measured both of these variables for the counties where an insurer product operates, weighting by county population.

Lastly, we included state and year fixed effects. Medicaid expansion is captured by state fixed effects, with the exception of the two states that expanded from 2019 to 2020. For this, we included state-by-year fixed effects.28

Statistical Analysis

The insurer product, observed by state-year, is the unit of analysis. Using multivariate regression analysis, we modeled health plan quality measures, ranging from one to five stars, as a function of non-mutually exclusive insurer-product types (PSHP, MMC, BCBS, non-profit, and PPO) and market-specific factors. We clustered standard errors at the insurer-product level. Model coefficients thus represent absolute differences in mean quality scores associated with belonging to a given insurer-product type, relative to not belonging to that insurer-product type. We conducted all analyses in Stata-SE 17.1. This study is exempt from IRB approval because it involves only publicly available information.

RESULTS

Our analytic sample consisted of 381 insurer products, representing 184 unique insurers in 50 states and the District of Columbia. Among the 381 insurer-product observations, 35% are PSHPs, 19% have a Medicaid focus, 28% are Blue Cross Blue Shield members, 57% have non-profit ownership status, and 22% are PPOs.

Figure 1 summarizes the distribution of global ratings by insurer type, with ratings ranging from one star (lowest) to five stars (highest). Provider-sponsored health plans had the highest overall quality performance—70% of plans had star ratings of four or five, followed by non-profit insurer products (54%). In contrast, 82% of Medicaid-focused insurers received three stars or fewer.

Figure 1
figure 1

Insurer-product overall star rating by insurer type. CMS only calculates a global score and assigns a star rating when the clinical quality management summary indicator has a score and at least one of the other two summary indicators has a score. If a plan’s global score cannot be calculated due to its inability to pass this scoring rule, the plan receives a “No Global” (NG) code and is unrated. Fewer than 5% of observations received a NG code in this analysis. The QRS Methodology can be found in the CMS Quality Rating System and Qualified Health Plan Enrollee Experience Survey: Technical Guidance for 2020. Insurer-product type indicators were not mutually exclusive in this analysis. More details can be found in the Appendix Table A2.

Full size image

Table 1 presents unadjusted summary statistics for the QRS hierarchy component ratings, categorized by insurer-product type. For the three summary indicators, plan efficiency, affordability, and management had an average rating of 3.46 (SD = 0.84), followed by clinical management (mean = 3.2; SD = 0.86) and enrollee experience (mean = 3.03; SD = 0.86). Within clinical quality management, clinical effectiveness, patient safety, and prevention domains had mean ratings of 3.21 (SD = 0.88), 3.72 (SD = 0.78), and 3.64 (SD = 0.83), respectively.

Table 1 Summary Statistics of Marketplace Insurer-Product Characteristics and Quality Measures
Full size table

Statistical Analyses

Table 2 reports adjusted differences across summary ratings by insurer type. For clinical quality management, PSHPs and non-profit organizations were associated with higher star ratings by 0.36 (95% CI = 0.11 to 0.62; P = 0.005) and 0.30 (95% CI = 0.10 to 0.51; P = 0.004), respectively. Also, PSHPs (95% CI = 0.03 to 0.50; P = 0.03) and PPOs (95% CI = 0.05 to 0.47; P = 0.02) exhibited positive associations with enrollee experience, whereas Medicaid-focused organizations were negatively associated with this outcome. Finally, non-profit insurer products were associated with a 0.24 (95% CI = 0.05 to 0.42; P = 0.01) higher rating with respect to plan administration. Ratings for plan administration were associated with lower ratings for Blue Cross Blue Shield, PPO, and Medicaid-focused organization types.

Table 2 Associations Between Insurer Type and Summary Ratings
Full size table

We report associations between clinical quality management domains and insurer-product type in Table 3. Similar to the summary indicator models, PSHPs were associated with a 0.33 (95% CI = 0.07 to 0.59; P = 0.01) higher rating than non-PSHPs in the clinical effectiveness domain. They also were associated with a 0.48 (95% CI = 0.25 to 0.70; P = < 0.001) higher rating on prevention relative to non-PSHPs. Insurers with Medicaid focus were associated with lower ratings by 0.34 (95% CI = − 0.61 to − 0.06; P = 0.02) on the clinical effectiveness domain relative to non-Medicaid-focused insurers.

Table 3 Associations Between Insurer Type and Overall Domains of Clinical Quality Management
Full size table

Table 4 reports the model estimates for the clinical effectiveness domain composites, including asthma, behavioral health, cardiovascular disease (CVD), and diabetes care. Here, PSHPs were associated with a 0.48 (95% CI = 0.14 to 0.82; P = 0.006) higher rating in behavioral health, and a roughly 0.35 higher rating in both cardiovascular disease (95% CI = 0.10 to 0.61; P = 0.007) and diabetes care (95% CI = 0.08 to 0.55; P = 0.008). Blue Cross Blue Shield plans were associated with a 0.35 (95% CI = 0.03 to 0.68; P = 0.03) higher rating for behavioral health, while non-profit insurers had a positive association with diabetes care quality (95% CI = 0.10 to 0.55; P = 0.005). Medicaid-focused plans were associated with relatively lower quality for CVD care, with a 0.39 (95% CI = − 0.67 to − 0.10; P = 0.008) lower rating relative to non-Medicaid-focused plans.

Table 4 Associations Between Insurer Type and Composites of Clinical Effectiveness Domain
Full size table

Within the prevention domain analyses, PSHPs were positively associated with all four composite measures (Table 5). Non-profit insurers were associated with a 0.26 (95% CI = 0.04 to 0.49; P = 0.02) higher rating in cancer prevention and a 0.39 (95% CI = 0.16 to 0.62; P = 0.001) higher rating in staying healthy adult composite. Medicaid-focused organizations were associated with a 0.33 (95% CI = − 0.64 to − 0.02; P = 0.04) lower rating in cancer prevention and a 0.4 (95% CI = − 0.72 to − 0.07; P = 0.02) lower rating in maternal care quality.

Table 5 Associations Between Insurer Type and Composites of Prevention Domain
Full size table

We found limited evidence regarding the association between market-related factors and quality outcomes. Insurers operating in markets with a higher proportion of the population located in rural counties were associated with lower quality scores on the plan efficiency, affordability, and management summary indicator as well as the staying healthy adult composite in the prevention domain. Additionally, insurers operating in more competitive markets were associated with lower quality for prevention and maternal care quality outcomes.

DISCUSSION

The individual market is an important source of health insurance coverage for Americans who otherwise lack a source of insurance. The Biden administration has signaled its commitment to improving affordability and accessibility of individual market coverage through passage of the American Rescue Plan Act and administrative actions. While individual market performance with respect to plan choice and affordability have been studied extensively, much less is known about individual market plan quality.

Our retrospective, cross-sectional analyses suggest that PSHPs exhibit higher quality than non-PSHPs for both clinical quality management and enrollee experience summary indicators, though it is not clear from our analysis whether this is a causal relationship. PSHPs reported quality that was approximately 8 to 13% higher, on average, for the management of chronic diseases, including CVD, diabetes care, and behavioral health as well as care targeting disease prevention (see Table 1). With tighter linkages between the financing and delivery functions, it is possible that PSHP organizations maintain stronger alignment of incentives to coordinate and manage enrollees’ care, such as through data sharing to identify enrollees’ care gaps or care transitions. Our results are similar to those of previous studies of quality performance among Medicare Advantage plans and international comparisons.10, 14, 15, 29

Other organizational attributes exhibited associations with the QRS summary, domain, and composite indicators, though patterns are less robust. Notably, non-profit organizations were associated with higher clinical quality management and plan administration scores. In contrast, Medicaid-focused insurers were associated with lower performance on enrollee experience, plan administration, and various outcomes related to clinical quality. Our results did not show systematic patterns in quality performance by Blue Cross Blue Shield membership, PPO insurer-product type, or differences in the market environment. As noted above, organizational attributes may proxy for what health insurers optimize, including profits, market share, or quality reputation. With respect to ownership status, non-profit plans may invest more heavily in quality improvement activities to promote their reputation. For-profit plans, in contrast, may strategically underinvest in quality as a means of risk selection (e.g., discouraging quality-sensitive consumers with high expected medical care needs from enrolling their plan).30,31,32 This is of particular concern in the individual market because, unlike Medicare Advantage, federal payments are not linked to quality performance.

Our work is subject to limitations. First, as an observational study, we cannot be certain of the causal relationship between organizational attributes and plan quality. In particular, higher quality health systems may be more likely to offer a PSHP versus a PSHP organizational form producing higher-quality care. Relatedly, we cannot account for additional heterogeneity within PSHP arrangements that may be related to quality outcomes. Third, we are not able to fully control for differences in the demographic, socioeconomic, or health risk–related composition of enrollees for a given insurer product. This information is not available in the public domain.

The availability of individual market plan quality information has the potential to influence consumers’ plan choices, though evidence suggests that premiums are a dominant factor, given high consumer price sensitivity and a subsidy design structure which encourages insurers to offer lower-priced plans to increase market share.33, 34

The creation of the QRS offers policymakers the opportunity to more explicitly account for quality in the subsidy design, should there be public interest in ensuring that individuals use public funds to purchase high-quality healthcare coverage.36 For example, federal legislation or state waiver programs could alter subsidy calculations and premium differences between high- and low-quality plans to reward consumers for choosing higher-premium plans with higher quality ratings. More broadly, plan quality information may contribute to federal antitrust regulation when considering the competitive effects of vertically integrated organizations, such as PSHPs.35 Future research is needed to investigate relationships among insurers’ organizational structures, quality performance, and premiums.