Introduction

Mental illness profoundly impacts both personal health outcomes and the systems that work to support health. Worldwide, mental illnesses are the leading cause of reduced disability-adjusted life years1, 2; in the USA, nearly one in five (18.6%) adults likely experience a mental illness in any given year.3 Studies have consistently found that mental illness may precipitate chronic disease, and that chronic disease may precipitate mental illness.4,5,6,7,8,9,10 Mental illness is also correlated with a range of poor health outcomes, especially for those who also have other chronic health challenges.11,12,13,14,15,16 In short, health is a “whole person” affair, and health care systems must respond with structures that acknowledge this basic truth.

Behavioral health integration (BHI) has emerged as a core health care transformation strategy for this very reason. Many people use primary care settings as a foundational tool for managing their overall health, and efforts to integrate behavioral health care into primary care are well underway across the nation. Those efforts often hinge on the idea of positioning primary care clinics as “medical homes” that are prepared to address a wide array of whole-person health concerns through better coordinated care,17, 18 often by improving access to, and normalizing treatment for, behavioral health challenges.19,20,21,22,23 Other systems are working the problem from the opposite direction, integrating elements of primary care into behavioral health settings.

Research has examined a variety of integrated care models, with approaches ranging from enhanced offsite referral relationships to full on-site integration.24,25,26,27,28 Overall, the evidence supports outcomes such as symptom reduction and functional improvement,29,30,31,32,33,34 while conclusions around cost reduction have been less promising,35 though there is some thought that BHI efforts may be well-aligned with health care’s move toward value-based payments, possibly creating new paths to financial stability.36

One place where the literature has been thin: exploring how BHI impacts patients and their experiences. Many BHI studies have focused on clinician efficiency and revenue generation over patient outcomes37,38,39,40,41,42,43,44; those that do focus on patients often assess outcomes through a clinical lens rather than explicitly assessing how BHI impacts the outcomes that matter most to patients in their daily lives. As a result, for many health systems, BHI has come to be about the structural and organizational changes they are making rather than the experiences patients are having; improved patient experience becomes an assumed outcome rather than an active design principle.

In this study, 12 primary care clinics in varying stages of behavioral health integration were examined in Oregon, a state where changes to policy and incentive structures have created significant momentum for integrated, whole-person care. Oregon’s approach to integration focuses on five major domains: (1) integrated staffing, or building care teams that include both physical and behavioral health providers; (2) integrated training, including training providers how to approach patients with behavioral health challenges, often including trauma-informed care training; (3) data sharing, or ensuring that providers can see all the data they need to build a full picture of their patients’ health across physical and behavioral domains; (4) integrated workflows, or ensuring continuity and warm hand-offs between different types of providers; and (5) integrated financing, including payment reform that moves away from fee for service arrangements and places providers at increased risk for whole-person health outcomes.

These five domains represent broad goals of BHI efforts in Oregon. All 12 participating clinics were working in various ways, and had been working for varying amounts of time, to achieve those goals. Participating clinics were all primary care and safety-net clinics, so their efforts were primarily directed at providing a more comprehensive care experience for their enrolled patient panels. Information about each clinic’s progress toward implementing core elements of Oregon’s BHI model, data on key patient-reported outcomes over time using longitudinal surveys, and data from in-depth qualitative interviews were collected. The association between the adoption of BHI elements in the clinics and the outcomes that matter most to patients was then assessed. It was hypothesized that patient outcomes would improve more over time at clinics that had made greater progress on implementing the key elements of BHI.

Methods

Study design and population

The study employed a longitudinal mixed-methods observational design across 12 clinics in Oregon, collecting both survey and interview data at two time points. All clinics offered primary care, accepted a range of payers, and were located in both urban and rural areas (Appendix Table 5). All clinics reported engaging in BHI efforts prior to the start of the study; efforts ranged from the addition of behaviorists on staff to the provision of primary care within a facility designed to treat patients with severe and persistent mental illness.

Administrative records were used to construct a stratified sample of 11,648 patients from the 12 study clinics. Participants were required to be 18 or older, English or Spanish speakers, and to have had at least one visit to a participating clinic of any type in the past 12 months. Patients with behavioral health challenges were oversampled because it was anticipated that these patients might be especially likely to experience the benefits of BHI, but the sample was not limited to only those with behavioral health challenges. It was hypothesized that the benefits of BHI, such as more seamless data sharing between providers or better provider-patient communication as a result of training, might be evident for all types of patients.

Survey development, interview protocol, and survey fielding

Data was collected using three approaches: (1) a Clinic Audit Tool (CAT) designed to be completed by the clinic managers and capture adoption of key elements of integration; (2) a representative longitudinal panel of patients surveyed on two different occasions using a Health Care Experiences survey (HCES); and (3) a series of in-depth follow-up interviews with patients designed to contextualize results.

To ensure that the study incorporated a patient-centered lens, a Patient Advisory Team (PAT) was organized that consisted of 14 active patients with at least one each of behavioral health and chronic physical health conditions and lived experience navigating the care system for themselves, family members, and peers. Members were recruited from both rural and urban areas and varied in age, gender, and sexual identity. The PAT helped develop the study aims, selected the patient-centered outcomes to be measured, assisted in the development of study materials including protocols and survey measurement domains, helped design of participant outreach and fielding protocols, and assisted with the interpretation and dissemination of results for the patient and advocacy community.

The clinical audit tool

The CAT was used at baseline to assess adoption of key BHI elements in the study clinics. As part of previous work under its State Innovation Model (SIM) grant, Oregon’s Health Authority partnered with researchers to develop a measurement tool that could assess clinics and health systems’ adoption of these key transformation elements.45 The CAT tool was used in this study because it was specifically designed to measure Oregon’s integration domains and, since it had been used before in the state, would be relevant to state policymakers and systems leaders.

The CAT is a 25 question assessment tool designed to “score” clinics along the five key domains of integration integral to Oregon’s approach: integrated care staffing, integrated care training, integrated data sharing, integrated workflow and collaboration, and integrated financial arrangements (Table 1). The tool is explicitly designed to assess progress toward these key integration goals, but does not necessarily seek to understand how clinics were working toward those goals—a key distinction in interpreting its results, since it may illuminate important elements of integration but not the tactics best used to achieve them. The scoring approach is further described below.

Table 1 Measures of integration at participating study clinics
Full size table

Health care experiences survey

The HCES was designed in collaboration with the PAT, and included questions assessing need, access, and satisfaction in regard to physical, mental, dental, and substance use care, as well as a range of self-advocacy and health efficacy questions, subjective health and wellness outcomes, and experiences of stigma. The PAT selected the primary outcomes based on their assessment of the outcomes that would matter most to patients in an integrated system of care. The survey underwent cognitive testing with a small sample of patients before being fielded.

Opt-out postcards were mailed out to sampled patients at the participating clinics. After exclusions for individuals who moved out of state, died, or refused participation, the pool of potential study participants was 11,323. Survey data was collected from this pool using a combination of mail, telephone, and door-to-door outreach. Once participants were in the study, an aggressive panel retention strategy was developed which included periodic address updates and outreach to third-party contacts (provided by each participant upon initial enrollment) to locate individuals lost to follow-up. In year 1, a total of 4368 participants (39% of the eligible pool) completed the survey. Of the year 1participants, 2524 (58%) responded in year 2.

Initial survey outreach and panel recruitment was conducted via a multi-wave, mixed-mode protocol: participants were contacted several times by mail and asked to complete a paper survey. Outreach included telephone and email reminders. Non-responders were advanced into a more intensive follow-up protocol. This included up to five telephone attempts to complete the survey using a the survey tool accompanied with a telephone script, and in-person, door-to-door outreach effort that utilized the paper survey, but was hand delivered and retrieved by study staff. Surveys were available in English and Spanish, and all patients received a modest stipend for their participation.

Qualitative interviews

A total of 85 patients were interviewed during the study period. Patients were eligible for the qualitative portion of the study if they indicated at the end of the survey their willingness to participate. Of those expressing interest, respondents who self-reported needing both physical and mental health care were purposefully oversampled. Then, the sample was stratified by responses to survey items about access to care, providers and clinical staff, and stigma within a care setting. Interviews were recorded and transcribed for analysis.

Data collection began in September 2015, and included both the fielding of the CAT and the first HCES survey, which assessed patient experiences over a 12-month recall period. The second round of data collection was initiated approximately 1 year later. Qualitative interviews followed closely after each survey period.

Key measures

Integration score

The primary independent variable was an Overall Integration Score derived from the CAT, which consists of 25 questions about BHI practices or elements that a clinic could adopt. Clinics receive points for each question: 0 if they have not adopted any of the indicated integration component, 1 if they have partially implemented the indicated component, and 2 if they already have mature systems in place regarding that aspect of integration. Questions are summed to produce scores within each domain, as well as an overall integration score that ranges from 0 (least integrated, with no changes adopted at all relative to Oregon’s integration elements) to 50 (most integrated, with all elements fully developed and implemented). Thus, higher scores in the integration variable represent clinics with more mature systems of integration.

Patient experience outcomes

The study team and the PAT worked together to select a set of outcomes that matter most to patients in the context of integrated, whole-person care. These outcomes acted as the dependent variables in this study. The PAT recommended that the study focus on four key integrated care experiences:

  • All Integrated Care Needs Met: Patients who, in the last 12 months, “always” (vs “sometimes” or “never”) got ALL of the care they needed across the health continuum, including physical, mental health, and substance use care.

  • Timely Access: Patients who, in the last 12 months, were “always” (vs “sometimes” or “never”) able to get appointments for all three types of care—physical, mental health, and behavioral health—as quickly as needed.

  • Hassle-Free Care: Patients who reported that it was “never” (vs “sometimes” or “always”) a hassle to get ALL of the care they needed in the last 12 months; and

  • Providers Communicating Well: Patients who reported that, in the last 12 months, it “always” (vs “sometimes” or “never”) felt like their medical, mental health, and other specialty providers were communicating and coordinating with each other.

Quantitative analysis

Multivariable logistic regressions with generalized estimating equations (GEE) were used to assess the relationship between the independent variable and each outcome of interest over time. GEE methods were used to allow for clustering of responses in the longitudinal framework, accounting for non-independent person-level responses. Estimates were adjusted for age, sex, ethnicity, race, self-reported behavioral health conditions, and the number of chronic physical and behavioral health conditions.

For analyses of effects over time, GEE approach (clustering by individuals over time) was used, which separately assessed three key terms of interest: the impact of integration score on outcomes, the impact of time on outcomes (looking at the differences between outcomes scores in year one and year two), and an integration × time interaction term representing the differential effect of time on outcomes of interest for patients in different integration settings. In the context of these models, the integration score term represents the “main effect” of integration on the outcome of interest independently of time, the time term represents the “main effect” of time on outcomes independently of a clinic’s level of integration, and the interaction term represents whether trends in patient outcomes differed over time at clinics with different integration scores. The latter is the primary outcome of interest; it was interpreted that a larger interaction term was evidence that different levels of integration resulted in differential changes in patient experience outcomes over time at the study clinics.

Analytic interpretations focused on the integration × time interaction term, rather than on the “main effect” term of integration itself. While the main effect term indicates the association between integration scores and patient outcomes irrespective of time, there is no way of knowing why that association exists—whether it is a byproduct of effects predating the assessment of integration scores, natural differences in population composition between clinics, or something else. Integration scoring was instead used to establish where clinics were on integration at the start of the study, then measured patient experiences at that baseline point and again at follow-up. By focusing on change from baseline, which is expressed through the interaction term, the hypothesis that patients in highly integrated clinics tend to experience better changes in their outcomes over time compared with patients in less integrated clinics can be tested.

These analyses were repeated for a subgroup of patients who reported at least one behavioral health condition, with the idea that patients with integrated care needs might be more sensitive to the impacts of BHI efforts. All models were also repeated for the overall integration score and for each of the five individual domains of integration to test the robustness of the study findings. Consistent findings across subgroups and individual domains of integration would validate the results for overall integration effects. P values are reported throughout the analysis, but were only used as rough guidelines for findings of interest and not to determine the clinical importance any finding. P values should be interpreted with caution when conducting multiple statistical tests and with samples of this size.

Qualitative analysis

The qualitative analysis was informed by two approaches: (1) a case study approach was used to understand an experience within a specific, bounded setting using multiple sources of data and (2) a phenomenological approach centered on the lived experience of the participant with a specific object, or phenomenon.46 A team of trained qualitative researchers collaborated on the data review and analysis. Interviews were coded using an a priori dictionary that was refined over the course of the study as new thematic information emerged. A trained scientist reviewed a portion of coded documents for quality control. Patterns and themes were identified using an applied thematic analysis approach.47 While many themes emerged from the data, the themes outlined here provide more insight into the quantitative results presented in this study.

Results

Clinic integration

There was reasonable variation across most integration domains, although in some cases, especially the integrated workflow and integrated staffing domains, study clinics were mostly concentrated in the mid and higher scoring range. Table 2 shows the distribution of integration scores from the CAT across the participating clinics, including variation in the overall integration score and for each of the five distinct integration domains.

Table 2 Variance in integration scores across 12 study clinics
Full size table

Demographics

Table 3 highlights the demographics of the survey panel participants for years 1 and 2. Just under half of the initial sample were age 55 and above, two-thirds were female, and the majority were non-Hispanic (87%) and white (86%), reflecting Oregon’s population composition. About half self-reported at least one mental or behavioral health diagnosis. Year 2 respondents did not differ from year 1, suggesting that panel attrition did not substantially alter the sample composition from baseline.

Table 3 Demographic characteristics of survey panel participants
Full size table

Quantitative results

Table 4 details primary results for the analysis of the impact of overall integration score on patient outcomes over time. The main effect of BHI is negligible, while the time term shows some increase in the all care needs being met outcome (OR=1.81, 95% CI 1.02–3.21, p=0.04), suggesting that patients’ success getting their care needs met tended to grow with more time at the clinic. However, this was not necessarily due to behavioral health integration: adjusted odds ratios for the interaction effect indicate that the primary outcome of interest—the integration × time term—hovered near 1.0 and were not statistically significant. This suggests that overall integration scores were not associated with changes over time across any of the four key patient experience outcomes: all care needs being met (OR=0.99, 95% CI 0.97–1.0, p=0.136), timely access to care (OR=1.00, 95% CI 0.98–1.02, p=0.91), hassle-free care (OR=0.99 95% CI 0.98–1.01, p=0.35), or providers communicating well (OR=1.00, 95% CI 0.98–1.02, p=0.97). Taken as a whole, Table 4 suggests that there is insufficient evidence to suggest overall integration scores impacted patient experience outcomes in the study.

Table 4 Association between overall integration score and patient experience outcomes over time
Full size table

Additional analysis was conducted on a subgroup of patients reporting at least one behavioral health condition, under the idea that they might be the most likely to notice the impacts of BHI. Results were nearly identical to those shown in Table 4 (data not shown), providing reassurance that the BHI impacts were not being masked by including all patients, including those without behavioral health needs, in the primary analysis.

Finally, Appendix Table 6 highlights the results for analyses of the impact of domain level integration scores on patient outcomes over time. This approach was repeated with five models, using scores for each individual domain of integration as the primary predictor to see whether specific aspects of integration might be driving improvements in patient outcomes, even in the absence of overall effects. The main effect of BHI remains minor and inconsistent across these domains, and the interaction effect is similar to the analysis in Table 4. Taken together, this analysis suggests little empirical support for the idea that specific dimensions of BHI implementation activities might have driven better patient experience outcomes over time.

Qualitative results

Results from the qualitative interviews brought some degree of clarity to these results through the emergence of two key themes.

Key Theme #1. Patients’ lens on integration is experiential, not structural

First, results suggested that patients are generally indifferent about integration as health systems see it: For a majority of interviewees (91%), the concept of “integration” does not fully resonate with them and the work of integration is largely invisible. Instead, patients care about having a doctor or care team who is collaborative and compassionate:

My doctor really hones in on the issue. [Provider is] very – what’s the word? [Provider] shows compassion. [Provider] shows true care, [and] is concerned with this issues I have. [Provider is] very supportive.

I’ve been with [provider] a long time. I had a concern about the amount of time [provider] was able to spend with me, but I voiced that concern. And so it’s changed – it seems to have changed. [Provider] seems a little less rushed.

Some participants did speak about BHI structural domains as essential components to strengthening the patient’s clinic experience. However, few interviewees connected these directly with the work of integration, and whether those things happened in a clinic that was “integrated” was not of particular importance to them:

[The care team] works as a team, and it’s real clear how and why they do. They do a really good job. They watch out for me when, in and out of the hospital, it’s a brand new thing.

I’m pretty sure that [provider 1] and [provider 2] talk to each other… I think it is better that way. I think communication is super important, so if they can talk to each other and I don’t have to keep repeating things over and over and over again, then I like it.

Key Theme #2. Patients see a potential downside to integration.

While patient interviews did reveal positive takes on some aspects of integration, at least in terms of how patients might experience it, they also revealed a strong potential downside: A quarter of interviews (25%) discussed an undercurrent of concern about stigmatization and lack of compassion from providers and care staff. Sometimes this concern was about the possibility that integrated health care systems create transparency that patients are not always comfortable with:

I saw this psychologist at the same doctor’s office, and in a way it’s nice because they can talk to each other and coordinate my care. At the same time, I didn’t like that some of my stuff was being put in my chart, because then I was in the hospital, and I’d have nurses or doctors come ask me about – refer to things that were in my chart that I was kind of like, ‘well, I don’t really want you talking about [that].’

In other instances, patients were worried that integration will push them to get more of their mental and behavioral health needs met in primary care settings—the very settings where they have historically felt the most stigmatized and misunderstood:

In [primary care provider’s] records of whatever conditions I have, I’m listed as an alcoholic. And I resent that. […] But I think my history, even though I overindulge, is indicative of willpower. […] There’s a sense of condemnation [from primary care provider], you know, that – the head shaking. You know? ‘Alcoholic.’

[My primary care provider] thinks I’m a complete user. And she stereotyped me. And I told her I don’t. I’ve taken several urine tests after that, and I tested clean. And I don’t understand it. […] They don’t pay attention. They don’t read their notes. And only the bad stuff on it. […] They’re not taking care of me and my health. They’re trying to overload themselves as being god, I feel. And to tell you what you can and can’t have.

And when it comes to connecting with a [primary care provide], it’s very mechanical. And it’s an automatic, ‘Okay, well go to physical therapy.’ And then physical therapy is really expensive and I can’t pay for it up front, you know, and I realize that physical therapy is a great thing. But it’s not something I can afford in my life. And it’s always the answer. And when I say that I can’t afford it, they say, ‘well that’s what you need to do.’ And that’s the end of the conversation. Right, so it’s mechanical, it’s this is what you do. And if you can’t do that well you’re screwed. Right?

Discussion

A patient-informed research design was used in this study to better understand the effects of integration on patients’ experiences of care and to reveal which organizational efforts most address patient’s priorities and promote a patient-centered experience. Longitudinal surveys were used to assess whether primary care clinics who had implemented more elements of behavioral health integration saw better improvements in key patient experiences over time, then used in-depth qualitative interviews to explore and contextualize those results.

Overall, these findings suggest that behavioral health integration, at least as measured in the study, did not improve patients’ key integrated care experiences. Patients were asked to help identify what better care in the context of BHI might look like, then measured those outcomes over time to see if patients in more integrated settings saw better improvements over time. In general, the results suggest that they did not—neither for patients in general, nor for patients who reported having at least one behavioral health condition.

There are several ways to look at this primary finding. First, though integration did not shape patient experience outcomes, it may still have had positive impacts on other types of patient outcomes, including quality metric scores derived from administrative data, provider satisfaction, or even other patient experience measures that were not included in the study. Additionally, the measure of integration—adapted from a tool Oregon has used to assess adoption of key BHI practices—may not have been sensitive to the elements of integration that most strongly drive patient experiences, or the impacts of integration over time may take longer to realize than the 1 year that passed between the first and second survey periods. And because the study period is just one window into a long BHI journey for participating clinics, it is also possible that there were early improvements in patient outcomes, but that the gains were already evident and stable by the time the study started.

Regardless, the findings may question some fundamental assumptions about BHI. Health leaders and policymakers have posited that increasing the capacity of primary care services to include behavioral health care will improve access to and quality of care in ways that improve patient experiences. For many health systems, this means integration has become structural and organizational work: changing workflows, hiring and training new workforce members, creating care teams, and bridging information systems. But for patients, integration is entirely experiential, and in this study, when BHI was measured as adoption of structural changes health systems define as integration, there was no evidence that adopting them was associated with improvements in patient experiences. These results are consistent with, and further, some previous research that has shown mixed evidence on whether structural components of BHI improve patient outcomes.37,38,39,40,41,42,43 Furthermore, the findings complement other research that suggests a variety of contextual factors—including workforce shortages, poor information exchange between providers, and reimbursement challenges—may influence a patient’s acceptance of behavioral health services.38, 48

Following up with patients in greater depth, many were indifferent to the idea of structural integration and really just wanted to feel respected, valued, and heard by their providers, regardless of what kind of structure existed at their clinics. Others were concerned about integration because they did not feel welcome at their primary care office or were worried about carrying stigma into other health care settings, thus negating the potential effectiveness of the better integrated care system. In previously published findings from this study, it was reported that nearly three in ten patients reported experiencing such stigma, and that perceived stigma was strongly associated with patient experience outcomes.49

Limitations

This study has several limitations that are important to consider. First, an observational design approach was used because BHI is widespread in Oregon and all site clinics had varying levels of integration at study initiation, leading to a lack of any true “control” clinics that represent a complete absence of integration work. Second, the tool used to assess integration (the CAT) has been used to assess Oregon’s integration work before, but it has not been formally validated and may not be applicable to integration efforts in other contexts. Third, the CAT was only employed once—at the start of the study—to assess BHI progress, and some clinics may have made more progress over the study period. Additionally, the study is designed to only measure changes that occurred after the CAT was administered, and all of the clinics included were doing at least some BHI work before the study started, so it is possible that integration work had strong early effects that were already securely in place by the time the study started.

In addition to the limitations outlined above, this clinic-based sampling approach is vulnerable to confounding factors such as payment arrangements, provider mix, organizational culture, or others that were not measured in the design. Using self-reported survey data for many of the key measures also creates a risk of response and recall bias, and while qualitative results provided useful context, purposive sampling methods, small respondent groups, and low response rates limit the generalizability of the findings. And finally, the work was centered on Oregon, and results from Oregon may not translate in other policy settings.

Implications for Behavioral Health

Overall, little evidence was found to support the claim that the structural elements of BHI—the organizational and system change work necessary to integrate staffing, provide integrated care training, connect data, create shared workflows, and integrate financing that supports whole person care—had an impact on outcomes that matter to patients. Patients want to feel heard, respected, and supported by providers, but they are largely indifferent to the structural context where that happens. If systems have been hoping that implementing behavioral health integration structures would lead easily to positive changes in those patient experiences, there may be more work to do.

The results of this study may seem discouraging to health systems that are putting significant effort into BHI implementation, but the results should not be interpreted as an indictment of the concept. Health systems should build on these findings to re-think how they approach BHI. Many systems tend to focus their efforts on the mechanics of integration—how to get data, how to do internal and external referrals, and so on—rather than the experience they are trying to create through integration. Systems that are not making explicit efforts to address patient experience, and especially stigma, as part of their BHI transformation plans could consider enhancing those aspects of their work. The results suggest that better patient experiences do not automatically follow structural change, but it may be that the structural work systems are doing around integration acts as a valuable precursor for work that focuses on patient experiences and completes the journey toward a more integrated health care system.