Abstract
Background
Emergency department (ED) visits contribute substantially to health care expenditures. Case management has been proposed as a strategy to address the medical and social needs of complex patients. However, strong research designs to evaluate the effectiveness of such interventions are limited.
Objectives
To evaluate whether a community-based case management program was associated with reduced ED utilization among complex patients.
Design
Patients whose risk exceeded a threshold were randomly assigned to a group offered case management or to the control group. Assignment occurred at five intervals between November 2017 and January 2019. Program effectiveness for all assigned patients was assessed using an intention-to-treat effect. Program effectiveness among those who received treatment was assessed using a local average treatment effect, estimated using instrumental variables. Both estimators were adjusted for baseline characteristics using linear models.
Participants
Adults over age 18 with at least one health care encounter with Michigan Medicine or St. Joseph Mercy Health System between June 2, 2016, and November 27, 2018.
Interventions
Intervention arm participants (n = 486) were offered coordinated case management across medical, mental health, and social service organizations. Control arm participants (n = 409) received usual care.
Main Measures
The primary outcome was the number of ED visits in the 6 months following randomization into the study. Secondary outcomes were 6-month counts of inpatient and outpatient visits.
Key Results
Of the 486 patients assigned to the intervention, 131 (27%) consented to receive case management. The intention-to-treat effect on ED visits was + 0.14 (95% CI: − 0.27 to + 0.55). The local average treatment effect among those who consented and received case management was + 0.53 (95% CI: − 1.00 to + 2.05). Intention-to-treat and local average treatment effects were not significant for secondary outcomes.
Conclusions
The community case management intervention targeting ED visits was not associated with reduced utilization. Future case management interventions may benefit from additional patient engagement strategies and longer evaluation time periods.
Trial Registration
Clinicaltrials.gov Identifier: NCT03293160.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
INTRODUCTION
Complex patients contribute disproportionately to costs and service use in the emergency department (ED)1, 2. Much of this service use results from the exacerbation of chronic health and social problems and is considered preventable3,4,5,6,7. Health care systems, which increasingly bear financial risk for patient health spending and have increasingly recognized the importance of social determinants of health, are seeking strategies to address the complex needs of these patients8, 9. Case management has the potential to address the needs of complex patients, reducing service use and improving quality10, 11.
Effective strategies to identify, reach, and deliver case management services remain elusive12. These strategies depend on the social and health care needs of specific communities13. For instance, interventions to reduce ED use related to addiction disorders may involve a Screening, Brief Intervention, and Referral to Treatment protocol14. Interventions to address ED use related to diabetes may involve increasing access to appointments and standardized scheduling14. The capacity of health care and social service agencies to coordinate care to address patient-centric needs also varies across communities. While a number of studies have reported success in reducing ED use through case management,11, 15,16,17,18 the evidence suffers from methodological flaws, including uncontrolled pre-post studies susceptible to mean reversion12. Randomized studies are extremely rare in community-based case management interventions19. Strong study designs are essential to build the knowledge base about effective and context-specific case management strategies.
The Livingston and Washtenaw County State Innovation Model was initiated in 2015 to address the challenges of caring for complex patients. Part of a federally funded State Innovation Model20 program, the intervention sought to bridge the siloed provision of medical, mental health, and social services to address the complex and cross-sector needs of patients. The intervention used a predictive model to identify patients at risk for future ED use, triaged patients to the appropriate health or social resource, and followed through with assigned case management.
In this study, we describe the results from a randomized trial in a community-based setting of complex patients. The aim of the study was to evaluate whether the intervention reduced ED use in the 6 months following assignment.
METHODS
Setting and Population
The intervention was funded through the Centers for Medicare and Medicaid Services State Innovation Model (SIM). The goal of the State Innovation Model is to partner with states to test health policies and regulations that improve population health, lower spending, and improve quality. CMS awarded the State of Michigan $70 million to be distributed over 4 years and five Community Health Innovation Regions within the State21. These regions were tasked with improving the health of their geographic regions, including a focus on reducing the ED visits of frequent users. As of 2018, the US Census estimated that Livingston and Washtenaw Counties had a combined population size of 562,187, with a predominantly (about 81%) white population and high median income ($78,430 in Livingston and $65,618 in Washtenaw) relative to the state median22, 23 ($52,668)24. Roughly 5% of the population of Livingston and 13% of the population of Washtenaw lived below the poverty line.
The intervention focused on complex patients at high risk for ED use in Livingston and Washtenaw Counties. While not a criterion for enrollment, many of these patients were presumed to be covered by Medicaid, or uninsured25,26,27,28. Michigan Medicine (University of Michigan) and St. Joseph Mercy Health System were partnering organizations; collectively they operated all licensed EDs in the two counties. IHA, a multi-specialty group of outpatient practices, was also a partnering organization. Clinical data related to ED, inpatient, and outpatient use were used both to identify patients eligible for the intervention and to assess study outcomes. Eleven social service agencies were also partners in the intervention, providing a range of services (Table 1).
This study was approved by the University of Michigan Institutional Review Board.
Trial Registration: clinicaltrials.gov Identifier: NCT03293160.
Intervention
The intervention was implemented between November 1, 2017, and May 27, 2019. It included two components: (1) identification of target patients with a predictive model; (2) triage to case management agencies and patient-centric case management.
Identification of Target Patients with a Predictive Model
To be eligible for our evaluation, a patient needed to be at least 18 years of age, randomized prior to December 31, 2018, and have at least one encounter with Michigan Medicine, St. Joseph Mercy Health System, or IHA between June 2, 2016, and November 27, 2018.
We used electronic health record (EHR) data from these two health systems to develop and validate a random forest model to predict emergency department visits29. Random forest is a non-parametric predictive modeling method for classification (binary or multinomial outcomes) and regression (continuous outcome) models30. The model utilized 12 months of baseline EHR data to predict the number of ED visits in the following 6 months for each resident in the dataset. We limited the model development to residents of Livingston and Washtenaw Counties. We validated the model using fivefold cross-validation and temporal validation. During the study period, the data for the model were periodically refreshed to incorporate recent patient data.
Patients were randomized to treatment (the intervention group) or delayed treatment (the control group). Those in the control group were offered the case management intervention after 6 months. Patients had an equal probability of assignment to either arm. Patients were identified for randomization based on a rank-ordering on their predicted ED use. The original protocol called for a pre-specified tranche of patients to be randomized every month. For instance, if the tranche included 40 patients, the 40 patients with the highest predicted ED use would be randomized. In the subsequent period, the tranche of patients with the next highest predicted use who had not yet been randomized would be identified. As the intervention was rolled out, the size of the tranche and periodicity of identification were varied based on the capacity of case management agencies, which led to variation in the number of patients that were randomized in different periods (Table 1). Patients were ultimately randomized during five separate periods between November 2017 and January 2019. During this time, providers were also able to refer patients into the case management program. To preserve the integrity of the randomization, referred patients were excluded from the analysis.
Triage to Case Management Agencies and Patient-Centric Case Management
The names and contact information of the patients who were randomized into the intervention group were distributed to 11 community-based case management agencies. Table 1 displays the core activities and target populations of each case management agency. Patients were assigned to agencies that had previously established relationships with these patients; approximately 60% of patients had a previous relationship with an agency. Patients who had no pre-existing relationship with a case management agency were assigned to case management agencies with the capacity to absorb new patients. Patients who were randomized to the intervention group were contacted by the case management agency assigned to them and invited to participate in the intervention. No monetary compensation was offered. Agency case workers were instructed to contact patients within 30 days and attempt to gain consent for the intervention. Most of the attempts to contact patients were conducted by calling patient phone numbers obtained from Michigan Medicine and St. Joseph’s electronic health records data. After making contact, case workers made an appointment with patients to conduct an inventory of the patient’s social and medical needs. The case worker then contacted appropriate community resources, such as food banks, housing support services, and integrated clinical teams, to assist patients in navigating health and community systems.
Usual Care
Patients who were randomized into the control group continued to receive usual care for a 6-month period beginning with the date of randomization. After the 6 months were over, these patients were offered the same case management services as the intervention group.
Outcome Measures
Our primary outcome was the count of ED visits in the 6 months following randomization. Secondary outcomes were 6-month counts of inpatient and outpatient visits.
Sample Size and Enrollment Targets
The enrollment of patients occurred in five waves, with the first wave on November 27th, 2017, and the last wave November 27th, 2018. A total of 1256 patients identified as high predicted users of the ED were ultimately randomized across the five waves. After removing duplicates, selecting those with randomization dates prior to December 31st, 2018, and confirming that all patients were aged 18 or older, our final sample included 486 intervention patients and 419 control patients for a total of 905 (Fig. 1).
Statistical Analysis
Balance of observed covariates across patients assigned to treatment and control was evaluated with standardized differences. Two estimation strategies were used to estimate the effect of the intervention on study outcomes. First, the intention-to-treat (ITT) effect was estimated, classifying the treatment group as all patients assigned to treatment regardless of participation. To correct for residual imbalance in covariates, we estimated linear models that adjusted for age, sex, enrollment wave, and utilization of health services from the 6 months preceding randomization. Second, we estimated the local average treatment effect (LATE). Since not all patients assigned to treatment consented to treatment, we used an instrumental variable approach to capture the unbiased effect of the intervention on patients who consented to treatment. This method uses each patient’s random assignment (i.e., to the treatment or the control arm) as the instrument for receipt of treatment, which produces a better estimate of the effect in the context of imperfect compliance with initial assignment.31 We adjusted for the same covariates as in our ITT approach. Our estimates of the treatment effect were robust to alternative methods, including propensity score nearest-neighbor matching and inverse probability weighting. Confidence intervals and p values were not adjusted for multiple comparisons. Results for the secondary outcomes (inpatient and outpatient visits) should be considered exploratory.
Limitations
There was often a delay between the date of randomization and the first visit with a case management agency, meaning that, during the first several weeks of a patient’s treatment period, no services would have been received, effectively diluting the treatment. However, due to the nature of outreach to complex patients, we expect that such delays would be present in most implementations of similar interventions. Second, one of the enrollment waves assigned a larger number of patients to the treatment group compared to the control group; however, in sensitivity analysis, we found that excluding this wave did not change our results. Third, our findings are context-specific: Michigan’s Livingston and Washtenaw Counties may not resemble other regions in terms of demographics, transportation, housing services, and accessibility of high-quality health care. Finally, we evaluate only utilization measures and do not address care quality or patient satisfaction.
RESULTS
At baseline, the intervention and control groups were an average of 55 and 56 years old, with 40 and 45% of patients over the age of 60 (Table 1). Women comprised 54% of treatment patients and 52% of control patients. Most covariates were closely balanced at baseline. In the 6 months prior to assignment, intervention patients had an average of 4.05 ED visits (compared to 3.88 among control patients) and 12.45 outpatient visits (compared to 10.75). However, there was residual imbalance for inpatient visits, where intervention patients had 1.53 inpatient visits compared to 2.11 among controls (standardized mean difference = − 0.21).
Of the 486 patients randomized to the intervention group, 131 (27%) consented to participate in case management (Table 2). Of the 355 patients who were assigned to the intervention but did not join the consented group, about half explicitly declined to join (n = 175). Another 40% either could not be contacted (n = 96) or were deceased (n = 48). Case workers deemed another 30 individuals ineligible for the intervention, and the remaining 6 were not consented for other reasons. It took a median of 27 days for staff to contact a patient and confirm their intent to participate (or not participate) in the intervention.
Within those assigned to the intervention group, we also compared patients who consented and those who did not (see Online Supplement Table 1). The consenting group had a somewhat larger proportion of women (57% compared to 53%). The two groups were most different in the number of prior ED visits: consenting patients had an average of 5.2 ED visits, compared to 3.6 among non-consenting patients (standard mean difference of 0.31). Consenting patients also received more inpatient and outpatient care in the 6 months preceding randomization (1.8 to 1.4, and 12.5 to 10.7).
After adjusting for age, sex, enrollment wave, and prior utilization, the ITT effect for ED use in the 6 months following assignment was 0.14 (95% confidence interval [CI]: − 0.27 to 0.55) (Fig. 2, panel a). This result implies a non-significant increase in ED use among the case management group relative to the control group. ITT estimates for inpatient use (− 0.21; 95% CI: − 0.60 to 0.18) and outpatient use (− 0.13; 95% CI: − 1.17 to 0.82) in the 6 months following assignment were also non-significant. The LATE estimates were 0.53 for ED use (95% CI: − 1.00 to 2.05), − 0.79 for inpatient use (95% CI: − 2.26 to 0.68), and 0.47 for outpatient use (95% CI: − 3.21 to 4.15), all non-significant.
DISCUSSION
In this randomized trial of case management to reduce ED use for complex patients, we report three main findings. First, the research design successfully identified patients likely to be frequent users of the ED and randomization resulted in balance across most covariates. Second, among patients assigned to the intervention, only 27% consented to participate. Third, intervention led to relatively small and non-significant changes in ED use (the primary outcome) and inpatient and outpatient use in the 6 months following randomization.
Our study is the largest randomized trial examining the effects of case management on ED use (905 total patients, 486 treated patients). Our findings are similar to those reported by Finkelstein and colleagues, who found that the Camden Coalition of Healthcare Providers’ Core Model program did not reduce re-hospitalizations19. Nonetheless, our study evaluates a somewhat different intervention, on a different outcome, in a different community context. The Camden model was implemented within an economically depressed community with considerably higher rates of poverty than in our study. Furthermore, to join the sample, Camden patients were required to have a hospitalization in the preceding 6 months, have at least two chronic conditions, and have other high-risk medical or social traits; we did not select for these characteristics in our intervention. Notably, Camden patients were recruited during their hospital stays, and randomization occurred after patients had consented. In our intervention, we did not recruit patients from hospitals, but from community settings, and we randomized patient names first, then invited those patients to join the study. Lastly, the Camden model primarily examined hospital readmission, while we targeted ED use. Given these important differences, this paper contributes to the evidence that targeting high-utilizing, complex patients with case management is challenging and perhaps less effective than previously thought.
Our results conflict with much of the recent literature suggesting that case management can improve quality and reduce ED and acute health care use. A recent review of the effects of interventions to reduce acute spending identified results from 17 case management interventions which connected patients with community resources to address broad health and social needs12 to reduce ED use. While 13 evaluations found positive effects on ED use, they were compromised by uncontrolled pre-post designs. Of the three randomized trials, two programs showed a reduction in ED use. Seaberg and colleagues examined a patient navigation program among 282 patients (148 treatment) in the Erlanger Health System in Chattanooga, Tennessee32. In the 12 months following assignment, ED visits declined by 13.2% among treated patients and 8.8% among control patients (p < .0001). Yet this paper reports implausibly precise effect estimates, casting doubt on the inferences. Shumway et al. examined a case management program among 252 (167 assigned to treatment) patients in San Francisco General Hospital15. They found that the program was significantly associated with lower ED use in the 24 months following assignment. Yet effect magnitudes are not reported, statistical methods are unclear, and significance tests are likely compromised by multiple testing. Zulamn and colleagues assessed the effects of an intensive outpatient program for 583 veterans (140 assigned to treatment) at the Palo Alto Veterans Affairs Health Care System. An intention-to-treat analysis found that the intervention led to a non-significant reduction of 0.12 ED visits in the 17 months following assignment. Due to methodological shortcomings, the literature supporting the effectiveness of case management in reducing ED visits is perhaps weaker than previously understood.
There are four key reasons why the Washtenaw-Livingston Counties State Innovation Model may not have reduced ED use. First, only 27% of patients randomized to the intervention consented to participate. Specific strategies for engagement, including financial incentives, might have increased intervention uptake33. Evidence suggests that in-person engagement in hospitals and EDs may also be an effective means for increasing uptake. Second, of those that did choose to participate, the 6-month post-intervention exposure period may have simply been too short to reduce ED use. Case management organizations may need more time to establish processes and build patient rapport that result in measurable differences in utilization. We cannot correct for this possibility since, in our study, the control group also began receiving the intervention after 6 months. The other trials of case management examined post-intervention periods between 12 and 24 months. Third, the program required substantial coordination between community partners, many of whom have different organizational cultures, to identify, target, and manage complex patients. It may have taken more time than anticipated for community partners to develop effective operating procedures and norms to address the needs of complex patients. Fourth, given the complex needs of the study population, some patients in both intervention and control groups may have already been receiving some form of case management. While such services would not have been as coordinated across social service agencies as the case management available in the intervention, existing case management may have muted the differences between intervention and control groups.
CONCLUSION
The Livingston-Washtenaw State Innovation Model sought to engage complex patients in community-based case management services. For complex patients, case management may improve certain aspects of care delivery or the patient experience by more holistically addressing social and environmental needs. However, in our study, case management did not reduce ED utilization relative to a control group. Our findings highlight the importance of understanding how to effectively engage complex patients in community interventions, and further underscore the need to use high-quality evaluation designs to build evidence of program effectiveness.
References
Cunningham A, Mautner D, Ku B, Scott K, LaNoue M. Frequent emergency department visitors are frequent primary care visitors and report unmet primary care needs. J Eval Clin Pract. 2017.
Hasselman D. Super-Utilizer Summit: common themes from innovative complex care management programs [Internet]. Super-Utilizer Summit. 2013. Available from: https://www.rwjf.org/content/dam/farm/reports/reports/2013/rwjf407990
Byrne M, Murphy AW, Plunkett PK, McGee HM, Murray A, Bury G. Frequent attenders to an emergency department: a study of primary health care use, medical profile, and psychosocial characteristics. Ann Emerg Med. 2003.
Hunt KA, Weber EJ, Showstack JA, Colby DC, Callaham ML. Characteristics of frequent users of emergency departments. Ann Emerg Med. 2006.
Lucas RH, Sanford SM. An analysis of frequent users of emergency care at an urban university hospital. Ann Emerg Med. 1998.
Sun BC, Burstin HR, Brennan TA. Predictors and outcomes of frequent emergency department users. Acad Emerg Med. 2003.
Williams ERL, Guthrie E, Mackway-Jones K, James M, Tomenson B, Eastham J, et al. Psychiatric status, somatisation, and health care utilization of frequent attenders at the emergency department. A comparison with routine attenders. J Psychosom Res. 2001.
Althaus F, Paroz S, Hugli O, Ghali WA, Daeppen JB, Peytremann-Bridevaux I, et al. Effectiveness of interventions targeting frequent users of emergency departments: a systematic review. Ann Emerg Med. 2011.
Mann C. Reducing nonurgent use of emergency departments and improving appropriate care in appropriate settings [Internet]. 2014. Available from: https://www.medicaid.gov/Federal-Policy-Guidance/Downloads/CIB-01-16-14.pdf
Mann C. Targeting Medicaid Super-Utilizers to Decrease Costs and Improve Quality. 2013.
G.S. K, R. K. Effectiveness of case management strategies in reducing emergency department visits in frequent user patient populations: a systematic review. Journal of Emergency Medicine. 2013.
Iovan S, Lantz PM, Allan K, Abir M. Interventions to decrease use in prehospital and emergency care settings among super-utilizers in the United States: a systematic review. Medical Care Research and Review. 2019.
Doupe MB, Palatnick W, Day S, Chateau D, Soodeen RA, Burchill C, et al. Frequent users of emergency departments: developing standard definitions and defining prominent risk factors. Ann Emerg Med. 2012.
Solberg LI, Maciosek M V., Sperl-Hillen JM, Crain AL, Engebretson KI, Asplin BR, et al. Does improved access to care affect utilization and costs for patients with chronic conditions? Am J Manag Care. 2004.
Shumway M, Boccellari A, O’Brien K, Okin RL. Cost-effectiveness of clinical case management for ED frequent users: results of a randomized trial. Am J Emerg Med. 2008.
Pope D, Fernandes CM, Bouthillette F, Etherington J. Frequent users of the emergency department: a program to improve care and reduce visits. CMAJ. 2000 Apr;162(7):1017–20.
Witbeck G, Hornfeld S, Dalack GW. Emergency room outreach to chronically addicted individuals. A pilot study. J Subst Abuse Treat. 2000.
Okin RL, Boccellari A, Azocar F, Shumway M, O’Brien K, Gelb A, et al. The effects of clinical case management on hospital service use among ED frequent users. Am J Emerg Med. 2000.
Finkelstein,1. Finkelstein A, Zhou A, Taubman S, Doyle J. Health Care Hotspotting - a Randomized CTNEJM 2020;382(2):152-162. https://doi.org/10.1056/NEJMsa190684. A, Zhou A, Taubman S, Doyle J. Health care hotspotting - a randomized, Controlled Trial. N Engl J Med. 2020 Jan;382(2):152–62.
Centers for Medicare and Medicaid Services. State Innovation Models Initiative: General Information [Internet]. 2019 [cited 2019 Oct 22]. Available from: https://innovation.cms.gov/initiatives/state-innovations/
Michigan Department of Health and Human Services. State Innovation Model.
United States Census Bureau. QuickFacts Livingston County, Michigan. 2018.
United States Census Bureau. QuickFacts: Washtenaw County, Michigan. 2018.
United States Census Bureau. QuickFacts: Michigan. 2018.
Hsia RY, Brownell J, Wilson S, Gordon N, Baker LC. Trends in adult emergency department visits in California by insurance status, 2005-2010. JAMA - J Am Med Assoc. 2013.
Cunningham P, May J. Insured Americans Drive Surge in Emergency Department Visits [Internet]. 2003. Available from: http://www.hschange.org/CONTENT/613/index.html
Overuse of Emergency Departments Among Insured Californians [Internet]. Oakland; 2006. Available from: https://www.chcf.org/wp-content/uploads/2017/12/PDF-EDOveruse.pdf
Tang N, Stein J, Hsia RY, Maselli JH, Gonzales R. Trends and characteristics of US emergency department visits, 1997-2007. JAMA. 2010 Aug;304(6):664–70.
Brannon E, Wang T, Lapedis J, Valenstein P, Klinkman M, Bunting E, et al. Towards a learning health system to reduce emergency department visits at a population level. AMIA. Annu Symp proceedings AMIA Symp. 2018;2018:295–304.
Breiman L. Random Forests. Mach Learn [Internet]. 2001;45:5–32. Available from: https://doi.org/10.1023/A:1010933404324
Sussman JB, Hayward RA. An IV for the RCT: using instrumental variables to adjust for treatment contamination in randomised controlled trials. BMJ. 2010 May;340:c2073.
Seaberg D, Elseroad S, Dumas M, Mendiratta S, Whittle J, Hyatte C, et al. Patient navigation for patients frequently visiting the emergency department: a randomized, controlled trial. Acad Emerg Med. 2017.
Kullgren JT, Williams GC, Resnicow K, An LC, Rothberg A, Volpp KG, et al. The promise of tailoring incentives for healthy behaviors. Int J Work Heal Manag. 2016;9(1):2–16.
Acknowledgements
The authors would like to thank Jeremy Sussman for his helpful comments on methodological approaches.
Funding
Authors Lapedis, Büyüktür, Singh, and Teske were supported by Grant Number CMS-1G1-14-001 from the Department of Health and Human Services, Centers for Medicare & Medicaid Services. Authors Ryan and Post were supported by a grant from the Community Foundation for Southeast Michigan.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
The authors declare that they do not have a conflict of interest.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Post, B., Lapedis, J., Singh, K. et al. Predictive Model-Driven Hotspotting to Decrease Emergency Department Visits: a Randomized Controlled Trial. J GEN INTERN MED 36, 2563–2570 (2021). https://doi.org/10.1007/s11606-021-06664-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11606-021-06664-1