Background

Significant health policy shifts in the last decade have spurred renewed attention to behavioral health service outcomes among child behavioral health policymakers and administrators across the United States. The passage of the Affordable Care Act (ACA) in 2010 enabled states to design and test innovative financial and organizational arrangements to reduce costs and upgrade quality and effectiveness through increased care coordination across providers and service sectors in their public health insurance programs (Mechanic, 2012). States are increasingly using performance-based purchasing arrangements through managed care organizations (MCOs) in their health programs to address the rising costs for children’s behavioral healthcare covered under Medicaid and the Children’s Health Insurance Program (CHIP) (Graaf & Snowden, 2020; Houston et al., 2022). These performance-based payment models provide financial incentives for providers to meet specific quality and outcome standards in care delivery (Graaf & Snowden, 2020; Hyatt et al., 2021). As a result, monitoring and measuring behavioral health outcomes has become an essential aspect of behavioral health service delivery.

Simultaneously, policymakers, advocates, and researchers increasingly promote the need for patient-centered health and behavioral health care that centers the perspectives and values of youth, families, and caregivers in child and adolescent behavioral health care (Huang et al., 2005; Kitson et al., 2013). Thus, building behavioral health services that reflect the preferences and priorities of youth and families requires creating policies that incent providers and service systems to deliver the outcomes that matter most to children and youth with behavioral health needs and their families. Funding and producing research evidence about the system and treatment interventions that achieve those outcomes is also needed to support providers in achieving the goals set by policies (The Lancet Psychiatry, 2020).

Several existing models and frameworks enumerate important child and youth behavioral health outcomes, but most have not been generated in partnership with the youth and families being served or seeking services in behavioral health systems. The National Institutes of Health’s Patient Reported Outcomes Measurement Information System (PROMIS) provides person-centered approaches to evaluating the physical, mental, and social health of children—including those with chronic conditions. However, validation of PROMIS excluded the perspectives of youth and caregivers of children and youth with mental health conditions (DeWalt et al., 2007). Further, because PROMIS was developed for clinical research, it does not provide measures for system-level outcomes that impact the care experiences of children, youth, and their families (Greene et al., 2012). System level measures include assessments of service use and availability, costs, and mental health system inter-organizational integration, relations, and coordination (Hoagwood et al., 1996). Other outcome frameworks used in research or policy—which do include system level outcomes or consumer perspectives—include the Symptoms, Functioning, Consumer Perspectives, Environments and Systems (SFCES) Model,1 the Mental Health Statistics Improvement Program (MHSIP) Consumer-Oriented Mental Health Report Card (1996) (Lutterman et al., 2003) and the U.S. Department of Health and Human Services’ 2021 Core Set of Children’s Health Care Quality Measures for Medicaid and CHIP (Child Core Set) for behavioral health care (Zima et al., 2019). These models, though, were primarily constructed by “expert” panels of researchers and policy makers, again omitting critical family perspectives.

Current knowledge of the behavioral health service outcomes most desired by youth and families is limited. The handful of studies capturing youth and family outcome priorities are constrained geographically—taking place in single communities both within and outside the United States—and draw from populations who are currently engaged in youth behavioral health services (Garland et al., 2004; Krause et al., 2018, 2021). These research designs limit the applicability of findings to specific locations or service settings. These studies also exclude the perspectives of youth and families who seek behavioral health care but are unable to obtain it or who have dropped out of or graduated from services. Other studies draw on chart reviews of treatment plan goals (Jacob et al., 2016), report only the “target problem” identified by service users (Hawley & Weisz, 2003; Weisz et al., 2011), or youth or parent reported positive or negative service experiences (Aarons et al., 2010; Narendorf et al., 2017). While these studies report findings that can be extended to understanding service user outcome preferences, they do not directly capture participants’ report of prioritized service outcomes.

This descriptive qualitative study aimed to develop a clear understanding of the behavioral health service outcomes prioritized by youth and young adults (YYA) with a wide range of behavioral health needs and service experiences, as well as by the caregivers of such children and youth. To achieve this, a professional association of family-run organizations serving these families co-led the study, in partnership with family-run organization, We conducted a series of focus groups in six communities nationwide to identify the behavioral health service outcomes YYA and caregivers hoped to achieve from behavioral health service use. Study findings will provide guidance (1) to policy makers and service funders about the outcomes that services should be targeting, (2) to service providers to better understand the outcomes they should be targeting in service delivery, and (3) will support researchers in developing evidence about interventions that achieve the outcomes that matter most to youth and families receiving behavioral health services.

Methods

The research team was co-led by Family Run Executive Director Leadership Association (FREDLA). FREDLA is a national network of FROs fully staffed by caregivers of children and youth with behavioral health needs. FREDLA provided significant guidance to the research team in expanding the accessibility of the study experience for family run organizations and their member participants. This was critical to maximizing meaningful study participation for project participants. FREDLA’s participation encouraged flexible approaches for data collection and analysis (e.g., use of broad discussion prompts, open and in vivo coding) to center family voice and minimize research bias in study results.

Sampling and Participant Recruitment

The team selected a purposive sample of six family-run organizations (FRO) based on geographic, clinical, racial, and ethnic diversity criteria to assist in sampling and data collection. FROs are family-led organizations that support families caring for children and youth and young adults (YYA) with behavioral health challenges and the agencies serving them. These organizations are governed and guided by the lived experience of families, encompassing the collective experience of families in their communities. Over 100 local and statewide FROs reach over 100,000 families annually through training, providing family peer support services, and consulting with local organizations on interventions to best support these families.

FRO Inclusion Criteria

Criteria for participating FROs sought a balance of three factors: (1) FROs with an ongoing youth-led program or an established partnership with a local YYA-led organization, (2) FROs with the demonstrated capacity to participate in complex projects (e.g., responsive to emails, consistent participation, follows through on assigned tasks), and (3) FROs which represented a range of regional and demographic populations (e.g., eastern, midwestern, southern, western; or rural, urban, or frontier populations). These criteria resulted in the selection of FROs from the following states: North Carolina, Arizona, Nevada, Washington, Pennsylvania, and Mississippi. Partner FROs received monetary compensation for their efforts and involvement in the project. The university-based research team had a pre-existing research-based relationship with FREDLA prior to the beginning of this study. FREDLA has ongoing professional relationships with all participating FROs. However, the research team had no existing relationship with the participating FROs or the participants they recruited prior to the study.

Participant Inclusion Criteria

FROs and YYA-led organizations recruited YYA participants ages 13 to 26. This age range includes ages in which behavioral health concerns are most likely to occur (Kessler et al., 2005) and this range is often considered to include youth moving through the transition to adulthood (Wilens & Rosenbaum, 2013). FROs recruited parents or caregivers, of any age, whose children or YYA who experienced behavioral health challenges before age 21. Participants were required to have prior or current experience of using any type of behavioral health service from any sector as a recipient of services or as a caregiver of a child receiving services. Possible service sectors included, but where not limited to, juvenile justice, child welfare, schools, community behavioral health centers, private therapists or psychiatrists, or psychiatric hospital or residential care. Potential participants were connected to FROs through current or past service use, employment through the FRO as a parent or youth peer support provider, or participation in or facilitation of family support or advocacy groups.

Sample Recruitment

We asked FROs to assist the research team in organizing and conducting one, two-round focus group for caregivers and one, two-round focus group for YYA in each state. This resulted in 24 separate focus groups: 2 (round 1 and 2) for YYA and 2 (round 1 and 2) for caregivers in six states. We asked partner FROs and their collaborating YYA-led organizations to identify approximately six to eight potential participants to join the study, with the goal to have approximately six participants in each focus group. Our target number of states, and focus group participants in each state, was guided by evidence, across multiple studies, which suggests that meaning and code saturation is often reached between three and five focus groups (Coenen et al., 2012; Guest et al., 2017; Hennink et al., 2019).

YYA-led organizations and FROs reached out to potential participants individually—via email, in person, or by phone—sharing the project summary provided by the research team and inviting them to participate in the project. The project summary conveyed the same study goal to participants as is stated in this report: to identify the behavioral health service outcomes that matter most to children and youth with behavioral health needs and their families when seeking services. Participants were asked to participate in two rounds of one-hour virtual focus groups approximately two months apart. Participants received a $50 (YYA) or $75 (Caregiver) gift card as an incentive to participate in each focus group. The co-primary investigator’s affiliated university’s Institutional Review Board reviewed and approved the study’s protocol.

Data Collection

Dates and times of focus groups were chosen in partnership with FROs based on meeting times identified as working best for their YYA and families. FROs were responsible for hosting, sharing the meeting link, and confirming participation with their members. Focus groups took place on Zoom between February and May of 2023, and included 34 YYA with behavioral health needs and 42 parents of children or youth with behavioral health needs. Total participants in a focus group ranged from 3 to 7 for YYA and 4 to 8 for caregivers. Participants were asked to participate in both rounds of focus groups, though some attrition did occur from Round 1 to Round 2. In several cases, FROs invited a new participant to replace participants who could not participate in Round 2. Five caregivers and eight YYA participated only in Round 1; Three caregivers and three YYA participated only in Round 2. An attrition analysis was not conducted because the sample size was too small to generate meaningful findings. Each focus group was collaboratively facilitated by a researcher, a local YYA-led or FRO staff member or volunteer, and the FREDLA Project Director or designee.

The research team developed focus group protocols in collaboration with FROs to appropriately identify language and norms to foster active participation and discussion. The protocol was loosely structured around capturing outcome domains specified in the SFCES Model (Hoagwood et al., 1996): (1) Symptoms and diagnosis (e.g., anxiety, depression), (2) functioning (e.g., capacity to adapt to environment), (3) consumer perspectives (e.g., satisfaction with care, family strain), (4) environments (e.g., marital relationships, social supports), and (5) systems (e.g., care coordination, restrictiveness of care). Prompts were phrased, with the guidance of FROs, to use accessible language to capture participants thinking about outcomes in each domain. Table 1 details the guiding questions for the first round of focus groups and the domains each question reflected, as well as the guiding questions for the second round.

Table 1 Focus Group questions
Full size table

At the beginning of each focus group, the facilitator explicitly defined “behavioral health services” to encompass all forms of behavioral health support—ranging from respite and peer support, therapy, medication management, and case management—delivered by various providers, such as schools, community behavioral health centers, private therapists or providers, hospitals, and residential facilities.

Round 1 Focus Groups

These questions guided semi-structured discussions as the facilitation team sought to elicit participants’ expectations for themselves, their families, and their service use experience. After the first few focus groups were conducted, the research team noted that participants—particularly in caregiver groups—responded to the first three protocol questions with answers about the quality of behavioral health services rather than with answers about results from services. To ensure that caregiver participants reported answers about their priority outcomes beyond service quality, the caregiver focus group protocol was amended to include an additional “magic” question: “Imagining that the quality and delivery of services was perfect—seamless coordination and communication, etc—what would you hope to see in yourself, your child, or your family as a result of participating in behavioral health services?”

Round 2 Focus Groups

Two research team members synthesized the data from the first round of focus groups and prepared summaries, in the form of word clouds, for examination and discussion during the second round of focus groups. Word clouds were generated from code applications to Round 1 transcripts (the analytic process is described in detail below), highlighting the most common themes within participants’ top three prioritized outcomes. The visual representation in these clouds used varying font sizes to indicate the relative frequency of each code. Word clouds was reported by family partners to be an easily understood and accessible format for participants. Clouds for youth and young adults (YYA) and parents and caregivers were generated separately to account for the diversity of responses between these two groups.

In Round 2, the word clouds were presented and guided discussions by visually representing the findings from the first round. The facilitators introduced these word clouds, one for youth and one for caregivers, explaining their underlying concept. Following any participant queries for clarification, the facilitators posed targeted questions to gauge agreement with the displayed data, identify any omission, and determine the three most critical outcomes within the cloud. Time permitting, the facilitation team also engaged participants in refining the phrasing and meaning of terms to inform the final structure of the coding scheme and its application.

Data Preparation

Participants were encouraged to participate via chat functions in both rounds, and Jamboards (virtual board where participants add virtual “sticky notes” with their responses) were used in round 1—with one board provided for each question. The facilitators recorded all focus group sessions, and one research team member produced verbatim transcripts from these recordings. Transcripts, including timestamps, were generated by running audio recordings through Microsoft Word transcription processes. Transcripts were then cleaned and finalized by cross checking and correcting transcripts with audio recordings. Timestamps in transcripts were cross-referenced with Zoom chat transcripts to determine what chat responses corresponded to which focus group prompt or question. The final dataset for analysis included time-stamped transcripts, Zoom chat transcripts and completed Jamboards.

Data Analysis

Two research team members conducted two rounds of inductive qualitative coding on the transcripts of all Round 1 focus groups using MAXQDA software (VERBI Software, 2021). To comprehensively identify the behavioral health outcomes of greatest concern to YYA and parents/caregivers, the inductive coding process allowed for incorporating youth or family perspectives that existing frameworks might not have previously recognized. The team employed in vivo coding in the first pass to assign initial codes, which they then refined, merged, and reorganized for a second pass of coding. MAXQDA was used to generate word clouds for Round 2—after two passes of code applications to Round 1 transcripts—highlighting the most mentioned themes within participants’ top three prioritized outcomes. Participant responses in Round 2 were used to refine the scheme further. The refined coding scheme was then applied to the Round 2 transcripts, adding new codes as necessary and conducting another coding pass to refine and collapse codes.

By integrating the transcripts from both rounds, the team used responses to the Top Three questions to construct a preliminary list of behavioral health outcomes. The research team discussed this list with partner FROs to fine-tune the terminology and gather input on the format and substance of the final list of outcomes. After these consultative sessions, the team formulated a comprehensive coding scheme based on the initial coding efforts from both rounds and FRO feedback. The two coders employed the finalized coding scheme to analyze the transcripts from both rounds, complemented by supplementary auto-coding and code-merging processes, applying codes to data units ranging from short phrases to multiple sentences. The two coders performed all coding tasks independently, achieving an intercoder reliability (kappa) of 0.82.

Each coder generated a code relationship matrix displaying the frequency of codes from the Top Three responses. We calculated the final code frequencies by averaging the individual counts from each coder and then ranked the YYA and caregiver outcomes based on the total average frequencies. We identified outcomes that featured predominantly in both YYA and caregiver rankings, signifying their cross-group importance, and collated them into a new, combined list of shared outcomes. The top three outcomes from this shared list, determined by the highest code counts, are presented here as the principal shared outcomes for both YYA and caregivers. Additionally, we ascertained the top three unique outcomes, those exclusively reported by either YYA or caregivers, by selecting the three most frequently occurring outcomes within each respective group. Full prioritized lists of desired behavioral health service outcomes, and participants’ rationales for their priorities, are reported elsewhere for caregivers (Graaf et al., 2024) and YYA (Graaf et al., In Progress).

Validity and Reliability

We enhanced transferability of study results by recruiting a national sample, representing six states from diverse regions, including urban, rural, and frontier areas. We achieved dependability by maintaining a detailed audit trail documenting the research process, including the development and application of the coding framework and the changes made based on participant feedback (Miles et al., 2013). We cross-checked the analysis by independent coders, resulting in a high intercoder reliability score, indicating objectivity in the coding process (Saldaña, 2013). We established confirmability through rigorous reflexivity processes, where the researchers continually scrutinized their biases related to their position as researchers (three team members), a social work student researcher (one team member), individuals with lived experiences of supporting family members or close friends with behavioral health needs (all team members), and former providers of behavioral health services for children and youth (two team members) (Olmos-Vega et al., 2023). Regular team meetings included reflexive examination of the influence of each of these roles in the team’s emotional and psychological responses to participant narratives and its influence on data analysis and interpretation of participant responses (Bieler et al., 2021). We ensured credibility in this study through member checking, where we invited focus group participants and FRO team members to review and validate preliminary findings, providing their perspectives on the accuracy of the data interpretation (Varpio et al., 2017). Finally, study conclusions were finalized after presenting draft findings and conclusions to participating FROs for feedback and refinement. These practices collectively reinforce the reliability and validity of the qualitative findings in this study.

Results

Basic participant demographics are presented in Table 2. Cells with sample sizes less than 11 are suppressed for confidentiality. Among YYA participants, about half the sample identified as women; there were also men and nonbinary participants. Racial and ethnic groups were evenly represented across the YYA sample, and almost half of YYA participants lived in rural or frontier settings. Most caregivers identified as female and were the biological or legal parent of the child with behavioral health needs, with an average age of late 40s. Racial and ethnic minority parents comprised nearly half the sample, with over 60% being from rural or frontier areas. On average, participants had about 7.5 years of experience in children’s mental health services use, with most reporting the use of at least two or more service sectors. Most often, participants were involved in both special education programs and community-based mental health care; many also reported experience with child welfare, juvenile justice systems, or residential psychiatric care.

Table 2 Participant characteristics (N = 76)
Full size table

Table 3 summarizes the top three behavioral health service outcomes prioritized by YYA behavioral health service users, caregivers of children and youth with behavioral health needs, and those identified by both YYA and caregivers. They are listed in each column in the order of how frequently they were mentioned when participants reported their three highest-priority behavioral health service outcomes. These outcomes and what they mean to respondents are described in detail below.

Table 3 Top outcomes from behavioral health services that matter to youth and young adults and caregivers
Full size table

Outcome Priorities for Youth and Young Adults

For YYA, being understood by others, improving their interpersonal relationships, and feeling heard were identified as the top three highest-priority behavioral health service outcomes.

Being Understood

The most frequently cited desired behavioral health services outcome for YYA was being understood, especially by their parents, caregivers, and providers. Youth reported wanting their providers to understand their perspectives and motivations for their behaviors and feelings: “Like, understand me first, allow me to know that it’s an opportunity for me to get back to normal without the medication.” (YYA 3.3) Youth also expressed the desire for their parents to understand that they are different from their parents, siblings, and others. They wanted their caregivers to understand, notice, and respond to their behavioral health needs and feelings and to see and accept their unique perspectives on life—whether they agreed with them or not. “…our guardians understanding us and accepting how we feel and accepting us and our personalities.” (YYA 3.2) Finally, this outcome was also discussed in the context of mutual understanding with their caregivers and peers; YYA expressed a desire to understand their caregivers’ perspectives and needs and achieve a shared understanding that could enhance their relationship: “Well, I hope that it shows results that I can see [things] from their perspective, and I hope that they can see [things] from my perspective also.” (YYA 5.1).

Improved Relationships

Another outcome highlighted by YYA in the focus groups was their desire for improved relationships. They expressed a hope for better relationships with their parents or caregivers, families, and peers. Improved relationships included spending quality time with their caregivers and families: “I feel like quality time is definitely important.”; (YYA 3.3) “…for us to, like, have more time to talk to each other.” (YYA 6.1) It also included being able to express care for their family members and caregivers, as well as feeling cared for by them. “I think there’s some results that would be—that would show that I also care about them.” (YYA 5.1).

YYA expressed a desire for less conflict, anger, and fighting in family and peer relationships, to have fewer negative emotional and verbal responses to their peers and family members, to manage conflict more effectively, and to “get along” better with their caregiver and siblings. Participants also reported wanting healthier boundaries with peers, caregivers, and their families. This desire included understanding what healthy boundaries look like, being able to enforce health boundaries, and wanting caregivers and providers to respect their boundaries: “As for our parents…they are really quick to want to just take over and protect us, but that’s not always what we need.” (YYA 6.3) Trusting others, being understood, and feeling accepted were themes frequently co-occurring with Improved Relationships, as youth perceived these elements to be critical in supporting improved relationships.

Being Heard

The final of the top three outcomes emphasized by YYA was Being Heard. Youth asserted that being heard is different from being understood – that feeling heard comes from having space to talk about your needs and perspectives and receiving feedback that the listener is paying attention and listening. Participants asserted that being understood—while it can result from being heard—is distinct from being heard: understanding means that listening results in learning more about the youth and seeing their perspective. “I just feel like—I don’t really know how to state this—but sometimes I just don’t feel like I’m being heard with what I’m trying to say to them, you know, and like, they’re not really understanding it.” (YYA 4.6) They expressed a desire to be heard, not just by their providers but also by their parents or caregivers. “I mean, sometimes there’s a major problem with families, you know, they—your family members care about you. But do they ever really listen to, like, what’s in your heart? Like when you try and tell them stuff?” (YYA 4.1).

Outcome Priorities for Caregivers

For caregivers of children and youth with behavioral health needs, having accessible services that meet their needs, having providers that collaborate effectively with parents and other service systems, and experiencing consistent and continuous behavioral health care for their child were the top three most important behavioral health service outcomes.

Accessible Services

The most frequently cited outcome that parents and caregivers identified as important was accessing the behavioral health care services that meet the needs they identify for their child and their family. “I would be afraid that someone would look at my family and say, oh, this is what you need as opposed to me saying, ‘No, this is what [I need].’” When discussing this outcome, caregivers often commented that providers were not delivering the services in their child’s treatment plan. “We have a behavior plan from our mental health [provider] that says that my children should be receiving a specific amount of life skills, right? We have yet to receive any of that.” (Caregiver [CG] 1.1)

Other caregivers talked about the need to access a range of community-based supports rather than relying on residential solutions: “So we need to get real about community-based services. Our kids shouldn’t have to live outside our home for 15 months.” (CG 5.6) Caregivers mentioned the need for the full range of youth and family supports, including specific services—school-based behavioral health care, respite, peer support, wraparound, services to support youth transitioning to adulthood, and vocational support for YYA.

Provider Collaboration

The second most reported outcome identified by parents and caregivers was provider collaboration. They described this outcome as a desire for increased cooperation and communication between multiple providers within the behavioral health system and other provider systems—especially schools. This outcome code was frequently cross-coded with effective communication and access to care—as caregivers described that a lack of communication across systems contributed to their child not receiving the support(s) they needed. “I’m being told, ‘Oh, I’m sorry we dropped the ball with the insurance. Oh, I’m sorry we didn’t follow through with getting the referrals.’ So, (my child) is back in their room again and doing the same thing they did before I hospitalized them.” (CG 5.3).

Participants frequently used the phrase “being on the same page” when speaking about what they wanted from providers: they wanted all providers, teachers, and school officials to have a shared understanding of what their child’s behavioral health needs were and how they were going to be met and accommodated. Participants shared many examples of times when this was not the case. “Everybody’s working on a different skill, or everybody’s working on a different issue because everybody thinks that their issue is primary.” (CG 8.1) Additionally, they wished for providers to work more closely with themselves and their children, emphasizing the importance of partnering with youth and caregivers in creating and implementing their care plans. Some participants explicitly referred to this as the need for youth and family-driven care.

Consistent and Continuous Care

Caregivers also reported the need for consistent and continuous care. They highlighted the importance of consistency for their child from providers across the behavioral health system. They specifically mentioned the value of maintaining the same provider so that they and their child would not have to continuously retell their history to a new provider: “My child doesn’t want to keep telling their story, which is traumatic for them, over and over and over and over again to different people. It’s traumatizing.” (CG 2.1) The need for consistency was emphasized by some caregivers who felt that new providers would attempt interventions and strategies that had already been tried unsuccessfully by previous providers—wasting time and resources. As one parent stated, “I don’t want to go back to square one that we already tried and didn’t work, you know, let’s start from where we’re at now.” (CG 6.4).

Caregivers also reported the need for services that could support their child in transitioning across levels of care (e.g., moving from inpatient or residential care back to the home and community setting) so their child doesn’t “fall through the cracks.” (CG 6.6; CG 5.4) They also wanted services that could support their children as they transitioned to the adult behavioral health care and educational systems.

Outcome Priorities for both Youth and Young Adults and Caregivers

Obtaining the knowledge, resources, and tools they needed to support their own or their child’s behavioral health needs, increasing effective communication with providers, within their family, and with peers, and experiencing less judgment and stigma from providers and their community were the top three outcomes important to both YYA and caregiver.

Knowledge, Resources, and Tools

The top outcome identified by both youth and caregivers was obtaining knowledge, resources, and tools to support the behavioral health needs of themselves or their children. YYA wished their providers would educate them about their behavioral health generally: “Like, there’s no like, teaching. It’s just kind of like, diagnosed, or here’s the services or like, ‘This is what I think that you need.’ But there’s no, like, ‘Here’s the research, and this is what I think that you have, and this is why I’m coming to this conclusion.’” (YYA 3.1) Participants also wanted providers to give them tools, exercises, or techniques to better manage their emotional or behavioral challenges. “There needs to be a lot more—I don’t want to call it training, but in a way, training, of how to use the things that they’re teaching you and how to apply them into your daily life and how to make them work for you.” (YYA 4.3) Desired training areas included coping skills, social skills and knowledge, problem-solving skills, conflict management, and skills for independent living such as self-organization and planning, effectively interacting with service systems and public spaces, and self-advocacy.

Youth also want providers to offer their parents or caregivers education about behavioral health and help them learn skills for better supporting the behavioral health needs of their child. “Psychoeducation for the person receiving services, but also psychoeducation for the people that are also involved…. the caregiver should be receiving the same level of education and information about what’s happening.” (YYA 3.1).

Caregivers expressed a desire for providers to offer ideas and strategies to help them support their child’s emotional and behavioral needs, manage their crises, and support skill development. This included learning skills to help their child cope with challenges and stress, supporting them in applying those skills daily, and understanding their child’s triggers for anxiety, depression, or dysregulated emotional responses. “A better understanding of why behaviors or issues are happening and how to address what is triggering the behavior instead of viewing the behavior itself as a negative…” (CG 7.5) and “…it will help us be a better parent to them and to change our ways of holding things together.” (CG 5.2) They also wanted providers to share knowledge about their children and youth’s behavioral health and well-being, educating them about the community resources available to support their children. For this reason, this outcome often co-occurred with access to care. “When I ask questions, the people on the team should be knowledgeable about the resources in their communities.” (CG 8.1).

Effective Communication

Another outcome cited by youth and caregivers was more effective communication. YYA desired better interpersonal communication with their caregivers, family members, and peers. They wanted to talk openly with their caregivers and family members about their problems without conflict. “I hope that I am overall able to communicate with my family as a whole so we aren’t always yelling at each other and there is a mutual understanding of everyone’s needs.” (YYA 2.5) They also wanted to feel confident and competent in engaging socially with their peers and other adults. For youth, this outcome frequently co-occurred with acceptance, improved relationships, and being understood.

YYA additionally wished to have clear and open communication with their provider, and for their provider to communicate clearly and effectively with their caregivers and families to enhance personal understanding, collaboration, treatment effectiveness, and safety.

“I think with mental health providers, there’s a lot of just unknown. So, I think a lot needs to be—a lot more information provided to families. There’s definitely, especially when you go into inpatient, or you go to residential, there’s never enough information provided, and you kind of go into it blindsided, and that can be scary for a lot of families.” (YYA 4.3).

Caregivers emphasized the importance of functional communication from their children’s providers—including teachers—about the treatment process, their child’s needs, plan of care, behavior, and school progress. For this reason, this outcome frequently co-occurred with access to care and provider collaboration. “I also value having a time each week or month to talk to the provider individually without my child in the room to really discuss the progress and struggles and work out a plan to tackle them as a team.” (CG 4.5) Caregivers also wanted more open, honest, and effective communication with their children and their whole family. “I would like for the family to be able to tell each other when something’s wrong or bothering them without the explosion and triggering.” (CG 8.1).

Less Judgment and Stigma

Another critical outcome identified by both YYA and caregivers was less judgment and stigma from others toward mental illness. This outcome was frequently cross-coded with access to care and feeling accepted, suggesting that participants perceived stigma as playing a role in their ability to access services and the level of acceptance they felt from their community. YYA articulated a desire to experience reduced criticism and judgment from their providers and parents or caregivers. Many YYA indicated feeling judgment from providers, stemming from mental health stigma or stigma towards youth. They emphasized the need for providers to be open-minded. “Like, the last thing I need is for the person I’m trying to confide in to judge me…when I’m trying to explain these things to someone…” (YYA 6.3) Participants also hoped for less stigma from broader society. Youth cited the need for education for other youth, their caregivers, and society as a critical pathway to this outcome. “Every type of education that is needed for everyone. OK, so, worldwide education… So, I feel like it needs to be…. publicly addressed.” (YYA 2.2).

Caregivers expressed a desire for reduced judgment of themselves as parents, their children, and their child’s behavioral health needs from providers, schools, extended families, and society. Many participants discussed experiences of providers blaming them for their children’s emotional and behavioral struggles, and some reported threats of being reported to child protective services: “It’s blame game when it’s the parents. Oh, this is your fault for the way your child behaves and things like that.” (CG 2.1) Other caregivers reported being told that their child’s behavioral health needs were a parental discipline issue:

“One thing, if I could have a wish, I would just wish better understanding amongst people, like I had said before, extended family members, but not only that, other caregivers too. I had somebody tell me that they thought I was too easy on my child, and I should beat my child and discipline them when he doesn’t behave.” (CG 8.5).

Discussion

This study is among the first to report the behavioral health outcomes that matter most to youth and young adults (YYA) and caregivers who have utilized a broad array of children’s behavioral health services for a wide variety of behavioral health concerns. Findings are derived directly from YYA and caregiver’s discussions, words, and sentiments and represent the perspectives of families both currently and formerly using behavioral health services, living in communities across the United States.

Though only one of the four questions presented at the focus groups was about the service experience, the caregivers’ most highly valued behavioral health service outcomes reported were about the process of receiving services. This was even after the research protocol added a “magic question” asking caregiver participants to imagine “perfect” service delivery. Caregivers reported outcome priorities that primarily related to the quality and accessibility of services: having accessible services that meet their needs, having providers that collaborate effectively with parents and other service systems, and experiencing consistent and continuous behavioral health care for their child.

Conversely, none of the top outcomes reported by youth were explicitly about the service process. YYA reported hoping to gain skills that would help to enhance their interpersonal interactions: being understood by others, improving their interpersonal relationships, and feeling heard. These outcomes were referred to in the context of personal relationships and their experience with their providers. This finding suggests that youth also place great importance on their experience of services but that they may evaluate the quality of services differently from caregivers—basing it primarily on the perceived quality of their relationship with their providers. Caregivers—as the driver, navigator, and coordinator of care—may have more exposure to systematic failures than youth. In contrast, YYAs’ primary experience of their services system may be derived from their direct provider interactions.

Other studies examining families’ goals in behavioral health services draw their findings from recorded service plan goals, family and youth reported presenting problems, service experience desires, or expressed priority service outcomes (dosReis et al., 2018; Evans et al., 2023; Garland et al., 2004; Jacob et al., 2016; Narendorf et al., 2017; Ross et al., 2015; Weisz et al., 2011). These studies found, similarly, that youth or children share some priorities with parents or caregivers, but that variation exists in how each group prioritizes these outcomes. Generally, our studying findings are consistent with existing research that suggests children and youth prioritize social and family relationship-related service goals (dosReis et al., 2018; Gibson et al., 2016; Gibson & Cartwright, 2014; Krause et al., 2021; Narendorf et al., 2017). Findings for caregivers in this study diverge from the differences observed in prior studies. In existing research, caregivers are more concerned with reducing symptoms (particularly problem behavior, aggression, depression, or anxiety) and increasing functioning (daily life skills, independence) (Garland et al., 2004; Hawley & Weisz, 2003; Jacob et al., 2016; Krause et al., 2021). Previous research also reports that caregivers place more value on the characteristics of the provider system than do youth (Aarons et al., 2010)—which may be reinforced and further explained in the current study.

Our study reports caregivers expressing great concern about the provider system’s quality, characteristics, and behaviors—whereas youth care more about the behaviors and characteristics of—and their relationship with—their individual provider. It should be noted, too, that both caregivers’ priorities for service quality (e.g., service coordination, provider communication, respecting youth and caregivers perspectives) and YYA’s priorities for individual provider qualities (e.g., being understood, being heard, non-judgment) are represented in common best practices for behavioral health care such as the system of care principles (Stroul et al., 2021) and the common factors of psychotherapy (Wampold & Imel, 2015). However, the narratives of families and youth in this study suggest these practices may not be present in many of their behavioral health care experiences—which is consistent with findings from other studies evaluating behavioral health quality experiences for children and their families (Williams et al., 2023; Zima et al., 2019).

Youth and caregivers also seek many of the same outcomes from behavioral health services. YYA and caregiver participants both expressed wanting more effective communication and less judgment and stigma from providers and other professionals. They also reported wanting this from their family members, peers, and their communities. Because families universally agree upon these outcomes, they may hold the most promise as the highest priority outcomes. Providers and service systems may find that focusing service planning around these shared goals, at the individual and community level, may help to establish unity and provide opportunities for small wins that the whole family and their community can celebrate.

Another outcome identified as important by both YYA and caregivers was obtaining the knowledge, resources, and tools needed to support their or their child’s behavioral health needs. Participants commented on the need for providers to be more knowledgeable about resources and skills that can help families, including local community resources that can provide additional supports. More detail is needed from youth and families to clarify what types of resources they hope to learn about in their community. This may include community resources like vocational supports, respite care, mobile crisis services, peer support, or legal guidance. More clarity is also needed about the specific challenges families and YYA face in their daily lives that they hope to develop skills in managing. Families may hope to develop skills in managing behavioral health symptoms, family conflict, behavioral concerns at school, or behavioral health crises, as well as effectively navigating complex child serving systems such as education. Understanding the specific needs behind this outcome priority can guide provider organizations’ professional development investments to better equip their direct service providers to teach families needed skills and coordinate with local community resources and supports.

Positioning Family Priorities in Behavioral Health Policy, Services, and Research

Examining how these outcomes fit into the current policy, research, and service system landscape is vital in identifying the next steps to integrate youth and family priorities into these arenas. The extent to which the outcomes reported here are currently used in federal, state, or local behavioral health system planning, funding, and accountability should be assessed, and mismatches identified. For example, the finding in this study that youth and caregivers highly prioritize the reduction of stigma and judgment from providers and their larger communities raises an essential point of inquiry. Assessing the match between the priority placed on stigma reduction efforts by system administrators and policy—as observed through the allocation of staff, funding, and activities aimed at mental health stigma reduction—and the priority placed on this outcome by families may find an imbalance.

Aiming for greater alignment between policy, administrator, and research priorities and family priorities may increase the efficiency and effectiveness of behavioral services by directly targeting the needs and pathways to healing identified by service users. Further, aligning resources to develop services that deliver the outcomes most desired by families may increase service engagement (Coulter, 2017), reduce dropout rates (Hawley & Weisz, 2005), and graduate families from services more quickly (Edbrooke-Childs et al., 2016). For example, payment systems may provide incentives or bonuses to providers for achieving specific metrics that demonstrate high quality service delivery, service recipients’ improvement in interpersonal relationships and communication, or increases knowledge or skills in managing behavioral health symptoms. To support service providers in achieving these metrics, researchers can target these outcomes in intervention development, testing and comparative effectiveness studies, and implementation scientists can identify strategies for most effectively scaling up the interventions that deliver these outcomes.

In this study, outcomes classified by researchers and policymakers as “system outcomes” or “consumer perspective outcomes” (Hoagwood et al., 1996)–which measure the outputs or experiences of the behavioral health service system—are among the most critical outcomes for cargivers using behavioral health services. However, in a review of outcomes used in behavioral health services research from 1980 to 2011, children’s behavioral health services research examined service impacts on clinical outcomes at twice and three times the rate that consumer experience or service system outcomes were examined; functional outcomes were examined at twice the rate of consumer perspectives and approximately 50% more than service system outcomes (Hoagwood et al., 2012). While this study is now over ten years old, it is the most recent examination of child behavioral health outcomes use in research. It suggests that, historically, most child behavioral health services research efforts and resources have not been targeting the priorities of service users—particularly those of caregivers.

These study findings can be shared and discussed with behavioral health system policymakers, administrators, providers, research funders, researchers, and services funders to understand the extent to which the outcome priorities reported here align with their outcome priorities. Published research studies, program evaluations, calls for research and services funding proposals, and federal, state, and local policy documents can be systematically reviewed to understand current outcome priorities for behavioral health services for children and families. This review may reveal divergence between stakeholders’ reported values and the enactment of those values in the outcomes deployed in policy, funding, and research. Identifying these gaps can provide a framework for understanding and addressing the extent to which certain family priority outcomes are used in these efforts. To advance the broader use of behavioral health outcomes that matter to families, barriers to their use in research and policy making must also be identified and addressed. Understanding the challenges to incorporating these outcomes in research and policy is critical to identifying supports and adjustments needed to expand their use in these activities.

Going Deeper: Making Meaning and Measures

Understanding service users’ priorities for behavioral health outcomes can guide state administrators and researchers in understanding which outcomes to target with behavioral health services research, system accountability, and quality improvement efforts. However, due to the exploratory goals and related limited scope of the study reported here, these findings only provide preliminary groundwork for understanding youth and family outcome priorities. Further work and research is required to make our results fully actionable.

First, a more nuanced understanding of these outcomes is needed. Many outcomes reported in focus groups overlap conceptually, and some are grouped into one concept that may benefit from disaggregation. Researchers made decisions to combine outcomes conceptually, with guidance from participants and FRO partners, that should be revisited with youth and family participants in more focused, deeper and more detailed conversations. Participants can provide greater insight into the meaning they ascribe to priority outcomes, how reported outcomes are differentiated from others, and how they understand outcomes to relate to each other. For example, understanding how stigma overlaps with feeling respected, if cultural responsiveness is a key feature of well-trained providers or if it should be assessed separately, or the extent to which professional communication is conceptually distinct from interpersonal communication are important distinctions. Such insights will guide researchers and policy makers in knowing how outcomes should be measured and reported.

Further, a deeper understanding of these outcomes and the beliefs and values driving YYA and caregivers’ prioritizies is needed. One pathway to understanding the reasons for youth and family priorities may be through uncovering youth and families’ theories about the role that achieving one outcome may play in achieving other outcomes. For example, caregivers may believe that reductions in their child’s behavioral health symptoms is not possible without consistent providers and high-quality coordination of care. The extent to which these theories match the theories of policymakers, system administrators, and researchers should also be assessed.

Next, a clear strategy to operationalize these outcomes is needed. Existing means of measuring these outcomes should be identified, evaluated, and vetted for face and content validity by youth and families. Existing valid and reliable measures should be reviewed with youth and families to evaluate their accuracy in capturing family priorities and measuring the experiences that matter most to them. They must also be reviewed with service providers to assess their usability in practice settings. This process will help to identify gaps in appropriate measurement, and measures to fill those gaps can be developed collaboratively with youth, families, providers and administrators, researchers, policymakers, and funders.

These steps are critical to identifying and developing a comprehensive set of valid and reliable measures to assesse family priority outcomes. However, these processes will be time consuming, complex, and, at times, possibly contentious. It will involve facilitating engagement and agreement among diverse service system constituents with varying priorities and frames of reference: families and youth, direct service providers, behavioral health organizational leaders, service funders, federal, state, and local policy makers and behavioral health system administrators, researchers, and research funders. However, undertaking such efforts is necessary for crafting service systems and organizations that deliver the outcomes prioritized and sought by youth and their families.

Limitations

This study offers perspectives from a range of YYA with behavioral health needs and caregivers of children and YYA with behavioral health needs. Participants represent a wide range of regions, geographies, and policy landscapes across the U.S. They have experienced a broad array of behavioral health diagnoses, diverse levels of care with a variety of behavioral health supports, from several different child serving systems. However, the generalizability of these findings is limited in several ways. First, the sample in this study was not randomly selected. Partnering with FROs with the capacity to actively participate in the project excluded the participation of YYA or caregivers in communities or states with no active FRO or FROs with very small operations. Since some state behavioral health administrations support FROs through funding and collaboration (Hoagwood et al., 2008), while other states do not, this exclusion may bias findings towards the perspectives of YYA and caregivers living in states with more robust funding for youth behavioral healthcare or more infrastructure to support youth and family voice. Because YYA and caregivers involved in peer support organizations or services are more likely to be trained in system advocacy and have extensive experience interacting with the mental health system (Crane et al., 2016; King & Simmons, 2018), reliance on FROs to recruit participants (which relied on their connection to and familiarity with participants) may also contribute to response bias in the data.

Further, the use of more flexible and open approaches to data collection and analysis is critical to amplifying the perspectives of youth and families to generate practice-relevant and translatable knowledge (The Lancet Psychiatry, 2020). However, while these methods may reduce the risk of study findings and interpretation of results being influenced by biases of the research team (Anderson, 2010), they create risks to data reliability and analytic precision. To control for these potential study limitations, the research process was rigorously managed through careful documentation, multiple rounds of coding with multiple coders, regular team reflexive practices, and member checking.

Conclusion

Behavioral health research and policies should be designed to address the outcomes valued by youth with behavioral health needs and families using behavioral health services. This study identifies the service outcome priorities of children and youth with behavioral health needs and their families. This evidence should guide the alignment of service and research funding outcome priorities and is essential to building a youth- and family-driven mental health care system. While additional stakeholder-engaged research is needed to operationalize and accurately measure family priority outcomes, these findings can support the development of behavioral health services that deliver outcomes valued most by children, YYA, and their families.