Introduction

Mental health conditions (e.g., depression, anxiety) in children are associated with increased risk of difficulties at home, school problems, and mental health conditions in adulthood and impact up to 20% of children in the US (Bitsko et al., 2018; Centers for Disease Control & Prevention, 2013; Melnyk et al., 2015). However, less than half of all children who need children’s mental health services (CMHS) receive them (Centers for Disease Control & Prevention, 2013).

The goals of HealthyPeople 2030, set by the Office of Disease Prevention and Health Promotion (2021), include CMHS as one of their priorities. Two specific goals are to “increase the proportion of children with mental health problems who receive treatment” and “increase the proportion of children and adolescents who get appropriate treatment for anxiety and depression.” Importantly, the gap between the need for mental health services and the use of services by children and adolescents continues to widen, especially given the recent increase in need driven by the COVID-19 pandemic (López-Castro et al., 2021; McKune et al., 2021; Yarrington et al., 2021). Furthermore, the number of children receiving mental health care in the US varies significantly from state to state (Mental Health America, 2019; Sturm et al., 2003; Whitney & Peterson, 2019) and research suggests that sociodemographic characteristics only explain a small proportion of this variation (Ghandour et al., 2012, 2019; Sturm et al., 2003). Additional barriers to accessing CMHS include stigma, lack of socioeconomic resources and social capital, and lack of insurance or inadequate insurance coverage (Owens et al., 2002; So et al., 2019).

Analysis of the nationally-representative 2016 National Survey of Children’s Health found that the proportion of children with a mental health condition who did not receive treatment varied from 29.5% to 72.2% across states (Whitney & Peterson, 2019). Such variation creates inequities in access to needed mental health services based on a child’s state of residence, places an undue burden on families, and exacts societal costs because of untreated conditions that can increase the risk for later problems including unemployment, under-employment, and shortened life-spans (Bitsko et al., 2018; Jonsson et al., 2011; Olfson et al., 2015).

One approach that can be used to rectify the inconsistencies in the need and the use of services is through policies at the state level. Policies, defined as any instruments that could be applied by an organization (e.g., agency, non-profit, school) or government (e.g., municipal, county, state, or federal) to influence use of psychological services (Raghavan et al., 2008), may restrict or facilitate the utilization of mental health services by children who need treatment (Ghandour et al., 2012; Sturm et al., 2003). State-level policies may provide valuable insight into understanding inequities between states in the utilization of CMHS.

State mental health agencies (Bruns et al., 2019; Hernandez et al., 2017; Purtle et al., 2021a) and advocates (Bushouse & Mosley, 2018; Kingdon & Stano, 1984; Teater, 2008) play a crucial role in shaping the children’s mental health agenda in their states through their involvement in the decision-making and implementation processes for policies that leverage a federal budget $125 million dollars allocated to CMHS (Department of Health & Human Services, 2021). Despite their significant influence on the policy design and decision-making process, few studies have explored how these stakeholders perceive the importance of potential policy levers that may impact the use of CMHS in their states.

This study aims to examine state-level policies and policy domains that policymakers and advocates believe impact the utilization of CMHS and to identify the policies and policy domains perceived as most important for utilization of CMHS. Note that this study is not focused on identifying policy levers that improve the overall mental health of children. We used a modified Delphi technique to engage an expert panel of state-level children’s mental health policymakers and advocates in a three-round consensus building process composed of two web-based questionnaires and an interview. The Delphi technique was developed to work with experts to achieve reliable consensus and has been used across the healthcare sector (e.g., implementation science, vocational rehabilitation, health care systems) for identification and prioritization of items (Coller et al., 2020; Okoli & Pawlowski, 2004; Powell et al., 2015; Roux et al., 2018).

Methods

We used a modified Delphi technique during February and April 2021 to develop consensus among state-level policymakers and advocates on state policies that impact the use of CMHS. The Delphi technique is used to gather information from individuals within their domain of expertise through a series of questionnaires and structured feedback cycles that provide individuals the opportunity to understand the group’s perspective and adjust their views accordingly (Okoli & Pawlowski, 2004). This method is often used to identify or prioritize issues in a specific domain, ends in group consensus (Hsu & Sandford, 2007; Okoli & Pawlowski, 2004) and modified for use in health services research and policy (Coller et al., 2020; Degeling et al., 2019; Hasson & Keeney, 2011; Khodyakov et al., 2020; Powell et al., 2015; Roux et al., 2018).

Traditionally, the Delphi technique uses one round of qualitative interviews with experts to solicit information, consists of three or more rounds of soliciting feedback and ranking to reach group consensus, and is administered via mail or in-person (Hasson & Keeney, 2011). However, the Delphi technique is often adapted based on the situation and the research question (i.e., offered electronically vs. in-person) (Hasson & Keeney, 2011). We used a modified Delphi technique because our research was conducted online (e.g., web-based survey and videoconference), we provided the experts with a pre-selected group of state policies in Round 1 based on a rapid review and we completed interviews in Round 3 (Fig. 1). The original Delphi technique was modified to fit our study design and allow for elaboration and expansion on how experts’ state policy context potentially influenced their rankings. However, experts were given the opportunity to provide additional input via the Round 1 web survey, despite not having an initial interview round.

Fig. 1
figure 1

Overview of modified Delphi research design

Full size image

Our study followed a QUAN → qual mixed methods design in which quantitative data was collected in Rounds 1 and 2 through a web-based questionnaire and qualitative data was collected in Round 3 for the primary purpose of confirming, elaborating, and expanding on the rating of policies from Round 2 (Palinkas et al., 2011). Group consensus was defined as 70% agreed that the list of state policies captured “the main policies that impact the use of children’s mental health services.” Experts were offered a $25 gift card for completing Round 1 and a $75 gift card after completing Rounds 2 and 3.

Selection of Expert Panelists

Twelve states were selected for participant recruitment to ensure a diverse sample from state-policy contexts with the goal of one policymaker and one advocate per state. Research from the National Academy of Sciences suggests the importance of anti-poverty policies in improving children’s health (National Academies of Sciences, Engineering and Medicine 2019), thus, the sampling frame was designed based on the percent of unmet need for CMHS in the state based on the National Survey of Children’s Health data from 2018 and the presence of a state-level earned-income tax credit (Child and Adolescent Health Measurement Initiative Data Resource Center for Child and Adolescent Health, 2018; Urban Institute, 2021). States were dichotomized across two domains: unmet need for CMHS (above or below mean) and state-level earned income tax (yes/no). We then used a factorial design to sample three states within each of the four state strata (high unmet need, no state-level earned income tax credit; low unmet need, no state-level earned income tax credit; high unmet need, state-level earned income tax credit; low unmet need, state-level earned income tax credit). The 12 states included in the final sample frame were Arizona, Tennessee, Texas, Washington, New Jersey, South Carolina, Montana, Pennsylvania, South Dakota, Colorado, New York and Illinois.

Approximately two policymakers and two advocates from each state with active roles in policies or programming related to children’s mental health were invited to participate (n = 45). We used a purposive sampling technique that began with an initial list of policymakers in children’s mental health from a prior study on research evidence use by policymakers in children’s mental health, mainly state mental health officials (Purtle et al., 2021b). A list of advocates was generated by the study team, in which we targeted several groups based upon their expertise in mental health policy, including state-level National Alliance on Mental Illness (NAMI) chapters, Mental Health America chapters and experts associated with advocacy organizations that were involved in state-level initiatives focused on children’s well-being. The initial group of nominees were encouraged to identify other members of their organization or partner organizations with appropriate expertise for participation. Experts were contacted three times via email in a 2-week time period before an alternative was selected.

Of the initial list of 45 policymakers and advocates, 62.2%, or 28 experts (n = 14 policymakers; n = 14 advocates) agreed to participate in at least one round of Delphi process (Table 1).

Table 1 Composition of expert panel
Full size table

Delphi Survey Development

We developed an initial list of 30 state-level policies based on a rapid review of CMHS literature that was modified through e-mail correspondence with two external state children’s mental health experts for relevance, redundancy and completeness. We developed our initial list of policies starting with a 2019 literature review (So et al., 2019) on evidence for specific policy approaches to promote utilization of CMHS which drew theoretically from Roberts’ Five Control Knobs for Health Services Reform (e.g., organization, regulation, finance, community education, and payment) (Roberts et al., 2003). Then we reviewed other reports from the National Academies of Sciences, Engineering and Medicine (2019), the National Center for Children in Poverty (2021), Smith et al., (2017, 2020), and the National Association of State Boards of Education (2020) to ensure all other relevant policy sectors were included. Twenty-four policies were identified and organized into four domains to represent our initial list of state-level policies that may impact the use of CMHS:

Insurance Coverage and Limits

This domain included aspects of a state’s Medicaid plan that might affect utilization (e.g., Medicaid reimbursement for telemental health services, Medicaid limits on mental health service provision in various settings, or coverage of health navigators).

Mental Health Services

This domain included policies that specifically focus on mental or behavioral health services that could impact utilization of services based on provider supply, access, or quality (e.g., scope of practice for mental health providers, requirements that mental health providers use evidence-based services).

School-Based

This domain included policies that encourage the use of mental health services in a school setting (e.g., policy support for school-based mental health promotion or intervention programs, state mandated classes on social–emotional learning in schools).

Social

This domain included policies that may indirectly affect the utilization of CMHS through the social determinants of health (e.g., state-funded housing assistance programs, paid family leave policy).

Two mental health stakeholders who did not participate in the study reviewed the initial questionnaire and provided feedback on the policy levers, their definitions and the policy domains.

Modified Delphi Process

Round 1

The first of two web-based questionnaires were conducted between February and March 2021. In Round 1, experts (n = 28) were asked to rate and rank 24 policies (definitions provided in Appendix A and Round 1 questionnaire provided in Appendix B) based on their importance for improving the utilization of CMHS. Ratings were based a 7-point Likert scale (1 = not important, 7 = extremely important). Experts were given the opportunity to comment on the definitions of each policy provided and to suggest up to five additional policies. At least two respondents completed the survey from each of the twelve states. Univariate descriptive statistics (means and standard deviations of the ratings) were calculated using STATA 15 for use in the Round 2 questionnaire.

Round 2

The second round was completed by 18 experts (n = 9 policymakers, n = 9 advocates). The group mean importance scores for each policy and newly generated policies were added to the Round 2 questionnaire (Appendix C). Additionally, de-identified comments and feedback from experts in Round 1 were aggregated and listed verbatim at the beginning of each policy domain. Comments are not part of the rating process, however, they are part of the feedback loop that is a crucial part of the Delphi technique and the consensus building process (Mead & Moseley, 2001). Experts are given the opportunity to change their ratings with the benefit of knowing how other experts in the group rated and critiqued each policy during the consensus building process.

Experts were asked to rate 33 policies (nine new policies generated in Round 1) on the perceived importance of the policy on the use of CMHS using a 7-point Likert scale (1 = not important, 7 = extremely important). Then, they were asked to “select five policies, looking across all policy domains, that are important for increasing the use of children's mental health services.”

Additionally, experts were asked two questions about their agreement with the list of state policies: “The list of state policies captures the main policies that impact the use of children’s mental health services,” and “The domains (i.e., insurance limits and coverage, mental health services, school and social) are useful for thinking about policies to address the use of children’s mental health services” using a 7-point Likert scale (1 = strongly disagree, 7 = strongly agree).

Round 3

Of the 18 participants that participated in Round 2, 15 individuals agreed to participate in a 30-min semi-structured Zoom-video interview during March to April 2021. These interviews were conducted to understand (1) agreement or disagreement with the most important policies rated by the group, (2) reasoning for personal top policy ratings, (3) their role in what their state legislature prioritizes, and (4) the impact of behavioral health provider shortage on the use of CMHS and the feasibility of certain policy options in their states (Appendix D). Prior to their interview, experts were emailed a list of the policies rated most important by the group as a whole and the policies they individually rated as most important. All interviews were conducted by the lead author and were audio recorded and transcribed.

Data Analysis

Quantitative Analysis

Data from Round 1 and 2 questionnaires were exported from Qualtrics and entered into STATA 15. Univariate descriptive statistics (means, standard deviations) were calculated for policies and policy domains in Rounds 1 and 2. Group frequencies were calculated for each policy for the question “select five policies, looking across all policy domains, that are important for increasing the use of children's mental health services,” and the most important policies were those with the highest frequencies. The items assessing agreement in Round 2 were dichotomized and responses of “strongly agree” and “agree” were coded as “agree.”

Qualitative Analysis

The lead author reviewed the transcripts and used matrices to code examples of expert perspectives on important policies borrowing techniques from the RADar (rigorous and accelerated data reduction) technique (Watkins, 2017) for qualitative analysis. Data was extracted from interview transcripts in response to the questions “In the questionnaire you indicated that [insert their five most important policies] were the most important for increasing the use of children’s mental health services you state. Can you explain your reasoning?” and “Do you agree with the top three policies selected by the group? Why or why not?” and put in Microsoft Excel. This information was coded by policy, stakeholder type, and state. Next, quotes were reviewed and reduced by selecting specific pieces of each quote and removing non-exemplary quotes. Finally, this table was reduced based on stakeholder type and state to ensure a representative, concise representation of confirmatory examples for each of the group’s most important policies (Watkins, 2017).

Mixed Methods Analysis

Our study used an explanatory sequential design (QUAN → qual) design (Fetters et al., 2013; Palinkas et al., 2011). We utilized a building integration methods approach where the quantitative data from the questionnaires was analyzed and used to inform the development of the interview guides (Fetters et al., 2013). This approach to data collection had three main functions: (1) to confirm the data collected in Rounds 1 and 2, (2) elaborate and provide additional context for the decision-making in the questionnaire process, and (3) expand and answer questions raised after analysis of the questionnaire data (Palinkas et al., 2011). In the analysis phase, we integrated the quantitative data from both rounds of questionnaires and the qualitative data from the interviews by connecting the data to use qualitative data to corroborate the findings from our quantitative data, as well as, enhance the meaning of and explain the context of the quantitative policy rankings (Palinkas et al., 2011).

Results

Expert panelists provided several comments and policy suggestions to the initial list. Nine new policies were suggested in Round 1: increased Medicaid reimbursement rates, value-based payments for integrated care, mental health workforce development, certified behavioral health centers, anti-stigma campaigns, age of consent for treatment, enforcement of Individualized Education Plans, non-emergency medical transportation and Special Supplemental Nutrition Programs for Women, Infants, and Children (WIC). Much of the feedback provided in Round 1 was not focused on how policies were operationalized, but rather, commentary on a specific policy and how it was relevant in their state or requests for more detailed information on a policy. For example, for the policy Medicaid carve-in/carve-out, experts noted “Carve in or out is not as important as the network of providers available and the rules around access. Payment rates are also important,” or “Not sure about the carve in and how important that is.” Feedback for each policy domain was integrated at the beginning of each policy domain and the definition for Medicaid carve-in/carve-out was clarified in the Round 2 questionnaire (Appendix C).

Seventy-eight percent of the experts (consensus defined as > 70% of agreement) that completed Round 2 agreed the final list of 33 state policies captured “the main policies that impact the use of CMHS” and 89% of experts agreed that “the domains were useful for thinking about policies to address the use of children’s mental health services.”

State Policy Domains

Table 2 reports the results from Rounds 1 and 2 of the modified Delphi process and Table 3 provides illustrative quotes of support for or against the most important policies rated by the group from the confirmatory interviews. “School” policies had the highest average importance rating in both rounds (Round 1 M = 6.15, Round 2 M = 5.89). “Insurance limits and coverage” had the second-highest average importance rating in Round 1 and the third-highest average importance rating in Round 2 (Round 1 M = 5.95, Round 2 M = 5.42), while the “mental health services” domain (Round 1 M = 5.24, Round 2 M = 5.27) had the third-highest rating in the Round 1 and the lowest rating in Round 2. Finally, “social” policies had the lowest average importance rating in Round 1 and the second-highest average importance rating in Round 2 (Round 1 M = 4.66, Round 2 M = 5.43).

Table 2 Compilation and ranking of state policies: results from Rounds 1 and 2 of the modified Delphi process
Full size table
Table 3 Illustrative quotes from policymakers and advocates in reference to the three highest ranked policies
Full size table

State Policies

The top-rated policies for school domain were school-based mental health services (M = 6.67, SD = 0.67) and school service referrals (M = 6.06, SD = 1.31). The top-rated policies for the insurance coverage and limits domain were increased Medicaid reimbursement rates (M = 6.44, SD = 0.76) and Medicaid reimbursement for telemental health services (M = 6.29, SD = 1.07). The top-rated policies for the mental health services domain were state mental health parity (M = 6.61, SD = 0.67) and mental health workforce development (M = 6.28, SD = 1.04). The top-rated policies for social domain were state-funded housing assistance programs (M = 6.22, 0.63) and paid family leave policy (M = 6.12, SD = 0.76).

Important Policies and Confirmatory Interviews

The only policy domain that did not contain a policy rated in the top three was the social policy domain. The most important policies overall were school-based mental health services (M = 6.67, SD = 0.67), state mental health parity (M = 6.61, SD = 0.68), and increased Medicaid reimbursement rates (M = 6.44, SD = 0.76).

In the confirmatory interviews, experts that included school-based mental health in their most important policies focused on providing CMHS “where kids are” (Participant 11) and felt that it was a policy that garnered bi-partisan support because “both sides of the aisle believe that kids can’t control the cards that are dealt to them” (Participant 1) (Table 3). Those who did not include school-based mental health in their most important policies expressed concerns over “infrastructure to bill medical claims” (Participant 12) and the difficulties that come with involving families in school-based mental health care, despite “all our most effective interventions…[being] family-based” (Participant 13). Experts who included state parity in their most important policies felt it existed in their state, but was not enforced, while experts who did not include state parity in their most important policies felt state parity was less of an issue in their state. Most experts agreed that increased Medicaid reimbursement rates belonged in the most important policy list, stating “I agree with [that policy]” or “I’m not surprised to see [it]” (Participant 6). One expert who agreed that increased Medicaid reimbursement rates were an important policy suggested that higher rates could “help bolster the community mental health workforce because you can attract talent with more competitive salaries and benefits” (Participant 7) while another expert mentioned that “… [Medicaid] reimbursed so low for psychiatric residential treatment facilities that our providers have to take kids from outside [the state] to subsidize the kids that are seen in [the state], which reduces the number of beds available for our state’s kids” (Participant 9).

Discussion

This study identified discrete state-policy levers that are perceived to impact the access to and utilization of CMHS systems using a national sample of state-level children’s mental health stakeholders. The mean range for individual policies and policy domains were relatively high for Round 1: (M = 3.87, 6.59; M = 4.66, 6.15; respectively) and Round 2 (M = 4.44, 6.67, M = 5.27, 5.89; respectively), suggesting that most experts perceived most policies in the list as important drivers for the use of CMHS. There was a noticeable shift in means between Rounds 1 and 2 which is to be expected due to the feedback mechanism inherent in the Delphi technique that provides experts the opportunity to reflect and revise their initial ratings based on group means and group comments from Round 1.

School policies had the highest average importance of the four domains, while school-based mental health services were the most important policy. This is in line with a 2020 systematic review and meta-analysis by Duong et al. that a higher percentage of youth in the general population (7.3%) and youth with elevated mental health symptoms (22%) received mental health services in schools than outpatient, primary care, and inpatient settings (Duong et al., 2020). However, a study using the National Survey on Drug Use and Health found a decreasing trend in the receipt of mental health care in schools from 49.1% in 2005 to 45.4% in 2018 (Mojtabai & Olfson, 2020). Despite the importance of school-based mental health services suggested by this study, in 2019 a majority of states “encouraged” but did not “require” school-based mental health services (National Association of State Boards of Education, 2020).

Our findings identify gaps in expert perceptions of potential social policy interventions. Social policies that impact upstream determinants of health and, ultimately downstream health behaviors, are also important to consider when seeking to understanding barriers to CMHS utilization (Andersen, 1995, 2008). For example, research suggests that social determinants like living in a high-poverty neighborhood (Chow et al., 2003) or socioeconomic status (Newacheck et al., 2003) can impact the use of CMHS. However, perceptions of the importance of social policies that may address barriers related to social determinants of health like paid family leave, state-funded housing programs, or Special Supplemental Nutrition Program for WIC, were the lowest of all policy domains in Round 1. This finding is important to note in the context of increasing federal funding opportunities that encourage states to develop sustainable mechanisms for cross-sectoral support to improve the well-being of children, like the Center for Medicare and Medicaid Innovation’s new program, Integrated Care for Kids Model, that requires select states to develop a child-focused alternative payment model and aligns the child and family support provided across multiple state agencies (Centers for Medicare & Medicaid Services, 2021). This study suggests variable stakeholder understanding of the potential influence of social policies on the use of CMHS. State-funded housing programs and paid family leave policy received higher ratings in Round 2 and several experts noted their disagreement with the lack of social policies in the group’s most important policies in their interviews, despite these social policies being the lowest rating policies in Round 1 of the Delphi process. Future research should explore the relationship between state social policies and access to or use of CMHS, in addition to, the potential impact of a peer-feedback mechanism aimed to improve perceptions of social policies.

State parity laws were a top-rated state policy by experts. States significantly influence the use and cost of CMHS by the extent to which they enforce the Mental Health Parity and Addiction Equity Act (Azrin et al., 2007; Barry & Busch, 2007, 2008; Kennedy-Hendricks et al., 2018; Perrin, 2018; So et al., 2019). A study by Kennedy-Hendricks et al. found that insurance plans that were subject to parity were associated with lower annual out-of-pocket spending (e.g., $140) for children’s mental health conditions (Kennedy-Hendricks et al., 2018). However, despite widespread state adoption of parity laws, the implementation and enforcement of mental health parity laws vary tremendously (Cauchi & Hansom, 2015; Douglas et al., 2018). As such, federal legislation that strengthens the enforcement of mental health parity laws, like the Parity Enforcement Act of 2021 (“Parity Enforcement Act of 2021”), could have significant implications for the affordability and use of CMHS in states that lack strong parity enforcement.

Experts also identified Medicaid reimbursement rates for mental health services as an important state policy for CMHS and discussed how Medicaid reimbursement rates were used to incentivize (or inadvertently disincentivize) pieces of the CMHS system. A report on reimbursement rates for commercial insurance plans found that the reimbursement rates for primary care office visits were reimbursed 30–50% higher than behavioral health office visits (Davenport et al., 2019). Particularly in rural or frontier areas, our findings suggest that providing competitive Medicaid reimbursement rates, especially in comparison to neighboring states, could help address disparities in access to and use of CMHS. More research is needed to document the potential relationship between state-level Medicaid reimbursement rates and access to or use of CMHS.

Experts did not perceive state policies mandating the use of evidence-based practices for mental health clinicians as being impactful on the access and use of CMHS. This could be due to this policy being most impactful for children who are already receiving mental health services and an unclear connection between how this could impact the access and use of CMHS. However, this may be indicative of a larger trend in state policymakers’ lack of understanding around the importance of evidence-based practices in CMHS. Research conducted with state and county policymakers found that the de-implementation of non-evidence-based programs for children’s mental health and substance use was consistently viewed as a low policy priority (Nelson et al., 2021; Purtle et al., 2021a). Previous research also found significant state-level variation in the use of evidence-based treatments for children (25–50%) (Bruns et al., 2016). This suggests an opportunity for more targeted dissemination to state policymakers on the importance and benefit of evidence-based practices given their crucial role in securing funding.

This study provides important insight into how key stakeholders in the policy context or outer setting perceive policies that can impact the successful implementation of evidence-based practices. Several implementation science frameworks (i.e., Consolidated Framework for Implementation Research, the Policy Ecology of Implementation framework and the EPIS Implementation Framework) acknowledge the impact of the external environment (e.g., service policies, funding constraints, social policies) on the receipt and the sustainability of evidence-based practices (Aarons et al., 2011; Damschroder et al., 2022; Raghavan et al., 2008). Our work provides insight into challenges and opportunities in the outer context for CMHS for stakeholders to consider as they work to implement evidence-based practices in this sector.

The findings from this study have important implications for the CMHS system policy design process. This study can serve as the basis for a novel policy tool that supports evidence-based policymaking by leveraging the experiential knowledge of policymakers and advocates, while also applying methodological rigor. First, the findings from this study can be used to tailor the dissemination of relevant information and increase policymakers’ and advocates’ understanding of the existing evidence-base for specific policies. This study identifies gaps in perceptions of evidence-based programs or policies that improve the utilization of CMHS by highlighting evidence-based programs or policies that are poorly perceived. For example, state mental health agencies or advocates can use this group of policies to identify gaps in knowledge within their own state’s children’s mental health stakeholders through an online survey or a facilitated conversation.

Second, CMHS stakeholders can use the results of this study to jumpstart the policy design process in their state. The substantive component of policy design is composed of a set of potential policy levers that stakeholders believe are capable of addressing the key policy problem (i.e., unmet need to CMHS) (Howlett, 2019a). Consequently, an important, early step in the policy design process is the analysis of different types of policy levers that impact the output or outcome of interest (Howlett, 2019a). The final group of policies identified by the experts in this study can be used as a starting point for states undergoing the policy design process with a CMHS-focused outcome interest and the domains can help stakeholders identify policies that may be more relevant in their existing state policy landscape.

Third, stakeholders can view and select complementary policies using these policy domains (i.e., insurance limits and coverage, mental health services, school, and social) to build upon existing state-level policy and programming in CMHS systems. As noted in the public policy literature, the exploration of multiple options to address a problem is a critical piece of policy design (Howlett, 2019b). These results can help states consider other polices and policy domains through which the state can address social determinants of mental health.

Finally, given that over three-quarters of the study’s experts felt the final list of policies encompassed the most important policies that impact children’s mental health, these results can guide state policy surveillance work—the systematic collection, analysis and dissemination of information about laws or policies (Burris, 2014; Burris et al., 2010, 2016)—focused on CMHS. Policy surveillance is a crucial tool to support evidence-based policymaking, or the process of using high-quality data and analysis of those data to inform decisions that are made about policies through different levels of government (Bipartisan Policy Center; Pew-MacArthur Results First Initiative, 2014). Policy surveillance is used across a wide range of health disciplines including access to contraceptives (Merz et al., 1995) and syringe exchange programs (Burris et al., 2002). Advocacy groups like Mental Health America and NAMI have taken some steps towards policy surveillance in the CMHS field. For example, Mental Health America, a national mental health advocacy organization, completed a project titled “Ranking the States,” in which states were ranked on youth mental health for seven measures (Mental Health America, 2020). However, only two of the measures looked at indicators (i.e., children with private insurance that did not cover mental or emotional problems and students identified with emotional disturbance for an Individualized Education Program) that could be impacted by policy change and did not include any indicators of state-level policies. Additionally, NAMI published a report on trends in state mental health policy that includes trends in all the themes identified in our study but does not track the presence of each policy across every state (NAMI, 2019). The results of a policy surveillance study can improve stakeholder awareness of a state’s current policy landscape, identify policy opportunities, and analyze state policy trends (Burris, 2014; Burris et al., 2010, 2016).

Limitations

The Delphi process was carried out during the Spring of 2021, during the COVID-19 pandemic. Unfortunately, due to the significant burdens that people were experiencing during this time, we unable to retain all original 28 experts in all three rounds of the modified Delphi process. However, our response rate for Round 3 (53.6%) is within the range of other Delphi studies with three rounds (between 45 and 93%) (Gargon et al., 2019). This is still an adequate sample size for achieving consensus (Hsu & Sandford, 2007) given that the final sample of 23 was still representative of the initial sample of experts (e.g., 50% advocates, 11 out of 12 states represented). Additionally, of our initial list of 45 experts, 28 experts (62%) agreed to participate and we did not have a policymaker or advocate representing each of our twelve states. This has implications for the representativeness of our initial list of policies and final policy rankings as these may have differed based on a larger, more diverse sample of experts. We sought to create a sample that was representative of state policymakers and advocates from different policy contexts, however, the results of this study are not necessarily generalizable to all US states. We provided an initial list of policies with definitions that were updated based on expert feedback from Round 1; however, there may still be policies that were misinterpreted or unclear. Lastly, though our questions were grounded in access to CMHS (Appendices B–D), experts may have selected policies based on a policy’s ability to improve overall mental wellbeing in children, not access specifically.

Conclusion

Engaging CMHS stakeholders using a modified Delphi technique generated a detailed, relevant, and expanded group of policies that are used to design state CMHS systems. State mental health agencies and advocates in CMHS could leverage this group of policies to understand current policy landscape in their state and identify gaps in policy domains (e.g., school, social) to create a more comprehensive system to address children’s mental health from a holistic, evidence-based perspective. Research and dissemination of evidence on the impact of social policies on CMHS utilization may be effective at improving policymaker and advocate perceptions of and integration of social policies in children’s mental health system design. More research is needed to demonstrate the relationship between (1) specific policies or (2) domains of polices and CMHS outcomes.