Introduction

Engaging family members in adolescent substance use treatment has long been considered a best practice and vital to successful treatment outcomes. Increased family treatment engagement, broadly defined as a family member’s behavioral, attitudinal, and affective involvement in adolescent treatment, has been associated with greater adolescent treatment engagement and adherence (Kumpfer et al. 2003) and reduced rates of treatment dropout (Liddle 2004). Further, greater engagement of family members is associated with increased length of sobriety (Liddle 2004; Steinglass 2009) and lower numbers of relapses (Steinglass 2009). Family engagement in substance use treatment can also improve parent mental health (Copello et al. 2005; Smith et al. 2004) and substance use–related family problems such as parent–child conflict (Steinglass 2009), indicating that family treatment engagement is advantageous for both adolescents receiving treatment and their family members. These findings have led researchers and organizations such as the Substance Abuse and Mental Health Services Administration (SAMHSA) to devote considerable resources to identifying best practices for engaging families in adolescent substance use treatment.

Substance use providers also recognize the importance of engaging family members in substance use treatment, as well as the challenges in doing so. In a study of 17 adolescent treatment programs, staff reported that parent engagement can be limited by parents’ beliefs that they are not responsible for their child’s change, parents’ views that substance use is episodic rather than chronic, scheduling and transportation issues, parents’ own mental health and substance use, and parents’ beliefs that teen drinking and drug use are normal and acceptable (Rieckmann et al. 2011). A number of strategies have been developed to address these challenges, such as providing fact sheets and education about addiction and the treatment process, offering a direct family program phone line (NIATx 2009; Gosnold n.d.), and offering intervention services to help family members get their loved ones into treatment, such as Community Reinforcement and Family Training (CRAFT; Smith et al. 2004). The effectiveness of these strategies differs across agencies and populations, making it important for providers to regularly assess whether an implemented strategy or intervention is successfully increasing family treatment engagement.

In order to assess their family treatment engagement efforts, substance use providers need quality measurement tools. In particular, providers will benefit from measures that are easy to administer, can be applied in diverse treatment contexts to monitor ongoing engagement, and that assess multiple domains of family engagement. A common approach to measuring engagement is to track treatment attendance and dropout. While this approach is an efficient way to assess the “bottom line” of engagement, counts of attendance and dropout suffer from several weaknesses, including that they (1) do not provide sufficient warning of emerging engagement challenges, (2) confound a number of potential barriers to engagement (transportation, family conflict, treatment attitudes, etc.), (3) are inappropriate for treatment settings in which regular attendance by family members is not expected, and (4) do not capture other domains of treatment engagement (attitudinal and affective) recognized as important by providers and researchers (Hornberger and Smith 2011; Rieckmann et al. 2011; Staudt 2007). For these reasons, substance use providers will benefit from measurement tools that provide more nuanced information to guide clinical practice, such as self-report and observational approaches.

Reliable, valid, and theoretically grounded measures of family treatment engagement are also critical for advancing research on the predictors and consequences of family member engagement in treatment, as well as assessing outcomes in engagement intervention studies. This latter point was highlighted by a recent review of approaches to engaging family members in treatment, in which the authors noted that the outcomes of family engagement interventions were difficult to interpret and compare due to the inconsistent conceptualization and measurement of family treatment engagement across studies (Kim et al. 2012).

A number of theories have expanded the concept of engagement to include attitudinal and affective domains. This distinction between the behavioral domain and attitudinal and affective domains is essential since family members may perform engagement behaviors, such as attending treatment sessions, without ever fully investing themselves in treatment (Gopalan et al. 2010). An important next step is to empirically test these theories using measures of family engagement that include these domains (Becker et al. 2013; Staudt 2007). Yet, the extent to which existing measures assess these domains is unknown. Identifying the congruence between existing measures and this multi-dimensional conceptualization of family treatment engagement will help researchers select instruments that best represent their intended conceptualization of family engagement. It will also help identify gaps in the conceptual coverage of current measures and potential needs for future measurement development. Unfortunately, while Tetley et al. (2011) conducted a recent review of client engagement measures, we were unable to find any such reviews of measures that assess family treatment engagement.

The landscape of treatment engagement research is characterized by a wide variety of engagement definitions and conceptualizations. Researchers have conceptualized treatment engagement as an individual trait or state (Hogue et al. 2006), a characteristic of the provider-client relationship (Horvath and Greenberg 1994), a set of behaviors by the treatment provider (Hogue and Liddle 2009; Nock and Kazdin 2005), and an early stage in the treatment process (Germain and Gitterman 1996). It is important, therefore, for researchers to clearly articulate their conceptualization of family treatment engagement before a meaningful investigation can take place. We offer the following assertions to summarize our conceptualization of family treatment engagement.

Family treatment engagement pertains to family members and concerned significant others who are close to the adolescent in treatment Family treatment engagement is often assessed using measures designed for the client, here defined as the individual whose change is the primary focus of treatment. Yet this approach does not capture the engagement of family members in the context of adolescent substance use treatment (Accurso et al. 2013). In contrast, we define family as those significant others who are not the direct focus of treatment but whose engagement is integral to creating and sustaining the client’s positive change. For this reason, family treatment engagement is inherently triadic, and exists between the family member, the adolescent client, and the treatment provider. This distinction has important practical implications for measuring family engagement in different treatment settings. For example, a review of family interventions for alcohol and drug problems distinguished between three types of interventions: (1) those that work with family members to promote treatment engagement of individuals with substance use disorders; (2) those that involve family members in the treatment of individuals with substance use disorders; and (3) those that focus primarily on the needs of family members (Copello et al. 2005). Family members participating in the second category of interventions would be most appropriately assessed by family engagement measures, while those in category 3 would be best assessed using client engagement measures.

Family treatment engagement exists in relationship to the treatment and begins when treatment begins This distinction is important to make, particularly with regard to individuals’ attitudes toward treatment. Some conceptualizations of treatment engagement have included the attitudes of individuals before they begin treatment, such as openness to treatment and treatment expectations (Randolph et al. 2009). Others have said that treatment engagement begins with the recognition or identification of a problem (McKay and Bannon 2004). Consistent with engagement researchers such as Tetley et al. (2011), we consider these pre-treatment attitudes to be indicators of treatment readiness, rather than treatment engagement.

Family treatment engagement is a characteristic of individual family members and includes attitudinal, affective, and behavioral domains In contrast to being a relational stage, family treatment engagement, as we conceptualize it, refers to an individual characteristic that exists in relation to the treatment and the adolescent client. We also consider family treatment engagement to have attitudinal, affective, and behavioral domains. The attitudinal domain includes family members’ cognitions and beliefs about the treatment itself, their beliefs about the clients’ involvement in the treatment, and their own roles in the treatment process (Staudt 2007; Yatchmenoff 2005). An example of attitudinal engagement would be a family member’s belief that the treatment will help the adolescent end their substance use. The affective domain of engagement refers to family members’ experience and expression of emotion in relation to their involvement in the clients’ treatment (King et al. 2014). Examples of affective engagement include the family member’s motivation to participate in treatment and expressions of enthusiasm during the treatment process (Cunningham and Henggeler 1999). The behavioral domain of treatment engagement includes behavioral indicators such as attendance, participation, and adherence to treatment protocols (King et al. 2014; Randolph et al. 2009); efforts to communicate with the treatment provider; and actively supporting the client’s engagement in treatment (e.g., transporting the client to treatment or helping the client complete out of session work).

The purpose of this study is to review existing measures of family treatment engagement and to assess their utility for adolescent substance use treatment settings. The previous paragraphs have articulated how we conceptualize family treatment engagement and defined the scope of this review.

In particular, we seek to:

  1. 1.

    Determine which domains of engagement (attitudinal, behavioral, and affective) are captured in existing measures and assess the congruence of these measures with current multi-dimensional conceptualizations of family engagement.

  2. 2.

    Examine the clinical utility of existing measures (ease of administration and scoring, applicability to substance-use treatment settings, availability of normative data).

  3. 3.

    Describe the psychometric properties data that are available for the measures.

  4. 4.

    Make recommendations for the future measurement of family engagement in substance use treatment settings.

The results of this review will help practitioners select appropriate measures to monitor and evaluate the effectiveness of their family engagement strategies. These findings can also inform future research by describing the sufficiency of existing measures for assessing current conceptualizations of family engagement and assisting researchers in theoretically based selection of family engagement measures.

Method

Search Strategy

A systematic search strategy was used to identify measures of family treatment engagement used in substance use and mental health settings. While originally we planned to focus only on substance use settings, an initial scoping review of the substance use literature revealed few measures of family engagement. Therefore, we decided to include the closely related field of mental health in this review.

We focused our review on measures that could be completed by family members’ self-report, client report, or by treatment providers. We conducted a broad search that included closely related concepts such as treatment alliance, treatment adherence, and treatment involvement. Regardless of the measure title, we conducted an item-by-item assessment to determine if the measure was consistent with our multi-dimensional conceptualization of family treatment engagement.

To be included in the review, a measure had to: (1) focus on the engagement of family members or concerned significant others on behalf of a child or adolescent (younger than 18 years old); (2) be published in a peer-reviewed publication; (3) assess engagement during treatment; (4) be a rating scale or questionnaire measure of family engagement; and (5) be used in a substance use or mental health treatment setting. Measures were excluded from the review if they: (1) took place in a non-treatment related setting, (2) measured engagement using behavioral counts (e.g., attendance, dropout), (3) measured the engagement of the immediate client, rather than a family member or concerned significant other supporting the client, (4) assessed pre-treatment characteristics, or (5) were non-English language.

Six electronic databases (Social Work Abstracts, Social Services Abstracts, PsychINFO, Academic Search Complete, CINAHL, ERIC) and three measurement clearinghouses (Health and Psychological Instruments, PSYCTests, Mental Measurements Yearbook) were searched for peer-reviewed studies published between January 1997 and September 2013. We chose this date range because it coincides with the emergence of the articles that formed the basis for our multi-domain conceptualization of engagement. Various combinations of the following search terms were used: Adolescent, child, family engagement, intervention, family involvement, substance use, treatment, alliance, adherence, participation, measure, and scale. This systematic search strategy yielded 3645 results.

One author screened titles and abstracts of all electronic search results to determine initial study inclusion. If the title and abstract indicated the measurement discussed might meet inclusion criteria, the full article was obtained and examined to determine whether the inclusion criteria were met. Twenty-two relevant studies were included based on assessment of abstracts and titles for possible alignment with inclusion criteria. Additionally, reference lists from these 22 studies and topical literature reviews were examined, generating 36 additional studies that potentially met the criteria for this review.

The research team then reviewed the full-text of all 58 articles. Of those, only eight studies utilized measures that met the inclusion criteria for this review. The other fifty were excluded because they did not meet inclusion criteria; specifically, 27 studies did not include a measure of family engagement, 11 studies were not used in mental health treatment or substance use settings, 11 studies focused on client engagement rather than family member engagement, and one study utilized a measure that was designed to assess pre-treatment attitudes and behaviors.

Once a final list of measures was identified, data extraction tables were used to compile relevant information across the eight included measures. This allowed for coding of the instrument format and administration procedures; the populations and treatment settings in which the measure was used; whether behavioral, attitudinal, and affective domains were measured; and measurement reliability and validity (see Tables 1, 2). To determine the treatment engagement domains assessed by the measures and ensure consensus among raters (Sandelowski and Barroso 2003), the first two authors independently examined each of the items in the measure to determine item fit with the definitions of behavioral, attitudinal, or affective domains of engagement described above. The first two authors then met and discussed discrepancies until consensus was reached. Because the measures were assessed at the item level, the existing measure as it was reported may not have included a subscale or scoring mechanism for the domain.

Table 1 Characteristics of measures
Full size table
Table 2 Psychometric properties assessed in the original measurement reports for family treatment engagement measures
Full size table

Results

This review of family treatment engagement measures in substance use and mental health settings resulted in the identification of eight measures: the Credibility Expectancy Questionnaire-Parent Version (CEQ-P: Nock et al. 2007), the Therapeutic Alliance Scale for Caregivers and Parents (TASCP: Accurso et al. 2013), the Multisystemic Therapy-Caregiver Engagement (MST Engagement: Foster et al. 2009), the Child and Adolescent Level of Care System/The Child and Adolescent Service Intensity Instrument (CASII/CALOCUS: Fallon et al. 2006), the Parent Rating of Parent Involvement (Noser and Bickman 2000), the Family Engagement Questionnaire (FEQ: Kroll and Green 1997), the Parent Motivation Inventory (PMI: Nock and Photos 2006), and the Vanderbilt Therapeutic Alliance Scale-Revised (VTAS-R: Shelef et al. 2005). In this section, we summarize (a) the clinical populations and settings studied, (b) the domains of treatment engagement assessed, (c) the clinical utility of the measures, and (d) the psychometric properties.

Clinical Populations and Settings Studied

All of the measures were designed for the parents or caregivers of children. Three of the studies included children from early childhood through early adolescence (ages 2–13), four focused exclusively on adolescents (ages 12–18), and one included both children and adolescents (ages 6–18). Two measures were administered in a substance use setting (MST-CE, VTAS-R), six measures were administered in outpatient mental health settings (TASCP, CEQ-P, PMI, parent rating of involvement, CASII/CALOCUS, MST-CE), and one was administered in an inpatient mental health setting (FEQ). Reasons for treatment among study participants included substance use disorders (n = 2); oppositional, aggressive, and other disruptive behavioral problems (n = 4); mental health challenges (n = 1); and emotional disturbance (e.g., Major Depressive Disorder, PTSD; n = 1).

Domains of Engagement Assessed

Our first aim was to determine whether the identified measures assessed the three domains that were determined to be relevant to this review, specifically attitudinal, affective, and behavioral domains. The behavioral domain includes treatment participation and adherence to treatment protocols, efforts to communicate with the treatment provider, and actively supporting the client’s engagement in treatment (King et al. 2014; Randolph et al. 2009). The attitudinal domain includes beliefs about the treatment itself, the clients’ involvement in the treatment, and family members’ own roles in the treatment process (Staudt 2007; Yatchmenoff 2005). The affective domain includes family members’ experience and expression of emotion in relation to their involvement in the clients’ treatment (King et al. 2014).

Each measure was examined item by item to determine whether all or part of the measure assessed the domains (see Table 1). Four measures contained items assessing attitudinal engagement (FEQ, CASII/CALOCUS, PMI, VTAS-R), five measures contained items assessing affective engagement (TASCP, CASII/CALOCUS, FEQ, PMI, VTAS-R), and six measures contained items assessing behavioral engagement (MST-CE, parent rating of involvement, FEQ, CASII/CALOCUS, VTAS-R, TASCP). With regard to coverage across domains, three measures assessed only one domain (CEQ-P, MST-CE, parent rating of involvement), three measures assessed two domains (TASCP, FEQ, PMI), and two measures assessed all three domains (CASII/CALOCUS and VTAS-R). A number of additional concepts were covered that did not relate directly to treatment engagement. Examples include the bond between parent and therapist, agreement on treatment goals, and problem acceptance.

Clinical Utility

Our second aim was to assess the clinical utility of the measures. Clinical utility refers both to the readiness with which practitioners can use the scale in real world practice settings and the likelihood of the measure to produce information that guides clinical decision-making. All of the measures are relatively brief (2–24 items). For half of the measures, the clinician reports their observation of the family members’ treatment engagement after a treatment contact (MST Caregiver Engagement Scale, CASII/CALOCUS, FEQ, VTAS-R). The other half of the measures are self-report questionnaires (CEQ-P, TASC-P, parent rating of involvement, PMI). The TASC-P also has a parallel form that can be completed by the clinician. Most of the measures are able to be scored by hand with simple calculations. In clinical practice, being able to compare client scores against normative data and/or clinical cutoff scores is important for interpreting the meaning of scores and making treatment decisions. We were unable to find normative data or clinical cutoff scores for any of the included measures. The majority of the measures are formal instruments developed for research and clinical use (n = 7). The exception is the parent involvement measure (Noser and Bickman 2000), which was comprised of two items derived from a project-specific satisfaction scale. To determine the accessibility of the measures by substance use providers, two graduate students attempted to find the measures and locate information about ordering, scoring, and interpreting the measures using general Internet search engines. They were unable to locate information for any of the measures. However, when we contacted the article authors directly, they provided information about cost and scoring. Practitioners seeking to obtain the measures will need to contact the developers directly.

Psychometric Properties

Our third aim was to describe the psychometric properties of the included measures. The methods by which reliability was evaluated varied by measure. Reliability was assessed through intra-class coefficients (ICC), Cronbach’s alpha coefficient, test–retest scores, and inter-rater reliability (IRR) scores. Cronbach’s alpha coefficient was the most commonly used method to assess internal reliability among the included measures (n = 7). Cicchetti and Sparrow (1990) suggest that alpha coefficients below .70 have an unacceptable level of clinical significance; scores between .70 and .79 are fair; .80 and .89 are good; and an alpha coefficient of .90 and above indicates excellent clinical significance. Alpha coefficients reported for the included measures (CEQ-P, FEQ-P, TASCP, PMI, VTAS-R, MST Caregiver Engagement, Parental Involvement) ranged from .73 to .93 with the exception of the FEQ-P (α = .61–.66). However, it is important to note that lower alpha values are associated with fewer instrument items (Graham 2006), and the FEQ-P only consisted of three items, which may have impacted the subscale alpha coefficient. ICC scores were provided for three of the measures (TASCP, CASII/CALOCUS, MST Caregiver Engagement). Scores ranged from .75 to .89 indicating excellent agreement between raters (Cicchetti 1994). One measure (FEQ-P) had an IRR score of .733 indicating adequate but not excellent inter-rater agreement (Hallgren 2012; Nunnally and Bernstein 1978). Test–retest reliability scores were only available on three measures (CEQ-P, TASCP, PMI) and ranged from .76 to .82.

Across measures, there was wide variability in the validity of demonstrated by the measures. Four measures exhibited construct validity (CEQ-P, VTAS-R, FEQ-P, CASII/CALOCUS), three exhibited predictive validity (CEQ-P, VTAS-R, TASCP), while one was documented as not possessing predictive validity (PMI). Two measures demonstrated discriminant validity (TASCP, FEQ-P), two demonstrated criterion validity (FEQ-P, CASII/CALOCUS), and one measure each demonstrated content validity (MST Caregiver Engagement), convergent validity (TASCP), and concurrent validity (VTAS-R). Overall, the TASCP appears to be the most psychometrically sound measure of family engagement; however, this conclusion is based wholly on the extent and rigor of the psychometric testing conducted on the instrument.

Discussion

Theorists emphasize that there are multiple domains of treatment engagement that will influence the treatment process and client outcomes (Becker et al. 2013; King et al. 2014; Staudt 2007). Therefore, a full understanding of family treatment engagement in research and practice settings requires measures that capture these domains. This review sought to identify existing measures of family treatment engagement and to evaluate their ability to assess the attitudinal, affective, and behavioral domains of engagement. Further, we sought to determine the potential utility of these measures for assessing family engagement in adolescent substance use treatment. It is important to note that we did not set out to review measures of client treatment engagement broadly (e.g., Tetley et al. 2011). Rather, this study focused on measures of the engagement of family members and/or concerned significant others in the clients’ treatment. We identified eight measures from mental health and substance use settings that assessed one or more domains of family treatment engagement. The implications of this review for the conceptualization and measurement of family treatment engagement in adolescent substance-use treatment settings is discussed below.

Conceptual Coverage of Family Engagement Measures

We conducted an item-by-item review to determine whether the included measures assessed the attitudinal, affective, and behavioral domains of treatment engagement. Overall, we found considerable gaps in coverage of the three engagement domains. Our results indicate that only two of the measures, the CASII/CALOCUS and VTAS-R, have items that cover all three domains. Yet the CASSII/CALOCUS uses a single observational item [“Parent and/or primary care taker acceptance and engagement”(p. 23)] that encompasses a number of indicators that are consistent with all three domains of treatment engagement (Huffine et al. 2010). For example, the indicators for “optimal” engagement include actively engaging in positive relationships with staff, sensitivity to the child’s needs and strengths, sensitivity to the child’s problems and how the pertain to recovery, and active and enthusiastic participation in assessment and treatment (Huffine et al. 2010). A limitation of this approach is that the rating itself cannot distinguish which domains are problematic for the family member. The VTAS-R contains several items consistent with the domains; however, its primary purpose is to assess the parent-therapist alliance, and it does not have an existing subscale related to one of the domains. This is true for a majority of the measures, which may have had one or more items determined to be consistent with a domain, yet would not yield a scale score that practitioners can use to assess the domain. In fact, none of the measures could provide interpretable data about all three domains. It is important to note that this is not an inherent limitation of the measures themselves, which may not have been designed for this purpose. However, this lack of conceptual coverage is a significant limitation of the current state of family engagement measurement broadly, and highlights the need for measures of family treatment engagement that can assess the quality of an individual’s attitudinal, affective, and behavioral engagement.

Three additional findings from this review warrant attention in future treatment engagement research. The first is that most studies of family engagement do not distinguish between individual-based and family-based treatment approaches (Pinsof et al. 2008). This distinction is important because the nature of a family members’ treatment engagement, and therefore its measurement, will differ considerably depending on whether they are intended to be active participants in the treatment or not (Nock and Photos 2006). In fact, in some family-based approaches to treatment, a parent’s engagement in treatment may be better thought of as client engagement, rather than family engagement. A second finding is that the included measures implicitly represent engagement to be dyadic—between the family member and the clinician. However, one of the primary reasons stated for engaging family members in treatment is to mobilize them to help create and maintain change for the adolescent client (Hornberger and Smith 2011; Smith et al. 2004). This way of thinking about family member engagement is inherently triadic (between family member, client, and provider). This is particularly important for family members of adolescents in individual-based treatment, where the primary manifestations of family members’ treatment engagement in treatment (behavioral, attitudinal and affective) happens outside of the treatment setting as they seek to encourage and support the adolescent towards treatment goals (Liddle 1995). Some example items that reflect this triadic relationship are “to what extent have you been able to encourage your family member to meet his treatment goals?” (Behavioral), “how involved do you feel you should be in your family member’s substance-use treatment at this point?”(Attitudinal), and “how would you describe your level of enthusiasm about your family member’s participation in treatment?” (Affective). A related improvement to existing measures would be to include the adolescent’s report of family member engagement in treatment. The measures reviewed only elicited the perspective of the clinician and/or family member. Understanding the adolescent perspective may be a particularly effective measurement approach for individual-based treatments in which family members attend treatment less frequently.

Utility for Adolescent Substance Use Treatment

Several factors are likely to influence the utility of the included measures for adolescent substance use treatment. The first is that the majority of included measures were administered in mental health settings. As such, measures may need to be created or adapted to be applicable in substance-use settings. Further attention needs to be paid to articulating the treatment-related and contextual factors that differ in substance-use settings so that measures can be adapted and/or created for substance use–specific treatment. For example, some potential differences might relate to the length and course of treatment, the nature of parental involvement, and family member attitudes towards substance use versus mental health. Future research can examine these differences and incorporate them into family treatment engagement measures that assess and reflect the specific characteristics of substance-use treatment settings.

It is important to recognize that no single measure will fully capture the diversity of substance-use treatment settings and expected family engagement roles. Practitioners must consider whether the treatment is residential, outpatient, or intensive outpatient, as well as the nature of the treatment, and the role of the treatment provider. All of these factors will have bearing on the utility of a measure. Measures included in this review assessed a variety of family engagement–related attitudes, emotions, and behaviors. They also varied in their length and whether they were completed by the family member (e.g., PMI), the clinician (e.g., FEQ), or a trained observer (VTAS-R). All of the included measures shared several limitations for practice settings. First, to our knowledge, none of the included measures are publicly available outside of research databases. Practitioners wishing to use the measures must contact the developers/authors directly. Additionally, we noticed that few studies reported the scoring procedures for the measures. We were also unable to find normative data for the measures, which will make it difficult for practitioners to use the scores they obtain to guide clinical decisions. In the future, researchers should be cognizant of these limitations and ensure accessibility to practitioners and provide scoring procedures. Finally, there is a lack of validity exhibited by current measures of family treatment engagement. An unclear conceptualization of family treatment engagement makes confirmation of validity challenging. This highlights the need not only to develop measures but also conduct further psychometric evaluation of existing measures to determine their utility in measuring the described domains.

Implications for Substance Use Treatment and Research

Family engagement is critical to successful adolescent substance use treatment. It includes attitudinal, affective and behavioral domains and is inherently triadic (between family member, adolescent, and treatment provider). For providers, measures of family engagement may be most helpful when they are used to detect more nuanced signs of treatment disengagement that may be early warning signs for treatment dropout or poor adolescent treatment outcomes (Nock and Photos 2006). With this information, practitioners can address emerging engagement problems with families, thereby increasing treatment retention and improving adolescent outcomes. In addition, multi-domain measures will allow substance use agencies to evaluate the effectiveness of their engagement-enhancing efforts and demonstrate the value of these approaches to funders. The results of this review reveal that currently, there are few multi-domain measures of family engagement designed specifically for adolescent substance use treatment settings. Providers seeking to assess family members’ attitudinal, affective, and behavioral engagement will have to compromise and select a measure that assesses only one or two domains with questionable construct validity and no normative data to compare their scores against. They may also consider adapting and using the measures described in this review, or even developing their own measure. In any case, the selection of a family engagement measure should take into account factors such as whether the treatment is residential, outpatient, or intensive outpatient, as well as the nature of the treatment (individual-based vs. family-based) and the role of the treatment provider. It is important to note that no single measure will capture the diversity of substance-use treatment settings and family engagement expectations or roles.

This review also highlights several important implications for family engagement research. First, there is a need to develop measures that explicitly assess the attitudinal, affective, and behavioral domains of family engagement. Such measures are necessary for assessing the effects of engagement-enhancing interventions and policies. Further, measures that assess multiple domains of engagement are important for furthering research about which approaches influence which domains of parent engagement and the relative influence of these domains on adolescent treatment outcomes (Copello et al. 2005). Second, substance use researchers and providers should work together to articulate the contextual factors specific to engagement in substance use treatment settings and use these to guide the development or adaptation of family engagement measures. Third, future research should aim to develop normative data for measures and cut-off scores that indicate potential problems or predict treatment dropout. This information will allow providers to detect cases with emerging engagement problems and intervene before the client drops out or fails to benefit fully from treatment. An example of this approach is the Session Rating Scale (SRS), which is used to monitor therapeutic alliance at each treatment contact and has clinical cut-off scores that indicate when a client is at risk for poor outcomes or treatment dropout. (Duncan et al. 2003). Finally, there is a need for substance use researchers to develop a level of agreement about the best approaches for measuring family treatment engagement. The use of common measures will allow researchers to obtain strong psychometric information for these measures and will facilitate meaningful comparisons across engagement intervention studies.

Limitations

There are several limitations of this review that should be noted. The first relates to the conceptual ambiguity inherent in family treatment engagement research. Given the conceptual overlap between constructs such as treatment alliance, treatment involvement, treatment participation, adherence, and engagement, it is possible that other researchers may have pursued different conceptualizations of family treatment engagement and thereby may have ended with a different set of measures of the construct. On the other hand, our inclusion of all of these constructs increased the breadth of our search and is a strength of the study. Similarly, our assessment of the consistency of the measures with attitudinal, affective, and behavioral engagement domains was guided by our conceptualization of these domains. Researchers with other conceptualizations of engagement may draw different conclusions. Another limitation is that by focusing our search on mental health and substance use treatment settings, we may have excluded measures of family member engagement in non-treatment settings (e.g., schools) that may have been useful for substance use treatment. However, we concluded that because the nature, intensity, duration, and expectations of relationships in these settings are different than substance use treatment, the nature of family engagement and its measurement would be less applicable. We also limited the review to measures published in peer-reviewed articles. Therefore, our findings are susceptible to publication bias and should be interpreted in light of this limitation. It is possible that otherwise promising measures were unpublished due to factors such as the excessive lag between submission and publication, insignificant findings, or lack of resources to publish the study (Bartolucci and Hillegass 2010).