Introduction

Self-reported consumer satisfaction with mental health services is a commonly reported indicator of the quality of mental health services for managed care companies and for state mental health systems. The Substance Abuse and Mental Health Services Administration (SAMHSA) includes perception of care as one of the National Outcome Measures (NOMS) for state- and federally-funded mental health and substance use disorder programs (Brunk et al. 2000; CMHS 2006). Although research on treatment outcomes has not indicated strong correlation between youth or family satisfaction and clinical change over the course of treatment satisfaction with services may be valuable as an indicator of social processes such as treatment engagement (Garland et al. 2000a, b; Lambert et al. 1998; Turchik et al. 2010).

Over the past two decades in the field of children’s mental health, the roles of family and youth consumers in shaping the mental health service system have expanded. To a large extent, this expansion has been driven by a growing body of evidence showing that providing services with family-centered values is associated with positive outcomes for children (Hoagwood 2005; McKay et al. 2002; Epstein et al. 1998; Gopalan et al. 2010). More recently, children’s mental health services have included access to family and youth peer-advocates as adjunctive services. Advocates provide a mechanism for youth and families to seek information and support from their peers in mental health service settings.

In New York State, the Office of Mental Health (NYSOMH) has had a unique emphasis on credentialing and supporting peer family and youth advocacy. In 2002, the NYSOMH created a network of family support programs consisting of approximately 200 separate family support programs and approximately 400 professional family peer advocates (NYSOMH 2008). New York State is in the process of expanding the number of family and youth peer advocates, has implemented a manualized training curriculum to establish certification and professionalization of this workforce and is developing quality indicators for family support (Olin et al. 2010, 2013).

No studies were identified that examined the relationships between satisfaction with mental health services and access to a peer advocate. This study examines the associations between having access to an advocate (youth or family) and youth/or caregiver satisfaction with mental health services. We hypothesized that youth and caregivers with access to advocates while receiving mental health services would have more positive assessments of their services and have improved outcome in terms of satisfaction with functioning and social connectedness.

Methods

The study sample included youth and caregivers of youth who participated in a youth assessment of care satisfaction survey (YACS) (N = 768, response rate 68 %) or a family assessment of care satisfaction survey (FACS) (N = 1,231, response rate 44 %) in 2012. Surveys were implemented during a two-week period in the spring of 2012 with youth ages nine and older and/or caregivers of children of all ages who were in care and had been receiving care for their emotional or behavioral issues for more than 1 month in selected programs. Programs selected included both state and locally operated inpatient and outpatient mental health service agencies in NYS. Surveys were distributed by program staff at the point of service or were mailed to homes by the programs. Surveys were completed anonymously and returned to NYSOMH for processing.

The NYS youth (YACS) and family (FACS) satisfaction surveys were derived from the national satisfaction surveys with local input (YSS and YSS-F, CMHS 2006). The national surveys are the standard used by most state mental health authorities to fulfill reporting requirements. The YACS contains twenty-seven items related to satisfaction with seven sub-scales: appropriateness of care, cultural sensitivity, access, participation, outcomes/functioning, medication management and global satisfaction. The FACS contains thirty-four items related to satisfaction with the same seven sub-scales plus one additional sub-scale to assess caregiver social connectedness.

The medication management sub-scale was designed with local input and is not found in the national surveys. This sub-scale includes questions related to understanding medication side effects, having choices about taking medications, and perception of helpfulness of the medication. The NYSOMH surveys also contain a question on whether the youth or caregiver had access to a peer advocate during this service episode. This question is also not found on the national surveys. Additional service and demographic information on the youth receiving services is also collected on the surveys, including: persons in need of supervision (PINS) status, arrest status, school expulsion and school attendance, program length of stay and youth demographics (age group, gender, race, ethnicity, where youth lives). NYSOMH survey forms are available to the public on the world-wide-web: http://bi.omh.ny.gov/kids/.

Analysis

Dichotomous indicators of satisfaction items were constructed by collapsing the four point response scale into two points (positive = agree vs. negative = agree slightly, disagree slightly and disagree). Dichotomous domain scores were computed as positive if greater than or equal to 50 % of the items in the domain were positive. Satisfaction domains included: access to services, appropriateness of services, cultural sensitivity, global satisfaction, psychotropic medication comprehension, functioning outcomes, participation in services and social connectedness [Caregiver version only (FACS)]. Demographic and services variables not already structured as yes/no responses were dichotomized as follows: age as below or above 12 years, race as white versus non-white; living situation as at home with parents or relatives versus foster or residential program; days in school as greater versus same or less; length of time in the program as less or greater than 6 months; PINS as yes versus no/unsure. Access to a peer advocate was dichotomized as yes versus no or unsure.

Associations between having access to a peer advocate (yes/no) and satisfaction agreement on survey items and domains were analyzed using separate Chi square comparisons. Associations between having access to a peer advocate (yes/no) and demographic and service elements of the youth receiving behavioral health services were also compared. Significance tests were performed at the Pr < 0.05 level. Cronbach’s alpha reliability estimates for all sub-scales exceeded 0.65 indicating a high level of sub-scale reliability (Gliem and Gliem 2003). SAS version 9.2 was used to perform all quantitative analyses (SAS Institute 2002–2003). The NYSOMH institutional review board (IRB) deemed that this study did not require human subject review as part of ongoing program management and quality improvement undertaken by the NYSOMH.

Results

Nearly 75 % (N = 899/1,200) of caregivers indicated they had access to a family advocate while their child was in care. Only 33 % (N = 246/737) of youth indicated they had access to a youth advocate while they were receiving services. There were no age, gender, race or ethnicity differences with respect to caregiver report of having access to a family peer advocate. However, older youth (12 and older) and youth of color (non-white) were more likely to report access to a youth peer advocate (Table 1).

Table 1 Comparisons of demographic and devices characteristics of caregivers and youth by access to peer question advocates
Full size table

In terms of service and family related indicators, how long the youth was in the program, caregiver relationship to the child and where the child lived were not associated with having access to a peer advocate (Table 1). For caregivers, the indicator of arrest since receiving services was associated with having access to a peer advocate (Pr < 0.0063). For youth, greater attendance in school was inversely related to having access to a peer advocate (Pr < 0.0063). Caregivers with access to a peer advocate were more likely to have an identified diagnosis for their child (Pr < 0.0029) and to have been informed of therapy options for identified emotional health reasons (Pr < 0.0001). For youth, having an identified diagnosis was not associated with access to a peer advocate but being informed of therapy options was associated with having access: 92 % of youth with access to a peer advocate versus 82 % of youth without access reported that they had been informed of therapy options (Pr < 0.0004) (Table 1).

Having access to a family advocate was positively associated with both youth and caregiver perceptions of access to care, appropriateness of care, participation in services and global satisfaction with care. Although satisfaction with the cultural sensitivity of the program was rated highly positively from the perspectives of both youth (81 %) and caregivers (95 %) this was not associated with having access to peer advocates from either perspective (Table 2).

Table 2 Comparisons of caregiver and youth satisfaction domains by access to peer advocates
Full size table

Caregivers reported a low score (56 % positive, overall) on the social connectedness sub-scale. However, caregivers with access to family advocates were more likely to respond positively (60 %) compared to caregivers without access (46 %) (Pr < 0.0001) (Table 2). Caregivers with access to family advocates compared to caregivers without access were more likely to respond positively to each question within the domain: having more social activities 43 versus 32 % (Pr < 0.002), having someone who would help in a crisis 80 versus 64 % (Pr < 0.0001), having support needed from others 78 versus 65 % (Pr < 0.0001), better able to handle stress 44 versus 36 % (Pr = 0.0099), having better family relationships 57 versus 48 % (Pr < 0.0096) (data not shown).

Youth with access to a peer advocate were more likely to respond positively to the medication management sub-scale compared to those without access (OR 1.46; 95 % CI 1.06, 2.00). For caregivers, no association was found between having access to an advocate and responding positively to the medication management sub-scale. In terms of the functioning outcomes sub-scale, caregivers with access to peer advocates were more likely to report greater satisfaction (OR 1.47; 95 % CI 1.12, 1.92) whereas no associations were found for youth. (Table 2).

In order to better understand the differences in youth and caregiver perspectives on the medication and functioning sub-scales, patterns of association between access to a peer advocate and satisfaction at the item level were examined (Table 3). Youth with access to a peer advocate compared to youth without access reported significant differences in satisfaction for the following medication management items: medications were explained in an understandable way (79 vs. 68 %, Pr <. 0038), knowing what side-effects to watch for (70 vs. 59 %, Pr < 0.0078), having choices about taking medications (58 vs. 46 %, Pr < 0.0045) and feeling comfortable about taking medications (67 vs. 57 %, Pr < 0.0151). Overall, caregivers rated the set of medication management questions more positively compared with youth and no associations between having access to a peer advocate and positive report on any of the medication items were found from the caregiver perspective.

Table 3 Comparisons of caregiver and youth satisfaction with functioning outcomes and psychotropic medication comprehension by access to peer advocates
Full size table

In terms of satisfaction with functioning outcomes, youth tended to rate these items more positively than did caregivers (Table 3). Youth with access to a peer advocate compared to youth without access reported significant differences in satisfaction on being able to face challenges (60 vs. 52 %, Pr < 0.0325) and being able to make friends (72 vs. 63 %, Pr < 0.0158). Caregivers with access to a peer advocate compared to caregivers without access reported significant differences in terms of their youth being happier (49 vs. 41 %, Pr < 0.0234), being better able to make friends (51 vs. 42 %, Pr < 0.008) and being able to get along better as a family (53 vs. 43 %, Pr < 0.0022).

Discussion

This study found that the majority of caregivers and approximately one-third of youth who participated in the satisfaction surveys reported having access to a peer advocate. From the perspectives of both youth and caregivers, positive ratings on four of the core satisfaction sub-scales (appropriateness, access, participation, global) were associated with having access to peer advocates. These satisfaction domains tap core service constructs related to engagement in care such as getting services that were helpful, being able to get services when needed and being included as a partner in planning services. Wisdom et al. (2011) found that sharing communication between families and providers and coaching are key parts of peer advocacy service provision.

This study also found that having access to a peer advocate was positively associated with caregiver social connectedness. Previous research has indicated that common service provision content for peer advocates such as support/education, service coordination, liaison and direct services to children may be directly linked to this increased perception of social connectedness (Olin et al. 2010; Davis et al. 2011). It is also possible that access to peer advocates is a mediator of other change processes that occur for the individual caregiver and for their family unit which result in more positive perceptions of social connectedness such as having social activities, being better able to handle stress and having better family relationships. Previous work has also posited that family support services in children’s mental health may be understood as change agents within a conceptual framework of behavior change theory (Olin et al. 2010).

Having access to a youth peer advocate was positively associated with youth comprehension of their psychotropic medication choices. Choices about using psychotropic medications are complex. Previous work has indicated that improving shared decision making among youth, parents and practitioners enhances psychotropic medication adherence (Charach and Fernandez 2013). It is possible that youth with access to peer advocates encountered a means to have their medication questions addressed by discussing with the peer advocate or by being empowered through working with the advocate to address questions with practitioners themselves. The impact that youth peer advocates may have with respect to youth adherence to a recommended medication treatment regime is an important area for continued research.

Mental health satisfaction survey data are reported annually by State Mental Health Program Directors to the Center for Mental Health Statistics as part of the Mental Health and Substance Abuse National Outcomes Study (http://www.nri-inc.org). NYSOMH is unique in having added a question on state satisfaction surveys to measure family and youth access to peer advocates so that associations between having access and satisfaction can be better understood. In the future, NYSOMH plans to add additional questions related to access to family or youth peer support to these satisfaction surveys to better determine the extent to which services were utilized and the impact these had on perceptions of care.

Study Limitations

The cross-sectional nature of the survey limits our ability to determine if access to a peer advocate caused the more positive assessments noted. Access to an advocate was measured as a self-report ‘yes/no’ type of question so we are not able to determine the extent to which youth or caregivers actually worked with a peer advocate or with an advocate. In addition, the sample selected represents a convenience sample of youth and families who participate in mental health services in NYS so the findings may not be generalizable. The high rate of access to peer advocates found in this study may be due partially the inclusion of all Home and Community Based Medicaid Services Waiver programs in the sample which includes family peer advocacy as a billable Medicaid service component. The low response rate for the family survey (44 %) may introduce bias the nature of which cannot be determined.

Conclusions

This study adds to the growing evidence regarding the importance of peer advocates in terms of enhancing attitudes and expectancies about mental health services for caregivers and youth. The potentially important relationships peer advocates may play for caregivers perception of social connectedness and for youth perception of psychotropic medication comprehension and adherence will be explored in future studies.