Introduction

Parents play an important role in preventing youth violence and promoting positive youth development. Parents shape their child’s behavior with their parenting practices (e.g., monitoring, involvement, discipline styles) and in their relationship and communication with their children. Inconsistent, harsh, and/or coercive parenting styles have long been associated with increased aggression in children (Deater-Deckhard et al. 1996; Dodge et al. 1994; Patterson et al. 1992). Adolescents with low parental monitoring display more delinquent and aggressive behavior (Griffin et al. 2000; Loeber and Stouthamer-Loeber 1987) and select more deviant peer networks (Dishion et al. 1991). In contrast, a positive caregiver-child relationship characterized by parental warmth, cohesion, support, and closeness can protect youth and reduce their engagement in aggressive behavior (Andreas and Watson 2009; Gorman-Smith et al. 2000). Caregiver involvement with children increases adolescent prosocial activities, which are, in turn, associated with decreased externalizing behaviors (Chen et al. 2000; Pursell et al. 2008; Zhou et al. 2002). Thus, teaching caregivers to increase monitoring and involvement, develop specific parenting practices, and improve communication with their adolescent is one method of reducing youth violence and other risk behaviors.

This article highlights successes and challenges associated with engagement in family-based parent training programs in urban and rural settings. We begin with a discussion of the literature on parent engagement, as well as structural, attitudinal, and interpersonal barriers specific to recruitment and retention in parenting interventions. Examples of implementation strategies used in two Centers for Disease Control and Prevention (CDC) funded projects are analyzed and lessons learned provided. Specific examples related to urban and rural implementation of the interventions are highlighted. This discussion is a synthesis of more than five years of implementation of parenting programs in two major urban and rural initiatives for youth violence prevention. Quantitative data-based program evaluations of these programs are available elsewhere (CITATAIONS). This article is informed by quantitative and qualitative data from participating families, from parenting group feedback, from group facilitator reports, and from the experiences of scholars doing community-based participatory research. The authors embarked on a series of in depth conference calls between implementation sites to share experiences and synthesize common and divergent themes. The following discussion is an articulation of lessons we learned concerning recruitment, retention, and implementation of parenting programs.

Effectiveness of Family-Based Parent Training Programs

Research has demonstrated the effectiveness of family-based parent training programs for reducing adolescent risk behaviors and promoting adolescent health (Forehand et al. 2007; Haggerty et al. 2007; Murry et al. 2007; Prado et al. 2006; Smokowski and Bacallao 2010). Reviews suggest that family-based prevention programs can lower aggression by reducing risk factors and promoting protective factors for the adolescent and parent (Kumpfer and Alvarado 2003; Lochman 2000). Family-based parent training programs also contribute positively to broader adaptive behavior and the development of social and emotional competencies in youth (Lochman 2000; Spoth et al. 2003). For example, positive changes associated with such programs include increased caregiver responsiveness and organization and decreased family conflict.

Family-based parent training programs have been identified as an effective approach for preventing child abuse and neglect (Fortson et al. 2016), youth violence (David-Ferdon et al. 2016), and violence against children globally (World Health Organization 2016). Parent training works to prevent violence, as it interrupts the coercive cycle of escalating negative parent and child behaviors (Patterson 1982; Fagan and Catalano 2013). Parent training interventions usually utilize behavioral modification and/or relationship enhancement to disrupt the coercive family cycle (Forehand et al. 2014). Using these approaches, parents learn appropriate ways of attending to their child’s negative behaviors and the child learns ways to gain positive parental attention, thereby strengthening the parent–child relationship and improving family interactions.

Results of meta-analyses indicate that, in general, parent training interventions have moderate to strong effects (Kaminski et al. 2008; Lundahl et al. 2006; Sandler et al. 2011). A meta-analysis by Lundahl et al. (2006) suggested that many of the positive effects of parent training interventions found at post-test were maintained at follow-up, although the effect sizes were smaller than at post-test. Major weaknesses of parent training evaluation studies include a lack of longer-term follow-up assessments beyond immediate post-test (Kaminski et al. 2008), as well as a lack of comparison groups at follow-up (Lundahl et al. 2006).

Parent Engagement in Family-Based Programs

There is difficulty in engaging parents and comparison groups to participate in parent intervention studies. Parent engagement has long been a focus of family-based programs. Research has consistently demonstrated an association between engagement and behavioral outcomes (Connell et al. 2007; Stormshak et al. 2009). Yet, engaging parents in prevention programs is not easy. Recruitment and retention are two elements of engagement that are often the focus in family-based parent training programs (Moran et al. 2004). Recruitment refers to identifying and successfully enrolling families in programs, while retention refers to the sustained attendance and participation of parents in programs (Moran et al. 2004). Common parental barriers to recruitment and retention in parenting programs include difficulties in accommodating work schedules, finding transportation, securing childcare, and overcoming language barriers (Finigan-Carr et al. 2014; Spoth and Redmond 2000). Attendance has often been used as a marker of retention (Kazdin 1996; Mendez et al. 2009); however, level of participation, satisfaction with the intervention, and understanding of the material are increasingly used as markers of retention and engagement (Bamberger et al. 2014; Byrnes et al. 2010). Further, researchers have recognized that what influences families’ recruitment in programs may not be the same factors that motivate them to continue participation in a program (Prado et al. 2006). Engagement is likely to change across the duration of family-based programs based on variables, such as demographics and family functioning, which interact with life circumstances and serve as barriers or enhancers to parent retention (Coatsworth et al. 2017). Unfortunately, engagement in parenting interventions remains a challenge. Recruitment rates for family-based prevention programs range from 3% to 35% (Chacko et al. 2016; Meek et al. 2004). Even when parents are successfully recruited, many family-based programs have a high dropout rate. In a review of 262 studies of engagement in behavioral parent training programs, Chacko et al. (2016) found that 26% of families drop out before completing treatment. Retention of parents is important because greater attendance is associated with better youth and family outcomes (Baydar et al. 2003; Pantin et al. 2003; Spoth et al. 1999). For example, members of the Multisite Violence Prevention Project team (Quinn et al. 2010) found that the impact of the selective family intervention that was part of the universal school-based program showed positive benefits on aggression and family processes for those most likely to participate in the intervention. Some research suggests, however, that parents of high-risk youth may be more likely than other caregivers to inconsistently attend family-based sessions or drop out altogether (Brody et al. 2006).

In trying to codify recruitment and retention strategies, Kazdin’s (1996) barriers-to-treatment model suggests that barriers to treatment can be structural, attitudinal, and/or interpersonal. Structural barriers include instrumental difficulties such as scheduling appointments, finding transportation, and acquiring appropriate childcare. Attitudinal barriers encompass beliefs about services and providers that may impact engagement, such as perceptions of the relevance and demandingness of the intervention. Finally, interpersonal barriers include poor relationships with providers, which may damage families’ participation in services.

Predictors of Engagement in Family-Based Programs

Early studies examining predictors of engagement in family-based programs focused on sociodemographic factors such as race and ethnicity, socioeconomic status, parental age, and parent education (e.g., Spoth and Redmond 2000). Results were mixed, making it difficult to draw conclusions about the influence of such factors. For instance, Frankel and Simmons (1992) found no evidence that marital status, educational level, or parent age predicted engagement in a sample of parents seeking behavioral parent training. Similar non-significant results were found by Dumas et al. (2007), as parent engagement was not associated with ethnicity and socioeconomic status. In contrast, Coatsworth et al. (2006) found that non-engagement was associated with a greater number of individuals residing in the participants’ homes, a lower household income, and being African American.

A broad body of literature also has focused on the influence of parent and child factors on engagement (e.g., Kazdin 1996; Kazdin et al. 1997; Kazdin and Wassell 2000; MacNaughton and Rodrigue 2001). For instance, MacNaughton and Rodrigue (2001) found that the number of barriers parents reported, which most commonly pertained to access problems or negative attitudes and beliefs, predicted overall compliance with treatment recommendations. Notably, quantity of barriers was a significant predictor of adherence, whereas type of barrier was not, suggesting a cumulative rather than qualitative effect on service utilization. Kazdin et al. (1997) reported that parent perceptions of intervention participation difficulties, including stressors and obstacles associated with treatment, perceptions that treatment is not very relevant, and a poor relationship with the provider, influenced who dropped out of a family intervention for child antisocial behavior. Kazdin and Wassell (2000) found that parent improvements in an intervention were predicted by fewer perceived barriers with the intervention. In a sample of urban families, Finigan-Carr et al. (2014) found that some families failed to enroll in a parenting program because they did not view their child as having a problem that needed to be addressed, despite the preventive focus of the intervention. Conversely, parents who engage in prevention programs (versus those who have low engagement) have children with higher levels of externalizing problems (Gorman-Smith et al. 2002). Parenting skills and behaviors prior to initiation of family-based programs also appear to influence engagement. Parents who reported high levels of monitoring were less actively engaged in one study (Gorman-Smith et al. 2002), while other research has found that the presence of positive parenting is related to higher engagement (Lefever et al. 2013; Schoenfelder et al. 2013). In other research, engagement has been linked to both positive (e.g., low baseline levels of parenting avoidance) and negative (e.g., higher baseline levels of perceived negative affective quality in the parent-youth relationship) parenting behaviors (Coatsworth et al. 2017).

Other studies have examined the influence of family variables on engagement, with particular emphasis on family tension/conflict and family organization (e.g., Coatsworth et al. 2006; Bamberger et al. 2014; Prado et al. 2006). For example, Bamberger et al. (2014) found that chronic family tension was associated with lower baseline levels of engagement but not change in engagement over time. Further, chronic tension moderated the relation between session-specific tension and engagement, thereby highlighting the importance of ongoing family dynamics, as they may influence how families experience time-limited stressors or barriers to treatment.

In their 2006 study, Coatsworth et al. found that family-level factors best predicted engagement, as dropouts reported lower family organization and more barriers to service utilization. Similarly, Perrino et al. (2001) examined family systems variables as predictors of treatment engagement and found that family order and family communication/shared views significantly predicted initial engagement, with greater organization and better communication related to higher likelihood of engagement.

Factors associated with program implementation also have been examined as predictors of participant engagement. For instance, referral source has been considered as a predictor of engagement (e.g., Breland-Noble et al. 2012). Prinz and Miller (1994) found that engagement of families was decreased (i.e., families were more likely to drop out) when referrals were made through a community-based agency rather than a school. Implementation fidelity also has received attention as a predictor of engagement, as poor implementation may undermine efforts to engage participants in interventions (Smith et al. 2013). Findings regarding the relative importance of program fidelity for parent engagement are mixed. Byrnes et al. (2010) explored the relation between fidelity and engagement and found that fidelity was unrelated to engagement. In contrast, Smith et al. (2013) found that improved fidelity was related to better parent engagement.

Efforts to Improve Engagement in Family-Based Programs

A number of efforts have been made to improve engagement in family-based parent training programs, including collaboration amongst service providers (e.g., McKay and Bannon 2004) and an emphasis on family systems dynamics (e.g., Cannon and Levy 2008; Ingoldsby 2010). Collaboration between providers has been widely promoted to enhance engagement, as families and youth benefit from coordination of care and a network of supports. Becker et al. (2015) conducted a study of engagement in school mental health services, and focus group participants frequently recommended that providers work together to facilitate goal setting and communication. In their review of family and youth engagement efforts, McKay and Bannon (2004) highlighted the importance of case managers and facilitators for families pursuing youth mental health services. These professionals often provided emotional and informational support, helping families better navigate the system, resulting in longer treatment engagement.

In a review of 17 randomized controlled trials (RCTs) examining family engagement and retention in child mental health programs, Ingoldsby (2010) identified four practices that were helpful in engagement and retention: brief early-treatment engagement discussions, motivational interviewing, use of a family systems approach, and enhanced family stress and coping support strategies. Similarly, interventions that incorporated motivational interviewing strategies to reconcile discrepancies between families’ goals and their engagement level were generally effective in enhancing participant engagement. This finding has been replicated with mothers of middle and high school youth who participated in a multi-systemic intervention aimed at improving youth functioning (Sterrett et al. 2010).

Ingoldsby (2010) determined that engagement was further enhanced when families’ barriers (structural and psychological) to service utilization were addressed multiple times throughout treatment by eliciting examples of obstacles they encountered and working jointly to generate solutions. Further, across studies, there was an emphasis on personalized care as families responded best when treated as unique entities by providers. Although Ingoldsby’s (2010) review was helpful in identifying useful engagement strategies, engagement had a very limited definition and only included participation and attendance, overlooking other dimensions such as satisfaction or material comprehension. In addition, in most of the research focused on engagement in treatment, as opposed to prevention programs, there was a lack of attention to ethnic minority populations, and the research did not examine family retention over longer periods of time. Further, the geographic location of the 17 studies reviewed by Ingoldsby varied greatly (e.g., Canada, Germany, Australia, urban US clinic, and one rural US school district). Much of the past research on family recruitment and retention has been limited to families in urban areas, thereby limiting our understanding of rural dynamics. The current article addresses this discrepancy by discussing recruitment and retention in both urban and rural areas.

In an effort to examine domains of engagement separately, Becker et al. (2015) reviewed 40 RCTs to examine how different engagement practices related to attendance, adherence, and cognitive preparation. Becker et al. found that effective interventions targeting attendance often incorporated an evaluation of barriers to treatment; those aimed at improving adherence frequently included homework assignments; and interventions focused on cognitive preparation involved expectation setting and modeling. These findings are particularly important as they provide insight into potentially efficient ways to enhance engagement across participants enrolled in universal prevention efforts, as well as a means to improve specific dimensions of engagement that may be lacking in selective interventions.

Summary

The extant research highlights many challenges with implementation of parent training programs and suggests that the barriers to recruitment, retention, and engagement can be categorized as structural (e.g., number of individuals in the home, lower income, African American race, referral source), attitudinal (e.g., perceptions of intervention participation as not valuable, views of child issues and difficulties, opinions of implementation quality), and/or interpersonal (e.g., parental skills, positive/negative parenting behaviors, family tension/conflict). In the sections that follow, we will use these categories to highlight the specific successes and challenges in implementing parent training programs in two CDC-funded youth violence prevention research centers (YVPCs). Unlike past research that has been heavily weighted towards urban experiences, we balance our discussion between urban and rural implementation sites. The parent training programs were implemented in a disadvantaged, rural county in southeastern North Carolina and in a moderate sized urban area in Richmond, Virginia.

Rural and Urban Experiences Implementing Evidence-Based Family Programs

Through a five-year cooperative agreement with CDC’s National Center for Injury Prevention and Control, the University of North Carolina YVPC (NC-YVPC) and Virginia Commonwealth University Clark-Hill Institute for Positive Youth Development (VCU Clark-Hill) worked to reduce youth violence in two high-risk communities. Each site implemented and evaluated parent-training programs in their communities as a strategy for preventing youth violence. Each site implemented a different suite of programs; however, sites included at least one of three parenting programs (Parenting Wisely [PW], Family Check-Up [FCU], or Staying Connected with Your Teen® [SCT]) as an essential component of their work. Table 1 provides an overview of the three parenting programs, including the program content, population, and targeted outcomes. While quantitative data were collected during the three years of program recruitment and implementation (see e.g., Corona et al. 2009; Cotter et al. 2013, 2018), the purpose of this manuscript is to highlight qualitatively the barriers and lessons learned across sites.

Table 1 Overview of the Family Interventions Delivered Across Sites
Full size table

NC-YVPC Experiences

Rural implementation occurred in southeastern North Carolina from 2010 to 2015. Implementing a parent training program in a rural community presents unique challenges. Recruiting and retaining parents is usually difficult, particularly in prevention programs, and often results with fewer than half of eligible families participating (Hooven et al. 2011; Ingoldsby 2010). Rural families have limited access to transportation and other resources and often have to travel long distances for services. These challenges are exacerbated in Robeson County, NC, which is one of the most socioeconomically disadvantaged and ethnically diverse rural counties in the nation (median income $30,608 vs. $46,868 for North Carolina; United States [US] Census Bureau 2016a). Close to 26% of the 133,235 residents of Robeson County are under 18 years of age (2016b), and 41.9% lived in poverty in 2015 (US Census Bureau 2016c). The area is made up of approximately equal proportions of African Americans, Caucasians, and Native Americans, with a growing Latino population, and a violent crime rate (VCR) of 859 per 100,000 in 2014 (about 2.5 times the VCR for the state; 333 per 100,000; North Carolina State Bureau of Investigations 2015).

The NC-YVPC offered a flexible and interactive version of PW. In addition to the traditional online format, the NC-YVPC team offered PW in a variety of delivery formats. Groups included the following: (1) parents only 5-week group; (2) parent and adolescent 5-week group; (3) parent online group; (4) parent and adolescent online group; and (5) parents only intensive workshop (Cotter et al. 2013). Anecdotally, we noted that parents were much more interested in participating in the interactive groups, with or without their adolescent children present, than in completing the program online. Through partnerships with community organizations, 23 agencies collaborated with the NC-YVPC to facilitate implementation of PW in a variety of locations throughout the county. The program was offered in the Department of Social Services, Boys’ and Girls’ Clubs, schools, churches, housing authorities, and in local non-profits across the 924 square miles of Robeson County. PW staff established partnerships with: Housing Authorities, the Lumbee Tribe, 15 elementary and middle schools, Parents’ Association of Robeson County, Mental Health Association of Robeson County, 4-H Extension Clubs, Easter Seals-Family Services, the Healing Lodge for Interfaith Ministries, and many individual churches. Parents were recruited with flyers, radio and television interview advertisements, by ministers in churches, and through word of mouth. PW staff worked with school leadership teams, coaches, guidance counselors, community center managers, and Teen Court coordinators. Rather than having strict eligibility criteria, any parents who had concerns about adolescent behavior management were invited to attend PW. This high level of community buy-in, coupled with the variety of delivery formats and interactive program content, contributed to the success of the program. Program sessions were offered during weekends and in the evenings to provide convenient locations and scheduling. This flexibility allowed parents with transportation and/or time limitations to participate in the program. In most instances, PW was offered at a community center within walking distance during various times when the parent was not working.

As noted in the literature cited previously, referral source can be a barrier for parental recruitment, engagement, and retention in parenting programs (Breland-Noble et al. 2012). In the work of the NC-YVPC, parents of adolescents referred from juvenile courts were particularly hard to reach and even harder to retain because of family instability and risk. Families referred from Child Protective Services were mandated to participate but brought attitudinal barriers (e.g., questioned relevance of services) to group participation. NC-YVPC engaged an array of different agency partners to maximize referral sources and convenience in location for implementation. This collaboration between providers and the project team enhanced engagement, as families and youth benefitted from coordination of care and a network of supports, reinforcing previous research (McKay and Bannon 2004).

Over four years of implementation, 367 parents were served across the different implementation delivery groups. The sample was exceptionally racially diverse and 46% identified as American Indian, 36% as African American, 6% as Hispanic/Latino, 7% as White, and 4% as mixed race. Parents remained invested throughout the program and more than 99% of the recruited participants (364 out of 367) finished the program. In many cases, the NC-YVPC team was able to overcome significant attitudinal barriers and highlighted to parents the value and relevance of the program content, especially when parents wished to modify child behaviors. Relative to a no-intervention comparison group, parent participation in PW sessions was associated with improvements in family problem solving, family roles, family involvement, parenting self-efficacy, parenting sense of competence, and decreased adolescent violent behavior (Cotter et al. 2013). Effect sizes varied by delivery format, with larger program effect sizes (i.e., 0.85) when parents participated in weekly action-oriented groups (compared to the intensive workshop (Cotter et al. 2013).

VCU Clark-Hill Experiences

Urban implementation occurred in Richmond, VA from 2010 to 2015. In 2010, Richmond had an estimated population of 207,878, including 34,556 youth aged 10 to 21. Richmond’s overall population is 49% African American and 6% Latino (Statistical Atlas 2017). Richmond has several notable economic, social, and political forces that impact youth development and violence prevention efforts. A large percentage (48%) of youth 12- to 24-years-old in Richmond live in poverty; this rate is three times the state average (American Community Survey 2008). Violence, particularly among youth, is a serious problem in Richmond. Between 1999 and 2006, 92% of all intentional injury deaths were among youth between 15 and 24, and homicide was the leading cause of death for this age group. From 1999 to 2006, the homicide rate among 15- to 24-year-olds in Richmond ranged from more than five times to nearly nine times the national average (WISQARS 2010).

VCU Clark-Hill implemented three different parent training programs: (a) FCU+SCT; (b) SCT only; and (c) PW only. VCU Clark-Hill implemented parent training intervention components that initially included two evidence-based interventions (FCU and SCT) for youth in grades 6-8 who had been referred by school personnel for truancy concerns. Families were first invited to complete the Family Check-Up (FCU), a three-session strengths-based family assessment designed to motivate caregivers to improve their parenting behaviors and engage in family-based services (Dishion and Kavanagh 2003). Once they completed the FCU, families were invited to participate in the self-directed Staying Connected with Your Teen (SCT) program.

Engaging families of high-risk youth (i.e., youth referred because of truancy issues) proved to be challenging for a variety of reasons (e.g., referral source difficulties, attitudinal barriers). As noted previously, these challenges were not unique to this project but are consistent with the literature, which suggests that barriers can arise based on referral source and questions about the relevance of the program. Although a rigorous evaluation was underway, the VCU Clark-Hill study team recognized the problem early in the process and implemented solutions to address these barriers. The team decided to eliminate the FCU component to reduce the total number of family intervention sessions and discontinued the requirement that only families of youth referred for truancy were eligible to participate (i.e., families could now self-refer for services). If the previously established recruitment protocol was followed, few parents would have been successfully recruited and retained in the programming. This change occurred after the first few months of recruitment challenges and led to the later success of the programming. When the eligibility criteria were expanded, the VCU Clark-Hill team noted that 18% of the families referred were Spanish-speaking parents. To serve these families, the VCU Clark-Hill team implemented the Spanish version of PW, which covers many of the same topics as SCT but is not available in Spanish. In addition, the VCU team began implementing SCT and PW in a group setting at schools and in the community, including in a local social services office and an apartment neighborhood’s community building. A family consultant was assigned to families who preferred the self-directed format but required additional in-person support for completing sessions.

Over the four implementation years, 549 youth were referred for the family intervention (either SCT or PW), including 101 Spanish-speaking families. Reasons for referrals included the following: (a) 17% for truancy, (b) 24% for discipline-related concerns, (c) 7% by school staff for other concerns, (d) <1% through community outreach, and (e) 51% through self-referral at school-wide events. Before the VCU Clark-Hill team began accepting self-referrals, the rate of ineligibility was 24%. After including self-referred families, participation increased. Of the 549 youth referred, 145 of the families participated in the family intervention, including 53 Spanish-speaking families.

Of the 145 youth, 50% were male and 50% were female with an age range of 11-16 years (M = 12.6). The sample had a high degree of transience with some youth changing schools (36%) and/or moving residences (46%) at least 1–2 times in the past year. Sixteen percent of youth had taken medication for emotional or behavioral problems at some point in their lives, 27% of caregivers had sought counseling for the child, and 21% of caregivers had participated in a parenting program or workshop before. Most caregivers who participated were female (87%) and the biological mother of the referred child (77%). Less than half of participating families (43%) completed all sessions, but all families completed at least one session. Although the completion rate is low, it is consistent with prior research with urban families, which documented retention rates between 30% and 65% (Finigan-Carr et al. 2014; Small et al. 2015).

At the VCU Clark-Hill site, 25 families completed SCT (9 in individual sessions and 16 in groups) and 38 completed PW (27 in individual sessions and 11 in groups). Eighty-three families (57%) partially completed the programs (i.e., completed at least one session but did not finish), including 68 SCT (42 in individual sessions and 26 in groups) and 15 PW (9 in individual sessions and 6 in groups). For Spanish-speaking families who participated in PW, 9% dropped out after the first program meeting and 72% of families completed all sessions. It took families who completed the SCT program an average of 123 days to finish, while families participating in PW completed the intervention in an average of 45 days. Barriers to recruitment, engagement, and retention identified at the NC-YVPC and VCU Clark-Hill sites are discussed below, as well as strategies used to address the barriers, when implemented.

Addressing Engagement Barriers Common to Rural and Urban Implementation

As noted above, Kazdin’s (1996) barriers-to-treatment model suggests that barriers to treatment can be structural, attitudinal, and/or interpersonal. This model is used to frame the discussion of structural, attitudinal, and interpersonal barriers to engagement in family interventions encountered in the NC-YVPC and VCU Clark-Hill sites (see Table 2).

Table 2 Engagement barriers common to rural and urban implementation of family interventions
Full size table

Structural barriers include instrumental difficulties in scheduling appointments, finding transportation, identifying appropriate childcare, accommodating work schedules, and addressing language barriers (Spoth and Redmond 2000). Both the NC-YVPC rural site and the VCU Clark-Hill urban site addressed structural barriers (Table 2). For example, both sites addressed language access barriers by hiring bilingual staff to implement PW with Spanish-speaking families. The VCU Clark-Hill site also maximized convenience for families by offering to implement SCT and PW in participants’ homes and apartment residences using DVD discs and program workbooks. The team also purchased DVD players for families who may not have had any other way to play the DVDs in their home.

Although PW can be implemented using the Internet instead of DVDs, the Internet option (at the time of program implementation) was a significant barrier in the widespread rural environment NC-YVPC staff aimed to cover. Poor cellular coverage at the time also limited the use of smartphones and mobile broadband; thus, NC-YVPC staff used the PW DVD discs, rather than streaming the latest PW version over the Internet. Moreover, the team brought parents together in housing authority community centers, Boys and Girls Clubs, churches, and other accessible locations. Transportation issues, which did not exist for VCU Clark-Hill’s home-based implementation, were addressed by NC-YVPC staff, as parents were picked up and brought to group sessions. It was advantageous to organize participant car pools, as parents would begin supportive interactions on their way to groups, and driving together exerted positive peer pressure for drivers and passengers to attend the meeting.

NC-YVPC and VCU Clark-Hill staff addressed convenience of scheduling and parent time management concerns (i.e., “I just don’t have the time”) by offering the parent training programs in a variety of different formats. In the rural setting, NC-YVPC parents could choose from five formats (outlined earlier). The least popular option from among the optional formats was the online parent only program, which is the intended format of the program to maximize convenience for parents. Completing the program online with the adolescent helped to decrease parents’ difficulties with the technology; at times, adolescents would even remind parents of program content during stressful situations at home.

Although the variations in PW delivery format in the NC-YVPC site were included to maximize recruitment and retention and increase convenience for participants, the different formats also were evaluated, thereby further contributing to implementation science. In initial pre- and post-test analyses, group delivery in person with parents and adolescents together was the most effective implementation format; however, effects did not hold for the 6-month follow-up, as outcomes associated with delivery format did not differ, except for the brief workshop format, which was less effective compared to other formats (Cotter et al. 2018). This pattern of effects suggests that program delivery can be flexible to emphasize participant convenience without sacrificing long-term program effectiveness.

The VCU Clark-Hill site also offered the parenting programs in different formats to facilitate engagement. Specifically, parents could participate in the self-directed SCT or PW programs with telephone support or with in-home support from a family consultant or they could participate in group formats of either program. While the VCU Clark-Hill site did not conduct an evaluation of each format, the research team reviewed implementation records and noted that very few families chose to participate in the self-directed option with telephone support. Even when families chose that option initially, they frequently required in-home support to continue engagement in the sessions. Of note, however, is the fact that this support did not increase retention and engagement above and beyond that observed in prior research but rates of retention remain comparable to rates observed in other research with urban families (Finigan-Carr et al. 2014; Small et al. 2015). Although the VCU Clark-Hill site had family consultants available to implement the programs individually in families’ homes, the family consultants reported encountering scheduling difficulties. The family consultants and research team noted that many families were transient, with frequent address changes and disconnected phones, while the youth in the families experienced frequent changes in school placements. Family consultants reported numerous missed appointments and instances where they would find the family in too much distress to be able to focus on session material. As an example, one mother was experiencing suicidal ideation and having trouble finding mental health care in her community without insurance. She could not focus on intervention material under her current stress. Unfortunately, the communities within which families resided often lacked resources to meet the families’ needs, and families sometimes expressed distrust of the services that were available (an attitudinal barrier). In cases where resources were available, families were provided with information.

At the VCU Clark-Hill site, as noted previously, it took families who completed the SCT program an average of 123 days to finish, whereas parents receiving the PW intervention were finished in about 45 days. More parents were retained in the PW program, which had less content and was shorter in duration, than the SCT program. It is unclear whether the difference noted in past research, which suggested that African Americans may be less likely to complete parent training programs, (Coatsworth et al. 2006), was an issue in the current study, as participants were not randomly assigned to the two parenting programs.

Both rural (NC-YVPC) and urban (VCU Clark-Hill) sites served dinner or snacks to engage participants. Rural participants also received $10 compensation for each completed assessment package (pre-test, post-test, 6-month follow-up), as well as $40 for completing all sessions of PW. Urban participants who attended group sessions were eligible to win raffle prizes (e.g., gift cards) over the course of the intervention (Table 2). While incentives are often important in engaging families in prevention programs, they may not always be possible when these programs are offered through community organizations outside of the scope of a research project. Accordingly, the VCU Clark-Hill team decided on a lower cost incentive (i.e., raffles) that may be more likely to translate into community uptake of the intervention. The lower retention rate may be attributable to the lower incentives at the VCU Clark-Hill site; the lower incentives may not have been enough to motivate participants. Moreover, raffles, which were used as the incentive, are intermittent, rather than consistent, reinforcement because individuals do not always win. If this less intense reinforcement mechanism is used, the size of the prize may need to be modified (e.g., participants might come for $100 prizes, but not for $5 prizes). Other research suggests that the use of incentives does not lead to greater enrollment, attendance, or active participation in parenting group sessions (Dumas et al. 2010). Indeed, NC-YVPC staff did not believe that the incentives influenced program attendance as much as completion of lengthy research measures. Instead, these staff members thought that the food for the groups was more important because it facilitated informal conversation and bonding over dinner. During dinners with parents and adolescents, parenting across families and positive social support were evident as participants shared their experiences. This group processes complemented the actual program curriculum. For community agencies, providing this type of weekly dinner can be a low cost pot luck where everyone contributes something small to share. Thus, the use of incentives, the amount and their frequency will need to be considered in future parent training work. For both sites, missed sessions were made up with a staff member outside of the group and/or by telephone. As a package, these implementation accommodations decreased structural barriers to recruitment and retention.

Across the two sites discussed herein, a number of structural barriers were identified; however, as noted based on the strategies described, these barriers can be overcome with flexible delivery options and locations, staff effort and attention (e.g., picking up participants, doing home visits for make-up sessions), and program resources (providing food, raffle prizes, incentives when possible). Engaging families in program selection may also help facilitate engagement. These strategies, which may be helpful in increasing recruitment, engagement, and retention, should be implemented at the beginning of a project to ensure consistent implementation of programs and limit effects on evaluation protocols.

Attitudinal barriers include beliefs about services that may impact engagement, such as perceptions about the demands of the program and that the program may not be relevant to the child’s or family’s problems. The VCU Clark-Hill site had limited success in implementing SCT as a self-directed program and in the group format. The research team and family consultants thought that this may be related to the program’s perceived relevance to the families invited to participate, an important factor identified in previous research (Kazdin et al. 1997). For example, the family consultants observed that families referred to the program required a higher level of support than initially expected; this was especially true of families whose youth were initially referred because of truancy and/or disciplinary problems at school. Specifically, family consultants noted that families referred for the SCT faced multiple stressors such as transient housing, joblessness, food insecurity, and community and domestic violence. It was not always easy for parents to focus primarily on preventing their child from skipping school or having troubles in school given everything else parents were facing. This finding harkens back to the importance of the number of barriers that families face, as discussed previously. In addition, SCT follows a specific sequence with the theoretical foundations presented in the beginning and problem solving addressed in the later sessions. Implementation records show that some families (77%) who seemed initially enthusiastic about the program quit after the first or second session. Family consultants reported that many had pressing problems with their children that required immediate attention and may have felt that these needs were not being addressed by the intervention. The research team and family consultants thought that the initial sessions on theoretical content, while important to program designers, may lack salience to families seeking quick solutions to pressing problems. This also underscores the importance of parent attitudes concerning the relevance of the program.

Although a large proportion of families in the VCU Clark-Hill site quit the program after the first or second session, the site was able to combat some of the attitudinal barriers by engaging parents in multi-family SCT groups. Groups were facilitated by two family consultants, usually of different races and ages, which created opportunities to relate to the family consultant depending upon one’s preference. Family consultants noted that the groups also allowed families to rely on one another for support rather than the facilitators alone, which often reduced stigma. In addition, the family consultants observed that the groups provided an opportunity for families to provide examples more relevant to their lives than examples included in the SCT training materials.

In contrast, PW, which had much higher retention rates in both urban and rural sites, immediately addressed common parenting problems. Theoretical content is de-emphasized, as it is presented in a supplemental workbook that is used between sessions. Moreover, family consultants reported that mothers appeared uncomfortable during the introductory session of SCT when they were asked to look at their own teenage selves versus what they hoped for their children, especially when they saw the child falling into the same patterns. As they went through the early exercises of SCT, they seemed helpless to affect any changes. Parents noted that their children were already getting into trouble and they were having limited control and influence over their behavior. In these instances, parents were looking for action, not introspection offered by program content. The team noted the mismatch between program content, even though it was well intentioned, and parents seeking behavioral strategies may be a key to the high dropout rate for high-risk parents. Lower risk parents may have more tolerance for spending time on introspection of intergenerational family processes. The team noted that the theoretical content may have been irrelevant or too demanding for high-risk parents. This issue highlights attitudes about program relevance (Kazdin and Wassell 2000). Although looking at intergenerational patterns of behavior in families has been salient in family therapy for at least fifty years, practitioners would be wise to consider the close match with participants’ immediate and long-term needs when selecting evidenced based practices. The match between participant needs and program content also underscores the importance of Ingoldsby’s (2010) emphasis on brief early-treatment engagement discussions, motivational interviewing, use of a family systems approach, and enhanced family stress and coping support strategies to make sure program content meets family needs. Lack of match between content and participants’ priorities is rarely discussed in the research literature and warrants more examination.

The NC-YVPC rural implementation of PW was very successful despite significant attitudinal barriers. Program staff encountered negative attitudes at different levels; within community level agency administration there were poor attitudes toward new programs, residents of the rural county were wary of “outsiders” and “research” (explained below as an interpersonal barrier), and parents were wary of participating if they did not hear about the program from someone they trusted. The team countered these attitudinal barriers by forming strong working relationships with many different community stakeholders (see interpersonal section below) and proving to participants that the PW content provided valuable skills to help with adolescent behavior management. Once they decided to participate in PW, parents remained invested throughout the program and more than 99% of the recruited participants (364 out of 367) finished the program. One parent expressed “[My daughter and I] have really been through some hard and stressful times, especially for a 12 year-old middle-schooler. The lessons and techniques taught in this program will be very useful to us as we continue to practice and use them regularly.” In many cases, as evidenced by this quote, the NC-YVPC team was able to overcome significant attitudinal barriers and highlighted to parents the value and relevance of the program content, especially when parents wished to modify child behaviors. Like the VCU Clark-Hill site, NC-YVPC staff encountered more attitudinal barriers from parents experiencing crises and high stress. NC-YVPC engaged an array of different agency partners to maximize referral sources and convenience in location for implementation. Diverse referral sources, however, also meant referred families brought a variety of different attitudes. Families referred from Child Protective Services were mandated to participate but brought attitudinal barriers (e.g., questioned relevance of services) to group participation. These mandated families were quieter during groups and exuded more anger over having to attend. They were also more guarded in providing examples of their own experiences. PW facilitators had to win them over, showing that the conent was useful in making their lives better. Parents referred from a Teen Court program that NC-YVPC also ran also brought challenging attitudes. These parents believed that their adolescent children had committed offenses and did not see why they had to attend parenting groups when the child was the one causing trouble. Once again, as these reluctant parents started the PW groups, facilitators worked hard to win them over and program content was immediately focused on practical management of adolescent problem behavior.

Having multiple family members attend the PW groups (i.e., at least two—one parent and an adolescent) also helped decrease attitudinal barriers when a single member negatively viewed the program. The family member who was harder to engage often came just to support the second member. This reluctant attendance often transformed once they jointly engaged in the material. Further, PW facilitators continuously worked to address attitudinal barriers by gathering participants’ feedback on lesson content and process and using a 10-item Consumer Satisfaction survey (CSQ) to assess parental satisfaction with the program (Gordon 2011).

NC-YVPC facilitators also overcame attitudinal barriers by allowing time in the group process to tailure the content to parents’ unique needs and experiences. Group administration included two facilitators; one focused on the PW material and the other solicited examples from the parents and adolescents in the group to further illustrate parenting strategies. In this way, the standard PW skills and strategies were placed in context for these rural families and made more relevant through use of real examples from the group. This heightened group interaction, trust, disclosure, and social support across the families.

It is often necessary to tailor evidenced based programs to increase the fit with local contexts. In this instance, we needed to make PW content palatable for rural, American Indian (Lumbee) families. If facilitators do not attend to this contextualizing, attitudinal barriers can increase, leading to poor retention. For example, one PW vignette shows an adolescent swearing at his mother. This shocked parents in the initial groups because their cultural, church, and community values absolutely forbid this type of disrespect. Our facilitators understood these attitudes and found a different way to teach the skills and content of this lesson without using the offensive vignette with adolescent swearing. We found that attitudinal barriers decreased when we fully attended to group concerns, made the program material highly relevant, and were sensitive to the local context.

Interpersonal barriers, which include relationships with providers that affect families’ participation in services, are the final component in Kazdin’s model. For the rural NC-YVPC, interpersonal barriers arose because the researchers were not from the target community, but rather were coming from the University of North Carolina at Chapel Hill, which was two hours away. There are profound differences between the rural and university communities in terms of socioeconomic status, race/ethnicity, political ideology, and many other areas. The NC-YVPC team noted that the fact that outsiders who were also researchers sponsoring the programs precipitated many trust issues. The NC-YVPC team had heard that community members did not trust researchers because some had collected data, shown the community in a negative light, and then disappeared in the past. Others did not want to be treated as “guinea pigs.” NC-YVPC leaders spent the first year of the project talking with community advisory boards about how the research experience would be different from past ones. They emphasized that the research was necessary for the grant funding but was secondary to providing long-term programs in the community. Hiring program staff from the community and ensuring these staff reflected the ethnic and racial diversity within the community helped to change some of these negative attitudes and built strong interpersonal ties. For PW recruitment, NC-YVPC staff hired from the county reached out to community partners in existing agencies across the county. Through partnerships with community organizations, NC-YVPC offered the PW program in a variety of locations across the county, as noted previously. Diverse partnerships helped to address challenges related to referral sources that has been identified in past research. This flexibility also allowed parents with transportation and/or time limitations to participate in the program, while also creating local advocates for the program that potential parents knew and trusted. PW staff also spent considerable time talking with pastors in the community about the program. Once pastors trusted the staff, they referred families with high-risk adolescents, provided a public endorsement, urged the congregation to accept the outsiders, and occasionally attended the program sessions for their own training. This was invaluable in building trust with participants and the target community to overcome interpersonal barriers to program participation. The NC-YVPC team concluded that the keys to building trust to counteract interpersonal barriers were consistency in communication, listening to community members’ concerns, forming partnership alliances, being reliable, and showing participants that staff cared and the program content was worthwhile.

As noted above, the NC-YVPC site found that parents were more engaged and interested in the PW program when delivered in the group format than when it was self-directed. The NC-YVPC PW program director was a certified psychodramatist with experience facilitating action-oriented simulations. Psychodrama is a psychotherapeutic method used in a group setting to encourage participants to act out scenes from their lives. These dramatic enactments allow participants to reflect upon their lives and recognize and process emotions (Kellerman 1992; Kipper and Ritchie 2003; Oxford and Wiener 2003). Delivering the program in interactive groups with role-playing to supplement the content was innovative and kept the parents interested. This action-oriented format allowed the facilitator to cover program content, while also tailoring the program to the needs and preferences of participating families and the community as a whole. This personalization of program content kept parents invested and the social support from other parents in the group reinforced program activities.

The VCU Clark-Hill site had better success implementing the PW program with Spanish-speaking families than the SCT program with English-speaking families, which is likely related to reducing all three barriers (i.e., structural, attitudinal, interpersonal). The rapid growth of Spanish-speaking families in the urban area resulted in barriers related to accessing bilingual services (e.g., mental health services, prevention intervention services) in schools and the community (Corona et al. 2009). Thus, by hiring bilingual family consultants to implement PW, the VCU Clark-Hill site addressed a community-level structural barrier and was one of only a few service providers offering parenting and prevention intervention services in Spanish. The team believed that this helped with family engagement in PW because parents had so few services available to them and their children. As discussed above, the content of PW may have better resonated with Spanish-speaking families because it is more flexible in implementation approach thereby reducing attitudinal barriers. Finally, the team also noted that participating family relationships and/or alliances with the family consultants may have affected participation rates. Unlike the family consultants for SCT, the main family consultant for the PW program was a trusted and well-known community member whose primary place of employment was with the City of Richmond’s Multicultural Office. This connection meant that she was a visible community member with an already established level of perceived trust with Spanish-speaking families in the urban community. This shared trust enabled the facilitator and participants to overcome interpersonal barriers to engagement and retention.

Conclusions

In a major implementation of evidenced-based family programs funded by CDC, rural and urban sites experienced common structural, attitudinal, and interpersonal barriers to participant recruitment and retention (Ingoldsby 2010; Kazdin 1996; McKay and Bannon 2004). These barriers were largely overcome through thoughtful use of resources (e.g., providing transportation, food, childcare), tailoring of program content to fit participants’ needs and concerns (e.g., variable program formats), and building trust with participants and community collaborators. Some challenges, such as the difficulty recruiting families with the greatest need and truant adolescents, persisted despite the urban research team’s best efforts. Engaging families high in need but low in organization and stability remains an ongoing challenge for all family programs (Coatsworth et al. 2006; Perrino et al. 2001). The rural environment posed special challenges, such as how to gain credibility when seen as untrustworthy outsiders. Yet, with attention to common and unique community barriers, recruitment and retention rates for PW exceeded expectations.

Researchers have recognized that what influences families to initially participate in programs may not be the same factors that motivate them to continue engaging (Prado et al. 2006). Engagement is often highest when providers are able to minimize the impact of cumulative, chronic stress, but this can be challenging when working with multi-problem, highly stressed families (Finigan-Carr et al. 2014; MacNaughton and Rodrigue 2001; Spoth and Redmond 2000). In our urban setting, families with truant adolescents were transient, hard to contact due to housing instability, and more concerned with basic needs than parenting skills. Previous research has underscored the difficulty of working with families in constant crises and multiple barriers to participation (Brody et al. 2006).

Although lower stress families are easier to retain, parents with high cumulative, chronic stress may be effectively engaged if program content clearly meets their needs. The VCU Clark-Hill site was able to engage Spanish-speaking families better than English-speaking, African American families. The research team and family consultants believed that this was likely the result of the content of program being implemented being better matched to parents’ needs and parental perceptions of what they were “gaining” in return. The urban site in Virginia was very successful in retaining impoverished Latina/o immigrant families, which the team believes was largely because the Spanish version of PW provided specific parenting strategies and the community had few other services for this target group.

While prior work has demonstrated the effectiveness of SCT in self-directed and group formats, the VCU Clark-Hill site encountered retention difficulties, which the team believes (based on their experiences when they tried to implement sessions with families) may have arisen because program content did not match parents’ needs to address other chronic and/or more pressing stressors. This mismatch resulted in high participant attrition. The extant literature often attributes attrition to family barriers (i.e., problems plaguing participants) and seldom considers whether program content is off the mark (Kazdin et al. 1997; Kazdin and Wassell 2000). Our experience suggests that program providers share the blame for high attrition and should regularly examine whether program content is strongly aligned to family characteristics and circumstances.

Participants who remained engaged and finished the program embraced the PW program’s focus on skills rather than introspection. Similarly, in rural North Carolina, severely impoverished families all completed the PW program, which the research team believes was largely due to the shared experience in the groups and the strong relationship with the program staff. Regular program content was supplemented with role-playing centered on similar concerns brought up by parents in the groups. The team noted that this brought the local context and culture into the PW sessions and made them personal. This active group process was similar to Ingoldsby’s (2010) suggestion to elicit examples of obstacles parents encounter and work jointly to generate solutions, except that it was done using role-playing techniques. Consistent with prior research, which has found that parental practice of skills during parent training sessions is associated with better outcomes (Kaminski et al. 2008), we recommend skills practice through role-playing for future program implementation efforts. Moreover, our experiences suggest the need to identify strategies that match program content to participants’ needs and encourage other researchers and practitioners to do the same to maximize recruitment and retention in future parent training research.

PW was designed as a computer-based program to maximize fidelity (e.g., the computer always delivers the content in the same way) and allow simple administration anywhere there is an Internet connection. Both urban and rural sites found that participants, in general, did not want to participate in the computerized version of the program. Recruitment, engagement with the program, and retention were highest with a facilitator, who used the “old-school,” low tech, group facilitation format to make the content more personal to the participants. This is an important observation for educators and practitioners who are relying more and more on online administration of programs. Not having a human facilitator may cut financial costs and boost fidelity; however, there may be substantial additional costs when participant dropout is considered, undermining program effectiveness (Ingoldsby 2010). A seasoned facilitator minimizes attitudinal and interpersonal barriers to engagement and retention by making sure that program content is matched to participant needs. Computers cannot match this sensitive customization of program content. Perhaps artificial intelligence will someday allow computer administration to catch up in this area, but currently, there is no substitute for human contact.

It was important to compare and contrast urban and rural sites, especially in light of the dearth of previous research in rural areas (Ingoldsby 2010). In the rural community, families were strongly rooted, and exceptionally disadvantaged, yet they would only seek help if their pastors referred them to the program. Once we aligned our efforts with the network of ministers, they linked us with the highest need families, who were likely to benefit most from the services provided. Navigating the closed boundaries of a tightly knit, rural community was difficult and took time and effort, including significant time in a car driving between implementation sites. We had to demonstrate our long-term commitment and collaborate with key gatekeepers (e.g., pastors, principals, juvenile justice counselors). At the same time, a key strength in rural areas is the dense social network that allows many community leaders to come together in efficient collaboration if they buy into the initiative. It was important to eat together, to respect local culture, and to be seen as a consistent, contributing part of the community over time.

Finally, both the rural and urban teams had at least one person who was trustworthy in the community. This liaison or ambassador was crucial for overcoming the attitudinal and interpersonal barriers to recruitment and retention (Kazdin 1996; McKay and Bannon 2004). The NC-YVPC team hired staff members who lived in the community their entire lives and were well-known and respected by others. VCU Clark-Hill had a program facilitator on their team who was bilingual and bicultural and was a high-level link between the Latino community and a governmental agency. These community ambassadors helped to bring community partners together, found appropriate locations for program implementation (e.g., churches, community centers), lent local credibility to the project, assisted in making contextual adaptations to the program material, and built trust with participants. The liaisons and program facilitators were essential for the project teams, but the role of these individuals has received little attention in past published research on recruitment and retention.

There are limitations with the information presented herein. Both sites experienced barriers to recruitment and retention that were handled in team supervision and with group problem solving. Neither site systematically assessed barriers or quantified them as constructs for the research projects. This discussion describes the experiences of two sites implementing evidence-based parent training programs that are widely available and used in various geographic locations and with different populations. Parenting Wisely is included in the Substance Abuse and Mental Health Services Administration’s (SAMHSA) National Registry of Evidence-based programs and Practices (Substance Abuse and Mental Health Services Administration 2016), and Staying Connected with Your Teen is included in the registries of the Office of Juvenile Justice and Delinquency (2012) and the National Institute of Justice (2012). The experiences described herein may differ in other geographic locations, with different parenting programs, and with different populations of parents and their children. Other providers may make different implementation decisions. Despite these limitations, we believe this discussion of recruitment and retention is valuable. Future program providers could consider structural, attitudinal, and interpersonal barriers to participant recruitment and retention proactively during study design rather than in the field as implementation crises. Structural barriers are overcome by providing transportation, food, childcare, and incentives that are consistent and have value to participants. Attitudinal barriers need to be addressed through flexible program delivery (e.g., variable program formats) and most importantly by targeting of program content to fit participants’ needs and concerns. Finally, interpersonal barriers fade after building trust with participants and community collaborators. Continuing research on ways to address these barriers would be helpful for future parent training implementation efforts.

Disclaimer

The findings and conclusions in this manuscript are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.