Introduction

Meta-analyses of the effectiveness of youth mentoring programs in improving youth outcomes have shown a remarkably consistent and relatively modest range of overall effects over time, despite steady investments in mentoring-related research designed to improve program impact (Fernandes‐Alcantara 2018). DuBois and colleagues have published two comprehensive meta-analyses of youth mentoring programs, both of which showed relatively small effects of mentoring across outcomes (Hedge’s g values ranging from 0.18 to 0.21; DuBois et al. 2002, 2011). Likewise, a 2019 meta-analysis of 70 intergenerational, one-on-one mentoring program evaluations once again showed that mentoring yielded a range of significant, but small overall effects (Hedges’ g = 0.21; Raposa et al. 2019). In response to the lack of improvement in the effect sizes of youth mentoring over the past two decades, the authors called for “more rigorous adherence to evidence-based practices that target specific mechanisms underlying particular youth difficulties, rather than relying on a relatively low-intensity, non-specific approach with uneven adherence to practices that are research-informed” (Raposa et al. 2019, p. 438).

Nonetheless, the vast majority of mentoring programs provide non-targeted care, encouraging mentors to provide general friendship, support, and role modeling aimed at broad developmental goals (Garringer et al. 2017). This approach is based on the assumption that the mentor–youth relationship itself is the primary active ingredient of change. In particular, a close, supportive relationship with an adult is thought to provide youth with a “corrective experience” which, in turn is thought to lead to improvements in youth functioning across a broad range of developmental domains (Rhodes 2005). In fact, according to proponents of this non-specific, relationship-focused model of mentoring, targeted, skills-based approaches may actually be counter-indicated insofar as they may hamper relationship-building (Li and Julian 2012). Yet there is emerging evidence that more targeted, problem-specific approaches to mentoring may yield larger effects (e.g., Lyons et al. 2019). Programs that have taken a structured approach often do so in response to the fact that many youth who are referred to mentoring programs present with significant emotional, behavioral, or academic difficulties (Jarjoura et al. 2018). The current meta-analysis, which builds on the Raposa et al. (2019) study, contributes to this growing debate by examining whether non-specific approaches to youth mentoring show differential effects relative to more targeted, problem-focused approaches to mentoring.

The non-specific, relationship-focused approach to youth mentoring has been the dominant model of youth mentoring in the U.S. since mentoring programs first emerged in the early 1900s. In a recent study of thirty, nationally representative youth mentoring programs, nearly 2000 mentors were asked how they spent time with their mentees (Jarjoura et al. 2018). The responses reflected a non-specific approach to broad developmental goals. The most common response, “making time to have fun,” was followed by activities such as discussing important people or personal issues, going to cultural or other special events, and engaging in creative activities (Jarjoura et al. 2018). Likewise, in a national survey of 1271 mentoring organizations affiliated with MENTOR: The National Mentoring Partnership (Garringer et al. 2017), the most common program goals, according to program directors, were the broad development of life skills (53.9%), general youth development (51.2%), and providing a caring adult relationship (44.6%). In contrast, comparatively fewer programs endorsed more specific and instrumental goals, such as supporting college access (17.9%), violence prevention (5.2%), STEM education (6.3%), or substance use prevention (2.9%). Finally, in an evaluation of Big Brothers Big Sisters (BBBS) school-based mentoring programs, only 11% of mentors endorsed having an instrumental goal (e.g., improving academic or school behavior), while 79% endorsed broad developmental goals (e.g., being a friend or helping the child feel good about him/herself; Raposa et al. 2016).

Researchers and theorists have long advocated this non-specific, relationship-focused approach as the most effective way to connect with one’s mentee (Rhodes 2005). In an influential article, researchers Li and Julian (2012) compared the role of the mentor-mentee bond to that of fluoride in toothpaste, suggesting that other ingredients (e.g., color, taste) may add value, but they are not essential to the success of the match. These authors argued that the “enduring emotional attachment” is the only “active ingredient” in mentoring programs and that “scaled-up programs and policies serving children and youth often fall short of their potential impact when their designs or implementation drift toward manipulating other ‘inactive’ ingredients (e.g., incentive, accountability, curricula) instead of directly promoting developmental relationships” (p. 157). Other researchers have reached similar conclusions, noting that “the common feature of successful interventions across all stages of the life cycle through adulthood is that they promote attachment and provide a secure base for exploration and learning for the child” (Heckman and Kautz 2013, p. 2). This type of bond is thought to compensate for a lack of the close bonds of a nuclear family and set youth on a more positive life course (Rhodes 2005).

One factor that may also help to explain the persistence of non-specific mentoring approaches relates to the wide range of presenting problems endorsed by the families of youth referred to large programs like BBBS. Many of these larger programs work with youth at diverse developmental stages (i.e., early childhood through late adolescence), from diverse backgrounds (e.g., foster care, unaccompanied refugees, previously incarcerated youth), who are often dealing with a wide variety of problems (e.g., depression and anxiety, peer rejection, substance use). For these programs, a generic, friendship-based model is a scalable approach that can be delivered to all youth, irrespective of their particular issues. Such programs rarely have access to the wide range of targeted, empirically supported interventions that would be needed to address their mentees’ needs. Moreover, when they do specialize, they can only hit the mark with a subset of mentees. It is thus completely understandable that non-specific programs have defaulted to the common denominator, lighter-touch friendship models that can essentially be delivered to all youth, irrespective of their particular issues.

Yet, non-specific, relationship-based approaches alone may not adequately address the substantial emotional, behavioral, or academic difficulties that mentees face. Compared to national samples, youth who are referred to mentoring programs are often at significantly higher risk for a variety of difficult life circumstances (e.g., poverty, parental substance abuse), as well as behavioral and mental health issues like depression, anxiety, aggression, and attention difficulties (Jarjoura et al. 2018). Moreover, recent evidence suggests that families from marginalized communities often perceive mentoring programs as an alternative to professional healthcare services. For example, one study found that Black caregivers were twice as likely as White caregivers to turn to mentoring programs to address their children’s externalizing behaviors such as aggression, hyperactivity, or conduct problems (Vázquez and Villodas 2018). Given the diverse barriers to accessing high-quality professional mental health services within marginalized communities, including language differences, discrimination, and financial costs, mentoring is sometimes seen as a less stigmatizing and more culturally congruent approach to supporting youth with mental health needs (Cook et al. 2013; Vázquez and Villodas 2018). In general, prevention programs that work with youth who experience more acute behavioral, emotional, and academic problems tend to show stronger positive effects because, like a pendulum, there is more room for an upswing (Tanner-Smith et al. 2018). Despite this, non-specific mentoring programs have tended to yield relatively weaker effects than other interventions that serve youth at comparable levels of risk. It may be the case that, in the absence of specialized training and clear goals, many mentors in programs with a non-specific focus feel overwhelmed by their mentees’ difficulties (Spencer 2007). In fact, higher rates of mentee stress exposure at home and behavioral difficulties (e.g., delinquency, academic problems) are often key predictors of poor mentor–youth relationship quality (Raposa et al. 2016). Moreover, this mismatch between parent and youth needs and mentor training can lead to high proportions of early match closures and youth disappointment, particularly for youth with more severe behavioral and emotional difficulties (Bernstein et al. 2009; Kupersmidt et al. 2017).

Thus, although efforts to develop strong mentor–youth relationships are certainly necessary, it may be equally important to provide mentors with training in targeted approaches that can more directly address the needs of mentees. In fact, several recent evaluations of more targeted approaches to youth mentoring suggest their effectiveness. For example, Sowers et al. (2017) evaluated the efficacy of a STEM-focused mentoring program for students with disabilities. Mentees were matched with a mentor in the STEM field, and mentoring activities and discussions specifically revolved around education and career development in STEM. Findings revealed moderate to large effects for a range of outcomes such as STEM-related knowledge, engagement, confidence, and career planning confidence. Other research suggests that, when mentoring activities are calibrated and targeted to specific challenges, youth see even stronger positive effects in academic (Wellington-Johnson 1997), psychological (Jent and Niec 2006), and social (King et al. 2002) outcomes.

In contrasting targeted and non-specific approaches to youth mentoring, it is helpful to look to an analogous, and perhaps more developed, discussion in psychotherapy. For decades, researchers and clinicians have debated the “active ingredient” of psychotherapy, with one side arguing for that the relationship between therapist and client as the primary vehicle of change, and the other arguing for the importance of applying evidence-based interventions that have demonstrated empirical evidence of effectiveness. This debate has been a topic of extensive research comparing “usual care” approaches, in which therapists “use their clinical judgment as they [see] fit, not constrained by evidence-based interventions or manuals,” to evidence-based interventions, in which therapists apply empirically support therapy protocols (Weisz et al. 2005, p. 631). If the therapeutic relationship is the primary vehicle of change, one would expect “usual care” models to be as effective, if not more effective, compared to evidence-based approaches.

Yet, decades of studies have consistently shown that evidence-based practice shows superior effects compared to usual care. In a multilevel meta-analysis of 52 randomized controlled trials (RCT’s), Weisz et al. (2013) found that evidence-based psychotherapy was significantly more effective than usual care, with an average difference in effect size of 0.29. The same research group replicated this finding in a more recent meta-analysis, finding an average difference in effect size of 0.30 between evidence-based practice and usual care (Weisz et al. 2017). The authors note that usual care seemed to be more rigorous and effective than other types of control conditions, such as waitlists, psychotherapy placebos, and case management, and that usual care likely has some active treatment ingredients, such as a therapeutic relationship and consistent contact. However, the consistent difference in effect size indicates that evidence-based interventions add value and effectiveness to treatment, and that techniques that are empirically and theoretically matched to presenting problems are also essential active ingredients. Applied to youth mentoring, these findings suggest that providing mentors with training and supervision in techniques that target youth’s presenting difficulties, rather than relying on intuition and relationships alone, may enhance effectiveness. In practice, this would involve specifically targeting youth with particular presenting problems and/or assessing and identifying difficulties at the start of the program, and then applying interventions that theoretically, and (if known) empirically matched to these problems that can be delivered in the context of the mentoring relationship. As discussed above, although non-specific approaches have been far more common, some targeted mentoring programs have been developed and evaluated, presenting the possibility to conduct a meta-analytic comparison between targeted and non-specific approaches.

The Current Study

Although no studies to date have directly compared these two models of mentoring, the relative benefits of targeted and non-specific models can begin to be explored within the context of meta-analysis. The current study builds on a recent meta-analysis of 70 intergenerational, one-on-one mentoring program evaluations, representing more than 25,000 youth, that were conducted from 1975 through 2017 (Raposa et al. 2019). One potentially important moderator that this meta-analysis did not code or analyze involves whether the program took a non-specific, relationship-focused approach versus a more targeted, problem-specific approach. To address this gap in the literature, the current meta-analysis sought to examine the relative impact of these two distinct approaches to formal mentoring. The present study built upon the analyses of Raposa et al. (2019) in two ways: (1) an additional literature search was conducted to find articles published since the completion of this previous meta-analysis (up through September 2018); and (2) a rigorous coding procedure was applied to these studies in order to specify whether the evaluated program was non-specific and purely relationship-focused versus targeted and problem-specific.

Using rigorous inclusion criteria established by Raposa et al. (2019), analyses included all relevant outcome studies of intergenerational, one-on-one formal youth mentoring programs (i.e., offering the support of a caring relationship with a non-parental adult) written in English between 1975 and 2018. A multilevel meta-analytic approach was used to estimate the overall effect size of youth mentoring programs and assess whether the magnitude of these effects was moderated by the approach to mentoring (i.e., non-specific model versus targeted approach).

It was hypothesized that the overall effect size would be larger for targeted mentoring programs compared to programs taking a non-specific approach. Moreover, in light of the research regarding stronger effects for programs serving higher-risk youth, youth risk status was examined as a moderator. Finally, analyses explored whether mentoring approach (targeted versus non-specific) would significantly moderate five most common outcome domains in mentoring program evaluations (i.e., academic functioning, psychological symptoms, social functioning, health problems, cognitive functioning).

Method

Study Selection

The current analyses drew on an updated set of the original studies used in Raposa et al. (2019). As reported in Raposa et al. (2019), these studies were found through a comprehensive literature search of three online databases (i.e., PsychINFO, ERIC, ProQuest), identifying all evaluations of formal mentoring programs (i.e., peer reviewed articles, unpublished dissertations, technical reports) published prior to September 2017 using the following search terms and combinations of terms: Youth mentoring, Mentor + program, Mentor + evaluation, Mentor + intervention, Mentor + outcomes, Mentor + effects, Mentor + comparison, Big Brother, Big Sister, Protégé + mentor, Apprentice + mentor, Nonparental adult + mentor, Mentor mentee relationship, Mentor + randomized control trial, Mentor + RCT, and Mentor + experimental. Prior meta-analyses and qualitative reviews were manually searched to identify additional studies. Websites of multiple national formal mentoring programs, agencies that fund or evaluate mentoring research, and the Office of Juvenile Justice and Delinquency Prevention were searched for additional articles and reports. Additional research recommendations were requested from experts in the field of youth mentoring, including those involved with previous mentoring meta-analyses as well as those with ample experience conducting youth mentoring research. This search process resulted in 16,455 potential articles, dissertations, and reports. Duplicate studies were eliminated prior to evaluation for inclusion and the most comprehensive studies were utilized in cases where multiple evaluations included the same sample.

Original inclusion criteria from the Raposa et al. (2019) study were used and included the following: (1) A formal mentoring program with mentoring defined as a non-parental adult or older youth serving in a non-professional helping role for a specific youth to promote positive outcomes through the relationship. More professional relationships such as coaches or tutors did not meet inclusion criteria. (2) An evaluation with a comparison group, such as a randomized control trial and/or quasi-experimental design. Original study exclusion criteria included: (1) same-age peer mentoring, (2) exclusively group mentoring, (3) adult mentees (i.e., mentees older than 18 years), (4) inadequate treatment versus comparison group distinction (e.g., both groups received mentoring or the treatment group included a considerable amount of youth who did not receive mentoring), (5) evaluations in which mentoring was adjunctive, or not the primary or secondary intervention, (6) measured outcomes could not be categorized into one of the following broad categories: psychological, social, academic, health, or cognitive, (7) the mentoring program could not be categorized into either a non-specific or targeted approach, (8) inadequate data to compute an effect size, and the author was unable to be reached to provide necessary information or did not respond to requests for information within a specified timeframe, and (9) the study was written in a language other than English.

To update Raposa et al. (2019) database, an additional literature search was conducted to find articles published between September 2017 and September 2018 using the same databases and search terms. This resulted in a total of 2530 articles being screened, of which eight additional studies were identified according to Raposa et al.’s original criteria. Of these eight new studies, only four contained adequate information to be coded as “targeted” or “non-specific model.” Thus, the final analytic sample for the current meta-analysis included 48 studies and 535 effect sizes (see Fig. 1 for an overview of study selection).

Fig. 1
figure 1

Flowchart of the study selection process

Full size image

Study Coding Procedures

Procedures for coding youth outcomes were identical to the methods described in detail in Raposa et al. (2019). Of particular relevance for this follow-up analysis, five raters with advanced statistical experience and who were trained by a meta-analysis expert double-coded all selected studies. Raters met weekly with each other, and with meta-analysis and mentoring experts, to resolve ambiguities and discrepancies. Five broad categories of youth outcomes were coded in each study, including relevant statistical information (e.g., sample size, means, standard deviations, frequencies) to calculate an effect size for each outcome. These categories, established a priori to align with theories of developmental psychopathology and positive youth development, included academic functioning, psychological symptoms, social functioning, health problems, and cognitive functioning. The original meta-analysis also coded for several mentee characteristics. In addition to being coded for youth age, gender, and race/ethnicity, each study was coded for the type of youth population for which the mentoring intervention was designed. Studies were coded as to whether the program was designed to serve a “general” youth population or targeted one of five “at-risk” sample types: racial or ethnic minority youth, youth with single parents, youth from low socioeconomic status families, youth in foster care, or youth with multiple risk factors such as these. Given that Raposa et al. (2019) did not find a moderator effect for these five different risk groups, they were collapsed into one “at-risk” category in the current analyses.

Coding of Non-Specific versus Targeted Approaches

For this study, an additional coding procedure was implemented to identify whether each study involved a program that took a non-specific approach versus a targeted, problem-specific approach to mentoring (see Table 1). The senior author, who has over 30 years of experience in the mentoring research field, as well as practical experience as a match support specialist in a mentoring program, developed the initial coding scheme in consultation with other mentoring experts and based on the body of the research reviewed above. Studies were coded as tests of the non-specific model if they involved a program in which mentors were trained to act as a caring friend, primarily engaging in non-specific recreational activities with their mentee, rather than targeting a particular youth challenge or need. For example, a study of a Swedish mentoring program aiming to prevent substance use in youth had the program objective of “[establishing] a safe and supportive relationship, by which the youth is assumed to benefit in social, emotional, and academic development, and as a consequence, be less prone to use alcohol and drugs” (Bodin and Liefman 2011, p. 441).

Table 1 Article codes of mentoring approach
Full size table

In contrast, articles were coded as testing a targeted approach if the mentoring intervention both (a) targeted a specific youth population (e.g., trauma exposed youth) or challenge (e.g., depression, academic difficulties) and (b) implemented a mentoring intervention specifically designed to match the needs of the target population or challenge. Although it was not necessary for programs to use a manualized curriculum to be considered targeted, one example of this type of program included the Achievement Mentoring Program (Holt 2008), a manualized cognitive-behavioral mentoring program that seeks to enhance school-related cognitions and behaviors in students at risk of not graduating from high school.

Rigorous coding standards were implemented to reduce ambiguity and achieve clarity and precision about the nature of the mentoring program. For example, articles with the following features were excluded from analyses: (1) articles that did not provide sufficient information about the mentoring program; (2) non-specific mentoring programs that occasionally exposed their mentees to group-based, more targeted workshops; and (3) mentoring programs that aimed to target specific problems, but were “mismatched” in terms of mentee challenges and actual mentoring activities (e.g., targeting youth with academic challenges but the mentor is a “friend” who engages in recreational, rather than educational activities).

Each article was coded independently (i.e., double-coded) by two graduate research assistants who were involved in the development of the coding scheme and received training from the senior author regarding the coding scheme. These graduate research assistants then convened to discuss discrepancies until reaching agreement. In the event that an agreement was not reached, a consensus meeting was held with both graduate research assistants and the senior author to review the article and determine the final code. Inter-rater reliability tests demonstrated that the overall agreement across these two coders was 90%, representing almost perfect agreement (κ = 0.85; Cohen, 1960, as cited in McHugh 2012).

Effect Size Calculation and Data Analysis

Effect sizes were calculated by obtaining the standardized mean difference between the mentoring intervention group and the control group for each outcome, where a positive standardized mean value indicates higher performance for the mentoring intervention groups. To adjust for small sample sizes as well as differences in sample size across studies, standardized mean difference values were then transformed into Hedge’s g (Hedges and Olkin 1985). Study authors were contacted for more information in cases where necessary data (i.e., sample sizes, means, standard deviations, other values) was not reported.

Given that many studies assessed more than one youth outcome, multiple effect sizes were calculated for a majority of the included studies. To account for interdependency of these effect sizes, a three-level meta-analysis approach was implemented (Van den Noortgate et al. 2014). Three-level meta-analysis approaches are more rigorous than traditional meta-analysis methods as they allow for all effect sizes obtained from the same study (i.e., dependent effect sizes) to be included in analysis. In addition to this major advantage, three-level approaches to meta-analysis account for between- and within-study variability, increase statistical power, and allow for more moderator analyses than traditional approaches.

Three-level meta-analysis allows for modeling of three sources of variance, including (1) the sampling variance of the observed effect sizes, (2) the variance between effect sizes obtained from the same study, and (3) the variance between studies. The first level of variance (sampling variance of observed effect sizes) was estimated using a previously established formula (Cheung 2014). To determine whether there was significant variance present at the second (within-study) and/or third (between-study) levels, log-likelihood-ratio-tests compared the full-model deviance relative to the deviance of models excluding one of the variance parameters (Assink and Wibbelink 2016). Significant variance at level 2 or level 3 suggests there is heterogeneity in effect size distribution, indicating that the effect sizes cannot be considered estimates of a common effect size. In such cases, moderator analyses assessing variables such as sample characteristics, program characteristics, and/or study methodology characteristics can help explain heterogeneity in effect sizes.

All analyses were conducted in R (version 3.2.0) using the metafor package. Given the diversity in study samples, settings, interventions, and outcomes, a multilevel random effects model was estimated (Assink and Wibbelink 2016). All model parameters were estimated using the restricted maximum likelihood estimate. The Knapp and Hartung (2003) method was utilized to test individual regression coefficients of the meta-analytic models and to calculate corresponding confidence intervals (Assink and Wibbelink 2016). Dichotomous dummy variables were created for categorical variables (Tabachnik and Fidell 2013). Program type (targeted versus non-specific approach) was examined as a moderator for overall effect size of youth mentoring programs in addition to effect sizes for each outcome domain. Youth risk status was examined as an additional moderator of overall effect size to determine whether programs taking targeted versus non-specific mentoring approaches have different effects in at-risk and general youth populations. In order to increase statistical power to detect a significant moderator effect, and to prevent chance capitalization due to multiple testing, we collapsed five types of risk into one “at-risk” group.

Results

Youth sociodemographic information is presented in Table 2. Table 3 shows that the average effect size across all 48 studies and 535 effect sizes was g = 0.19 (p < 0.001; 95% CI: 0.13–0.24), which is a small effect and not significantly different from the average effect size found by Raposa and colleagues (g = 0.21; 95% CI: 0.14–0.28; 2019), given the overlapping 95% confidence intervals of the different meta-analyses. There was significant heterogeneity within studies (σ2 level 2 = 0.02, p < 0.001) and between studies (σ2 level 3 = 0.03, p < 0.001). A total of 30% of the variance among effect sizes was accounted for by the within-study level, and 48% by the between-study level, while random sampling error accounted for 22% of the variance.

Table 2 Descriptives non-specific versus targeted mentoring approaches
Full size table
Table 3 Overall effect of mentoring programs on youth outcomes
Full size table

Table 4 presents the results of the moderator analyses. The omnibus F-test for targeted versus non-specific programs proved to be significant (p = 0.011); the overall effect size for targeted, problem-focused programs (g = 0.25) was more than twice as large as that of non-specific programs (g = 0.11). In terms of clinical significance (Kraemer and Kupfer 2006), these statistics indicate that targeted, problem-specific programs, on average, tended to yield a 14% improvement in youth outcomes relative to no mentoring, while non-specific mentoring models predicted only a 6% increase in outcomes, relative to no mentoring at all.

Table 4 Targeted versus non-specific approach to mentoring
Full size table

Subsequent moderator analyses tested whether programs targeting a particular high-risk population of youth (e.g., youth in foster care, youth from families with low socioeconomic status) showed stronger effects when the curriculum was problem-specific, relative to programs treating a more general population of youth. Omnibus F-test results revealed a trend approaching statistical significance (p = 0.067). In general, the effect of the targeted approach to mentoring was larger in high risk samples (g = 0.27) and smaller, or sometimes not statistically different from zero in general population samples (g = 0.07).

Finally, analyses of the five outcome sub-domains revealed that the moderator effect for the targeted versus non-specific approach to mentoring was significant for academic functioning, (which includes outcomes such as academic performance and school engagement), psychological symptoms (which includes mental health related outcomes such as self-regulation and internalizing symptoms) and social functioning (which includes outcomes such as social skills and social support), but not for health problems and cognitive functioning.

Discussion

Although previous meta-analyses have examined the impact of mentoring programs on youth development, no studies to date have directly compared the influence of distinct models of mentoring in promoting youth outcomes. The purpose of this study was to compare the effectiveness of “targeted” mentoring programs (i.e., those that explicitly matched specific interventions to mentees’ presenting problems) to the effectiveness of “friendship-based” mentoring programs (i.e., those exclusively focused on relationship-building and non-specific recreational activities). The average effect sizes of targeted, problem-focused and non-specific programs were compared. Further, analyses examined whether the respective effect sizes of targeted and non-specific programs differed as a function of outcome domains (i.e., academic, psychological, social, cognitive, physical health) and mentees’ risk status. These analyses were conducted by drawing upon the most comprehensive, rigorous meta-analytic dataset on youth mentoring accumulated to date (Raposa et al. 2019), while adding new studies published after this dataset was originally compiled.

The meta-analysis yielded an overall effect size of 0.19, an impact not significantly different from that found in Raposa and colleagues’ meta-analysis (0.21) and one that corresponds with a conventionally “small” effect (Cohen 1988). However, when type of program was examined, targeted and more problem-specific programs had an average effect size of 0.25, which was more than double the average effect size non-specific programs (g = 0.11). These findings are in line with recent calls from mentoring researchers for stronger alignment with theoretical and evidentiary standards of prevention science (e.g., Cavell and Elledge 2015). Such standards typically require a close association between structured interventions and identified target problems in youth.

Across outcome subcategories (i.e., academic, psychological, cognitive, social, physical health) the effects of targeted programs ranged from 0.11 (cognitive) to 0.28 (academic), while the effects of non-specific programs ranged from 0.07 (psychological) to 0.13 (cognitive). Targeted programs were significantly more effective than non-specific programs in improving academic, psychological, and social functioning, but there was no significant difference in terms of physical health and cognitive functioning. This is perhaps not surprising, given that mentoring interventions are primarily psychosocial in nature. Indeed, most mentoring interventions coded as “targeted” specifically recruited youth with particular academic, psychological, or social difficulties, and equipped mentors with training and treatment approaches that directly addressed these respective issues (e.g., DeSocio et al. 2007; Karcher et al. 2002; King et al. 2002). In contrast, the majority of programs did not tend to target cognitive and physical health outcomes. Additionally, it is notable that relatively few studies—targeted or non-specific—reported on physical health outcomes at all, perhaps resulting in insufficient statistical power to detect a significant difference between program models in this domain.

Research suggests that significantly elevated rates of mentee stress exposure, as well as pre-existing emotional and behavioral difficulties, might be one reason why mentors struggle to successfully implement non-specific models of mentoring (Raposa et al. 2016; Spencer 2007). To examine the effects of targeted versus non-specific mentoring as a function of mentees’ presenting risk, a second set of moderator analyses was conducted. Although applying a targeted approach had a marginally greater benefit relative to non-specific approaches when samples involved higher-risk youth (0.27 versus 0.12) as opposed to a general population of youth (0.20 versus 0.07), results indicated that youth risk status did not statistically significantly moderate the impact of mentoring program approach (p = 0.067). However, these analyses used indicators of risk status that were primarily demographic (e.g., racial and ethnic minority status, low SES, single parent household) and, by nature of the meta-analytic approach, were at the program level rather than the participant level. These findings likely do not capture more acute problems often targeted by mentoring models (e.g., academic difficulties or delinquency). Future studies should include a more detailed assessment of youth risk at both individual and contextual levels.

It is important to note that relationship-building activities and training are not antithetical to targeted mentoring approaches. Indeed, many of the programs that were coded as “targeted” provide training to mentors aimed at developing sufficiently strong mentor–mentee bonds as the context and catalyst for the targeted intervention. Even the strongest proponents of more targeted approaches (e.g., Cavell and Elledge 2015) do not eschew the fact that mentoring is, at its core, a relational intervention and that overly prescriptive, rigid approaches could threaten relationship quality and mentees’ persistence in the intervention. This tension runs parallel to long-held debates in psychotherapy research about the impact of structured, evidence-based therapies relative to the non-specific or “common factors” of therapist warmth, empathy, and support provided across therapeutic modalities (e.g., Weisz et al. 2017). Meta-analyses of child and adolescent psychotherapy have consistently found that, across treatment modalities, therapist-youth working alliance has a moderate effect size on youth outcomes, even in the context of structured treatments like cognitive-behavioral therapy, and that certain relational variables (e.g., counselor empathy, genuineness, and warmth; counselor direct influence skills on youth; youth willingness to participate) significantly boost outcomes (Karver et al. 2018; Karver et al. 2006). Training in these universal characteristics of effective helping relationships can ensure a strong foundation for targeted skills development and remediation. However, the twofold difference in effect sizes between targeted and non-specific mentoring interventions in the current study suggests that relationship-building may be a necessary, but not sufficient, basis for helping mentees achieve change.

Study Strengths, Limitations, and Future Research Directions

The following limitations should be acknowledged when interpreting the study findings. First, evaluations of more targeted, problem-specific programs benefit from the fact that they measure a more restricted range of youth outcomes that are more directly aligned with the treatment goals. In non-specific programs, differential patterns of improvement can be essentially washed out when widely diverse outcomes are calculated into overall averages (Rose 2016). In addition, many other variables (e.g., sample, program size and length, implementation or evaluation quality) could account for different effects across these types of programs. It was not possible to estimate a multivariate model including all possible confounders of the targeted non-targeted effect because of missing data in many of these possible confounders; however, many of these variables (including many sociodemographic variables) did not emerge as significant in the meta-analysis by Raposa et al. (2019) in univariate analyses. Nonetheless, it will be important for future research to include randomized trials that control for these variables and directly compare the effects of non-specific to targeted program models.

It is also important to note that there was significant heterogeneity in outcomes both within and across studies, and a number of the targeted mentoring models examined in these analyses produced relatively small effects. Thus, although the meta-analytic data support targeted programs more generally, there are certainly instances where this is not the case. Future program evaluations should consider and measure the wide variety of mentor, youth, and program characteristics that may account for differences in program effectiveness (even within the more targeted, problem-specific category). It is also notable that programs were coded as targeted if they matched specific interventions to youth’s presenting problems, irrespective of the evidence base, rigor, and fidelity of their interventions (and only included studies that specifically delineated program practices such that they could be clearly categorized into one of two mentoring models were included). Again, it is important for mentoring programs to articulate the theories and evidence base from which they develop their interventions as well as the fidelity with which these interventions are carried out, and for future meta-analyses to examine these factors as moderators of program effectiveness.

The analyses only included studies that were written and published in English, and all but two (i.e., Simões and Alarcão 2014; Bodin and Liefman 2011) were conducted in North America. As a result, the results are not necessarily generalizable to mentoring programs operating outside of North America. Further, although the Raposa et al. (2019) meta-analysis from which the subset of studies was drawn did not find strong evidence of publication bias, it remains likely that this sample of studies is not representative of all mentoring programs being implemented and evaluated.

Despite these limitations, there are a number of strengths to the current study. This meta-analysis explored a critical, previously untested moderator of youth mentoring effectiveness. Previous studies (e.g., Raposa et al. 2019) have examined some program characteristics as predictors of program effectiveness (e.g., structured vs. unstructured vs. semi-structured, whether or not programs use a curriculum), but found no impact on outcomes. It is possible that these previous findings were null because these categorizations do not take into account whether structures and curricula are actually matched with youth’s presenting problems.

Another strength of the current study is the use of rigorous statistical methodologies, particularly multi-level meta-analyses to account for variability in effect sizes both within and across studies. The analyses suggested that a substantial portion of the variance among effect sizes was accounted for within studies (30%). That is, even within the same program evaluation, there were significant differences in the size of observed effect sizes for different outcome measures or constructs. This finding suggests that the multi-level approach to meta-analysis should be considered as a particular strength of this study, given that a traditional meta-analysis only accounts for between-study heterogeneity among effect sizes. Future studies in this area should use a multi-level approach to more accurately explore the specific outcome-level (e.g., self-report versus teacher-report) and study-level (e.g., demographic background of the youth in the program) constructs that might be accounting for all sources of variability in effect sizes.

Implications for Practice

The findings from the current study suggest several recommendations for future research and practice. Currently, there is inconsistency among mentoring programs in adherence to empirically supported program practices (e.g., recruitment and training strategies) recommended by organizations such as the National Mentoring Resource Center or MENTOR’s Elements of Effective Practice for Mentoring (Garringer et al. 2015). Implementation of evidence-based practices has the potential to increase mentoring match lengths (Kupersmidt et al. 2017), thus increasing the likelihood of positive youth outcomes (Grossman et al. 2012; Herrera et al. 2011). Moreover, mentoring programs and researchers can partner to shift from non-specific approaches and activities to targeted and precise interventions that have stronger theoretical and empirical foundations (McQuillin and Lyons 2016; Weiler et al. 2017). Still, it is important to acknowledge that even the largest effects of targeted programs found in the analyses (i.e., 0.28, on academic outcomes) are still considered small by standard conventions, suggesting the need for ongoing innovation, research, and implementation of evidence-based practices to increase impact for youth participants.

Of course, it will be important to introduce and implement changes to mentoring practices in ways that are compatible with existing organizational structures. In particular, it may not be realistic for large mentoring programs that serve a broad array of youth, like Big Brothers Big Sisters, to develop interventions and trainings that target every potential presenting problem. Rather, large programs can partner with other organizations and embed their volunteers into programs that already offer specialized, evidence-based interventions and deploy mentors to play supporting roles. Likewise, the burgeoning field of mental health apps and other technology-delivered interventions have the potential to provide large, non-specific programs with a growing array of targeted, evidence-based interventions. In this “blended” model, youth learn and practice targeted skills through youth-friendly apps, with mentors reinforcing this engagement. In both embedded and blended approaches, mentors encourage youth to complete lessons and activities and provide contexts for practicing new skills (Mohr et al. 2011).

Conclusion

In recent decades, mentoring programs have become increasingly prevalent interventions to support positive developmental trajectories for youth experiencing a range of emotional, behavioral, and academic difficulties. The dominant “non-specific” model of youth mentoring programs assumes that a supportive relational bond between volunteers and youth can promote positive developmental changes and prevent a range of negative outcomes. However, findings from years of systematic reviews and meta-analyses of mentoring programs yielding small effects on youth outcomes have led to growing calls for mentors to employ targeted skills designed to match the presenting concerns of mentees. Despite these recommendations, no studies to date have directly compared non-specific versus targeted approaches to mentoring. Results from the current meta-analysis of 48 mentoring outcome studies suggest that non-specific, relationship-based models of mentoring may be less effective than more targeted, problem-specific approaches in addressing the needs and challenges of today’s youth. Moreover, moderator analyses revealed that targeted approaches may be particularly effective for promoting important developmental outcomes such as youths’ academic performance, psychological wellbeing, and social functioning. Yet, as the field corrects for an overemphasis on non-specific approaches and moves in more targeted, evidence-based directions, it should resist veering too far from the relational component that sets the field apart from pure tutoring or other skills-based training. Programs should strive to find equilibrium between relational bonds and the delivery of more targeted and specific approaches to mentoring. When this balance is achieved, the mentoring relationship may be poised to better address the particular needs and challenges of today’s youth.