There is no shortage of information on the topic of youth mentoring. In addition to a growing number of academic books and peer-reviewed journals devoted to the topic, the sheer volume of articles and online reports is enough to numb even the most curious of minds. Despite this wealth of information, the base of evaluation findings on which policy and practical decisions rests remains curiously thin. Mentoring strikes deep emotional chords and has attracted powerful constituents who, at some level, look to evaluations to confirm what they intuitively hold to be true. Likewise, practitioners tend to value pure and simple findings that can be used to for action. Although it can be difficult to satisfy such appetites while remaining true to the evidence, a more nuanced, understanding of what it takes to deliver high quality, effective youth mentoring could, in fact, lead to allocations for program enrichments that would yield a higher return on investments.

So, what do we know about the efficacy of youth mentoring? From experience and the research that has already been compiled we know that, when done well, mentoring is an effective intervention strategy for some young people. Evaluations of formal one-to-one mentoring programs have provided evidence of their success in promoting better social, academic, and behavioral outcomes (DeWit et al. 2006; DuBois et al. 2002a, b; Grossman and Tierney 1998; Herrera et al. 2007; Karcher 2005; Keating et al. 2002). Yet such evidence is in relatively short supply. The most scientifically rigorous verdict on effectiveness was reached over 5 years ago, when a meta-analysis of 55 youth mentoring program evaluations was conducted (DuBois et al. 2002a). Findings from this analysis, as well as evaluations that have been conducted subsequently, will be described in later sections. To this end, the evaluation literature can be broadly defined as fitting into somewhat overlapping categories of reviews, program evaluations, and meta-anlayses.

Reviews

Several comprehensive reviews of the youth mentoring literature have emerged from the US, Canada, and the UK in recent years (see Table 1). Although such reviews can move readers beyond the more piecemeal approach of individual studies, mentoring programs can vary on a multitude of dimensions (e.g., duration, intensity, integration with other services, target populations, approaches) in ways that complicate global assessments of effectiveness. Similarly, although high quality work is often included, many reviews also contain a discouraging mix of flawed studies. It is not uncommon, for example, to see rigorously, peer-reviewed research placed on relatively equal footing with unpublished in-house reports. Moreover, reviews of overlapping bodies of work sometimes draw dramatically different conclusions (Boaz and Pawson 2005). For example, a recent review (Hansen 2007, p. 4) concluded that, “studies consistently find a broad range of positive outcomes from both community-based and school/site-based mentoring.” A survey of many of the same studies, however, led researchers (Roberts et al. 2004, p. 513) to conclude in the British Medical Journal (BMJ) that “mentoring programmes as currently implemented may fail to deliver on their promises.” This difference of opinion stems, in no small part, from how and what evidence is considered. For example, the BMJ review and others (e.g., Roberts et al 2004; Hall 2003; Philip and Spratt 2007; Liabo and Lucas 2006), place considerable stock in meta-analyses of program effects. By contrast, Hansen (2007) and others (Jekielek et al. 2002; Sipe 2002) put more weight on the 1995 evaluation of Big Brothers Big Sisters of America, which has been interpreted quite positively. More generally, reviews tend to differentially highlight potential iatrogenic effects and set different inclusion standards (i.e., strict evaluation versus a mix of evaluations, secondary analyses, and more qualitative program descriptions). Likewise, review articles and chapters in special issues of journals and academic handbooks, which summarize the literature as it bears on particular topics (e.g., gender, special needs) are only as strong as the research and evaluations on which they stake their claims (Pawson 2006).

Table 1 Summary of reviews
Full size table

Mentoring Program Evaluations

Evaluations of formal one-to-one mentoring programs have provided evidence of success at reducing rates of problem behaviors, academic difficulties, and psychological disturbances. Yet, these evaluations vary in their ability to rule out confounds and, as in all program evaluations, there exists a constant tension between the real and the ideal. Even when well conducted, findings from the evaluations that have been conducted since DuBois et al.’s (2002a) meta-analysis do not suggest the strong effects that are central to arguments for investment in mentoring initiatives. In some instances, negative or no effects have been found (e.g., Blechman et al. 2000), or effects have eroded to non-signficance within only a few months of program participation (Aseltine et al. 2000; Herrera et al. 2007). In fact, only one mentoring program, Across Ages, has achieved the status of “model program” on the Substance Abuse and Mental Health Services Administration (SAMHSA) Registry of Evidence-based Programs and Practices (NREPP), an online registry of independently reviewed and rated interventions.

Big Brothers Big Sisters of America (BBBSA) was listed on this registry as an “effective program,” a designation that stemmed, in part, from the landmark study of their community-based mentoring (CBM) programs (Grossman and Tierney 1998). Several widely cited, statistically significant differences in behavior, academic functioning between the mentored youth and the control group were uncovered after 18 months. Although promising, the standardized effect sizes across all outcomes was relatively small (.06)Footnote 1 (Herrera et al. 2007).

This same effect size was detected more recently, in a large randomized evaluation of BBBSA’s newer, school-based mentoring program (SBM) was conducted. In SBM, interactions between youth and mentors typically are confined to the school setting and the 1-year minimum commitment of mentors is shortened to the 9-month school year. Because SBM is linked to the academic calendar, the relationships tend to be less enduring than those forged through CBM. Indeed, the average length of the relationships in the SBM evaluation was just 5.3 months (compared to 11.4 months in the CBM evaluation), and nearly half (48%) of the relationships did not continue into the following school year. Overall, findings were mixed; at the end of the first school year, youth assigned to receive mentoring showed significant improvements in their academic performance, perceived scholastic efficacy, school misconduct, and attendance relative to a control group of non-mentored youth. Nonetheless, when youth were re-assessed a few months into the following school year, most differences were no longer statistically significant.

Despite these somewhat discouraging trends, the group differences that have been uncovered in the national evaluations do give grounds for cautious optimism about the potential viability of mentoring interventions. Matches vary considerably in their effectiveness, depending on the characteristics of the individuals involved and the quality of the relationships they form, in ways that affect outcomes. Indeed, secondary analyses of the SBM evaluation data revealed that mentees who experienced longer, higher quality relationships received bigger benefits than those in shorter or weaker relationships (Herrera et al. 2007). And, in Year 2, those involved in weaker relationships actually showed declines relative to their non-mentored peers. The same patterns have been found in community-based mentoring. When Grossman and Rhodes (2002) reanalyzed the BBBS community-based mentoring data taking the quality and length of relationships into account, wide variations in program effects emerged. But when all relationships are combined, as was the case in the analyses conducted for national evaluations, positive outcomes are easily masked by the neutral and even negative outcomes associated with less effective mentoring relationships. The challenge is to identify those program inputs and factors that can facilitate the formation of close, enduring, and, ultimately, effective mentor-youth ties.

Meta-analysis

A series of meta-analyses have permitted researchers to empirically summarize the results of mentoring across multiple studies and to statistically determine the strength of program-related effects. Although the ability to code such studies on important dimensions (e.g., relationship quality, intensity, and length, program approach) is constrained by whatever information is provided in the original study (see Cooper and Hedges 1994; Lipsey and Wilson 2001), comparisons across studies have revealed important patterns and gaps in the literature.

In their meta-analysis on youth mentoring to date (see Table 2), DuBois et al. (2002a, b) found favorable effects across relatively diverse types of program samples. Among the small number of studies that included follow-up assessments, the benefits of mentoring appeared to extend a year or more beyond the end of a youth’s participation in the program. As DuBois et al. (2002a) note, however, the magnitude of these effects on the average youth participating in a mentoring program was modest. Although there was considerable variation across studies, the effect size was relatively small (.14), particularly in comparison to the effect sizes that have been found in meta-analyses of other prevention programs for children and adolescents. For example, a meta-analysis of 177 prevention studies found effects ranging from .24 to .93, depending on program type and target population (Durlak and Wells 1997). Meta-analyses of youth psychotherapy, encompassing hundreds of studies, have reported even stronger mean effects, ranging from .71 to .88 depending on the age of the children being treated (Weisz et al. 2005). But, importantly, while the overall effect size of mentoring programs was modest, substantial variation in the effectiveness of different programs emerged across these studies. More structured programs, in which there were clear expectations, a focus on instrumental goals, and ongoing support to volunteers yielded notably strongest effects. Interestingly, a similar pattern emerged in meta-analyses of youth psychotherapy. Weisz et al. (2005, p. 631) note that, in studies of “treatment as usual in settings in which therapists were able to use their clinical judgment to deliver treatment as they saw fit, not constrained by evidence-based interventions or manuals, and in which there was a comparison of their treatment to a control condition” effect sizes were close to zero (see e.g., Weisz et al. 1995).

Table 2 Mentoring meta-analyses
Full size table

More recently, Tolan et al. (2005) conducted meta-analysis of 31 youth mentoring programs. Focusing on a more limited array of outcomes, the researchers found effect sizes of .24–.28 for delinquent and aggressive outcomes, respectively, while drug use (.08) and academic outcomes (.16) were somewhat smaller. The authors concluded that additional evaluations that include random assignment and growth measurement over time were needed. Jolliffe and Farington (2007) explored the effects of youth mentoring on recidivism among juvenile offenders. Their analyses, which were based on 18 evaluations, revealed a combined fixed effect of only .08. Again, significant variation emerged across studies; seven studies showed significant positive impacts on re-offending while an equal number showed negative (but not statistically significant) impacts. Programs that combined mentoring with other interventions, required weekly meetings for longer periods of time per meeting, and had more enduring relationships had the most positive effects on re-offending. Looking at a broader range of outcomes, Eby et al. (in press) conducted a meta-analysis of 40 youth mentoring evaluations, comparing them to 53 adult workplace mentoring and 23 college-level academic mentoring evaluations. Again, the effect sizes were generally small, with mentoring more highly related to some outcomes (school attitudes) than others (psychological distress). Interestingly, effect sizes were found to vary across the three types of mentoring, with absolute values ranging from only .03 to .14 in youth mentoring to .11 to .36 and .03 to .19 in academic and workplace mentoring, respectively. This relative ranking is consistent with the previous meta-analysis, and makes sense when one considers the greater challenges facing youth and the fact that academic and workplace mentoring includes a mix of assigned and natural mentors. Nonetheless, the authors conclude that, “we believe the results underscore the need to temper what are sometimes seemingly unrealistic expectations about what mentoring can offer to protégés, institutions, and society at large.” Finally, Smith (2002) reports an effect size of around .20 across 43 studies. Similar to Tolan et al. (2005), effect sizes varied depending on the outcome assessed.

Although less thoroughly explored than in the DuBois et al. study, the findings of the more recent meta-analyses suggest that the effects are likely to vary depending on an array of youth, mentor, and program characteristics as well as the quality of the evaluation methodology and outcomes measured. Given this variation, it is unfortunate that only two of the meta-analyses (DuBois et al. 2002a, b; Smith 2002) have conducted formal tests for moderators of program effects. A study that includes a systematic, up-to-date meta-analytic review of the current literature and a thorough test of the moderators would thus represent a significant contribution to the literature. Several well-designed evaluations of multiyear mentoring programs are underway or recently completed which, when combined with the smaller evaluations that have been conducted in recent years, will provide a better sense of the moderating variables and their association with outcomes. The inclusion of these additional studies will help practitioners and policy-makers establish more realistic goals and expectations concerning program scale, intensity, length and outcomes. For now, as unsatisfying as it may sound, the conclusion that “robust research does indicate benefits from mentoring for some young people, for some programmes, in some circumstances, in relation to some outcomes,” is probably the closest to a “bottom line” on youth mentoring that can be reached (Roberts et al. 2004).

Implications for the Practice of Youth Mentoring

The above review offers a somewhat sobering evaluation of the current state of evidence for youth mentoring, while pointing to strategies for improving programs, relationships, and outcomes (Weissberg et al. 1989). To a certain extent, however, the field of youth mentoring has taken on a public life of its own—a life that is, at times, removed from the scientific evidence. Despite expansive goals, there has been no clear road map for how to scale up this intervention approach in ways that provides high-quality mentoring relationships to all participants. Instead a relatively small base of evidence for quality community-based mentoring programs helped to galvanize a wide constituency of support for youth mentoring interventions. This support has stimulated aggressive growth goals, which have necessitated that mentoring be delivered more efficiently and less intensively (Rhodes and DuBois 2006).

Bringing an intervention to scale while retaining fidelity is costly and challenging, but it can be done. To meet this challenge, policy-makers and funders must demand greater adherence to evidence-based practice and rigorous evaluations to test the efficacy of existing programs and guide the development of new initiatives. Of course, as evidenced by this review, research findings tend to be complex and replete with qualifications and nuances that do not always lend themselves easily to advocacy and practice. Yet, if we are to champion this intervention strategy, we must be prepared to grapple with its complexities—even at the risk of learning that commonly deployed programs and practices do not always improve youth outcomes. To this end, prevention researchers have a central role to play in comparing methods of implementation, analyzing success and failure in different applications of mentoring, and effectively communicating these findings back to the field.