Introduction

Participatory action research (PAR) entails a cooperative, iterative process of research and action in which non-professional community members are trained as researchers and change agents, and power over decisions are shared among the partners in the collaboration (Israel et al. 1994, 2003). This approach intends to provide opportunities for community members—typically disenfranchised ones with little power to affect policies and circumstances that shape their lives—to work together to solve problems of concern to them, develop relevant skills, increase their understanding of their sociopolitical environment, and create mutual support systems (Zimmerman 1995).

While longstanding practice in community psychology, public health, adult education, and international development as a means of engaging marginalized populations in projects that address conditions of oppression, PAR is becoming increasingly common as a means of promoting urban young people’s engagement in improving their schools and communities (Cargo et al. 2003; Nieto 1996; Shor 1996) and improving youth development outcomes (Mitra 2004). For example, youth researchers have advocated for policy changes to improve neighborhood food access (Breckwich Vásquez et al. 2007) and reduce diesel bus emissions (Minkler et al. 2006); educated communities regarding childhood obesity prevention (Findholt et al. 2011) and the judicial system (Stovall and Delgado 2009); and participated in urban planning processes (Horelli and Kaaja 2002).

In “youth-led” or “youth-driven” participatory research programs (YPAR), young people are trained to identify major concerns in their communities, conduct research to understand the nature of the problems, and take leadership in influencing policies and decisions to enhance the conditions in which they and their peers live (London et al. 2003). In addition to improving community settings and resources—such as schools, neighborhoods, and agencies that serve youth—the YPAR process is intended to yield developmental benefits for the young people who participate. The potential benefits suggested by relevant theory and research include key attitudinal and behavioral aspects of psychological empowerment, such as the perceptions of control and efficacy in relevant domains; motivation to influence their schools or communities in constructive ways; decision-making and problem-solving skills; critical understanding of the sociopolitical environment; and participatory behaviors (Holden et al. 2005; Zimmerman 2000). Other gains observed in qualitative research on YPAR include increases in adolescents’ sense of purpose, perceived support from caring adults, and more positive attitudes towards education and school (Mitra 2004; Wilson et al. 2007).

Despite the promise of YPAR for fostering positive development among youth—particularly politically and economically disenfranchised youth of color—there has been relatively little study of its effects. One of the key challenges in evaluating the impact of YPAR is that—similar to other youth development and empowerment-focused interventions—the approach is generally practiced with small groups of young people who have volunteered or been selected by adult facilitators. It is certainly reasonable, from a program perspective, to recruit young people with existing interest and promise to become youth researchers. However, it is challenging to draw any inferences that youth actually benefit from their participation given the absence of a counterfactual condition and the high likelihood of selection effects entailed in picking youth who already show promise.

Thus far, only two published studies to our knowledge have included a control group in their evaluation of YPAR or similar empowerment-oriented programs for youth. The Adolescent Social Action Program used an experimental design to study the effects of this program based in Freirian and protection-motivation theory (Rogers et al. 1978) for reducing alcohol use and other risk behavior among low-income Native American, Latino, and European American youth (Wallerstein et al. 2004). The 7-week program engaged youth in dialogue with adults in prison and psychiatric hospitals with alcohol, violence, and other problems; this was followed by a social action phase in which youth advocated for healthier schools and communities. Although positive effects were found for perceived control, empathy, and self-efficacy for protection of family and friends, the authors reported that the impact of the program was limited by weaknesses in measurement and challenges in implementing the open-ended curriculum by college student facilitators. A recent quasi-experiment investigated the effects of participating in a youth research program focused on reducing adolescent risk behavior for urban teens; the program was conducted over the summer with an 8 month action follow-up (Berg et al. 2009). The study found reductions in marijuana use among the 114 youth research students, compared with control students; there were also increases in collective efficacy and non-significant decreases in alcohol use and the number of sex partners.

Goals and Design of Present Study

The goal of the present study was to investigate the effects of participating in YPAR on diverse urban teens, using a within-school experimental design intended to maximize the internal and external validity of the evaluation. Our primary focus was on the impact of YPAR on psychological empowerment, specifically on adolescents’ motivation to influence their schools and communities, sociopolitical skills, perceived control, and participatory behavior. These dimensions represent key potential outcomes of YPAR as an empowering process that guides young people in conducting action research to address problems of concern to them in their schools or communities. We also tested the potential effects of the intervention on self-esteem, primarily to differentiate these general psychological effects versus more specific dimensions of psychological empowerment that are theoretically linked to youth-led participatory research.

We utilized a within-school, within-teacher experimental contrast between “YPAR” (treatment) and “direct service youth development” (comparison). The direct service youth development class trained young people to serve as peer educators and mentors to their classmates. Both the treatment and comparison classes received the same initial 6-week training from their teachers that included intensive activities focused on team-building, communication, and examination of school and community problems from a social justice perspective. This evaluation design is thus highly stringent in its use of (a) experimental assignment of classes to condition and (b) another youth development intervention as the comparison condition. Both conditions receive a youth development “treatment;” the YPAR differs from the direct service in terms of YPAR’s emphasis on research training and broader change efforts as opposed to direct peer-to-peer helping. Our hypotheses were that participation in YPAR would be associated with increased levels of psychological empowerment, constituted by dimensions of motivation to influence their schools and communities, sociopolitical skills, perceived control, and participatory behavior.

Methods

Our present investigation is part of a five-year, mixed-methods intervention study in which YPAR projects were implemented and evaluated in collaboration with five public high schools and a community-based organization (Peer Resources) that co-funds and provides supervision for elective peer mentoring classes at these sites (see Ozer et. al 2008). Certified classroom teachers (“teacher-facilitators”) coordinated the YPAR projects in a daily elective class for two semesters of the school year, with technical assistance from their supervisor and the university team. The present study is focused on assessing individual-level quantitative outcomes for young people who participate in the YPAR projects. We also gathered extensive qualitative data from students regarding their experiences of the YPAR projects via interviews and participant-observation (described below) as part of our measure development efforts and assessment of school-level effects (Ozer & Schotland 2011; Ozer & Wright in press). Excerpts from these qualitative data will be presented here to help interpret and illustrate the quantitative findings.

Participants

Participants were 401 youth (178 in treatment group; 223 in comparison group) from 5 high schools in a major metropolitan school district in California. The sample was ethnically diverse with 35 % of the adolescents of Asian American ethnicity, 31 % Latino/Hispanic, 14 % African American, 7 % White or European American, and 10 % from other minority groups such as Native American or Arab American. The overall sample was 65 % female and 35 % male with an average age of 16.3 years (SD = 1.1).

At the start of the school year, participants were recruited from elective courses that were part of a Peer Resources program at their school; the curriculum for these classes trained students in helping other peers through activities such as mentoring and peer education. At least two classes at each school were randomized to either the experimental or comparison condition; additional control classes were added to the experiment when feasible to boost the sample size. Both the experimental and the control classes started the semester with the same 6-week introductory Peer Resources curriculum focused on team building, communication skills, and principles of social justice. The experimental class then received training to conduct research on a topic of concern to them; the comparison condition spent most of the semester doing direct service peer mentoring and peer education with no research component.

All students enrolled in these classes (a total sample of 61 classes across experimental and control conditions) were invited to participate in the study. Ninety-six percent of students agreed to participate and obtained positive parent consent. The majority of students (n = 373) completed the follow-up assessment at the end of the semester, a retention rate of 94 % in the experimental condition (n = 167) and 92 % in the control group (n = 206). Depending on their class schedules and the distribution of academic and elective credits permitted at the schools, some students had the opportunity to stay in the Peer Resources classes for two semesters in a row, thereby receiving greater exposure to either the experimental or comparison condition. Approximately half of the students in the experimental group (n = 90) and control (n = 107) remained in the study for a second semester. All of these students completed the follow-up 2 assessment at the end of a year. We compared the students who participated in the study for one semester versus two semesters using independent sample t-tests. We found that those who participated for two semesters were slightly older (by 2 months; t = 2.04, df = 400, p = 0.04), and were less likely to have participated in church or religious leadership activities before enrolling in the study (t = 2.09, df = 402, p = 0.04). There were no differences, however, in terms of grade in school, gender, or participation in other community service or student government leadership activities prior to enrolling in the study across those who participated for one versus two semesters. There was no significant difference in the proportion of participants from the experimental or control condition who stayed in the study for a full year; the two conditions were also similar in terms of baseline scores on all outcome measures.

School Sites

Participants attended five high schools in a large, urban area in California. Schools were diverse in terms of size, ethnic diversity and achievement level. School size ranged from approximately 200–2,000 students, with Latino/Hispanic students comprising the majority at two sites, Asian-American students comprising the majority at two sites, and African American students comprising the majority at the fifth site. Schools ranged from 577 to 938 on the 2010 Academic Performance Index (API), a statewide school-level indicator of student performance (range 200–1,000), and enrolled between 35 and 57 % of students qualifying for free and reduced lunch, an indicator of very low family income.

Overview of Projects

The problems studied and addressed in the YPAR projects were decided by the students with facilitation from their teachers, and ranged according to the school and cohort. Topics included the prevention of school drop-out; smoothing the transition to 9th grade; stress related to family, academics, or peers; improving the school lunch; cyber-bullying; sexual health; safety and hygiene in the school bathrooms; improving teaching practices to engage diverse students; and improving inter-ethnic friendships at the school. Each project lasted at least one semester; some projects continued for the entire year. At two sites, the subsequent year’s cohort decided to continue with the same topic. The curriculum used by the teachers was adapted by Peer Resources, based on existing YPAR curricula (London 2001; Sydlo et al. 2000). In the issue selection phase, the teacher-facilitators led multiple class sessions intended to help students decide on a topic as a group, and to pick topics that were within the scope of feasible action. The issue selection process started with students’ creation of an “issue tree,” consisting of branches of “leaves,” i.e. post-it notes each representing a problem that were organized in terms of domains and hypothesized “root” causes. These issues were generated by the students, based in their experiences and in initial informal interviews with students, teachers, and parents. In structured activities, students advocated and voted for their choice of topics, with the facilitator assisting the group in respecting differing views and working together to achieve consensus. With training and guidance from their teachers and the university team, students then engaged in a research phase to study and understand the problem, using a range of survey, interview, observational, and multi-media approaches for data collection as determined by each group of student researchers. In the action phase, the teacher-facilitators helped students to identify specific and feasible actions that they could take within the time frame to start to address the problem, with the understanding that it was likely beyond the scope of the project to fully solve it. The activities in the YPAR and the control classes were documented via a combination of interviews with teachers, focus groups, and observations by the research team.

Procedure

Data Collection

Research team members visited each participating class at the beginning of the semester to distribute parent consent/student assent forms. Students received a small incentive (water bottle or key chain) for returning the form regardless of whether they agreed to participate in the study or not. Consistent with district policy, only students returning consent forms completed surveys and interviews as part of this study; however, students remained in the Peer Resources class itself regardless of participation in the study. After consent forms were collected, one to two graduate-level research team members administered the 45-min survey during class time and assisted as needed. Students provided identifying information on the cover page of the survey, which was detached and handed in at the beginning of the administration period; follow-up surveys were linked through a unique barcode.

Quantitative Measures

The student survey measure assessed four dimensions of psychological empowerment and self-esteem. All measures provided a 4-point Likert-Scale response option. This scale eliminates the option of a middle or neutral choice which we have found in prior research to be a common default choice among this population (Ozer & Weinstein 2004, Ozer 2005, Ozer et. al 2008).

Psychological Empowerment

In prior research (Ozer and Schotland 2011), the research team developed and tested a psychological empowerment scale to assess four core conceptual areas: General socio-political skills (8 items; α = 0.81), motivation to influence one’s school or community (4 items; α = 0.80), participatory behavior (8 items; α = 0.83), and perceived control (6 items; α = 0.80). All four scales showed positive correlations with each other (0.59–0.66) and the other key measures of adolescent functioning assessed in this survey (0.32–0.72). Sample items include: “I can usually figure out how to get an adult to see my point of view, even if they don’t agree with me” (sociopolitical skills), “I want to have as much say as possible in making decisions in my school” (motivation to influence), “I have spoken with other students about issues that I want to improve at the school” (participatory behavior), and “Students have a say in what happens at this school” (perceived control). Please see Table 1 for scales and all items.

Table 1 Items in psychological empowerment scales
Full size table
Self-Esteem

The 8-item scale from the Self Esteem Questionnaire (SEQ; α = 0.75) was used to tap into global feelings of self-worth, such as “I am happy with myself as a person” (DuBois et al. 1996). The SEQ was developed for use with middle and high school students and has demonstrated excellent psychometric properties.

Qualitative Data

We conducted focus groups (40 in total) with the entire YPAR class—minus 1–2 absences per class—at the end of each semester. These occurred during class time; students were separated into 2 groups. These interviews started with open-ended questions to elicit the students’ “story” about the trajectory of the project and the decisions they made regarding their topic, research methods, and actions. We then asked questions regarding their perceived power, what they learned from the project, and their perspectives on their interactions with school and community stakeholders.

Data Analyses

The quantitative data were analyzed using Generalized Estimating Equations (GEE) in Stata Version 11 (StataCorp 2009), this analytic approach is robust with respect to the clustered, hierarchical structure of our data in which students are nested within classrooms and teachers at schools; we replicated our analyses in Stata using multiple regression for clustering data and found the same results. The qualitative data considered in the present study were analyzed consistent with guidelines for qualitative analysis (Miles and Huberman 1994), using verbatim transcriptions of the students’ group interviews. Transcripts were coded by at least two members of the research team (the study authors and a research associate with a B.A. degree and extensive experience in qualitative coding) to establish consistent application of codes to data. For the present study, we categorized students’ reports of their areas of growth and experience, based on the key dimensions of psychological empowerment assessed by the quantitative measure: Socio-political skills, motivation to influence school or community, participatory behavior, and perceived control. Here, the qualitative examples of these dimensions are used to help illustrate and contextualize the quantitative findings based on the youth’s perspectives, and will be presented in the Discussion.

Results

Descriptive Analyses

Despite randomization at the classroom level within schools, there were differences between youth in the experimental YPAR and control conditions (see Table 2). Students in the experimental group were slightly older, with more females and lower standardized test scores at baseline. The most marked difference between the two groups was that the experimental condition had a greater proportion of African American youth and a smaller proportion of Chinese American youth than the control; the reason for this is that the control classes were larger at the high schools with a higher proportion of Chinese American students. We conducted initial analyses to assess if any demographic or academic achievement variables were significantly associated with our dependent variables and should be included as covariates in our multivariate analyses. None of these variables with the exception of the students’ grade level was significantly associated with our dependent variables; thus only students’ grade level was included as a covariate in order to conserve degrees of freedom.

Table 2 Respondent characteristics at baseline
Full size table

Multivariate Analyses

Our hypotheses were that participation in the experimental YPAR condition would be associated with higher levels of psychological empowerment, in comparison with the control “direct service” condition. These hypotheses were tested using GEE analyses robust to the hierarchical structure of our data; results are presented in Table 3. For 3 out of 4 outcomes, our results confirmed our hypotheses in that we found modest, statistically significant effects for the treatment versus control condition at one or both follow-up time points: For sociopolitical skills at follow-up 1 (−0.081, p < 0.05, 95 % CI [−0.15, −0.01]) and follow-up 2 (−0.123, p < 0.01, 95 % CI [−0.19, −0.06]); motivation to influence their schools and communities at follow-up 1 (−0.104, p < 0.01, 95 % CI [−0.16, −0.05]); and participatory behavior at follow up 2 (−0.146, p < 0.01, 95 % CI [−0.23, −0.06]). We found no significant effects for perceived control at school or self-esteem at either time point (Table 3).

Table 3 Mean outcome scores and effect size for the change mean score between baseline and follow-up assessments
Full size table

Discussion

This study utilized a stringent experimental evaluation to test the contrast between youth-led participatory action research and a “direct service” condition with respect to the psychological empowerment of urban teens. Direct service programs such as peer mentoring or peer education are an approach commonly utilized in secondary schools. Our hypotheses that participatory research would be associated with increases in multiple dimensions of psychological empowerment were partially confirmed. Statistically-significant YPAR program effects were found for participatory behavior, socio-political skills, and motivation to influence their schools and communities; these effects were modest in magnitude. There was no YPAR program effect for perceived control or self-esteem. Below we consider our pattern of effects, drawing on qualitative data to illuminate and contextualize our findings in terms of the young people’s experiences of YPAR.

Our finding that participating in YPAR was associated with adolescents’ increases in socio-political skills and motivation to influence their schools and communities makes conceptual sense in light of the more explicit political focus of YPAR relative to direct service peer education. In the YPAR condition, students’ activities were focused on collective change efforts. Qualitative data from the focus groups in which students reported on areas of new learning help illustrate how these dimensions of psychological empowerment were experienced. Learning to work together with other students to do research and make change was a strong theme, e.g. “…we learned how to build community within our class and we also learned how to attempt to implement change, certain steps and procedures we have to follow,” and “everybody was like giving different ideas and like everybody wanted to fix different problems from the school and we all had to agree to it and we couldn’t but we finally did.” Students’ comments further reflected enhanced motivation to influence their schools and make a difference on a range of issues, e.g. “I think that maybe if we like spread the word to people like outside of this class maybe they’ll listen to us. And by the survey they’ll like agree with some of our things; maybe they would stop failing classes and think about their future or whatever.”

From a developmental perspective, reported gains in teenagers’ motivation and skills to influence their schools and communities are meaningful in that they can enhance positive identity formation and adolescents’ sense of purpose (Damon 2003). Recent large-scale survey research indicates that adolescents’ commitment to community improvement and social justice is enhanced by parents’ discussion of current events with their teens (Diemer 2012); the YPAR process studied provides a parallel process of change-oriented socialization in the school context. YPAR may be particularly salient for youth of color whose process of identity development entails the negotiation of social position and racism (García Coll et al. 1996), as well as their assessments of themselves and of the risk and protective factors that affect their lives (Spencer et al. 1997, 2003; Ozer et. al 2010). In addition to changes in self-assessments, the YPAR training pushed youth beyond individual-level explanations of problems faced by their communities to consider and investigate broader causal factors. As it was not feasible to follow up with the students in the ensuing years after their participation in YPAR and the comparison direct service condition, we make no claims regarding the effect of these experiences on their life trajectories. Future research should investigate the long-term impact of these developmental effects on multiple domains of young people’s lives, including mental health as well as educational and occupational outcomes.

Our finding of no significant differences in adolescents’ perceived control in their schools and communities between the YPAR and direct service conditions may emphasize a key dilemma in empowerment-oriented interventions that engage young people in working to make changes in the settings in which they live. Perhaps students in the YPAR conditions did not report any increases in perceived control because—in some cases—their experience taught them just how long and hard it actually is to make a dent in problems that concern them. Focus group data provide support for this view, as multiple students commented on the barriers they faced in their projects while at the same time emphasizing that it is important to make the effort, as in this response to what the student learned from YPAR: “I guess, uhmm, just learning that students could make a difference. We haven’t really made a difference but I mean the process of how to do all that stuff.” While extensive efforts were made by teachers to promote the likelihood that students would pick problems to address in which they could experience “small wins” within the semester or year time frame (Weick 1984), it is likely that students’ experience as action researchers as well as their initial Peer Resources training promoted a realistic understanding of the often-formidable barriers to change. In the YPAR projects studied here, barriers to change ranged from those of a political nature—such as taking on issues like the dress code or academic requirements that engendered resistance to action among school administrators and staff; or larger policies about school lunch determined at the district level; to practical constraints such as the inability to work on an anti-cyber-bullying Facebook page while at the school site. There were also numerous resource and time constraints linked to the school calendar and the demands on teachers’ time and focus due to budget cuts. Further, uncertainty about teachers’ job status and possible school closures created underlying conditions that impeded sustained actions at some schools.

Strengths and Limitations of Design

The design of the present study represents key advances in the empirical testing of YPAR in specific and youth development programs more generally, and confronted multiple challenges inherent in evaluating participatory research. First, designs utilizing a control condition and random assignment are rare in the evaluation of youth development and other empowerment-oriented practices (Berg et al. 2009; Wallerstein et al. 2004). As noted previously, selection effects are difficult to combat in studies of extracurricular afterschool or summer programs that are likely to recruit and attract youth already demonstrating activist “potential” or commitments. Here, participants engaged in YPAR and the control direct service peer education class as electives during the school day, rather than as an extracurricular program. We were fortunate to build an excellent partnership with the Peer Resources program to implement a design that embedded an RCT within the existing structure of their curricula at multiple schools. Students included in our study, in both conditions, were likely more service oriented than the general student population in that they had chosen to participate in peer education but they had not chosen roles as youth researchers or agents of broader change in their schools and communities.

Our collaborative arrangement enabled the study to engage the participation of a large number of experimental and control cohorts in a manner that was able to be sustained over time in the school settings. The extent to which RCT’s can lead to artificial, overly-standardized tests of complex and dynamic interventions is a source of legitimate concern in the community psychology and public health literatures (Hawe et al. 2004). Here, we submit that the embedding of this experiment within the school structure succeeded in maintaining the integrity of the complex and flexible YPAR intervention for the dozens of elective classes who participated at diverse school sites. We interpret the fact that Peer Resources teachers have chosen to continue to conduct YPAR projects at these sites after the end of the research study as further evidence for this claim.

The embedding of this experiment within real-world practice was not without its tradeoffs, however. As noted earlier, our experimental contrast was particularly conservative because our control condition also represented a potentially impactful intervention (peer education). Both of these leadership roles may have contributed to the similar increases in self-esteem reported for both groups. Further, students in both conditions participated in an initial 6-week Peer Resources training that engaged students in interactive workshops; these workshops encouraged team-building across diverse students, developed communication skills, and introduced analyses of social and health problems from a social justice perspective. Thus, although the direct service activities in the control condition were not overtly political, the initial 6-week curriculum did include training with a larger social and political emphasis. That both conditions constituted a youth development intervention was ideal from the standpoint of assessing more precisely the unique contributions of YPAR for the psychological empowerment of the teens who engage in it, but made for a weak experimental contrast that likely led to the modest effect sizes. This represented a disadvantage that stemmed from working with the existing structure of practice because the initial 6 weeks of the curriculum was considered core to the Peer Resources curriculum. Thus, it was not an option to deny this training to the control classes as this would have violated the philosophy of our collaborating partners.

Other limitations should be noted. Although our recruitment and randomization of intact classes likely reduced selection effects, the classroom-level randomization did not yield equivalent groups with respect to ethnicity and academic achievement—the YPAR group was more likely to be African-American and to have lower GPA’s than the control group. We attribute these differences to disproportionalities in the ethnic and achievement distribution in the school district and in our sample of schools. While the YPAR classes tended to be comparable in size at all sites (averaging about 12 students per class), our comparison classes were substantially larger at over-enrolled rather than under-enrolled high schools: The over-enrolled high schools were overall higher-achieving high schools with a higher proportion of Chinese American students. On the positive side, this suggests that the YPAR group was more educationally-disadvantaged and makes the gains that were found for the YPAR condition that much more meaningful and hard-won. It is important to note that none of the ethnicity or academic characteristics of the students were associated with our psychological empowerment variables in terms of main effects. We cannot rule out the possibility, however, of interactions between student characteristics and treatment condition that were not examined as we did not have sufficient power to test these interactions. Future research with a larger sample of classes and students should be undertaken to address this issue.

Although we interpret the conservative experimental contrast as likely accounting for the modest effect sizes found here, alternative explanations should be considered. For example, challenges in implementing YPAR in schools—such as the limitations of classroom time and the school calendar (Ozer et. al 2010)—may have undermined the intensity of the YPAR project at some sites. On the other hand, conducting YPAR within the school settings enabled almost-daily progress over a sustained period of time for an intact group of students, taught by a certificated teacher with extensive youth development training. In sum, this study provides evidence that YPAR can be implemented in a large-scale fashion over multiple cohorts in urban public schools, with beneficial effects on the psychological empowerment of diverse students who participate. The study also reflects an innovative embedding of an experimental design within the practice of participatory research in intact school settings, an approach to building capacity while enhancing rigorous evaluation that is firmly aligned with community psychology principles.