Many nations around the world lack access to efficacious and culturally congruent substance abuse prevention programs. Regardless of their prevention needs, low- and middle-income countries tend to have limited or no access to evidence-based interventions (EBIs) (Heikkilä et al., 2021; Mejia et al., 2020; Parra-Cardona et al., 2021). A meta-review of existing prevention programs with robust evidence of effectiveness identified only one that was originally designed and tested outside of the developed north (Tremblay et al., 2020). EBIs exported from developed nations into low- and middle-income countries often do not achieve their prevention goals due to a lack of cultural adaptation to the new context (Flores-Garza et al., 2021). There is an unmet need to culturally adapt programs to augment their effectiveness with diverse populations (Castro et al., 2021; Griner & Smith, 2006).

The rapidly increasing rates of substance use among youth in countries like Mexico call for the translation of evidence from other countries to increase the effectiveness of in-country prevention efforts (Vázquez et al., 2019; Villatoro et al., 2016). Mexico and the USA share a 1954-mile international border, 48 ports of entry, and large daily transborder movements of people in both directions. Due to their strong historical, cultural, geographic, and economic connections (Romero, 2008), the USA and Mexico are well positioned to jointly address substance abuse prevention gaps in Mexico.

Youth Substance Use in Mexico

The incidence and prevalence of substance use in Mexico have been increasing in the general population and especially among youth (Puyana et al., 2017), a situation that requires urgent attention by researchers, policymakers, and health providers. Recent national household surveys have reported an increase in alcohol and other drug use among youths aged 12 to 17 years (Villatoro et al., 2017). Among these adolescents, 39.8% had consumed alcohol at least once in their life, 28% had done so in the last year, and 8.3% engaged in heavy or binge drinking in the last month. As for tobacco, 4.9% had smoked cigarettes in the last 30 days, 6.4% had used illegal drugs or misused prescription drugs in their lifetime, and 3.1% had used illegal drugs in the last year. Surveys among middle and high school students in Mexico also report prevalent lifetime substance use (Villatoro et al., 2016), including alcohol (53.2%), tobacco (30.4%), and other drugs (17.2%). Both the general population and school surveys highlight the magnitude of the drug use problem among Mexican adolescents and the need to implement EBIs to prevent or delay substance use in this population.

keepin’ it REAL and Mantente REAL in Mexico

Although some school-based prevention programs exist in Mexico, in most cases they lack evidence of efficacy (CONEVAL, 2015). Prior pilot studies in selected Mexican cities and in other Spanish-speaking countries such as Guatemala, Spain, and Uruguay, produced significant desired prevention effects using Spanish language versions of keepin’ it REAL (kiR), an EBI for middle school students. They include a reduction of adolescent substance use, preserving or strengthening norms and attitudes against substance use, and an increase in the use of effective drug resistance strategies (Cutrín et al., 2021; Kulis, García-Pérez et al., 2021; Kulis et al., 2019; Marsiglia et al., 2015, 2018). These pilot studies not only showed the cross-national relevance and resonance of the kiR prevention model with Spanish and Latin American youth. They also suggested ways to increase the intervention’s effectiveness by aligning it with local social and cultural contexts.

These earlier studies provided a foundation for a more systematic cultural adaptation of kiR for Mexico, renamed Mantente REAL (MREAL) (Marsiglia et al., 2019), and its subsequent large-scale randomized controlled trial (RCT) in Mexico’s three largest cities. The RCT documented MREAL’s efficacy in preventing substance use and violence among early adolescents (Kulis, Marsiglia et al., 2021). The adapted version for Mexico retained the core components of the original kiR program: training in risk assessment, communication competence, and use of a repertoire of effective drug resistance strategies. Although previous pilot feasibility studies across Mexico had translated the kiR manuals into Spanish, the Mexican and US research teams in the most recent RCT made additional linguistic and cultural adaptations to increase comprehension and relevance. The culturally adapted MREAL version represents city, neighborhood, school, and family contexts where Mexican adolescents encounter substance use offers, often facing high pressure, intimidation, and threats. The adaptation also addressed the influence of gender roles in substance use and violence as a strategy to resist substance offers (Cutrín et al., 2022; Nuño-Gutiérrez et al., 2022). The adaptation process produced updated manuals and videos, which allowed students to explore alternatives to responding to substance offers in angry or violent ways, and equipped them with skills to assess and leave threatening situations (Marsiglia et al., 2019).

The Current Study

Beyond assessing the efficacy and effectiveness of MREAL, the present study investigates its social validity, and the program’s feasibility, acceptability, and utility (Humphrey & Wiglesworth, 2016). Demonstrating the social relevance of the goals, procedures, and impact of an intervention is essential for its successful implementation and sustainability (Soneson et al., 2020). Assessments of social validity are based on participants’ and implementers’ perceptions of the social relevance of the intervention goals, the appropriateness of its strategies, and the social significance of its impact (Murta et al., 2021). Social validity is stronger when community-based providers can implement an intervention with fidelity in real-world settings such as school teachers in classrooms (Horner et al., 2005).

MREAL was successfully implemented in middle schools across Mexico and the present study analyzed its social validity across three validated constructs (Pearson et al., 2020; Weiner et al., 2017): (1) feasibility is the degree to which an intervention can be successfully implemented within a given setting. (2) Acceptability is the perception among stakeholders that the intervention is agreeable, comfortable, or satisfactory. (3) Utility or appropriateness is the perceived fit, relevance, and impact of the intervention for the setting, providers, consumers, and the targeted problem. These constructs are benchmark preconditions for interventions to attain desired outcomes (Martinez et al., 2014).

We hypothesized, from a social validity perspective (Halle, 2019), that the culturally adapted MREAL version of the intervention would be feasible, well accepted and appropriate, and of utility in the Mexican context (H1). Students receiving the adapted MREAL version will find the curriculum to be more acceptable and of greater utility than students receiving the non-adapted kiR (H2). To incorporate the perspectives of different constituencies involved in MREAL and kiR, we analyzed reports on the implementation of the programs from participating students, teacher-implementers, assistant principals, and the field research teams’ fidelity and implementation observations.

Method

The study used qualitative and quantitative methods to assess the social validity of the intervention. The voices of participants and implementers were captured predominantly through qualitative methods (Leko, 2014). The quantitative data complemented those findings by confirming themes that emerged from the qualitative findings.

Participants

The sample of students, teachers, and school assistant principals came from the three largest metropolitan areas of Mexico: Mexico City, Guadalajara, and Monterrey. With the aid of the sampling statisticians responsible for national surveys of substance use in Mexico, a representative sample of public middle schools (secundarias técnicas) was drawn from all those in each metropolitan area, stratified by city (12 schools each) and whether school sessions were held in the morning or afternoon (equal numbers). The 36 selected schools were randomized within strata into three intervention conditions: adapted MREAL, original kiR (translated into Spanish but not culturally tailored), or treatment-as-usual control (see Kulis, Marsiglia, et al., 2021, for CONSORT details on the population-based sample selection). State-level secretaries of public education agreed to be part of the study and provided authorization to recruit schools. Local research teams visited school principals, and the principals recruited between 2 and 5 regular teachers of 7th grade students to be implementers.

This study limited analysis to surveys completed by students attending the 24 schools assigned to MREAL or kiR (N = 3607), teachers implementing the programs (N = 73), and assistant principals (N = 18) participating in in-depth interviews after the implementation. Participating students were enrolled in the first year of middle school, 49% female and with a mean age of 11.9 years. The teachers implementing MREAL or kiR in their classrooms were 64% female, 42.9 years old on average, and averaging 15 years of teaching experience. The participating assistant principals were 79% female and were 52 years old on average.

Description of the Interventions

Both kiR and MREAL are based on life skills training (e.g., emotional recognition and regulation, communication competence and assertiveness, risk assessment, and problem-solving skills), and teach specific behavioral skills to manage social pressure to engage in substance use or other risky behaviors (Gosin et al., 2003). The behavioral skills are strategies used most commonly by youth who successfully resist using substances (Kulis et al., 2011; Marsiglia et al., 2009), which correspond to the acronym REAL. Refuse refers to saying “No” verbally or non-verbally in a direct and respectful manner. Explain is to provide a reason to decline the offer. Avoid refers to not going to places, situations, or with people where alcohol or drugs might be offered available. Leave consists of exiting those situations.

The MREAL version of the intervention for Mexico incorporates culturally specific values and communication styles identified in the literature and in the focus groups with students and teachers (Marsiglia et al., 2019). It has two more lessons than the standard Spanish-language version of kiR, with additional violence and gender specific content.

Implementation

The binational research team conducted a 2-day-long teacher training in each city to assist implementers in delivery of the manualized curriculums with fidelity. The original curriculum specialist and master trainer of kiR conducted training-of-trainer sessions in preparation. Trained members of the local team in each Mexico site and bilingual members from the US-based team jointly implemented the training, which included a review of the underlying prevention approach, and practice in leading each lesson in an interactive manner. Teachers received a curriculum manual and copies of an accompanying student manual with in-class and homework activities for all their students. Teachers delivered the curriculum lessons between September and December of 2017, except in Mexico City, which finished 2 months later after delays caused by earthquakes that shut down the schools.

Teachers were generally able to deliver each lesson of both versions of the intervention, one lesson per week in one 45-min session. The four-core lessons covering the REAL strategies contain educational videos illustrating each strategy, which generated much discussion in class and sometimes required more time and a second session.

Data Collection Procedures

Information about feasibility, acceptability, and utility was collected through focus groups with teachers, interviews with school assistant principals, student surveys, fidelity checklists, and field notes on the implementation process gathered by field researchers.

Teachers’ Focus Groups

Teachers (N = 6) were from schools that implemented an interim version of MREAL in a pilot test (2015/2016) that was part of the adaptation process (the final version had only small changes). Teachers in the pilot were invited to participate in focus groups to provide information about the implementation process and opinions about the curriculum to help validate the adaptation of the intervention to the Mexican context. All provided oral informed consent, and they were assured their views would remain confidential by anonymizing transcripts. Focus groups were conducted in the summer of 2016 after the pilot study. The interviewers for the focus groups were three PhD research team members who received 5 h of training on how to conduct focus groups.

Teacher and Assistant Principal Interviews

Teachers (N = 12) and assistant principals (N = 18) of schools that implemented MREAL and kiR in the RCT (2017/2018) were interviewed after finishing the implementation. Local research team members, all with postgraduate training in psychology or nursing, went by appointment to the campuses to conduct the interviews. The teachers and assistant principals read and signed an IRB-approved informed consent. The interviews were conducted in a private office, audio-recorded, and lasted up to 60 min. The structured interview guide addressed their opinions about the quality of the MREAL/kiR curriculum and implementation processes, protective and risk factors in the students’ school and neighborhood environments, perception of youth substance use, problems on campus, and the need for prevention programs.

Student Surveys

Students in schools that implemented MREAL and kiR completed post-test surveys in the spring of 2018, about 3 months after their teachers finished delivering the last curriculum lessons. Students answered a series of questions evaluating the prevention program they received, including items assessing acceptability and utility. Parents provided active consent for their child to participate in the survey, and students read and signed an assent. Survey administrators informed students that completing the questionnaires was voluntary, and answers would remain confidential.

Fidelity Observations

Observers from the local research teams visited the classrooms of each implementing MREAL and kiR teacher three times to rate fidelity to the curriculum manual, including lessons which covered the same topic (a particular REAL resistance strategy) in both versions of the program. The observers completed standard forms with closed-ended ratings of various aspects of fidelity and student involvement, open-ended comments about implementation issues, and feedback provided to implementer teachers.

Field Notes

The field researchers kept a field implementation diary (bitácora) where they noted any implementation-related observations. It included impressions and observations at all levels of the ecological system in which the intervention was being implemented.

Human Subjects Protections

The responsible ethical research review committees for the Mexico and US research teams approved the study’s research design and all qualitative and quantitative data collection procedures, with no reports of adverse events. Teachers, students and their parents, and assistant principals gave their informed consent.

Qualitative Measures

Data sources included transcripts from the teachers’ focus groups, transcripts from the assistant principals’ interviews, and implementation narratives and blogs (bitácora) collected by the research teams in each city.

Feasibility

The viability of the intervention was investigated through the following factors that directly affect program implementation (Bird et al., 2014; Soneson et al., 2020). (1) Intervention fit: perception of the program as relevant to address the prioritized goals (substance use and violence) and appropriate to the target population (7th grade students). (2) Cost and resource implications: human and material resources needed to implement the program such as essential materials, technical issues in using the materials; staff and/or facilitator training prior to implementation; on-going supervision to aid the implementation, support the facilitation and/or increase the motivation of facilitators. (3) Implementation characteristics: complexity of delivery of the curriculum, flexibility of the curriculum to be adapted (ability to tailor it to fit schools’ and students’ needs), helpfulness of the manual, adequacy of time to implement the program. (4) Practicality: engagement in delivery by teachers and participation by students (Lohan et al., 2018). (5) Fidelity: results of adherence assessments and accuracy of implementation (Lopata et al., 2012).

Acceptability

The degree of acceptance of the program was explored through factors relating to its perceived appropriateness, fairness, reasonableness, and intrusiveness (Gadke et al., 2021; Nastasi & Truscott, 2000). (1) Satisfaction: overall satisfaction with the program. (2) Comfort with topics and activities: degree to which participants and implementers liked curriculum topics and activities. (3) Understanding of content: how well participants and implementers understood concepts and activities in the curriculum. (4) Willingness to use the intervention in the future, for implementers and administrators.

Utility

The degree to which the program is perceived as useful in producing desired individual and social changes (Murta et al., 2021) was explored through the following factors: (1) knowledge: information acquired and perception of useful contents; (2) applicability: level of connection to participants’ life, and authenticity of content; (3) impact: changes in the lives of participants and/or changes in the work of implementers.

Quantitative Measures

Closed-ended questions in the fidelity observation forms and the students’ post-test surveys were the main sources of quantitative data.

Feasibility

(1) Practicality: observers rated three closed-ended items to assess student involvement with the curriculum (i.e., participation, engagement, and attention to the videos). (2) Fidelity: observers rated implementer fidelity with measures of whether the teacher completed each of the nine curriculum items that were specific to the observed lesson (e.g., introduced a particular topic, completed an activity in the student manual). These were combined into an index counting the number completed, from 0 to 9. Additional single items assessed how well the teacher was prepared for the lesson, informed about its content, followed the lesson plan, gave clear instructions, and motivated students to participate (1 = not at all, 2 = somewhat, 3 = mostly, 4 = completely). Another item rated the teacher’s classroom management of group processes in the lesson (1 = poor, 2 = well, 3 = excellent). Two more items assessed whether the pace of instruction was appropriate (rather than too slow or too fast), and whether the teacher added content not in the manual (0 = no, 1 = yes).

Acceptability

(1) Satisfaction: in four survey questions, students reported how much they liked the prevention program they received (1 = did not like it at all, 2 = did not like it much, 3 = liked it, 4 = liked it a lot), including its various components (videos, homework, classroom activities) and overall. (2) Comfort with topics and activities: in four other questions, students reported whether the program was interesting, fun, easy to pay attention to, or boring (1 = strongly disagree, 2 = disagree, 3 = agree, 4 = strongly agree).

Utility

(1) Knowledge: students reported whether the program gave them useful information (1 = strongly disagree to 4 = strongly agree), and how much they learned from it (1 = not at all, 2 = a little, 3 = some, 4 = a lot). (2) Applicability: students assessed the authenticity of content in three items, whether it was “like my life,” “like youths I know,” and like situations that students they know get into. Two additional items assessed whether situations in the curriculum and the characters in the videos seemed “real” (1 = strongly disagree to 4 = strongly agree). (3) Impact: students reported whether they talked about the program with various people in their social network: parents, siblings, cousins, other family members, friends, and others. These were assessed separately (0 = no, 1 = yes) and as a count of the number of different categories of people they talked with (0 to 6).

Analysis Protocols

We followed a structured qualitative approach that combined discovery and imposition (Berniker & McNabb, 2006), and open to discovery of new themes and understandings (Glaser & Strauss, 1967), but aware that we had a priori categories and a specific purpose (Evered & Lewis, 1981). Thematic analysis was applied to identify codes and analyze patterns (themes) (Braun & Clarke, 2006) among the participants’ experiences with, and beliefs about, the MREAL program. Guidelines and recommendations of Gibbs (2018) and Rabiee (2004) were followed to organize and interpret the data throughout all stages of qualitative analysis (i.e., familiarizing with data; identifying a thematic framework; creating, indexing, comparing and hierarchically organizing analytical and theoretical codes; considering internal consistency across informants; and the frequency and extensiveness of comments). The initial coding scheme was developed from a literature review of relevant aspects of feasibility, acceptability, and utility. Variables were selected upon agreement of three researchers based on the literature reviewed and the codes were generated from keywords that define the selected variables. To ensure rigor and trustworthiness, the research team engaged in multiple strategies, including prolonging engagement, reducing researcher bias, and triangulating the data (Morse, 2015).

Focus group data were transcribed verbatim in Spanish by the local team responsible for data collection. Following the guidelines of a constant comparison analysis (Gibbs, 2018), two research team members conducted the thematic analysis of the transcripts independently, and met to review the coding process to increase consistency and decrease the likelihood of interpretive errors and misrepresentation of the data. When discrepancies in coding arose, they were resolved by clarifying nuances in semantics using a keywords-in-context procedure (Onwuegbuzie et al., 2009). The final results of the thematic analysis were shared with additional team members who were directly involved with the data collection process in order to solicit feedback and increase trustworthiness of the findings.

Assistant principals’ interview data were also transcribed verbatim in Spanish by the local research team responsible for data collection. The analysis procedure for the interviews followed these specific steps: (a) identifying questions that related to the prevention program and its implementation; (b) searching for keywords in the transcripts (i.e., “Programa,” “Mantente,” “Real”); and (c) selecting the relevant excerpts pertaining to the program.

Quantitative analysis was conducted in SPSS 25 to produce descriptive statistics and assess differences between MREAL and kiR implementation groups using t-tests.

Results

Qualitative Results

Feasibility

Teacher-implementers, assistant principals in implementing schools, and research team members gave overall positive feedback about the program. MREAL was perceived as a feasible program to implement in Mexican secondary schools. They viewed MREAL as fitting the main intervention objective of preventing substance use and violence among students, and that it is well positioned to address risky behaviors in adolescents.

The material is very well suited, very well thought out, very well positioned to achieve its intended objective (Teacher, Mexico City [CDMX]).

Teachers and field research team members thought that the teacher training was a positive and helpful resource to prepare for implementing MREAL. They assessed the on-going supervision by the researchers, including weekly check-ins with teachers and email delivery of a one-page reminder of key aspects of the coming week’s curriculum lesson, as a positive strategy to stay connected, support the facilitation, and increase teachers’ motivation and engagement to implement the program with fidelity.

The [teacher] training was very important for me because it taught me where to start and where to direct each lesson (Teacher, CDMX).

Other positive implementation characteristics perceived by teachers were the flexibility of the MREAL curriculum to fit the specific needs of schools and students, such as tailoring activities to address conflictive situations occurring within the school during the implementation, and its manualization. The teacher and student manuals were perceived to be very helpful and appropriate to implement the program.

I think the manual was easy to follow... it is very understandable... the manual is self-explanatory (Teacher, CDMX).

In addition, field research team members perceived the program as user-friendly and applicable, which facilitated high levels of involvement of teachers and students in its implementation. Teachers noted their own active engagement in delivery and also mentioned the high levels of student participation, especially in the five sessions that included videos.

The boys [and girls] are very interested and participate very much (Teacher, GDL). Everyone participated and collaborated (Teacher, Monterrey [MTY]).

Teachers and principals perceived the program to be cost-effective in terms of the limited resources needed to implement it relative to valuable potential benefits to the youth and society. Teachers and field researchers, however, identified some barriers that could affect program feasibility and fidelity. For example, they mentioned the lack of audiovisual equipment in the schools to show the videos and PowerPoint presentations. Other comments related to systemic barriers to implementation such as the high student–teacher ratio and the shorter school day in Mexico (two sessions), which could dampen feasibility and fidelity.

Something that did affect us a little, or a lot, was the use of technology, because we do not have the equipment or means in each classroom to show videos. I took my students to the library to see the video in lesson one and they could not hear it, the speakers did not work (Teacher, MTY). The times that start and end the school day do not work out for me with the students, because at the beginning they are asleep and at the end the children are excitable, there is already a lot of disorder (Teacher, GDL).

Acceptability

On the other hand, teachers and assistant principals confirmed the acceptability of MREAL as a prevention program for early adolescents in Mexico. They expressed high levels of satisfaction for having the opportunity to implement the program and for the adequacy of the curriculum. The high level of acceptability of MREAL is reflected in the strong willingness of teachers and assistant principals to use the program in the future.

I felt very happy working on this program... it was very satisfying, very enriching, very beautiful, very pleasant (Teacher, CDMX). To me the program is fabulous... I loved it (Teacher, MTY). I enjoyed working with the (local research) group (Teacher, GDL).

Teachers reported that they understood the curriculum content, had high levels of comfort with the topics and activities, and found their students were also comfortable with MREAL. Students were a bit uncomfortable with some activities related to expressing feelings and had some difficulty grasping new concepts for them (such as assertiveness). Teachers noted that students enjoyed the activities of the curriculum, especially those involving games, acting (such as role-playing) and watching the videos.

They also really liked the dramatization and all the role plays, they were happy and fascinated... that freedom that they had to be able to express their ideas about the situations or activities that were brought up in the manual are the ones that were very enriching for them, very significant, to have the opportunity to dramatize, to express, to lead with their own story (Teacher, CDMX).

Utility

Teachers and assistant principals championed the utility of MREAL at the start of secondary education (7th grade) in Mexico. Teachers perceived the situations presented in the program as similar to those that students experience in their daily lives. They highlighted the authenticity of the curriculum and how students noted the applicability of its content. Teachers reported that students acquired new knowledge, concepts, and skills that they identified as being useful, inside and outside school, including effects on self-esteem.

They [female students] said it was okay, that it was good and that it was going to help them to make decisions, not just in terms of drugs but in their personal life, and above all to strengthen their self-esteem (Teacher, MTY).

The perception that MREAL provides useful knowledge, applicable to adolescent lives, ultimately led to changes in their attitudes and behaviors. It also influenced the teachers’ teaching style, the way they communicate with students, and how they manage conflict in the classroom. According to teachers and assistant principals, the impact of MREAL on students went beyond substance use and violence and included general life skills to cope with interpersonal relationships and make decisions in conflictive social situations. They thought the program had an impact on the larger community outside the school.

They are saying it in their own words, they tell you: ‘teacher, it helped me, I applied it and I am liking it,’ so I think it's been very useful (Teacher, CDMX). It’s a program that’s going to benefit schools, it’s going to benefit the school community, it’s going to benefit the whole community, and this activity that you (local research team) do seems to me to benefit the school community, but at the same time it can have an impact on the outside community (Assistant principal, MTY).

Quantitative Results

Table 1 reports the teachers’ levels of fidelity in implementing the manualized intervention, and observer ratings of student engagement, separating the two versions, and testing for mean differences. Teachers of the adapted MREAL and the original kiR, on average, completed all or almost all the activities on the fidelity checklists. Teachers “mostly” or “completely” followed the lesson plans, and they were prepared and informed, gave clear instructions, and motivated student participation. Their classroom management was typically “good” or “excellent,” the pace of instruction was about right for three-fourths of them, and few (10% or less) inserted content not in the manual. The similar high ratings of fidelity in both groups limited statistically significant differences to two measures: (1) kiR teachers completed slightly more items in the fidelity checklist, and (2) had marginally higher ratings of being well informed about the curriculum. In addition, observers reported that students in both MREAL and kiR showed high levels of participation and engagement in the activities, with MREAL students showing significantly more attention to the videos.

Table 1 Implementation fidelity and student engagement in Mantente REAL and keepin’ it REAL
Full size table

Students’ quantitative assessments of the two versions of the prevention program are summarized in Table 2. Students participating in MREAL and kiR reported very positive evaluations of both curricula, as indicated by means very close to the maximum rating. As an indicator of program satisfaction, large majorities (over 75%) said that they liked or liked very much both programs overall, as well as their separate components. Over 80% rated both programs as acceptable: interesting, fun, and easy to pay attention, as well as not boring (about 70%). In assessments of knowledge gained, over 80% said the programs gave useful information, and more than half said they learned “a lot” (an additional one-third said they learned “some”). About half of the students agreed that the programs were applicable to their own lives, and about two-thirds agreed they applied to youths they knew. They endorsed the authenticity of the programs, with 80% or more reporting that the situations in the curriculum and videos seemed real. One indicator of the impact of the program was that most students (about 85%) talked about it with family members and/or with friends.

Table 2 Student evaluation of adapted Mantente REAL and original keepin’ it REAL
Full size table

Although the evaluations of both curricula were very positive, MREAL students reported significantly greater engagement with the curriculum on recalling videos and ratings of the enthusiasm of their teacher, but there were no significant differences with kiR students on degree of participation. Students reported significantly higher satisfaction with MREAL than with the kiR program on all measures. MREAL students also found the program more acceptable than kiR, and more interesting, fun, and easy to pay attention. MREAL students said that they gained more knowledge than those in kiR in terms of the amount and usefulness of the information. Students found MREAL more applicable to their lives and youths they know, and more authentic in the situations and characters represented. The impact of the program, as measured by whether the students talked about it outside class, was more widespread for MREAL students, who were more likely to talk with parents, siblings, cousins, and friends, and with a larger array of people in their networks. There were no significant differences between programs on whether lessons were boring, whether the students knew drug-using youths as presented in the curriculum, and whether they talked with unspecified “others.”

Discussion

This study adds to the body of prior evidence showing MREAL to be an efficacious school-based substance use prevention program for urban Mexico. Findings support the hypothesized social validity of MREAL as indicated by the high levels of fidelity, feasibility, acceptability, and utility reported by teachers, administrators, and middle school students in the three largest cities of Mexico (H1). Fidelity was high for both the MREAL and kiR versions of the manualized curricula. The structured training prepared the teachers well; they generally followed and completed all the content and activities, and they motivated student participation. Differences in fidelity between curriculum versions were few and substantively small. The students and their teacher-implementers reported high levels of satisfaction and acceptability with MREAL and kiR. Finally, students, teachers, and assistant principals reported the strong positive impact that the program had within the school and highlighted its potential impact outside school. Taking qualitative and quantitative results all together, the social validity of MREAL was supported by the key stakeholders in the three cities.

Findings support the hypothesis that the culturally adapted version of the intervention (MREAL) would be more better accepted by the students, and be viewed as more useful in Mexico than the original kiR program (H2). Quantitative results revealed differences between the adapted and non-adapted versions of the interventions. Students were more satisfied with MREAL than with kiR, and they felt they gained more knowledge, found it more acceptable, applicable, and authentic, and discussed it with more of their family members and friends. Preference for MREAL might be explained because Mexican youth were integral in its creation. The team developed all the program materials from the adolescents’ perspective, and considered their unique communication and decision-making processes and situations they must deal with in their daily lives (Marsiglia et al., 2019). The adapted videos illustrated how to apply the resistance strategies in real-life situations in Mexico. Mexican youth scripted, cast, and enacted these videos, and did camera, sound, and editing work, assuring that the end product reflected their cultural norms (Holleran et al., 2002). These adolescent-based materials reinforced learning and facilitated the incorporation of the REAL strategies in their daily lives. Previous research has found that when an intervention combines oral, written, and audiovisual support materials, its effectiveness increases (Espada et al., 2015).

From a social validity perspective, these findings suggest the importance of cultural adaptation as a means not only to increase the efficacy of interventions but also to increase their feasibility, acceptability, and utility. Efficacy and social validity are clearly related and future research could elucidate more clearly how. EBIs with high levels of social validity appear to be the most likely to produce strong public health benefits (Castro et al., 2021).

Limitations

Proper interpretation of the study findings requires consideration of some limitations. First, the maintenance of long-term program impacts on student behavioral change is not reported, as is common in studies of social validity (Halle, 2019). Second, teacher opinions in focus groups were collected for a pilot test before the curriculum adaptation for Mexico was finalized. Although there were only minor additional changes, teacher opinions might be somewhat different if they had implemented the final version of MREAL. Third, sample sizes varied for data collected from different stakeholders. Respondents included much larger numbers of student participants than of teacher-implementers or assistant principals. A final limitation was that the study collected teacher/administrator qualitative data only from schools implementing the culturally adapted version of MREAL.

Policy and Practice Implications

Findings have implications for practice, policy, and future prevention research for Mexico and for the potential generalizability of MREAL to additional middle- and low-income countries by attending to implementation barriers. There are few evidence-based prevention programs in Mexico and those that exist provide only limited evidence from non- or quasi-experimental designs (Parra Cardona et al., 2021). MREAL could contribute to the development of a prevention policy based on an educational curriculum within schools. A next step in accomplishing this task is building an effective strategic planning process in collaboration with Mexican educational authorities and policymakers. There is a need to establish its financial sustainability and long-term implementation supported by its documented feasibility, acceptability, and utility (SAMSHA, 2019). MREAL was identified as an important prevention program by the school communities where this study was conducted.

The study is well aligned with established principles for advancing prevention research and the dissemination of effective interventions beyond high-income countries (SPR Standards of Knowledge Task Force, 2011). It incorporated the vital steps of developing and adapting effective interventions through international research collaborations, expanding training in implementation, and gaining knowledge about delivery systems attuned to target populations and institutional environments. Educational authorities reported that MREAL fulfills a need for the school system to address sharp increases in drug use. Leading prevention researchers from Mexican universities were part of the research team and are interested in furthering the evaluation, implementation, and dissemination of MREAL and expanding ownership at multiple levels. This study thus illustrates a route for closing the gap between the development of efficacious interventions and community uptake by overcoming important barriers to implementing tested interventions in low-resource settings.

Conclusions

The study showed that MREAL had better social validity outcomes than kiR in several areas, supporting the value of cultural adaptations of prevention programs. The culturally adapted MREAL for Mexico was able to maintain efficacy while incorporating cultural elements from adolescents’ everyday life contexts. MREAL captured where and how offers of substance use occur. The inclusion of family, school, and neighborhood scenarios familiar to the students may have increased the acceptability, applicability, authenticity, and utility of the intervention, as well as satisfaction with it. The inclusion of teachers as the implementers of these programs facilitated the sustainability of the program and increased the schools’ capacity to offer evidence-based substance abuse prevention.