Introduction

The demand for science, technology, engineering, and mathematics (STEM) talent is growing globally and specifically in the USA. Between 2014 and 2024, computing jobs are expected to grow 19 %, advanced manufacturing jobs 16 %, and engineering positions 12 %. Yet the percentage of women and minorities in STEM has evidenced little change over the past 13 years. Women in the computing workforce stayed at 36 %, while those in the advanced manufacturing and engineering workforces decreased from 19 to 18 % and 25 to 24 %, respectively. Currently, African-American and Latino workers represent 29 % of the general STEM workforce (up from about 24 % in 2001), but just 16 % of the advanced manufacturing workforce, 15 % of the computing workforce, and 12 %of the engineering workforce (Change the Equation 2015).

According to Rising Above the Gathering Storm: Energizing and Employing America for a Brighter Economic Future (National Academy of Sciences, National Academy of Engineering, and Institute of Medicine 2007), it is not enough to increase the numbers of US citizens entering the STEM workforce in order to maintain global US leadership. The next STEM generations also need to generate entirely new STEM industries to compete effectively in future global markets. The ability of the USA to meet the demand for individuals with the knowledge, skills, curiosity, and creativity necessary to enter STEM-intensive careers is hindered by the lack of women and underrepresented minority populations in STEM education. In Confronting the “New” American Dilemma, Nicholas Donofrio proposes that “Diversity may be the trump card…. We are going to run out of talent unless we get more women and underrepresented minorities going to college to study STEM” (Frehill et al. 2008, p. 4). This resonates with the central argument of a more recent study, Expanding Underrepresented Minority Participation: America’s Science and Technology Talent at the Crossroads (National Academy of Sciences, National Academy of Engineering, and Institute of Medicine 2011), which notes that minority groups represent 28.5 % of the US population, but only 9.1 % of science and engineering professionals. With the overall need for increasing STEM professionals in the USA, the increase in minority race/ethnic groups in the US population, and the unique perspectives underrepresented groups bring to STEM, participation of women and minorities in STEM is of critical importance.

Ensuring that the USA, and specifically the state of Arizona, has a competitive STEM workforce requires that students become interested in STEM, engage in STEM education, and remain in the STEM educational pipeline. Educational interventions responding to the need to increase the number of students in the K-20 STEM pipeline must work toward engaging STEM students who offer dynamically new perspectives. While nearly 28 % of high school freshman in the USA declare an interest in STEM-related fields, over 57 % lose interest by the time they graduate from high school (Munce and Fraser 2013). Women and ethnic minorities are significantly underrepresented in STEM education. In 2014, only 15 % of Arizona high school females reported interest in STEM fields compared to over 40 % of male students (Alliance for Science and Technology Research in America 2014). Similarly, Hispanic students report interest in STEM fields at a rate of 8–12 % less than their Caucasian/Asian-American counterparts. While data on Native Americans in STEM are not always reported separately from other minority groups, the Sloan Foundation (2014) notes that while American Indians make up 1.2 % of the US population, they earned only 0.3 % of all doctorates in 2012—which is less than the 0.5 % earned 20 years previously. Moreover, in 2012, only 48 doctorates were awarded to American Indians/Alaska Natives in engineering and science fields other than social sciences. For states such as Arizona in which a relatively large percentage of Native Americans reside (5.3 % of the total state population; US Census 2013), inspiring Native American students’ interest in STEM and engaging and retaining them in STEM education is of utmost importance both regionally and more broadly at the national level. The issues of Native American Indian/Alaska Native postgraduate attainment in STEM fields begin at the K-12 level with support provided through high school graduation (National Action Council for Minorities in Engineering 2012).

Research indicates that effective practices that increase students’ interest in STEM throughout the K-20 pipeline include (1) participation of caring adults, (2) critical thinking, collaboration, and small group work, (3) content related to real-world application particularly with hands-on learning, and (4) provision of STEM opportunities (Bouvier and Connors 2011). Additionally, the importance of utilizing a theoretical framework to guide such practices is noted in the literature (González and Moll 2001). At the core of our project design, these effective practices are embedded within our approach, which specifically include (1) STEM mentoring and (2) informal science education experiences and are guided by a Funds of Knowledge theoretical framework.

STEM Mentoring

School-based mentoring has grown quite rapidly. In part, this is due to concerns about student performance and schools’ efforts to implement programs that address students’ challenges and foster academic success (Herrera et al. 2011). Past research indicates several variables related to effective mentoring, such as length of time mentor–mentee remain connected and strength of mentor–mentee relationship—with longer and stronger relationships yielding greater impacts (Herrera et al. 2011; Rhodes 2005, 2009; Stevens et al. 2008). Research also shows that simply matching a mentor and mentee is not enough. The need for training and ongoing support of mentors, mentees, and other stakeholders (e.g., teachers) is crucial to building and maintaining effective, meaningful mentoring programs (Henry 1994). Generally, outcomes from school-based mentoring have reflected improvements in academic performance (Diversi and Mecham 2005), self-perception (Bernstein et al. 2009), and school attitude/connectedness among mentees (Portwood et al. 2005) and have even positively influenced variables such as peer and parent relationships (Karcher 2005, 2008).

STEM mentoring differs from other mentoring practices as it specifically addresses engaging and retaining students in the STEM pipeline, often with a focus on females and minorities given their underrepresentation in STEM careers. For many students, socialization practices tend to instill a negative self-perception of their ability in STEM academic fields. Mentoring relationships help to alleviate anxiety and other common academic deterrents, including fewer networking prospects, insufficient preparation, and culture shock both as a student and as a member of a minority population (Brainard 2000; Brainard et al. 1998). Successful STEM mentoring programs have included different types of mentors such as peer, professional, and personal mentors (Dean 2009). Research on STEM mentoring of Native American students by non-STEM Native American adults is lacking. This mentoring model could be viewed as a “peer” mentoring model in that both mentor and mentee are learning STEM together. Yet could also be considered a “traditional mentoring model” given that the adult Native American mentor is senior to the mentee and is a respected and caring adult.

Informal Science Education Experiences

According to the US Congress Diversity and Innovation Caucuses’ 8-point policy agenda (2008), out-of-school STEM activities (i.e., informal science education) are an important strategy for increasing diversity in STEM. One of the group’s eight recommendations includes ensuring that students in high-need populations have access to hands-on laboratory experiences and informal learning opportunities that have been shown to increase interest and academic performance in STEM (U.S. Congress 2008). Programs which incorporate intensive out-of-school science activities and experiences are known to spark interest in STEM, develop students’ understanding of STEM, and increase the possibility of students’ commitment to pursue a career in STEM-related fields (National Research Council 2009).

Out-of-class activities offer students the opportunity to learn in a safe environment without fear of failure like in the classroom (PCAST 2010). Studies indicate that connecting out-of-school activities to in-school learning objectives is an effective way to increase self-efficacy as well as to improve overall learning and interests (Moore and Sandholtz 1999). Research also shows that out-of-school experiences are especially important for students who may be discouraged by school science or those who feel less skilled or have self-defined themselves as “not interested” in science. Often, these students are females and/or are from minority communities (Bouillion and Gomez 2001).

Theoretical Framework: Funds of Knowledge

When working with minority ethnic groups and specifically Native Americans, culturally relevant theoretical frameworks, such as Funds of Knowledge (FK), are called for. FK is defined as the historically accumulated and culturally developed bodies of knowledge and skills essential for household or individual functioning and well-being. FK is based on the premise that learning is mediated by social contexts, that is, social relations, practices, and artifacts that comprise human experience (Spear Ellinwood 2009). These social contexts are critical to in- and out-of-school learning by students from the non-dominant cultures (González and Moll 2001; González et al. 2005b). Elaborating on the significance of local context and geography to Native students’ academic success, scholars call for embracing ways in which Native worldviews coexist with Western paradigms with regard to the interconnections among peoples and with nature (Deloria and Wildcat 2001; Jacobs and Reyhner 2002). Furthermore, through the inclusion of mentoring relationships and out-of-school activities within the FK framework, a supportive link between traditionally marginalized funds of knowledge and academic funds of knowledge is made—contributing to the understanding how science learning involves learning to negotiate multiple texts, discourses, and knowledge (Calabrese Barton and Tan 2009).

Research findings from educational practices framed by FK have evidenced positive results in a number of areas including math education (Andrade et al. 2001; Ayers et al. 2001; Civil and Andrade 2002; González et al. 2001, 2005a). FK assists educators, mentors, and other caring adults in identifying, documenting, and building on the “hidden resources” found in the lifeworld of students for the purpose of helping them contextualize problems for ease of learning. By identifying students’ “hidden resources,” those working with students increase their understanding of students’ diverse cultural and linguistic backgrounds. With this knowledge, they are better able to facilitate learning activities that respond to the cultural, familial, and social strengths and interests of their students and/or mentees. This type of praxis also supports individual learning styles within a framework of expectation for success and challenges societal expectations of underperforming schools and minority students’ capabilities.

For Native American students living in southern Arizona, there is a gravitational pull to the home and community. Research indicates that parents and other family members of Native American students do not want their children to leave home, or if they do leave for educational advancement, they are expected to return to the community or reservation (Stevens et al. 2010). Thus, a focus on exposing Native American students to local STEM educational experiences and local STEM industries is critical for increasing Native American students’ interest and success in STEM fields and careers. Using local places and FK to explore what students know about STEM and STEM education can bring relevance to the science subject under study. For example, when iSTEM’s module on solar energy is facilitated, students can be asked about how the sun and heat of the southwest desert impact their ways of living, how their ancestors adapted to the intense heat of the desert, how water levels have changed over time, and how the sun has been used as an energy source (Gomez et al. 2015). Encouraging students to discuss these topics with family and friends is a way of fostering community interest and honoring familial funds of knowledge.

The iSTEM Project

The iSTEM project was funded by the National Science Foundation in 2012. It is a collaboration between the University of Arizona (UA), StrengthBuilding Partners (a youth mentoring program), the Pascua Yaqui Tribe, and public schools that predominately serve Native American and Latino youth. Using the FK framework, the iSTEM research design is based on the contention that a synergistic hybrid program combining two strategic approaches (in-school mentoring with out-of-school informal science education experiences) would be successful in engaging and retaining 3rd–8th grade Native American students in STEM education. This age/grade range was selected based on STEM literature that suggests that engaging elementary and middle school students has the greatest impact on closing the STEM educational gap (National Science Board 2010) and engaging these young students is critical in preparing them for relevant high school courses that then allows them to pursue a STEM career in post-secondary education (President’s Council of Advisors on Science and Technology (PCAST) 2010).

The public schools that sponsored iSTEM are predominantly attended by Native American and Hispanic/Latino students living near the USA–Mexico border. Originally, the project included two schools: an intermediate school serving 3rd–5th grade students and a traditional middle school serving 6th–8th grade students. The intermediate school expanded to include 6th–8th grades in the project’s third year. A third school, also a traditional middle school, was later included when budget cuts resulted in the closure of the original middle school and the majority of the students were transferred to the new school. iSTEM participants who transferred to the new school have difficulties adjusting to the new school and participation in the iSTEM dropped at this school, in part, due to students at the new school viewing iSTEM as a remedial mentoring program for underperforming students.

The selection of STEM topics for the iSTEM program was based on the four Grand Challenges for Engineering themes: (1) Energy and Environment, (2) Health, (3) Security, and (4) Learning and Computation (National Academy of Engineering 2009). Five themed modules per year (e.g., solar energy, global positioning system), each facilitated over a 2-month period, were developed and implemented. In-school “flash STEM” activities are completed by mentors/mentees during the students’ lunch periods and aligned with informal out-of-school experiences on the same topic. The content of the iSTEM activities is related to real-world application specifically attuned to Native American culture and ways of learning as well as the context and geography of southern Arizona. All iSTEM project activities are theory driven based on a Funds of Knowledge theoretical framework.

In-School Activities

Unlike many mentorship programs that occur outside of the traditional school day (generally as after-school programs), the flash STEM activities were integrated into the school day and took place during the student–mentee’s lunch hour every other week. Choosing to schedule these activities during the school day was a strategic decision based on the personal circumstances of the targeted student population. The majority of students who participate in this program live in rural locations, a significant distance from the schools they attend. Many of the students’ families do not have vehicles, and for those that do the driving, distance to and from the schools is too time-consuming and costly. Thus, the only reliable transportation to and from schools for the students is the school bus service. Scheduling flash STEM activities during the school lunch periods provided an opportunity to integrate these supplemental activities into the traditional school day and bypass transportation-related challenges.

Given time constraints at lunchtime, the in-school flash STEM activities are designed to last about 20 min and to be easy enough for the mentor to engage the mentee during completion of the activity (Gomez et al. 2015). Themed activities are prepackaged for the mentor and student mentee, requiring minimal setup or the need to review instructions. The activities are also designed to feel less like “schoolwork.” In particular, they are facilitated in a relaxed atmosphere—a designated, non-traditional classroom space in each of the schools, are “hands-on,” and aim to be as fun as they are educational. For the majority of activities, iSTEM staff are present to assist in a supportive role (e.g., with setting up the activities). For activities that require a higher level of background knowledge or expertise (e.g., GPS device-based activities), iSTEM staff play a larger role and more directly facilitate the activities between the mentees and mentors. The intent, however, is that the activities are self-paced, easy to follow, and focused on the module theme (e.g., solar energy, GPS). For example, making a solar bracelet is a pre-packaged flash STEM activity facilitated during the solar energy-themed module. Materials include an assortment of white solar beads and multi-colored pipe cleaners for making the solar bead bracelet. Sunglasses and other objects are used to test the intensity of the sun in changing the color of the beads from UV exposure. Students enjoy making the bracelet and are able to demonstrate and tell others about the principles of solar energy.

Every other week, “non-STEM activities” are built into the schedule to help facilitate a personal connection between the mentor and mentee by allowing them to choose their own activity—typically mutually held interests and hobbies—which may or may not be STEM related. The mentoring classrooms at the partnering schools are equipped with various materials that mentors and mentees can use during this time including a number of games, puzzles, art supplies, and various other activity items as well as activities associated with each of STEM-themed modules.

Field Trips

The iSTEM field trips were designed to directly relate to the modular themes students focused on during the flash STEM activities. During year 1, the five modular themes included: (1) solar energy; (2) optics; (3) flight and motion; (4) GPS; and (5) astronomy. Associated field trips included solar robotics at the UA College of Engineering, station exploration at the UA Flandrau Science Center, the Pima Air and Space Museum, GPS scavenger hunt at the partnering schools, and the UA Planetarium and a star party led by UA’s student astronomy club. Year 2’s modular themes offered (1) solar energy; (2) mapping and GIS; (3) space, earth and soils; (4) watershed; and (5) ecology. Field trips included solar robotics at the UA College of Engineering, GIS mapping activities on the UA campus mall, soil activities at the UA Agriculture Center, Sweetwater Wetlands and water treatment plant, Arizona-Sonora Desert Museum, and UA Science Sky School on Mt. Lemmon. At the start of the program, field trips were held at the end of the themed module, but the timing of the field trips shifted depending on availability at the field trip sites. Field trips that began a new unit or that occurred midway through a module contributed to mentee excitement in participating in the activities, while field trips that occurred at the end of the module provided a wrap-up. Field trips were facilitated on Saturdays. A chartered bus met the students at the schools, and given that many families lacked transportation, project staff also provided a shuttle service to and from home to the chartered bus for students without transportation.

To understand the impact of iSTEM, a theoretical and culturally driven hybrid program that combined mentoring with informal science activities, this study asks the following evaluation questions: (1) What was the level of student mentee enrollment and retention in iSTEM? (2) What was the level of mentor participation and retention in iSTEM? (3) What outcomes were evidenced for mentees’ (a) science beliefs, (b) satisfaction with the iSTEM (perceptions of their mentors; engagement in activities), and (c) school-related indicators?

Methods

Mentor and Student Mentee Recruitment and Expectations

Mentors

Mentors are recruited through a partnership team comprised of tribal members and tribal employees, school personnel, and iSTEM program staff; through outreach to UA colleges and departments; and through outreach to local STEM companies. iSTEM staff contact potential mentors and provide program information including mentor expectations: (1) a commitment of at least one academic year; (2) attending five field trips per year; and (3) participating with their mentee in the in-school lunchtime activities once a week. The mentors are expected to function as a reliable adult who is genuinely interested in the overall well-being of their mentee as well as a person who can inspire STEM learning. Mentors do not need to be STEM experts, rather all levels of STEM knowledge are welcome as long as mentors are enthusiastic about STEM.

If the person agrees to serve as a mentor, they are screened and assessed by staff employed by StrengthBuilding Partners, a local nonprofit agency that has expertise in providing mentoring programs (www.strengthbuilding.org/). Mentor screening includes fingerprinting and background checks, as well as an initial interview that incorporates an instrument for assessing a range of human intelligences: logical–mathematical, linguistic, musical, spatial, bodily kinesthetic, interpersonal, intrapersonal (e.g., multiple intelligence; Gardner and Hatch 1989). Results of this assessment inform mentor and student mentee matching. Once completed, mentors are then provided mentor training also through StrengthBuilding Partners. The 3-hour initial mentor training includes mentor expectations; basics of mentoring; mentoring research; school and iSTEM program rules; and practical information such as sign-in logs, location of room/materials, contact information, and communication expectations. Ongoing training is available on various mentoring topics. Neither mentors nor student mentees were compensated for their involvement in iSTEM. However, the flash STEM activity materials as well as paid entrance fees and lunch for the informal science field trips were provided by the project.

Student Mentees

Students are eligible for iSTEM if they attend one of the partnering schools and are in grades 3–8. Recruitment and enrollment occurs throughout the school year via flyers, word of mouth, and teacher referral. Some students participated in StrengthBuilding Partners non-STEM mentoring program prior to iSTEM, while others joined because they were interested in the iSTEM activities. Interested students are interviewed to ascertain their interest in the iSTEM program and are assessed (e.g., multiple intelligences) to inform mentor matching. Expectations are communicated to the student and include (1) a commitment of at least one academic year; (2) attending the five field trips per year; and (3) participating with their mentor in the in-school lunchtime activities once a week. Parents or guardians then complete the permission paperwork, and the student completes an iSTEM evaluation pre-survey, before being placed on the waitlist or being paired with a mentor.

Data Sources and Data Collection

The iSTEM evaluation is conducted by an external evaluation team which works closely with project staff to coordinate the evaluation activities. The iSTEM evaluation employs a mixed-methods design and includes a two-step (formative and outcome) evaluation. While student mentees are the focus of the iSTEM project evaluation, the perspective of the mentors is also obtained through mentor/student mentee observations and end-of-the-year surveys.

Prior to implementation of the iSTEM program, and on an annual basis, the evaluation protocols and assessments were reviewed and approved by the UA Human Subjects Institutional Review Board. Additional reviews and approvals were conducted by UA’s Native American Studies department head and the local school district research board. Prior to enrollment into the iSTEM project, students are required to have parental consent as well as provide their own assent to participate in the program and the evaluation component. Once the program coordinator for the mentoring program receives all completed paperwork, the student mentee meets with an evaluation staff to engage in the evaluation component of the project. Evaluation staff read the evaluation survey to students at their request (typically 3rd–4th graders) with 5th–8th graders choosing to complete the survey alone. Student mentee tracking and assessments include.

Student Mentee Waitlist, Participation Tracking, and Retention

After student mentees complete and return the enrollment paperwork, they are placed on a waitlist due to a higher number of students interested in participating in iSTEM compared to the number of committed mentors. This waitlist is maintained by StrengthBuilding Partners. Once enrolled in the project, the level of participation in iSTEM is tracked by the evaluation team through sign-in logs located in the iSTEM room at each of the schools. Retention of student mentees (e.g., number of years) in the iSTEM project is also tracked by the evaluation team.

Student Mentee Pre- and Post-program Beliefs Survey

Given our unique informal, hybrid program, the evaluation team identified a variety of available instruments to measure student changes in science and math attitudes, and interest in STEM activities and careers. No single survey met the needs of assessing change in our students based on the types of activities. Thus, survey items were drawn from many of the measures identified. At the beginning and end of each academic year, the pre- and post-surveys assess beliefs about school belonging, the importance of science and mathematics and of doing well in these subjects, interest in STEM activities, books, TV programs, and intention to pursue higher education, particularly in a STEM field of interest. The survey is completed by the mentees independently or with the assistance of the evaluation team (e.g., reading the questions to the student mentees). About two-thirds of students spent more than 6 months from their pre- to posttest, while one-third had five or fewer months between the two tests. Eight students joined in April prior to a May posttest. The survey is comprised of items from the following surveys: iSTEM School and Career Interest Survey (Kier et al. 2013); STEM Semantics Survey (Tyler-Wood et al. 2010); Science Curiosity Scale (Harty and Beall 1984); Sense of School Membership (Goodenow 1993); Hemingway Measure of Adolescent Connectedness (Karcher and Sass 2010); Frequency of Hands-On Experimentation and Student Attitudes Toward Science (Ornstein 2006); and Modified ATSI (Weinburgh and Steele 2000). Cronbach’s alpha reliability for the 39 items is .868. A factor analysis of the items found that all items loaded on a single factor explaining 59 % of the variance. A second factor explained an additional 8 % and included items related to participating and enjoying science outside of school, including their family encouraging them to participate in science.

Student Mentee End-of-Year Follow-Up Survey Regarding Their Mentor

A final data collection instrument is administered to student mentees about their mentor and the mentoring experience. Items included relational aspects about the mentor (i.e., feeling excitement, importance, or special), participation in activities and field trips, time spent with mentor, and mentor’s interest in student mentees’ activities, advice giving, and mentor’s visits. Items on the student mentee survey were modified and included items from: “Measuring the Quality of Mentor-Youth Relationships” (Jucovy 2002), as well as project-specific items regarding field trips and the iSTEM mentor. Cronbach’s alpha reliability for the 18 items is .925.

Mentor End-of-Year Follow-Up Survey Regarding Their Student Mentee

Mentors completed the same set of questions presented to student mentees with the exception of an open-ended question allowing the mentor to provide additional information. In this study, several open-ended responses are included in the results section. The survey results will be presented in a future article about mentors.

Student Mentee/Mentor Observations

The third author notified and received permission from the International Center for Leadership in Education to use the Student Engagement Walkthrough Checklist (Jones 2009) to record observational data. A member of the evaluation team observes and scores the engagement checklist as the mentor/student mentee pair engages in iSTEM activities. The observational engagement checklist assists in capturing data on positive body language, consistent focus, verbal participation, student confidence, and fun and excitement. The scale for rating ranged from very high to very low. Pairs were observed from 10 to 30 min at a time, and the timing of the observation could occur in the first 10 min, the middle 10 min, or the last 10 min of an activity. The time block was also recorded to provide additional context for the observation (i.e., reading instructions at the beginning or the early stages of the project look differently than at the culmination of the activity). Similarly, the level of engagement with the mentor can appear different from the beginning to the end, thus noting the time of observation provides this additional context to make inferences from the data. The second set of five items in the checklist measures individual attention, clarity of learning, meaningfulness of the work, performance orientation, and rigorous thinking. Cronbach’s alpha reliability for the ten items was .938.

Student Mentee Report Card Data

At the end of each academic year, copies of each student mentee’s report card are provided to the evaluation team by school staff. Science and mathematics quarterly grades as well as school absences are then recorded in the evaluation database.

Data from the measures described above are used to assess: the level of student mentee enrollment and retention in iSTEM; the level of mentor participation and retention in iSTEM; and outcomes evidenced for mentee’s (a) science beliefs, (b) satisfaction with iSTEM (perceptions of their mentors, engagement in activities), and (c) school-related indicators. While absences and grades were not intended outcomes of the program, the results are included as indirect changes associated with participation in iSTEM.

Results

Results of Student Mentee Engagement

Student Mentee Engagement and Demographics

During all 3 years of the iSTEM project, there was a waitlist for student mentees with the waitlist more than doubling in year three compared to year one. The waitlist was eventually capped at 30 students. During the 3 years of the program, a total of 78 students participated, with 12 students completing all 3 years, 31 students participating for 2 years, and 35 students joining for 1 year. At the start of year two, 22 of the original 37 students enrolled in year one continued in year two. After year one, eight students graduated 8th grade and two students left the school. Five students decided not to continue in the program during year two. Furthermore, the middle school was closed after year one due to district restructuring; the middle school selected to receive the majority of students from the closed school participated in years two and three. From year two to year three, 33 students returned and eight new students were added. A total of 14 students graduated from 8th grade, and 9 students dropped from the program. Table 1 shows the grade level of students for all 3 years. Table 2 shows the gender and ethnicity of the repeat mentees and mentors for the 3 years, while Table 3 provides the number of mentors for each mentor type. During the 3 years of the project, iSTEM enrolled 76 % female students and 96 % underrepresented minority students. The female student mentees in the program especially enjoyed hanging out in the iSTEM room, sometimes completing activities a second time even without their mentor. Female student mentees also recruited other girls to become participants through word of mouth more so than male students.

Table 1 Grade level of students
Full size table
Table 2 Number of repeat students and mentors by gender and ethnicity
Full size table
Table 3 Mentor types
Full size table

Results of Mentor Engagement

When proposing the iSTEM project, we expected to engage a total of 60 mentors over the 3-year period: 30 Pascua Yaqui Native mentors, 15 STEM professional mentors, and 15 university student mentors. As detailed in Table 3, the final number for each mentor group was 26 Pascua Yaqui Native mentors, 6 STEM professionals, and 13 university student mentors. Recruitment of mentors who could commit to once a week meetings presented challenges due to the distance to the schools in the middle of the workday as required for the in-school lunchtime flash STEM activities. Distance to the schools was typically not an issue for tribal employees and tribal community members. However, distance and lack of transportation was initially problematic for university mentors until project staff began providing transportation to and from the university. Recruitment of STEM professionals presented the greatest obstacle to participation with the distance to the schools (on average a 25-min drive) being a major barrier.

Each year approximately 360 school visits were made by Native mentors (45.55 %), followed by university students (30.86 %) and STEM professionals (25.41 %). However, considering the numbers of mentors for each type in the program, STEM professionals reported the most frequent visits. The six STEM professionals had an average of 16.6 visits; the 26 tribal members had an average of 8.5 visits, and the 13 university students had an average of 12.8 visits. Table 4 shows that the STEM professionals had a higher percentage of their visits (42.17 %) being iSTEM related; the university students had 32.17 %, and the Native mentors had 16.15 % of their visits as iSTEM related. The overall total percentage of visits conducting iSTEM activities was 27.30 %. Most of the iSTEM activities were accomplished in 20–30 min. Times were calculated based on mentor sign-in attendance. STEM professionals and university students spent more time (30 and 26 min, respectively) with their mentee, with Native mentors spending just over 17 min.

Table 4 Average number of visits by mentors to schools during lunch
Full size table

Results of Student Mentee Beliefs About Science, Satisfaction with iSTEM, and School-Related Variables

Student Mentee Beliefs About Science

When comparing pre- to post-survey results for all student mentees (62 students) across the 3 years, there is a decline in mean scores for many of the 39 items in the survey; however, when analyzing data for the 28 students that indicated wanting to have a career in science (36 % of total iSTEM population), the items show positive or stable growth for most items. Of the 28 students, 71 % are female, which closely resembles the sample distribution. Table 5 includes a sample of items from the beliefs survey showing the pre- to post-gains for these students. The final six items are asked only of middle school students in regard to college attendance and jobs in science; there were ten middle school students of the 28 who were strongly interested in STEM with pre- to post-data finding same pre- to post-mean scores or slight increases.

Table 5 Student beliefs survey for students interested in science careers
Full size table

Specific items showing an increase include doing well in school, having a career in science, enjoying science activities at school, the importance of math and science for their future, doing well in math and science, encouragement by their family to study science, and talking with friends about science. All student mentees maintained a consistent response for attending college with a slight increase for graduating college with a science degree and contributing to science in a meaningful way. The inability to collect end-of-year data from all 78 student mentees precluded a full analysis; thus, analyses include only student mentees completing both pre- and post-surveys.

Student Mentee Satisfaction

Student mentees rated their mentors on items such as feeling excited, relaxed, and special when with their mentor, having fun participating in the activities, enjoying field trips, and trusting their mentor’s advice. Given that differences between mentor type is the focus of a forthcoming manuscript, the scores listed in the Table 6 include the average across all student mentees completing the post-only survey regardless of mentor types. Note that the sample size varies each year based on number of students completing the survey. During the first 2 years of the project, mentors were consistent in their attendance and in their enthusiasm to work with the student mentees; however, by the third year, some of the student mentees who were less interested in pursuing a science career preferred to play games or talk with their mentors instead of completing iSTEM activities. Those interested in STEM persisted in their engagement with the iSTEM activities with their mentor. In addition, questions asking about the mentor returning or spending more time decreased in the third year since students knew the program might be ending. The Likert scale represented 1 as strongly disagree to 5 as strongly agree. Two items are negatively worded as noted by the “n” in parentheses following the question (items 5 and 14). These items score closer to the opposing scale of 1, representing strongly disagree.

Table 6 Mentee perceptions about their mentor
Full size table

Data from student mentees confirmed their satisfaction with the program as does the retention data for both mentors and student mentees. Satisfaction with the mentoring experience rated very highly. For example, across the 3 years, students rated the statement: “It’s fun to do iSTEM activities with my mentor” as a 4.25 (out of 5.0). Mentors did an exceptional job engaging the mentee in the activity by explaining the task or demonstrating the activity. In some of the modules (i.e., GPS tracking), a staff member facilitated the activity due to borrowed equipment from the UA. Some of the mentors particularly enjoyed these facilitated sessions, which typically contained challenging or unfamiliar science material. Mentors were asked to provide feedback about the iSTEM through a survey containing the same questions as the student mentees. Table 7 shows a comparison of mentor and mentee mean scores across all 3 years.

Table 7 Comparison of mentee and mentor responses to year-end survey
Full size table

Observational data served as the hallmark of program evaluation and allowed assessment of the mentee and mentor while engaged in the iSTEM activity or field trip. The use of the Student Engagement Walkthrough Checklist (Jones 2009) allowed for iSTEM activities to be coded by a member of the evaluation team. Ratings of iSTEM activities (presented in Table 8) exceeded a 4.0 (out of 5.0) on eight of ten items. The first five items were included for every observation; while the latter five were included when the evaluator could both hear and observe the interaction with complete clarity. Positive body language and consistent focus both received the highest score (4.5), while student confidence scored the lowest (4.2). The second five items from the checklist were not completed for every observation given the challenging nature of the items and the ability to observe and/or hear during 10–15 min of observation to assess these items with accuracy (e.g., rigorous thinking 3.7 and clarity of learning 3.8). These items are left blank and a comment added for the evaluation session that items within the observation checklist were not observed.

Table 8 Student engagement walkthrough checklist
Full size table

School-Related Variables

Rates of absenteeism for iSTEM mentees decreased from an average of 16 days to 9 days from the year prior to iSTEM to the completion of year one for elementary school students and 30 days to 9 days on average for middle school students. During the third year (2014–2015), the average absence was 11 days for elementary and 23 days for the middle school students. In part, the increase to 23 absent days for middle school students in year 3 was due to students who changed schools when their school closed. These students did not want to change schools, students at the receiving school were generally unwelcoming, and the distance to the new school was an additional 7 miles of travel. Science and mathematics grades stayed the same or improved for mentees with most mentees passing science and about two-thirds of students passing mathematics. The majority of elementary school students were passing both math and science. At the middle school level, 6th graders earned the highest grades, and seventh graders had the lowest grades, with 8th graders rebounding in math and science to earn mostly As and Bs with an occasional C, D, or F.

Discussion

Given the low percentage of Native Americans engaged in STEM education and STEM careers, programs for engaging Native youth in STEM are critical. This study suggests that using a culturally relevant theoretical model and approach to STEM including a hybrid model may result in increased engagement and interest in STEM, as noted by the number of participants each year and those on the waitlist. The program successfully engaged a majority of females, and almost all students were either Native American or Hispanic, addressing the underrepresented minorities in STEM. Outcomes of participants that indicated a preference toward STEM careers resulted in improvements to those participants’ science beliefs and school indicators such as school attendance and grades. Mixed outcomes were noted for those less interested in pursuing STEM; however, the introduction to STEM-based activities may prove beneficial in the long term as students’ mature and pursue new interests.

Of importance to note, at the beginning of the iSTEM project the two selected schools had been labeled “failing schools” and one of the two schools closed during the project period. Historically, many Native American students in the southwest USA have experienced intergenerational historical trauma and negative school experiences (Stevens et al. 2015). The boarding school experience of past generations of Native Americans has resulted in mistrust of public education. Thus, having engaging STEM programs that include and honor the knowledge of Native Americans is of utmost importance. Partnering with the Pascua Yaqui Tribe proved extremely beneficial not only for recruitment of mentors, advisement on science subjects, but also in terms of identifying what stories and knowledge can be included and which ones are sacred and are only allowed for ceremonial purposes.

Using an FK framework has helped guide the project activities. iSTEM activities related to Native Americans living in the southwest—including a historical perspective and current life context—made abstract science concepts personal. For example, during the module Space, Earth, and Soils, student mentees and their mentors engaged in the in-school flash STEM activities and a corresponding field trip to the UA Agriculture Center learning about Native Americans’ history of desert southwest farming and use of rain harvesting and irrigation canals (quite like today); how air and water move through different soils (contributing to which crops are effectively grown in the southwest); soil analysis; how to make edible soil; and Native American soil (sand) painting; as well as what one can do with a career in soil science. The inclusion of culturally relevant activities led to greater understanding by the mentees and the ability to re-teach others, thereby increasing their self-efficacy and their pride in their culture. Mentees reported discussing these and other iSTEM module topics with family and friends and reported family and community interest in what they learned.

Perhaps one of the components of the iSTEM project that contribute to its success is the inclusion of family. While parents and other family members were not formally involved in iSTEM, several of the field trips encouraged family involvement. Not only did many of the parents and guardians attend some of the field trips, but younger siblings frequently attended—bringing into question the ripple effect that the iSTEM field trips may have on these younger children’s STEM interest and engagement. Examples of family-focused fieldtrips include: the Star Gazing Party hosted by UA’s Astronomy Club where Astronomy Club members provided telescopes to look at planets and constellations along with several additional interactive activities, and the Mt. Lemmon field trip that brought families to the top of the local mountain, stopping to observe the variations of landscape as the elevation increased and ending at the UA’s Sky Islands headquarters. The focus on relevant science that is “close to home” and the development of a Family STEM Guide that includes information on STEM education opportunities and science-related careers in southern Arizona aims to intrigue and capture the attention of both student mentees and their family members. More research is needed on how to involve Native families in STEM education and the impacts of family involvement on Native American students’ engagement and retention in the STEM educational and workforce pipeline.

The inclusion of three mentor types to test the benefits of tribal community members, who typically do not have a STEM background, as mentors is in need of further exploration. The number of mentors who are tribal community members (n = 26) indicates a willingness of tribal community members to serve as a mentor—above and beyond STEM professionals (n = 6) and university students (n = 13). Further analysis of the iSTEM data needs to be conducted to examine the level of mentor involvement, comfort with mentoring, and mentor training needs by mentor type. Additionally, in some instances, iSTEM mentees placed on a waitlist for new mentors were paired with a program staff or “STEM guide” in order to participate in the iSTEM activities and field trips. This approach has potential for success, and further research on a STEM guide mentoring approach would add to our understanding of successful approaches to STEM mentoring and engaging and retaining Native American students in STEM. Additionally, the number of female mentors (n = 35) compared to male mentors (n = 10) was somewhat surprising; even more surprising is the number of female mentees (n = 60) compared to male mentees (n = 18). Why the iSTEM program appealed more to female mentors and mentees also needs further exploration.

Student mentee enrollment and retention was substantial and at times overwhelming with a waitlist that had to be capped at 30 students. Mentor challenges were present with each mentor type including, for example, recruitment of STEM professionals and Pascua Yaqui Native members engaging in limited STEM activities. Overall, however, the mentor component was successful. Finally, the student mentees interest and continuation in the program from year-to-year may be indicative of iSTEM’s success in maintaining the interest of this population.

Initial results from this study are encouraging, yet several study limitations are worth mentioning. While the evaluation team strived to maintain fidelity in data collection, the timing of mentee enrollment into iSTEM at times compromised the duration of time between pretest to posttest with one-third of mentees participating for five or fewer months. Some mentees (n = 10) completed the pretest in March or April of a school year and then were expected to complete the posttest by mid-May. This short duration often failed to result in changes in science beliefs given the time spent in the program which may have only included two or three iSTEM activities. In many cases, mentor recruitment for the school year had ceased; thus, these students were not paired with a mentor. These mentees’ interactions occurred with program staff “STEM guides” facilitating the iSTEM activities. In general, a lack of full participation in post-data collection resulted in small sample sizes each year. Different strategies (e.g., timing of data collection) were used each year in order to gather post-data from as many students as possible.

Additionally, across the 3 years, many students reported moderate to high ratings on the pretest, thus creating a ceiling effect limiting growth by the posttest. In part, these elevated scores could be due to attracting students already interested in science and/or students’ attempt to please the interviewer at the pretest. Moreover, the sensitivity of the created instrument may limit the ability to detect true changes in students’ growth, particularly given the other limitations mentioned. While we did not detect large pre- to post-changes, we did not generally detect decreases—a positive result among an age group (3rd–8th grade) and gender group (mostly female) in which there is typically a drop in science interest.