Introduction

There are several advantages to pursuing a career in science, technology, engineering, or mathematics (STEM). Careers in STEM are more lucrative than most professions, as recent data have shown that seven of the ten highest starting salaries require STEM degrees (National Association of College Employers 2015). Compared to other disciplines, STEM fields tend to be associated with both higher rates of job growth, lower rates of unemployment, and individual earnings that are on average, up to 40% higher than for those who pursue non-STEM degrees (Carnevale et al. 2013; Melguizo and Wolniak 2012; National Association of College Employers 2015). In addition to employment benefits, providing opportunities for engagement in STEM careers is a key strategy for solving societal problems such as combating disease, protecting the environment, developing new innovations, and addressing local and global needs (Atkinson and Mayo 2010), and for improving overall quality of life (Atkinson and Mayo 2010; Xie et al. 2015). Despite the numerous benefits associated with STEM careers, many individuals perceive a STEM trajectory as an unattractive option (Tytler and Osborne 2012) and those in underrepresented groups are even less likely to complete and pursue STEM careers (Landivar 2013a; Palmer et al. 2010).

Women (Miyake et al. 2010) and members of historically marginalized racial and ethnic groups (Landivar 2013a; Palmer et al. 2010) continue to be underrepresented across STEM majors and careers. African-Americans and Hispanics comprise approximately 11.7 and 14.6% of the US population respectively (Landivar 2013b), but these numbers do not correspond with STEM trajectories. Specifically, only 5.8% of African-Americans and 7.9% of Hispanics have a science or engineering degree, and only 4.8% of African-Americans and 6.1% of Hispanics are engaged in science and engineering careers (National Science Board 2016). While the number of women with science and engineering degrees and careers has doubled over the past decade (Landivar 2013a; National Science Board 2016), only 11 to 12% of females are employed among some of the highest paid STEM occupations including physicist, astronomer, aeronautical engineer, and computer engineer (National Science Board 2016). To exacerbate this problem, there is evidence that positive science attitudes decline as youth make the transition from elementary school to middle school, and again from middle school to high school (Archer et al. 2014; Tytler and Osborne 2012). Strikingly, some studies suggest that formal science classes may hinder rather than augment interest in STEM especially among members of historically underrepresented racial and ethnic backgrounds (Maerten-Rivera et al. 2010). Thus, science education faces a dilemma. There is compelling evidence that engagement in STEM trajectories can benefit young people in terms of social mobility and individual prestige. Yet, for many individuals, formal science education experiences lead to disengagement and ultimately divergence from a STEM pathway.

Informal science education (ISE), defined as voluntary participation in science learning during out-of-school time (OST) hours (Hofstein and Rosenfeld 1996), is an important vehicle for facilitating sustained interest in STEM trajectories (Bell 2009; Dierking and Falk 2010). Long-term access to ISE institutions, such as museums, science centers, botanical gardens, and zoos, may have greater impacts on participants’ STEM trajectories than formal experiences (Tytler and Osborne 2012) as there is compelling evidence that participation in informal science learning is associated with increased interest in science (Bell 2009). McCreedy and Dierking (McCreedy and Dierking 2013) suggest that OST informal science learning experiences may have cascading effects, and that the cumulative memories and experiences associated with long-term participation may longitudinally impact STEM outcomes. In support of this idea, recent studies have shown that participation in ISE programs help to generate and sustain interest in STEM majors and careers (Adams et al. 2014; Gupta and Siegel 2008) and more recently, impact on careers in the arts (Linzer and Munley 2015). While there is ample evidence that participation in ISE matters, knowing what features of the participation mediate the impact on STEM majors or STEM careers still remains quite elusive.

An important and unresolved question in the science education literature is whether participation in ISE OST programs is associated with future engagement in STEM majors and careers. A recent meta-analysis of 15 OST programs by Young et al. (2017) reported a positive effect on participants’ interest in STEM. Likewise, Dabney et al. (2012) found a significant association between students’ participation in OST activities and STEM career interest. However, to date, a paucity of studies has measured the impact of ISE OST programs on future engagement in a STEM trajectory (Bell 2009; Dabney et al. 2012). Promisingly, of these studies, many have reported positive results. For example, Fadigan and Hammrich (2004), in a retrospective study of the Women in Natural Sciences program, reported that 45 of 100 alumni (45%) were majoring in or engaged in a STEM-related career, and that more than half of the alumni attributed the program to influencing their career decisions. Further, a retrospective study of the Young Women in Science program (YWIS) (Schumacher et al. 2009) compared STEM major outcomes of program participants to students who applied, but did not participate in the program, and found that 29 of 44 (65.9%) YWIS alumni and 11 of 34 (32.4%) of the comparison group majored in science; this difference was significant. Moreover, Winkleby et al. (2009) studied the STEM outcomes of participants of the Stanford Medical Youth Science Program, a biomedical pipeline program for high school students that aims to diversify participation in the health professions: The authors found that 147 of 256 (57.4%) alumni were engaged in a STEM major and 28 of 129 (21.7%) alumni were engaged in a health-related career. Lastly, a review of six ISE OST programs designed specifically for girls reported that 52 of 108 (48.1%) alumni were engaged in a STEM career (McCreedy and Dierking 2013). Collectively, these studies suggest the possibility that ISE OST programs impact participants’ decisions to engage in a STEM major or career and the need for a landscape review of what the ISE OST field knows about the impact of these programs on participants’ STEM trajectories.

In this paper, we examine how participation in a longitudinal ISE OST program facilitated at a large natural history museum impacts the STEM major and STEM career outcomes of past participants. In particular, we connect data describing how the development of sustained social networks within a communities of practice framework (Lave and Wenger 1991) facilitates the development of social and academic capital (Bourdieu 1986) giving rise to participants’ self-efficacy to persist with STEM in college and beyond.

Curriculum

The Lang Science Program is a 7-year program at the American Museum of Natural History (AMNH) specifically designed to make STEM accessible to historically underrepresented 6th–12th grade youth. Students apply in the 5th grade via a competitive process comprised of written applications and an interview, and those accepted into the program continue participation until high school graduation. The program takes place at AMNH where youth meet on alternate Saturdays during the academic year and for 3 weeks during the summer for a minimum of 165 contact hours per year. The teaching staff is comprised of educators who hold degrees in a STEM field and AMNH scientists who serve as mentors and instructors.

The curriculum is modular, and designed with a scope and sequence that introduces youth to the key areas of AMNH research, which are the biological sciences, physical sciences, and anthropological sciences (Electronic supplementary material). Modules within these disciplines incorporate material from the over 40 permanent and special museum exhibitions. They also include a career exploration component in which AMNH scientists facilitate behind-the-scenes tours and provide information and answer questions about their respective career trajectories.

The entire 7-year curriculum, in alignment with the Next Generation Science Standards (National Research Council 2015), is designed to provide participants with repeated experiences for exercising science practices while engaging in different science topics and working alongside a variety of science experts. During the first 2 years of the program, youth in 6th and 7th grades are provided opportunities to practice science through participation in the curricular modules. Beginning in 8th grade, and extending throughout all 4 years of high school, participants are provided multiple opportunities to engage in authentic science investigations. Each summer, participants select their own research team and work with their peers in either a field- or a laboratory-based research project that parallels AMNH research alongside an AMNH scientist or a master science educator. While participating on a research team, the museum youth develop investigable questions; formulate hypotheses; collect, organize, and analyze data; formulate conclusions; and communicate findings. Thus, rather than simply participating in traditional, pre-fabricated science laboratory activities, museum youth engage in the practices of science that are authentic and personally meaningful. During this process, museum youth are afforded opportunities to communicate science to the AMNH community—their peers, their parents, and to the public—and some participants have published their work in scientific journals. The research experiences are rigorous in that youth learn to read scientific articles, interpret big data sets, and conduct literature reviews for their projects in ways that are low stakes, with the goal of helping participants to view themselves as people who know science and can do science (Adams et al. 2014). In the past years, research teams have focused on a variety of museum-related topics including astrophysics, cultural and physical anthropology, DNA barcoding, geology, and urban biodiversity.

Overall, there is specific intention in the design shift taken as participants transition from middle to high school in that as youth move out of their early adolescent years and into their later teen years, they want to be able to have more autonomy, and more flexibility in their academic pathways (Deschenes et al. 2010). To promote autonomy, Lang high school youth are offered elective courses, which provide “deeper dives” into select areas of science and connect students to topics that are currently studied by museum scientists. Additionally, high school youth participate in college and career readiness modules, which include college trips, tours of laboratories, and workshops designed to prepare youth for STEM trajectories. As of June 2017, the program has graduated 12 cohorts of young people for a total of 121 participants. Though graduation rates were lower for earlier cohorts, revisions in program design have brought the current retention rate to approximately 85% signifying youth completing all 7 years of the program.

Research Questions

In this study, we assessed how and to what extent long-term participation in an informal, museum-based OST program impacted the STEM trajectories of youth participants. We collected and analyzed both qualitative and quantitative data, in order to provide points of triangulation and to help us more confidently formulate conclusions on how a long-term, informal OST museum program can impact participants’ STEM outcomes (Creswell 2013; Fitzhugh 2009). Specifically, we investigated the following research questions:

  • In what ways does long-term participation in an informal OST museum program mediate changes in academic and social capital that contribute to persistence with STEM?

  • What hypotheses can we generate about program design principles based on analysis of participants of an informal OST museum program?

  • What are the STEM majors and STEM trajectories of long-term participants of an informal OST museum program?

  • How do the STEM major and STEM career outcomes of these participants compare to other college students and graduates?

Theoretical Underpinnings

The overarching conceptual framework of communities of practice (Lave and Wenger 1991) provides us the tools for understanding youth as they move through the described curricular experience. In a community of practice, individuals with varying skills and expertise engage in, on an ongoing basis, a communal endeavor using shared tools and vocabulary and adhere to key ideas specific to that endeavor. In the Lang program, as youth experience the foundational classes during their middle school years, they learn the disciplinary ideas and practices of science that they will need to engage in authentic scientific research later in the program. When participants transition into the 8th grade, they join research teams, and through to the 12th grade, they construct a community of practice that consists of three different tiers of stakeholders: (1) an expert educator or scientist (team leader), (2) near peers who are alumni of the program serving as teaching assistants, and (3) their peers also interested in the topic.

Participants in the community of practice bring their existing repertoire of knowledge and use of tools and skills and together define their activities. As they engage in the science research, they hone their skills and practice, potentially mediating changes in their academic capital (Bourdieu 1986). In the case of the Lang research groups, the adult expert and youth co-construct their scientific inquiry with the adult expert also serving as a facilitator of the group. With annual opportunities to join different research teams and to engage in authentic science research, museum participants develop relationships with all members of that community, grow their social networks, and mediate changes in their social capital. Thus, the Bourdieusian notion of social and academic capital (Bourdieu 1986) becomes a useful lens because the program aims to support youth in developing persistence with STEM that extends beyond the duration of the program. We posit that multi-year science learning experiences specifically designed to support the development of academic and social capital may be an approach to supporting youth to persist with STEM majors and careers.

The learning experiences at AMNH expose participants to many fields within each of the broader topics of biology, anthropology, and the physical sciences. With each exposure, youth experience the similarities among the sciences, but also learn about the unique attributes within those areas of study. With each science research experience, they are provided an opportunity to enact an identity as a science researcher—a trying on of a possible self—in a low-stake setting where collaboration is encouraged and failure is an option, with rich resources, and with caring and trusting adults and peers (Markus and Nurius 1986). Markus and Nurius (1986) define possible selves as the people we would like to become and claim. They describe an individual representation of possible selves as a “cognitive manifestation of enduring goals, aspirations, motives, fears, and threats” (p. 954) and propose that these manifestations are transformed by socio-cultural experiences throughout a young person’s life. In the case of the museum participants, their experiences of doing the work of science and engaging in authentic research put them in a position to identify as one who does science thus allowing students to view themselves as a science person, or a scientist (Adams et al. 2014; Gray 2013).

Anna Stetsenko (2008) reminds us that the construct of identity is fluid, transforming all of the time, and embedded in activity. It is in the doing of an activity that one is enacting an identity. Youth take on the identity of scientists within communities of practice (Lave and Wenger 1991) in the program. The structure of the program allows them to participate in different scientific communities allowing them to compare and contrast communities of practice of the biological sciences, the anthropological sciences, and the physical sciences, and through that process, develop their understanding of possible selves (Markus and Nurius 1986) as well as the skills and foundational knowledge needed to pursue those possible selves. At AMNH, such experiences can take form in a variety of ways. As an example, in one year, a museum participant can join a geology research team, and within a community of practice framework (Lave and Wenger 1991), this individual will work with near peers practicing and honing skills as an emerging geologist, using the tools of geology as they conduct authentic research, and building social capital (Bourdieu 1986) as they regularly interact with geologists and other science professionals. The same student, during the subsequent year, can join an evolutionary biology research team, and try on a new possible self (Markus and Nurius 1986) as an evolutionary biologist using the tools and skills of that community. Thus, each repeated experience affords museum participants opportunities to try on possible selves (Markus and Nurius 1986) in low-stake settings with the goal of helping students to view themselves as science persons (Adams et al. 2014; Gray 2013).

Methods

Study Sample

Our sample consisted of Lang alumni who responded to surveys about their respective majors and/or careers (66 of 74 participants; 89.2% response rate). Of the 66 alumni that were contacted, 62 indicated their major. Of these 62 respondents, 64.5% were females and 35.5% were male. Further, 32.3% self-identified as African-American, 29.0% as Asian, 22.6% as Latino/a, 12.9% as White/Non-Hispanic, and 3.2% as other. A purposeful sample of a subset of 21 of the 66 participating alumni was selected to participate in interviews based on the following criteria: (1) alumni who participated in Lang in both middle school and high school (minimum of 5 years of participation), (2) a diverse representation of Lang alumni who were present in college (i.e., 1 year removed from the program, 2 years removed from the program, etc.) or who had graduated from college (i.e., 1 year removed from college, 2 years removed from college, etc.) to better assess whether the impacts of participation were sustained over time, and (3) a diverse representation of racial and ethnic backgrounds with an emphasis on individuals who were members of demographic groups historically underrepresented in STEM careers (women and underrepresented ethnic and racial groups). Of the 21 participants in our purposeful sample, 71.4% were female and 28.6% were male. Further, 33.3% self-identified as African-American, 23.8% as Asian, 19.0% as Latino/a, and 23.8% as White/Non-Hispanic. Since our data revealed that most AMNH Lang alumni pursue STEM majors and STEM career trajectories, we also recruited disconfirming cases (Creswell 2013) to better understand why some participants did not persist in a STEM trajectory.

Interviews

Our interview recruitment process allowed us to identify alumni who would help contribute to the development of theory on how and to what extent long-term engagement in an ISE OST program at a museum influenced participants’ long-term STEM engagement. We recruited 21 Lang alumni to participate either in a focus group discussion or in a remote interview via FaceTime. Fifteen alumni participated in one of three focus groups held at AMNH and six alumni who were not living in New York City at the time were individually interviewed remotely. We facilitated semi-structured interviews that allowed alumni to reflect on their experiences at AMNH and to discuss their educational and career trajectories. We prompted discussion via questions related to science identity and social networks but allowed the conversation to move in the direction chosen by the alumni. We probed more deeply into themes that emerged during these discussions and contacted interview participants via follow-up e-mails and interviews to further delve into these categories. We digitally recorded and transcribed all discussions and stored these data in a secure database. To protect their identity, interview participants were numbered from participant 1 to participant 21 and are reported as such in this manuscript.

Qualitative Analyses

The semi-structured interviews were based on open-ended questions designed to elicit stories of the participants’ experiences as Lang scholars (Denzin 2001). The questions allowed for participants to reflect on their experiences and allowed us, as researchers, to engage in a dialogic process with the participants that afforded the emergence of themes during the interview process. This dialogic, narrative approach allowed us to identify and follow themes on how youth’s long-term participation in a museum-based ISE OST program impacted their motivation and interest in a STEM trajectory. To achieve this goal, first we reviewed all narratives from the semi-structured interviews and used the process of memoing (Strauss and Corbin 1998)—i.e., making margin notes, forming initial codes—to develop baseline data. We incorporated open coding and employed a constant comparative approach to identify emerging themes (Strauss and Corbin 1998). From this process, we identified a central phenomenon of interest—in this case, persistence in STEM—and used axial coding to provide insight on what factors caused this core phenomenon, what strategies were applied to impact persistence, what factors influenced these strategies, and what were the consequences of adopting these strategies (Creswell 2013). We applied selective coding to develop theoretical propositions explaining the interrelationship between long-term participation in an informal OST museum program and persistence in STEM and linked these emerging theoretical propositions with established theories (Creswell 2013). Our use of a narrative approach allowed us to uncover themes that interconnected emerging theory with three established theories—identity development (Adams et al. 2014), social capital (Bourdieu 1986), and communities of practice (Lave and Wenger 1991)—to help us illuminate how long-term participation in Lang motivated youth to view themselves as members of the scientific community and to decipher what practices and approaches, if any, influenced their career choices.

Quantitative Analyses

We contacted Lang alumni via email or phone and obtained information on each individual’s college experience. Specifically, we ascertained whether or not an alumnus was attending college or had attended college, and if so, we asked them to identify their respective college or university. For alumni who were attending or had graduated from college, we gathered information on each individual’s respective major and minor, their career interests, and whether or not they had conducted research or participated in extracurricular activities. For college graduates, we gathered data on whether they went to graduate school and we asked each alumnus to identify their present career. We quantified the number and percentage of Lang alumni who engaged in a STEM major or STEM career across demographic groups (sex, ethnicity/race). We used a generalized linear model (glm) framework and multiple comparison tests (Tukey) to compare Lang outcomes to local (City University of New York 2015) and national measures (Landivar 2013a; National Science Board 2016), and to youth who attended specialized science, mathematics, and technology high schools (Thomas and Love 2002). We represented each dependent variable (engagement in a STEM major; engagement in a STEM career) as a binary variable (STEM versus non-STEM); population type (Lang alumni; alumni of specialized science, mathematics, and technology high schools; local college students; national college students/alumni) was represented as the independent variable. We conducted our analyses using the stats package from the R Project of Statistical Computing (R Development Core Team 2017). Because the STEM major and STEM career outcomes were assessed differently by each population type, when comparing the outcomes of Lang alumni to these measures, we made sure our classification matched those of each study. Thus, we classified the STEM majors and/or STEM careers of Lang alumni based on three designations: (1) National Science Foundation’s Scientists and Engineers Data System (http://setstat.nsf.gov), which matched the categorization of national measures made by the National Science Board (National Science Board 2016); (2) Department of Homeland Security (https://www.ice.gov), which matched the categorizations made by the City University of New York (2015); and (3) the National Center for Education Statistics (https://nces.ed.gov), which matched the categorizations of alumni of specialized science, mathematics, and engineering high schools (Thomas and Love 2002).

Validation and Reliability

In an effort to increase the validity of our findings, we incorporated triangulation methods, which involved the use of multiple researchers to collect and synthesize multiple sources of qualitative and quantitative data (Creswell 2013; Stake and Mares 2001, 2005). To assess consistency across coders (reliability), we calculated inter-coder agreement using both percent agreement (Lombard et al. 2002) and Cohen’s kappa (Cohen 1968) applying methodologies established for semi-structured interviews (Campbell et al. 2013). Specifically, we applied an inductive approach to coding, and through a collaborative process, we identified emerging themes. An independent coder was provided with approximately 20% of the raw text that was previously coded and asked to assign each section of this text to the selected categories (Thomas 2003). Percent agreement was 77.5% and Cohen’s kappa (k) was 0.71 both indicative of substantial agreement (Cohen 1968).

Results and Discussion

The results and discussion below brought together what we have learned from both qualitative and quantitative analyses. First, we describe four key themes that emerged supporting our central thesis that long-term engagement in ISE OST programs fosters persistence in STEM (Table 1): (1) Practitioners of Science, (2) Social Networks, (3) Shared Science Identity, and (4) Discovering Possible Selves. Then, we provide comparison data with local and national statistics demonstrating how alumni are faring relative to their counterparts.

Table 1 Themes lending support that long-term engagement in a museum-based ISE OST program cultivated persistence in STEM
Full size table

Practitioners of Science

Throughout their Lang experience, youth were provided opportunities to engage in authentic experiences as practitioners of science including laboratory investigations, fieldwork, exhibition design, publication opportunities, and the use of AMNH collections to investigate research questions. Beginning in the 8th grade, youth participated in research teams and worked with mentors in either a field- or laboratory-based research project that connected to AMNH research. Participant 19 recalled her experience on an ecology and evolution research team in which her team conducted fieldwork by collecting fish from various state parks coupled with genetic research in an AMNH laboratory. She credited this experience on her decision to major in ecology, evolution, and behavior and her resolve to continue fieldwork in Africa as part of her senior thesis at an Ivy League university. Participant 12, a mechanical engineering graduate from a major northeast university, equated his experience in Lang to a PhD program: “Normally the duration of a PhD program is six years and to me, completing the Lang Science Program was almost the equivalent of completing a PhD program because every year we did…research projects, and participating in…Lang [gave] me research experience and I am grateful for that.”

As practitioners of science, Lang alumni described how specific skills they learned at AMNH prepared them for a STEM trajectory, and they discussed how they gained real-world experience doing science. Alumni reflected on being constantly encouraged to ask questions, to develop hypotheses, to design a research plan, and if necessary, “to tweak [the research] design or the questions that you were asking” (participant 4). They also discussed their experiences collecting and organizing data, conducting statistical analyses, and how presenting and submitting their work for publications validated their efforts. For example, one alumnus (participant 3) majoring in biochemistry at an urban Ivy League university discussed a DNA barcoding project that she completed at Lang, and her experience presenting her research at an AMNH exposition: “When we presented our research, there were actual scientists and they were respectful, actually asking me really hard questions, and they weren’t really treating me like a kid, which I really liked.” Youth were active members of a community of practice because they were using tools, languages, skills, and techniques alongside and with the support of practicing scientists. Many Lang alumni reported that, as a result of these opportunities to conduct authentic research, they were awardees at national science writing competitions on a diversity of projects including but not limited to the study of hamadryas baboon behavior in the Prospect Park Zoo (participant 10), a field biology project on the ecology of earthworms (participant 8), and a survey of trees and mosses of New York City (participant 12). These authentic science research experiences were set up longitudinally with the goal of providing enough time for museum youth to become a part of a community of practice, exercise the needed research skills, and experience the scientific setbacks and challenges that inevitably occur when doing science investigation work.

Shared Science Identity

While Lang youth differed in their geographical, socioeconomic, ethnic, and racial backgrounds, one commonality they shared was their shared science identity. For up to seven years, Lang youth from all over New York City converged at AMNH during various OST hours because of their common interest in science. As participant 2 expressed, “It was nice to have a group of people who were all interested in the same thing…” While a number of museum participants described their school-based peers as “not the best influences” (participant 17) or “not much interested in science” (participant 13), these youth described their AMNH peers as “the smartest people I know” (participant 11), “some of my best friends in the entire world” (participant 1), and “the friends that your parents like” (participant 17). Participant 8 expressed her participation in AMNH as her “outlet to be as nerdy as possible.” As youth continued their participation in Lang, they increasingly began to identify themselves as “science persons,” and there was consensus among them that the longitudinal duration of the program helped to foster their developing identities as people who do science. Youth reflected that their experiences occurred during “a very influential time” (participant 19), that it “[left] an imprint” (participant 13), and that “it was something I was doing for so long—it seemed natural to continue with it” (participant 9). Participant 13 reflected on how her informal science learning experiences at AMNH influenced her science identity, and ultimately her decision to engage in a STEM career:

I remember the hiking trips. I remember sitting in the rooms and meeting different scientists. I have a memory of…I want to have an ‘ist’ at the end of my name, you know, biologist, wildlife biologist. I was like, ah – I want to be a scientist. So I came to that goal…a veterinarian technologist [laughter]. At a young age, it helped me…made me more receptive to think that…my love for science can become a career… We met somebody who studies spiders. I just remember saying I want the ‘ist’ at the end of my name. I want to be a scientist, also. It seems like an amazing career to pursue.

Participant 2 stated, “We like got the same level of respect as real researchers even though we were so young.” Importantly, these museum-based, ISE experiences also gave Lang alumni a sense of self-efficacy, as summarized by participant 14: “I thought I could do anything. I was like – Lang…science research…There is no biology course that this college can throw at me that I can’t take [laughter].” Multiple participants recalled the rigorous work they completed at Lang and their investigations on a broad range of topics in science. Our data suggest that these experiences mediated changes in their academic capital, and for many museum youth, an increased confidence in tackling their science courses.

Social Networks

Lang youth credited their enduring relationships with museum educators, scientists, and peers for helping them persist in a STEM trajectory. Youth described how the relationships that they developed with scientists while doing science helped clarify what it was like to engage in a STEM trajectory. For example, participant 20, a research technician at a major medical institution in New York City who recently completed a degree in biochemistry and molecular biology, and who is entering graduate school next semester, recalled his experiences conducting research in the lab of an AMNH paleontologist. He credited his mentor for being “the one who first told me what the whole process is like about getting a PhD.” Moreover, alumni who were first-generation college students described their relationships with museum scientists and educators as playing “a pivotal role in my life” (participant 8), “the driving force that kept me going” (participant 11), and “impactful experiences” (participant 2). For example, participant 8, now a cancer researcher, stated: “I didn’t have that much resources at home and role models, I think it was really important for me to be around museum people. I can’t imagine what my life would be like…my work ethic would be like without being exposed to these people.” Participant 21 stated, “I think having the opportunity to work that close with somebody within science is just amazing, and I think when you are trying to pinpoint what you want to do in life…getting that advice and knowledge from somebody whose actively doing it is important.” We found evidence that these networks, which consisted of scientists, educators, and peers, were critical as museum youth developed more sophisticated understandings of both careers and the pathways to those careers. Our data suggest that these repeated and varied engagement afforded museum participants multiple opportunities to engage in STEM activities alongside and with various people, and that it led to the comfort necessary for developing social networks.

The social networks described by youth extended beyond their tenure at AMNH. Lang alumni described how they have remained in contact with each other and with their museum mentors. A number of youth stated that their cohorts established Facebook pages where they could sustain communication with each other. Many alumni also discussed how they continue to meet at AMNH annually both at formal events such as “alumni mixers” hosted by AMNH and informally by meeting up with their “lifelong friends.” Alumni described how their participation in Lang opened doors for them. Participant 21, while describing her multi-year engagement in Lang, discussed the prestige associated with AMNH, and stated: “…when you put that on a college resume, it shows commitment and determination.” Participant 7 credited his association with the Lang program with helping him “to land [his] first research internship.” Alumni also described how they maintained contact with their mentors well after graduation. Participant 11 stated that she was thinking of dropping out of college, and she credits her sustained communication with her museum mentors for helping her complete a degree in mathematics, and as she articulates, “making sure that I finish and finish well and finish strong.” Others described how their mentors assisted them by writing letters of recommendation, by continuing to provide college and career advice, and by introducing them to other STEM professionals. Altogether, students expressed that Lang provided opportunities for multiple quality relationships that have been critical in supporting them as they traversed across the STEM pipeline.

Aside from the networking benefits secured by Lang alumni via prolonged relationships with their mentors and peers, alumni noted that their associations extended to the next generation of Lang scholars. For example, many alumni were invited to return to Lang to lead discussions on what it was like to be a STEM major or engaged in a STEM career. Other alumni returned to AMNH following graduation to work as teaching assistants, or as one alumnus described, “…to help the younger generation” (participant 13). One alumnus described her experience as a Lang teaching assistant as “…one of my most impactful experiences” (participant 2). Now in graduate school, she recalls how this experience motivated her to become an advocate for getting more historically underrepresented youth interested in careers in environmental science. Participant 17, as a result of her teaching assistant experience, now has aspirations of becoming a STEM educator and is interested in an AMNH career. Others described how they helped lead college tours repeating an aspect of the program that they experienced when they were Lang scholars themselves. Only this time, they engaged in the role of the mentor rather than the mentee. An alumnus (participant 7) attending an urban university provided one example of this experience. He recalled how he conducted a tour of his university for Lang youth, and that he included a behind-the-scenes visit to the lab where he conducted research. He stated, “The program meant so much to me, I didn’t mind doing it.” Thus, the social networks that many Lang alumni attributed to their persistence in STEM may also have an intergenerational effect as graduates provide similar guidance and advice on STEM careers to the new generation of scholars.

Discovering Possible Selves

An emerging theme uncovered from the interview data was that participation in Lang helped youth develop their sense of possible selves fostering their preparedness for college and for a STEM career. In terms of college preparation, many scholars indicated that the program augmented their perceived college readiness, or as participant 7 stated, “[Lang] helped me figure out what I want to do in college.” Because youth attended Lang for multiple years, alumni indicated that their longitudinal participation increased their confidence in considering a STEM discipline as a major. Specifically, Lang scholars expressed a sense of preparedness relative to their peers in school. Participant 14, a biology graduate of a northeastern college, reflected: “[I was]…not panicking like my friends about all this college stuff because I’ve been talking about it in [Lang] since the 7th grade”. In the possible selves construct of identity development (Markus and Nurius 1986), having positive experiences associated with success is instrumental for youth for imagining possibilities, and for imagining “successful professional” types of possible selves. Markus and Nurius (1986) describe how in the face of challenges and obstacles, people who have negative experiences will struggle with imagining themselves having success, but people with a mindset of positive possibilities are more likely to see the possibility of being in that successful profession. Consistent with this idea, our data indicated that many Lang youth who experienced success with science developed mindsets of positive possibilities helping them persist through difficult times. For example, participant 9, a biomedical engineering graduate, expressed how her prior experiences in Lang helped her to persist in a STEM major even when faced with a highly competitive college environment. She stated that without her Lang background, her college experience “…would have been very detrimental” and she acknowledged that “…maybe I would have even switched majors because it was really hard to be in that kind of academic atmosphere.” Even alumni who chose not to major in STEM felt that Lang prepared them for these types of experiences in college. For example, one alumnus (participant 6) majoring in performing arts at a northeastern university stated that she felt like “such a well-rounded person,” and that if she chose to, she would feel very comfortable switching out of her major.

To facilitate participants’ discoveries of their possible selves, when Lang youth transitioned into high school, a college readiness curriculum was incorporated into the program. The curriculum includes seminars on a range of topics including financial aid, scholarships, college essay writing, and preparing for an interview. As part of this curriculum, Lang alumni who were STEM majors visit AMNH and provide informal lunch time seminars to Lang youth on what it is like to be a STEM major and they also provide advice on how to prepare for college. The college readiness curriculum is also comprised of a free SAT preparation course and an annual onsite college fair. Indeed, Lang scholars largely attributed their decision to attend a particular college or university, and their decision to pursue a STEM major on the college readiness component, and on the ensuing social interactions that occurred as a result of these experiences. Participant 5, presently a neuroscience major at an Ivy League school, recalled the following: “I think I remember the college component the most because that was the most helpful for me getting into college and having people help me write my essays and stuff and going on all those tours.” Each summer, Lang educators accompanied youth on college tours, many of which were led by Lang alumni. The college trips were scheduled to colleges and universities along the northeast corridor and included behind-the-scenes tours of research departments of various STEM disciplines. For example, youth toured the physics department at Stony Brook University, the anthropology department at New York University, and the ecology, evolution, and environmental biology department at Columbia University. Encouragingly, many Lang youth applied to and matriculated into the programs highlighted during these college tours. In a number of cases, these alumni were among the first in their family to attend college, and many attributed their special relationships with museum staff in helping them to navigate the college application process. Participant 8, whose parents were immigrants from Vietnam, stated, “My parents didn’t go to college...so I think I had a knack for really attaching or sticking with people I found as like a mentor.” Participant 14 attributed his decision to attend a college in upstate New York and to major in biology on interactions with a Lang educator who chaperoned him on two different occasions to the college, and on an older Lang alumnus: “He [the alumnus] had a big impact on me wanting to go there. He had a good time at Lang, and was able to do science-associated stuff at [the college]. I ended up working for him as a science teacher.” Thus, the social networks developed during the college readiness component were also important influences on youth’s decisions to attend college, to pursue a STEM degree, and to discover their possible selves.

The career readiness component was another important vehicle for affording museum youth opportunities to try on possible selves. Participants typically entered the program with limited awareness of STEM career options and generally perceived a career in medicine as “synonymous with science” (participant 16). Indeed, many Lang alumni indicated that participation in the program aided in their preparedness for a STEM career. Participant 18 reflected that before Lang, he was unaware that “there were scientists only for birds or only for insects” and participant 13 stated that she realized that “science doesn’t necessarily mean being premed or an engineer.” As Lang alumni progressed from middle school to high school, they were increasingly exposed to the unique resources of a museum—curators, collections, scientists, and laboratories—and expanded their sense of STEM career options. In high school, Lang scholars participated in career readiness workshops that were designed to prepare youth for STEM careers and that included a component in which guest scientists presented information and answered questions about their respective career trajectories. High school participants also had the option of engaging in intensive research experiences and were exposed to many different authentic science experiences including working in an astrophysics lab, conducting fieldwork, and participating in a DNA barcoding project. Multiple alumni attributed these research experiences for helping them realize that a science career was “more than working in a lab and taking measurements” (participant 1), and that there were “all different types [of careers] out there” (participant 19). Program participants described walking around the exhibits, interacting with different scientists, being exposed to a variety of STEM careers, and going “behind the scenes” to observe “all the scientists doing their research” (participant 5). Students recalled experiences that contributed to their awareness of STEM career options such as the talk from “the guy with leeches” (participant 4), the behind-the-scenes tour with “the insect guy…the one who had…bed bugs” (participant 2), and a scientist who spent six hours helping participant 8 skin a bat for a research project. Many stated that these experiences influenced their decisions to pursue a STEM career.

Persistence with STEM Majors and Careers

Our qualitative analyses showed that Lang alumni undergo particular experiences that set the stage for their success with STEM studies. However, the question remained whether these alumni actually persisted with their intended major and career. Based on definitions provided by the National Science Foundation’s Scientists and Engineers Data System (http://setstat.nsf.gov), 51 of 62 (82.3%) Lang alumni were engaged in a STEM major, a percentage that significantly exceeded national measures (glm: estimate = − 2.765; SE = 0.332; Z = − 8.317; p < 0.001; Table 2). Further, 87.5% of females and 72.7% of males that participated in Lang engaged in a STEM major, which was well above the national averages of 11.7 and 26.0% (National Science Board 2016). Likewise, the percentage of African-American (75.0%), Asian (77.8%), Caucasian (100.0%), and Hispanic (92.9%) Lang alumni engaged in STEM majors exceeded national measures (National Science Board 2016).

Table 2 Number and percentage of STEM majors of Lang alumni in relation to national statistics, local statistics, and specialized science, mathematics, and technology high schools
Full size table

We also compared the STEM majors of Lang college students to alumni of City University of New York (2015) based on the more stringent definitions provided by the Department of Homeland Security (https://www.ice.gov). Based on these categorizations, 40 of 62 (64.5%) Lang alumni were engaged in a STEM major, a percentage that was significantly higher than their City University of New York counterparts (glm: estimate = − 2.006; SE = 0.266; Z = − 7.557; p < 0.001; Table 2). Lastly, we compared the STEM majors of Lang college students to alumni of specialized science, mathematics, and technology high schools based on definitions provided by the National Center for Education Statistics (https://nces.ed.gov). Based on these categorizations, the percentage of STEM majors of Lang alumni (66.1%) exceeded the percentage of STEM majors among alumni of specialized science, mathematics, and technology high schools (61.7 and 56.5% respectively) across the USA (Thomas and Love 2002). However, this difference was not statistically significant (Table 2).

In terms of careers, based on categorization by the National Science Foundation’s Scientists and Engineers Data System (http://setstat.nsf.gov), 12 of 19 (63.1%) Lang post-graduates were engaged in a STEM career, a figure that exceeded the national average of 12.9% (estimate = − 2.446; SE = 0.476; Z = − 5.143; p < 0.001). Of these Lang alumni, 58.3% of females and 71.4% of males engaged in a STEM career, which was well above the national averages of 13.9% for females and 12.1% for males respectively (Landivar 2013a; Palmer et al. 2010). The percentage of African-American (66.7%), Asian (57.1%), Caucasian (75.0%), and Hispanic (50.0%) Lang alumni engaged in STEM careers also exceeded national measures (Landivar 2013a; Palmer et al. 2010). Lastly, based on definitions provided by the Department of Homeland Security (https://www.ice.gov) and the National Center for Education Statistics (https://nces.ed.gov), 11 of 19 (57.9%) of Lang post-graduates were engaged in a STEM career. However, to date, no data are available on the STEM careers of alumni of City University of New York or the alumni of specialized science, mathematics, and technology high schools (Erdogan and Stuessy 2015). Thus, we were unable to compare the STEM career outcomes of Lang alumni to these populations. Lastly, in a survey of a subset of Lang alumni not yet engaged in careers, 18 of 25 (72.0%) indicated an interest in pursuing a STEM career, and 13 of 26 (50.0%) indicated that they were participating in science research with a scientist at their college.

Research Questions Revisited

In our analyses, we assessed the STEM outcomes of long-term participants of a museum-based ISE OST program. We collected and synthesized both quantitative and qualitative data, to address four main research questions. The first two questions probed whether the engagement in a long-term science program led to changes in youth that could become critical supports in their trajectories and persistence with STEM, which allowed us to generate hypotheses on programmatic design principles linked to these outcomes. Our qualitative analyses revealed four key themes critical for cultivating persistence in STEM (Table 1): (1) Practitioners of Science, (2) Social Networks, (3) Shared Science Identity, and (4) Discovering Possible Selves. To address our third research question, we corroborated our quantitative findings with our qualitative data to assess how long-term participation in Lang mediated changes in academic and social capital that contributed to persistence with STEM. We investigated how and to what extent long-term engagement in Lang impacted participants’ STEM trajectories as measured by the STEM majors and STEM careers of program alumni. In response to this question, our quantitative analyses revealed that Lang alumni engaged in STEM majors and careers at rates that far exceeded national measures. The fourth research question pushed to address the issue of self-selection bias since so many Lang participants entered the program with an interest in and personal motivation in science. Thus, we compared the STEM majors of Lang participants to similar youth—alumni of specialized science high schools. Here, we found that the percentage of Lang alumni engaged in STEM majors were comparable to these students (Table 2). Collectively, these data suggest that long-term engagement in ISE OST programs matter and are potentially important pathways for fostering a community of practice, for building social capital, and for helping participants to develop a science-affinity identity.

From our narrative analyses, we developed a theoretical explanation of how youth’s long-term participation in a museum-based ISE OST program impacted their STEM trajectories (Fig. 1). We found that long-term engagement in an ISE OST program with structured opportunities to join scientific communities of practice (Lave and Wenger 1991) cultivated persistence in STEM, and that this effect extended well beyond the duration of the program as evidenced by both qualitative and quantitative outcomes. Our data revealed that three factors—long-term participation, informal nature of science education experiences, and the unique resources of AMNH—helped to foster a community of practice (Lave and Wenger 1991) and to build social capital (Bourdieu 1986). The program design principle of giving youth multiple opportunities to participate as practitioners of science gave them exposure to doing science, each time learning new skills, applying existing skills, comparing and contrasting scientific communities of practice, and imagining their own lives as scientists. Participating in the communities of practice framework (Lave and Wenger 1991) also meant that there were more social network opportunities available to youth as many formed enduring relationships with museum peers, educators, scientists, and curators that persisted beyond the program and that were often helpful as alumni navigated their college and career trajectories.

Fig. 1
figure 1

Conceptual Framework. Graphic depiction of how youth’s long-term participation in a museum-based ISE OST program impacts their persistence in STEM

Full size image

Lang scholars expressed a sentiment of belonging to AMNH both as a physical space and as a community of practice. As a physical space, AMNH provided youth with access to over 40 permanent and temporary exhibit halls, laboratory spaces, and collections. The informal environment allowed participants to explore questions of interest at their own pace and to “discover things on your own” (participant 11). Participant 14 stated that through interaction with his peers and “by just wandering around…you were learning.” Participant 9 distinguished her learning experiences at AMNH from her formal school environment: “It was more relaxed than school where it was very intense, competitive…everything revolved around grades and then in Lang it was all about learning so you were able to enjoy it and have fun.” As program participants transitioned from middle school to high school, their sense of belonging to AMNH became more pronounced as evidenced by reflections of alumni. “I think as when time went on I realized what a privilege it was to be part of an institution that is globally known,” stated participant 16. Participant 2 reflected that as she got older, she became more aware of the opportunities within the institution, including internships, research opportunities, and afterschool clubs, and that “the Museum name itself” carried a lot of weight and “really, really helps you out in the future.” Thus, by accessing and acknowledging the assets of informal learning experiences, youth built social capital and, over time, became aware of the opportunities these experiences afforded them as they perused the STEM landscape.

Conclusion

In this investigation, we embarked on a research study to assess how and to what extent long-term participation in an informal, museum-based OST program impacted the STEM outcomes of participants. In accordance with previous studies, we found that engagement in a long-term ISE OST program had cascading effects (McCreedy and Dierking 2013) that shaped participants’ science-affinity identities (Adams et al. 2014; Gray 2013) and impacted their STEM major and STEM career outcomes longitudinally. We found that the percentage of Lang alumni engaged in STEM majors well exceeded national and local measures. By design, youth who elect to apply for programs like Lang do so because they are presumably highly motivated youth. Encouragingly, we found that the percentage of STEM majors among Lang alumni were comparable to alumni of specialized science high schools. Collectively, these data supported our thesis that multi-year informal science learning experiences specifically designed to support the development of academic and social capital (Bourdieu 1986) augment the STEM trajectories of its participants.

Limitations

We recognize that there are limitations of our study. First, as many youth enter the program with an interest and personal motivation in science, there is the problem of self-selection of participants. Second, since youth are not randomly selected to participate in Lang, there is also the problem of selection bias by the institution. Third, because youth participated in Lang for so long, there might be a lack of critical distance; consequently, the alumni interviewed might identify themselves as representatives of the program and thus underestimate negative impact resulting in response biases. To circumvent the aforementioned problems, in this study, we compared the STEM major outcomes of Lang participants to youth who attended specialized science high schools because there is also self-selection and institutional selection bias inherent in these types of institutions. Moreover, our incorporation of triangulation methods (Creswell 2013; Fitzhugh 2009) helped us to more confidently support our conclusion that a long-term, informal OST museum program can impact participants’ STEM outcomes. Nevertheless, we recognize that without a randomized control trial, something that was not possible in the present study for ethical and logistical reasons, that self-selection, institutional selection, and response biases remain as limitations of this research study. Lastly, a fourth limitation of this study was our moderate sample size for the quantitative analyses. Of the 74 eligible alumni, 66 responded to our surveys for a response rate of 89.2%. As more cohorts graduate from Lang (as of June 2017, there are now 121 participants who have completed the program), it is our intention to build on the findings of this study, and to continue to longitudinally assess the impacts on participants using an ever-increasing sample size.

Future Research

Our study has prompted several ideas for future research. First, one research platform that we think would contribute to the field are studies in which participants of ISE OST programs are followed longitudinally. Specifically, researchers can assess a “continuum of persistence,” and measure whether participants’ interest in pursuing a STEM trajectory decreases, sustains, or increases over time (Dierking and Carroll 2008), and from these data, determine what factors impact such decisions. Importantly, these studies could also include interviews with students who started a longitudinal program and later dropped out allowing researchers insight on why some students, but not others, persist in ISE OST programs. Second, as self-selection and institutional bias remain as criticisms of studies of ISE programs (Bell 2009), future studies should work to circumvent these problems. To address the problem of self-selection, we recommend, where possible, that future studies of ISE programs compare the STEM outcomes of program participants to a comparison group (i.e., applicants who were not accepted into their respective programs (e.g., Jayaratne et al. 2003; Koch et al. 2010; Schumacher et al. 2009). To address the problem of institutional bias, one somewhat risky policy that could be adopted by ISE institutions is to randomly select applicants, and then compare program participants to those who applied but were not selected via random lottery (e.g., Hubelbank et al. 2007). Another option is to assess programs using alternative statistical methods such as propensity score analysis, which can be applied to simulate randomization and estimate causal effects (Rosenbaum and Rubin 1983). The aforementioned methodology has been used in recent years to compare treatment groups to comparative groups covarying in shared demographic characteristics (Hahs-Vaughn and Onwuegbuzie 2006). Third, as social networks was one of the major themes that emerged from our qualitative analyses, and from our interview data, we found that the quantity and quality of enduring relationships appear to be critical for helping youth persist in a STEM trajectory; we suggest that future studies incorporate social network analyses, which will allow researchers to compare the quantity and quality of the STEM relationships of ISE participants to comparison groups and will help the field gain insight on how these interactions impact STEM persistence (Carolan 2014). Fourth, one open question that emerged from this process was how many hours minimally are required to have this experience? Our data suggest that 64 contact hours conducting authentic research spread over five years, from grades 8–12, is certainly appropriate. However, as a field, establishing a better understanding of what constitutes a sustained meaningful experience and the minimum exposure necessary to affect sustained outcomes is a fruitful area for future research. Lastly, there are several important areas of ISE research that we believe will advance the field of science education and our understanding of how OST programs impact participants. Thus, we recommend that future research focus on motivations for participation (i.e., what motivates youth to attend and to persist in an ISE program), exposure to content (i.e., commonalities and differences between formal and informal curricula), influence on school grades and course choices, and understanding the nature of science.

Implications

This study has implications for how other ISE institutions could adopt and adapt the programmatic design principles identified from this study. Internationally, there is great interest in museum youth programs learning from each other and in particular, developing ways to document the design principles that make programs successful (Klein et al. 2017). As shown in Fig. 1, our triangulated data suggest that there are three factors—long-term participation, informal science experiences, and the unique resources of an ISE institution—that help to foster a community of practice (Lave and Wenger 1991) and to build social capital (Bourdieu 1986). The program design principles of (1) affording multiple opportunities for youth to engage as practitioners of science where they learn the tools and language of science; (2) providing exposure to and repeated experiences where participants actively engage in conversations, research, and workshops with other practitioners of science—that is, interactions with STEM professionals such as scientists, educators, and graduate students to build social networks; (3) furnishing opportunities for participants to make choices in terms of elective courses and research teams so that they can develop shared science identities with like-minded individuals; and (4) offering exposure to and preparation for a variety of STEM majors and STEM careers so that they can engage in discovering possible selves and in imagining new ways of being in science are important for augmenting academic and social capital and cultivating persistence in STEM. We suggest that these design principles that emerged from the analysis of our program would afford rich learning experiences in informal science learning contexts for young people towards persistence in STEM and diversifying STEM majors and careers.