Introduction

Integration and synthesis of theories, concepts, and data across the borders of disciplines has been a transformative force since the inception of science (see Gillispie 1960 for an historical view; Weingart and Stehr 2000; Abbott 2001; Rhoten et al. 2008 for contemporary perspectives). Psychology, biochemistry, molecular biology, and other fields and subfields of science arose through interdisciplinary integration and synthesis, often catalyzed by practical challenges or structural changes in academic organization, resources, or job markets (Ben-David and Collins 1966; Mullins 1972; Stokes 1997). Recognizing the power and possibilities of interdisciplinarity, policy makers and practicing scientists have invested, invented, and organized to create conditions amenable to integration and synthesis across disciplinary borders. For example, the National Center for Ecological Analysis and Synthesis (NCEAS), which was founded in 1995 with support from the National Science Foundation (NSF), has pioneered research that is transforming ecology and related fields of science (Hackett et al. 2008). NCEAS has, in turn, become a model for more than a dozen similar centers founded in other fields of science, in the U.S. and other nations. A complementary, simultaneous transformation is the emergence and institutionalization of integrative biology, which “incorporates the biological, physical, socio-economic, mathematical, engineering, and humanities components” to build explanations and solve problems that bridge disciplines, taxa, levels of biological organization, and time scales (from physiological through ecological to evolutionary) (Wake 2008).

The momentum of these successful scientific/intellectual movements (Frickel and Gross 2005), combined with escalating societal challenges that demand integrative strategies of remediation (e.g., climate change; new infectious diseases) and commitment to transformative research at the highest levels of U.S. science policy (Bement 2005; NSB 2007), ensure that the pace of integrative, synthetic science will accelerate. The most recent examples of NSF-level efforts to further interdisciplinary research include, for example, the Environment, Society and the Economy opportunity (NSF 09-031), the Virtual Organizations as Sociotechnical Systems program (NSF 08-550), and the Cyber-enabled Discovery and Innovation initiative (08-604). With these programs, NSF has explicitly sought to engage computer scientists, natural scientists and social scientists in an integrated team structure and to promote the tight coupling among teams thought to be essential to the analysis of complex interactions. The thematic foci of these programs reflect not just lingering intellectual curiosities but emerging “real world” practical concerns at the intersection of human, natural, and built systems.

In such purpose-driven transformations some theorists of science detect the emergence of a “second mode” of scientific discovery that complements theory-driven inquiry with a new form that is shaped by its applications, conducted across disciplinary borders, housed in new types of organizations, performed in a reflexive manner, and evaluated by a kaleidoscopically changing “peer” community using varied criteria (Gibbons et al. 1994; Nowotny et al. 2003). Mode 2 science has stimulated considerable theorizing, substantial criticism, and little empirical study, yet it remains a compelling way to characterize a science policy environment increasingly determined to apply scientific resources to societal concerns with ever greater purpose, precision, and efficiency.

In parallel to the emergence of these interdisciplinary research programs, there have also been investments in interdisciplinary education programs explicitly designed to prepare the next generation of scientists. In the United States, federal agencies like the National Science Foundation and the National Institutes of Health have allocated hundreds of millions of dollars to reform graduate education and training programs in ways that prepare students for new modes of interdisciplinary integration and synthesis (Martin and Umberger 2003). One of the most expansive and deliberate of these efforts is the Integrative Graduate Education Research and Training (IGERT) initiative. Implemented in 1997, the IGERT initiative challenges universities to create coherent interdisciplinary programs for educating the next generation of scientists and engineers to work across disciplines and employment sectors to address the deep intellectual and practical problems that lie ahead. To do so, IGERT programs must go beyond mere interdisciplinary graduate education to prepare scientists to collaborate across disciplinary borders, integrate and synthesize disparate ideas and results, work fluidly in a succession of varied organizations and sectors, and through all this to be guided by a sense of the greater public good and a reliable moral compass. Beyond its impact on participating students and faculty, IGERT aspires to “catalyze a cultural change in graduate education for students, faculty, and institutions by establishing innovative new models for graduate education and training in a fertile environment for collaborative research that transcends traditional disciplinary boundaries” (National Science Foundation 2002). The IGERT initiative, in effect, aspires to educate scientists capable of creating and flourishing in a varied and changing landscape of scientific research.

Most of the current literature on interdisciplinarity, whether supportive or critical, explains its rise as a broad cultural phenomenon—a spirit of the times—or as the calculated structural response of organizations to funding opportunities (Jacobs and Frickel 2009). Little empirical research has addressed the processes by which members of these new scientific/intellectual movements actually engage in integration and synthesis across disciplines to build explanations and solve problems of relevance to them and the broader society. Available research examines how organizational characteristics create a climate for interdisciplinary research collaboration that is conducive to the focused exchange and evaluation of ideas essential for synthesis to occur (see Hackett 2005; Hackett et al. 2008; Rhoten 2004; Rhoten et al. 2008; Rhoten and Parker 2005; Rhoten and Pfirman 2007). In this paper, however, we are concerned with whether interdisciplinary education programs can positively influence and perhaps even accelerate the preparation of students seeking to initiate or join scientific/intellectual movements that are interdisciplinary in nature. More specifically, we ask whether and how the IGERT program influences students’ ability to collaborate across disciplines and their capacity to do original, integrative, synthetic work in such collaborations.

We begin by outlining the intellectual forces surrounding the emergence of interdisciplinarity and then describe the institutional features of the IGERT program. The empirical heart of our paper focuses on the methodology and results of our novel social science experiment, which we call the Snowbird Charrette. This experiment, which was part of a larger empirical study of the workings of the IGERT program, compared the performance of groups composed of students enrolled in IGERT programs with that of groups enrolled in disciplinary graduate programs, both drawn from a national sample, with the goal of assessing whether and in what ways they differed in their approaches to interdisciplinary collaboration and capacities for synthesis and integration. We conclude with observations about the future of expertise and graduate education.

The Rise of Interdisciplinary Science

In the United States, the rise of interdisciplinarity has been influenced by four fundamental forces: “relevance, experience, liberation, and totality” and has “positioned itself as a direct critique of ‘old knowledge,’” with interdisciplinarians often distinguishing themselves from the “tribes and territories” of people who “occupy the physical and intellectual spaces” of disciplines (Bird 2001: 466–467; for background see Lipset 1972; Ladd and Lipset 1975; Jencks and Riesman 1977; Grant and Riesman 1978). In this regard, interdisciplinarity is often thought of as a “challenge to the limitations or premises of the prevailing organization of knowledge or its representation in an institutionally recognized form” (Salter and Hearn 1996, p. 43). Further, while interdisciplinarity may borrow and incorporate disciplinary approaches to knowledge when they are useful, it is not constrained by disciplinary methods and rules for the uses of such approaches.Footnote 1

From the perspective of disciplinarians, then, interdisciplinary research can appear unfounded, illegitimate, transgressive, and fundamentally challenging. But, for the members of these scientific/intellectual movements, interdisciplinary research is often thought to allow for more freedom and creativity by encouraging the scholar or student to range across different fields and experience several as s/he so desires, needs (Kavaloski 1979). The “[i]ntellectual cross-pressures generated by an interdisciplinary outlook liberate a person’s thinking from the limiting assumptions of his own professional group, and stimulate fresh vision” (Milgram 1969: 103).

Further, in contrast to resigning oneself to making nothing but incremental contributions upon which others will build and surpass without much recognition, interdisciplinarity is often promoted as offering scholars and students a greater chance of accomplishing something akin to novel breakthroughs and original insights. The assumption here is that, given the preponderance of intellectual research done within the separate and distinct disciplines, there is opportunity for unique and inventive knowledge to emerge at the interstices between the disciplines.

Finally, counter to the complaints that disciplinary science has become too abstract and idealized, a common positioning of interdisciplinary research is that it seeks to solve “real life” problems of society (Hansson 1999; Klein 2000; Roy 1979), revealing some of the contemporary enthusiasm for interdisciplinarity as driven by a sociopolitical strategy (Barry et al. 2008). Interdisciplinarity is endorsed as a means of transforming science from the realm of the general and abstract to the full complexity and specificity of concrete reality, and is thus imputed with the purpose of addressing socially relevant “real-world” problems whose solutions are beyond the scope of a single discipline or area of research practice (Funtowicz and Ravetz 1993; Klein 2000; National Research Council 2004). The rising tide of “applied” and “use-inspired” research along with the current push of technology has made interdisciplinary “problem-solving” a new form and focus of knowledge production for academic as well as extra-academic science (Klein 2000).

Despite these strong intellectual forces behind interdisciplinarity, scientific/intellectual movements centered on integrative, interdisciplinary science still face an uphill struggle because disciplines are powerful, interdisciplinary communication is difficult, integration and synthesis are elusive, and engagement with problems of the real world places extraordinary demands on the judgment and probity of scientists (Pielke 2007). Thus, while a zeitgeist may transform a culture, or isomorphic forces might change an organizational field, movements can effect change only through the perseverance of their members. It is those members that are of concern to us here. Who will do this new sort of science? Where and how will they be educated? How will they work, and will they succeed in producing new knowledge in new ways?

The Role of the IGERT Program

In 1997, the NSF implemented the Integrative Graduate Education and Research Training (IGERT) program, which is designed to “catalyze a cultural change in graduate education, for students, faculty, and institutions, by establishing innovative models for graduate education and training in a fertile environment for collaborative research that transcends traditional disciplinary boundaries” (NSF07-540:5). The first IGERT competition was held in 1998, with annual competitions continuing to the present. By December 2009 more than 200 IGERT programs have been funded at a cost of approximately $2M–$3M each, involving about 3,000 students at a total cost exceeding $500M.Footnote 2

The IGERT program espouses a distinctive model of graduate education that is delivered by a varied group of faculty who share an interdisciplinary theme and conduct a spectrum of innovative educational activities that integrate education with research, students with faculty, disciplines with one another, and academics with those working in other sectors (IGERT Program Solicitation 08-540: 6). IGERT aims to develop scientists and engineers capable of working across disciplinary boundaries, national borders, and economic sectors. In recognition of the ways such aspirations extend and entangle the professional principles that guide the conduct of scientists, every IGERT also includes training in research ethics and responsible professional conduct.

The program is highly selective, at both the institutional and the individual levels.Footnote 3 In recent funding cycles, hundreds of public and private universities competed through a two-stage process for about 25 awards of approximately $3M each that will support graduate training and related activities for a 5-year period.Footnote 4 Most of the money provides graduate students with a $30,000 annual stipend (plus benefits), tuition, and an allowance for travel, equipment, and related research expenses. With this desirable package of support, IGERT programs enroll students who are talented and motivated, and provide them with time and resources to do well.Footnote 5

IGERT programs offer a strategic research site for studying the education and research of young scientists and engineers who are preparing to bridge disciplines, built integrative theories, collaborate extensively, engage diverse publics, and span the gap between research and its uses. In our 4-year study of the IGERT program, we used surveys, interviews, site visits, and social network analysis to examine program design, institutional context, student and faculty performance, and scientific innovation and productivity. From this work we have learned about the motivations and aspirations of IGERT students, the satisfactions and frustrations of their faculty, and the networks of collaboration that emerge from interactions catalyzed by the program (Interim Report 2005 http://programs.ssrc.org/ki/fis/pubs/).

In accord with our results, a recent program evaluation, conducted by a team from Abt Associates and funded by NSF, “finds that doctoral students participating in IGERT projects receive different educational experiences than non-IGERT students…and that the IGERT program has been successful in achieving its goal of improving graduate educational programs in science and engineering” (Abt 2006: ix). That is true as far as the organization and delivery of the educational program are concerned. But what remains unknown is whether and in what ways IGERT students differ from others in their conceptual approaches, cognitive abilities, and collaborative behaviors. Do IGERT students do science differently than their counterparts in discipline-based programs?

To address such questions, which reflect the most important goals but least tangible outcomes of the IGERT program, we designed and conducted a novel, real-world experiment in collaborative interdisciplinary research conceptualization and design. We called this experiment the Snowbird Charrette.

Experimental Design of the Charrette

The term ‘charrette’ has evolved from a nineteenth century exercise at the École des Beaux-Arts in Paris where architectural students were given a design problem to be solved within a fixed period of time. When time expired, a charrette, or small cart, passed through the aisles to collect the students’ work. In our adaptation of this exercise, we formed groups of graduate students drawn from IGERT programs and from disciplinary programs, presented them with a research problem, and set them to work for two and a half days. Their task was to design the kernel of a research proposal responsive to the problem and to present their proposal in the form of a five-page document and a 20 minute presentation, which were then evaluated by a panel of experts in the environmental sciences. The research problem statement, developed by the panel using a modified Delphi process, was designed to be comparative, to involve both social forces and ecosystem processes and services, and to join analysis with action or policy (see “Appendix I”). A subset of the expert panel that designed the research problem traveled to Snowbird to judge the presentations and proposals.

Design and Sample

The experiment was designed primarily to compare groups of IGERT students, who had had explicit training in integrative, interdisciplinary approaches to research, with groups drawn from disciplinary programs. To control for the influence of duration in graduate school we separately grouped students in the early years of graduate school (years 1 and 2) from advanced graduate students (in year 3 and beyond). The design yielded a 2 × 2 table with one replication, as indicated in Table 1.

Table 1 Schematic design of charrette
Full size table

Potential participants were solicited through a national mailing to graduate departments in the environmental sciences.Footnote 6 Online applications requested information about students’ graduate program (IGERT or not), educational background and field of study, GRE scores, and a brief essay explaining why the student wished to participate. From 158 completed applications we chose a sample of 48 students—half from IGERT programs, half from other programs—that varied in geographic, disciplinary, and institutional origin. From these we formed eight groups of six students each, such that each group was homogeneous with respect to graduate program (IGERT or disciplinary) and graduate career stage (first and second year versus years three and beyond), but heterogeneous in disciplinary composition (each group included students from the life, physical, and social sciences) and balanced by gender (each group included at least two men and two women; in all, 23 men and 25 women took part in the study).

Conduct of the Charrette

The charrette took place from August 24 to 27, 2006, at the Cliff Lodge in Snowbird, Utah. Participants arrived Thursday afternoon, and the study began that night with a plenary dinner, an overview of aims and plans, a review of researchers’ ethical obligations and participants’ rights, and a brief initial meeting of the groups where they became acquainted with one another and with the research problem. Collaboration began in earnest Friday morning and continued through noon Sunday, followed by a plenary session of group presentations and expert questions and commentary.

In its work room each group found a round table, chairs, a sofa, computers, Internet connectivity, flipchart, tablets, and pens. Each room was also equipped with a video camera that recorded activities at the table, three microphones distributed around the table to capture discussion, and a trained observer who kept notes as unobtrusively as possible. Observers were instructed to limit their interaction with group members and were provided with a protocol that asked them to attend to matters of group socialization, identity formation, interaction patterns (leadership, challenges, exclusion), communication (especially cross-disciplinary questioning, explanation, and understanding), and skeptical or evaluative comments about ideas or research plans. Observers made systematic notes on a structured rating sheet every 20 minute and made continual free-form notes of group process.

The panel of experts who discussed proposal presentations at the Sunday afternoon plenary and rated the written versions during the following several days was composed of three ecologists, an atmospheric scientist, a mathematician, an economist, and a marine management official. During the plenary, the panel questioned presenters and offered comments and advice. After the plenary the experts provided written evaluations to the participants and to the researchers. Working independently and without consultation or attribution, experts rated fifteen aspects of the proposals, using criteria and five-point scales (1 = poor to 5 = excellent) that they had helped us to adapt from the work of Boix Mansilla and Dawes Duraising (2007; see “Appendix II”).

Observer notes, expert evaluations, our on-site focus group interviews as well as incidental observations, and the documentary record of proposals and presentations support the fact that the charrette was an intellectually and socially engaging experience. We report on the group processes of collaboration and discovery elsewhere (Rhoten et al. 2008), and so will offer some brief observations here. Perhaps the clearest overall assessment of the charrette experience was summarized by an observer who wrote:

In all this session seems quite unremarkable until you consider the fact that these six members, who have known one another for less than 24 h[ours], are now collectively engaged in coming up with a set of hypotheses for a proposal they haven’t even quite nailed down yet…all members are engaged in this collective task and appear to be participating equally. There is no conflict … and very little miscommunication.

Observers were unanimous in reporting that groups engaged the task fully and seriously, investing long hours of effort in their collaboration. While formally free to do as they pleased once they arrived in Snowbird—principles governing the treatment of human subjects demand that no one is compelled to participate, even after the study directors have invested in airplane tickets and hotel rooms—no one opted out and every member of every group remained fully involved in the task. In fact, in even so short a time a substantial amount of group identity formed, marked by social cohesiveness (groups often took meals together), group sociality (groups hiking, swimming, drinking, or partying together), and collective group identity (one group called itself the “Green Team,” another was the “Riverine Group,” and a third coined the term “aqualogy” for its ecological study of aqueous environments). Perhaps most importantly, the groups performed very well scientifically in the judgment of the expert observers, whose comments about the proposals included:

“compelling conceptual framework…disciplinary methods and techniques brought to bear on the question are very strong…the problem is generally well motivated…and the group gave a provocative and clear presentation.”

“impressive effort…strong conceptual model of resilience”

“a strong conceptual model [that] addressed a problem that is worldwide”

“ambitious but very well articulated”

“highly interdisciplinary, effectively weaving together both economic and ecological perspectives…tightly argued and well presented; this was very close to a final proposal”

“highly interdisciplinary…addresses a compelling and difficult challenge facing humans”

“a compelling problem…an impressive effort…a very well presented proposal…a problem of clear global importance”

“impressive effort…the structure is clean and the elements clear”

Every expert considered every proposal a strong effort, judged within the constraints of the charrette design. In fact, as might be expected of senior scientists evaluating junior scientists, ratings tended to be somewhat generous and tightly clustered, which presented a challenge for analysis. But taken as a whole, six observers, six expert reviewers, and two study directors concurred in the judgment that the charrette research problem and design accomplished its objective of presenting student-subjects with a research challenge that would engage their intellectual and collaborative abilities.

Expectations, Analysis, and Results

Our principal research question is whether groups of students trained in IGERT programs perform better on the design of interdisciplinary and collaborative research approaches than do groups trained in disciplinary programs. Since experience in graduate school may influence performance, both through the learning and experience students acquire and through the selective pressures that drive a substantial fraction of students to leave graduate school, IGERT and disciplinary groups will be compared within two levels of experience in graduate school: the first two years of graduate study (“junior” students) and three or more years of graduate work (“senior” students: rows 1 and 2 of Table 1). We expected IGERT groups to outperform disciplinary groups among students early in their graduate careers, and we expected larger differences among those more advanced in their graduate careers. Our expectations were based on the following evidence drawn from the Abt Associates study and our own fieldwork.

First, it is the mission and responsibility of IGERT programs to enroll students with interest in and aptitude for integrative, interdisciplinary research, so selective pressures should favor IGERT programs at both levels of seniority. Survey data support this expectation: About 85% of IGERT directors and 72% of department chairs surveyed in our study said that IGERT grants allow them to recruit more highly qualified students. The Abt Associates study concurs: faculty found IGERT students to be significantly more capable and promising than disciplinary students, and 75% of IGERT faculty “believe that the students in the IGERT program are better qualified than the usual department students in terms of their academic and research potential” (Abt 2006, p. 67).

Second, the educational content of IGERT training is more likely than disciplinary training to include multidisciplinary research (76 vs. 42%), team research projects (66 vs. 50%), research projects with students from other disciplines (64 vs. 36%), and training in communication outside the student’s home discipline (50 vs. 22%; Abt 2006: 33, 35). In consequence, IGERT students are more likely than their disciplinary counterparts to feel very well prepared to work in multidisciplinary teams (42 vs. 19%) and to communicate with people both inside their fields (52 vs. 41%) and outside (34 vs. 13%; Abt 2006: 34, 36).

Finally, the charrette research problem and evaluation criteria were designed specifically to assess IGERT’s stated aims. That is, charrette participants were given a research problem that spans disciplines, entails both scientific and practical considerations, and requires attention to matters of values and ethics. For these reasons, we expected groups of IGERT students to outperform disciplinary students in the charrette research task, and for the difference to be larger among senior graduate students than among those in their first or second years of study. All of these claims are based on IGERT program design and mission, and faculty and student reports about program content, quality, and impact. But what matters is how students work together and what they produce, and that is where findings from the charrette complement, and perhaps challenge, results obtained through other methods.

Analysis

The basic data of this study are expert judges’ ratings of 15 aspects of each group’s proposal, using 5-point scales with explicit verbal descriptions for each score (see “Appendix II”). Table 2 presents the mean ratings on each item for IGERT groups and disciplinary groups, organized by whether the students were in the early or latter years of graduate study. To focus and simplify the statistical analysis, 13 of the 15 items are combined into additive scales representing disciplinary quality (literature, knowledge, methods, and depth; α = .89), interdisciplinary quality (interdisciplinarity, integration, synthesis, breadth, and comprehensiveness; α = .96), and scientific reasoning (formulation, skepticism, rigor, and originality; α = .80). Two items—intellectual merit and broader impacts—stand alone because these represent the major dimensions of merit review employed by the National Science Foundation.

Table 2 Mean ratings for IGERT and disciplinary groups on outcome criteria and scales by year of graduate study
Full size table

The data presented in Table 2 show a surprising result: for students in the first years of graduate study, IGERT groups outperformed their disciplinary counterparts on all but one of the criteria (synthesis), with differences as large as a half-point on scales with ranges of only four points. Among students in the latter years of graduate study, however, the pattern is reversed: groups formed of disciplinary students outperformed groups of IGERT students on 13 of the 15 measures (exceptions were disciplinary literature, which was tied, and scientific skepticism). Several of the differences were a half-point or greater.

The magnitude and consistency of these data are reassuring, but the study design and small sample size do not meet the assumptions required for parametric statistical tests (particularly that the intervals of the scales are equal, that judges assign similar meanings to values on the rating scales, and that scores are drawn from an approximately normal distribution with a variance that can be estimated from these data; Freedman et al. 1991). Instead of comparing means using parametric statistics, we will transform the data in a way that relaxes assumptions yet allows our results to be compared with expectations derived from an explicit (binomial) probability model. We construct the comparisons in the following way.

Imagine that the four groups formed of students in the first years of graduate study are labeled A and E (for those in IGERT) and C and G (for those in disciplinary programs; see Table 1). When, for example, a particular judge scores the intellectual merit of the groups’ proposals, she is implicitly making 4 pair-wise comparisons: Group A to Group C, Group A to Group G, Group E to Group C, and Group E to Group G. If ratings were assigned to groups purely by chance, we would expect IGERT groups to outscore disciplinary groups about 40% of the time, for disciplinary groups to outscore IGERT groups 40% of the time, and for the remaining 20% of comparisons to result in ties.Footnote 7 On the assumption that judges assign scores independently of one another, which is true by design (the judges worked alone, without consultation, and each dimension scored was characterized distinctly, see “Appendix II”), then the expected number of successes (s) for IGERT groups in a set of (n) comparisons is:

s = np (where p = .50, since groups have equal chances of success), with

σ = √npq and a distribution that is approximately normal (Freedman et al. 1991).

Stating the statistical problem in this way provides a reasonable probability model for evaluating whether the pattern of results occurred by chance, while avoiding the strong assumptions imposed by parametric statistics, which are untenable with our study design and sample.

Table 3 confirms the pattern of results detected in the presentation of mean scores in Table 2: Among students in the first two years of graduate study, IGERT groups “won” 11 of 14 comparisons on intellectual merit and 7 of 9 comparisons on broader impacts (p < .05 for each). Among senior graduate students, in contrast, disciplinary groups won 6 of 8 comparisons on each of these two dimensions. For scales measuring disciplinary quality, interdisciplinary quality, and scientific reasoning there is a similar pattern with considerably larger differences: early-career IGERT groups won 30/37 comparisons on disciplinary quality, 29/46 on interdisciplinary quality, and 39/57 on scientific reasoning. In contrast, among graduate students in third year and beyond, disciplinary groups overwhelmingly received favorable comparisons on all three dimensions (disciplinary quality: 29/43; interdisciplinary quality: 49/57; scientific reasoning: 25/38).

Table 3 Comparison of IGERT and disciplinary groups on outcome scales by year of graduate study
Full size table

We asked if the results could be an artifact of differences in academic ability, as measured by the GRE. Table 4 shows virtual equality of average scores across all four categories of groups, so differences in academic ability, as measured by the GRE, do not explain the results obtained in this study.

Table 4 GRE scores for IGERT and disciplinary students by year of graduate study
Full size table

Collaboration Observed

Expert comments about the proposals that were rated most highly on all three measures—originality, interdisciplinarity and disciplinarity—used such phrases as the following in their review of the proposals: “impressive effort for such a short period of time,” “relatively unique framework,” “compelling problem for society,” “conceptual model was well-developed and well-presented,” “disciplinary methods and techniques … are very strong,” “good interdisciplinary thinking,” “well-motivated [and] carefully justified,” and “provocative and clear presentation.” By comparison, expert comments about the proposals that were rated most poorly on all three dimensions offered comments such as: “little in the methods,” “proposed study was original [but not] … well-posed,” “not convinced that it addressed an issue of the highest scientific and/or societal urgency,” “picked an over-studied system,” “proposal is too rigid in its approach,” “chose a topic with widespread pre-existing knowledge,” “naïve expectations,” “dominant role of one group member,” and “presenters had [difficulty] in stating the problem, in identifying the hypotheses, and in describing the broader impact.”

Observers’ field notes for three of the more successful groups (the two “junior” IGERT groups and one of the senior disciplinary groups) reveal aspects of the groups’ collaborative behaviors that may account for differences in the quality of their proposals. Both junior IGERT groups approached the initial problem identification (or idea generation) phase of the exercise by setting out to ask a ‘broad theoretical question, which would have findings relevant for both natural and social sciences’ (Student Quote, Field Notes, Group E). One of the junior IGERT groups, for example, started with the very broad notion of studying something to do with water. The group quickly moved beyond this topical focus, however, because while water covers much of the Earth and touches upon a broad range of disciplines and research issues, the group believed it should pose a ‘good scientific question’ rather than study a broad topic that merely engaged everyone’s expertise (Student Quote, Field Notes, Group E).

Having made that decision, the group soon hit upon the idea of ‘estuary resilience.’ This was a successful move because any effort to study the process of resilience requires framing the research with a model of a coupled natural-human system, which integrated the ecological and economic expertise of the students. Following a somewhat similar process, the second junior IGERT group was in the midst of discussing the theme of monoculture when the students found themselves arguing about biodiversity. While the group had been divided between land and water topics, the concept of ‘biodiversity’ brought these two domains together in what was again a coupled natural-human systems framework, much as their peers in the other junior IGERT group.

Similar to the two junior IGERT groups, the high-scoring senior non-IGERT group was drawn to the overarching theme of water because of its breadth and potential inclusiveness. Whereas the junior IGERT groups were less inclined to inventory their members’ skills and backgrounds, this group did so expressly to limit the selection of their problem to one that had ‘overlapping research questions and interests’ (Student Quote, Field Notes, Group D). As with the two junior IGERT groups, this senior IGERT group ultimately refined their overly broad theme to form an original topic, in this case one having to do with potable water. However, unlike the two junior IGERT groups, which followed a linear path of convergence to integrate their ideas, this senior IGERT group instead iterated ideas in a repetitive sequence of mini-cycles of divergence and convergence. Nevertheless, despite its different path, this group also ultimately arrived at a coupled natural-human systems approach to frame its problem.

Importantly, then, the most successful groups (as measured in terms of originality, interdisciplinarity, and disciplinarity by the expert panel) all focused on topics concerned with coupled natural-human systems and all used modeling as a tool to capture and convey the relational structures of the complex systems they proposed to study. In each case, the model became the ‘boundary object’ that integrated the group’s interactions, dialogue, and labor (Star and Griesemer 1989). The development and use of a boundary object by the groups not only enabled but also encouraged interdisciplinary acts of collaboration by virtue of its ability to “satisfy the informational requirements of each community of practice” (Bowker and Star 1999, 297). Moreover, when constructed correctly to be intrinsically plastic in nature, boundary objects actually foster the formation of conceptual feedback loops that then work to advance integration and synthesis of ideas on an epistemological level. In this way, the use of models allowed the members to ‘see’ the selected problem, identify the opportunities for and interdependencies of their individual contributions, and then restructure their own perspectives to combine knowledge and skills that might have otherwise remained divided along disciplinary lines.

In contrast to these high scoring groups, which collaborated either integratively or iteratively around an epistemological model of complexity, both low-scoring senior IGERT groups seemed instead to construct a research problem by collaborating performatively around a set of social tactics and prescriptive lessons about collaboration. That is to say, it almost appeared that they were following a script or imitating a normative model for collaboration. For example, while the junior IGERT groups accounted for members’ disciplines and considered their intellectual diversity, this was an internally-driven and organic process rather than a cultivated practice. By contrast, the senior groups were very focused on externally-oriented rules and practices. For example, one senior IGERT group focused on establishing criteria of disciplinary ‘legitimacy,’ ‘expedience’ and ‘applicability’ (Field Notes, Group F), while the other group “self-organized quickly, stating IDs, qualifications, expectations,” then were so “very concerned with efficiency that they planned out entire 2.5 days before reading the problem” statement (Field Notes, Group B).

These senior IGERT groups were self-conscious about ‘interdisciplinarity’ and ‘collaboration’ to the point of requiring that each student’s disciplinary expertise be addressed in the problem and of assessing each other’s anticipated contribution on the bases of their interdisciplinary qualifications (Field Notes, Group F and Group B). Ironically, however, while these groups were quite deliberately enacting an interdisciplinary collaboration, they were not particularly good at being an interdisciplinary collaboration. In both senior IGERT groups, students were hesitant ‘to go far outside the realm of knowledge, as it will take more time and effort’ (Field Notes, Student Quote, Group F) and were not willing to compromise his or her individual discipline’s methods, which wasted lots of time and prevented consensus (Field Notes, Group B). In the end, the senior IGERT groups appeared to be schooled in the rules of interdisciplinary collaboration but had not learned the roles of interdisciplinary collaboration. In fact, we found an inverse relationship between their use of social tactics and their ability to integrate or synthesize, leading us to argue that viable interdisciplinary collaboration requires mechanisms that operate at the epistemological level, not merely at the social level.

Discussion: Limitations and Implications

IGERT, a well-established program for offering interdisciplinary graduate education, serves as a strategic site for examining how an explicitly integrative and interdisciplinary graduate education influences the process and products of research. We have employed a variety of methods to do so, centered on a real-world experiment but including interviews, questionnaires, document analysis, and a blend of structured and opportunistic observation. From the charrette experiment we learned that presenting graduate student groups with a challenge of this sort is “real” in its appearance and consequences: students worked well and diligently on the task and produced smart, sophisticated science. The charrette worked so well in this regard that it may have broader applicability as an exercise in collaborative problem solving that may be built into educational programs.

The results of this experiment indicate that IGERT groups outperformed disciplinary groups among students in the first years of graduate study, but that the reverse occurred among students in the third year and beyond. The result for first and second year students conformed closely to our expectations, which were derived from an understanding of the IGERT program, but the result for students in the latter years of graduate study contradicted expectations. We expected initial advantages to cumulate as more knowledge and skills are acquired to fit into an interdisciplinary framework, and as the student’s integrative abilities increase with intellectual maturation.

The findings are statistically significant, substantial in magnitude, consistent in direction, and robust to reasonable challenges. Recognizing the limitations of the study—a one-shot design in a single domain, a small and self-selected sample in unusual surroundings, performing a creative task that imposes distinctive demands on group interactions their interaction—we provisionally accept the results and address their possible causes and implications.

IGERT programs often begin by plunging students into a mix of interdisciplinary educational and extracurricular activities at the outset of graduate study. Students become strongly engaged, both intellectually and socially, with interdisciplinary patterns of thought and behavior, and begin to shape a professional identity around them. In time, however, the level and intensity of interdisciplinary activity diminishes, and as that is happening disciplinary demands arise: qualifying exams, dissertation proposal requirements and evaluations, the dissertation itself and its defense—all demanding mastery and technical facility, and all starkly real in their consequences. After a pleasant initial immersion in pools of interdisciplinarity, the advanced graduate student encounters the more treacherous waters and institutional realities of a nascent professional career (culminating in the prospect of a discipline-dominated job market).

Disciplines offer reliable recipes for the production of certified knowledge: certain sorts of problems are understood to be worth attention and they may be addressed using an agreed conceptual language, empirical approach, analytic technique, and rhetoric which, in combination, reach an explanatory endpoint that may be published. Disciplines are so named for a reason: in addition to disciplining members (and candidate members) through a spectrum of rewards and punishments, they also discipline thought through the concepts, methods, and standards—the epistemic cultures—they instill in their members (Clarke 1998; Knorr Cetina 1999).

Students acquire the elements of disciplines during graduate training, and the discipline of rewards and punishments is employed to motivate learning. Observing and experiencing this in action (when colleagues fail exams or face technical difficulties in their dissertation work), advanced graduate students in IGERT programs may become more self-consciously disciplinary and less sanguinely interdisciplinary than they had been earlier in the graduate careers. They seek a method or pathway to success, and are too mature and perhaps too vulnerable to fail, and so become conventional in their thinking and behavior. In contrast, students in the first years of graduate study were more original in their charrette work than those more advanced, perhaps because originality had not been trained out of them, perhaps because they could afford to fail without shame, or perhaps because, like Heisenberg, they remained unaware of the “magnificent unity” of their fields. More advanced students, aware of this unity and of the method a discipline imparts to its disciples, looked to their training for a method or formula—a script—that would lead to successful interdisciplinary collaboration.

There is no such script. Instead, interdisciplinary research demands a leap of faith from the safety of disciplinary patterns into a new sphere of creative collaboration. Several groups in the charrette took this leap, working integratively or iteratively to fashion a workable research problem and plausible approach. Others held fast to the ways of their epistemic cultures, and some looked to their interdisciplinary training in collaborative behavior for guidance. But Archibald MacLeish’s assertion that “A poem should not mean/But be” applies to the art of collaboration just as it does to the art of poetry: collaborators must do the work of working together, not merely mean to collaborate. Senior IGERT groups, trained in the process of collaboration, were self-conscious about their process but insufficiently engaged in producing. A group absorbed in self-conscious reflection on the meaning and process of collaboration may neglect to do the real work of collaborating.

Support only in the early years of graduate education will likely be insufficient to effect the transformations that motivate the IGERT program. Additional resources, applied throughout the graduate career, would help graduate students to withstand the countervailing disciplinary forces that lie ahead in their doctoral programs. And as the student enters the early career, confronting disciplinary journals, disciplinary funding opportunities, and departmental (read, usually, disciplinary) standards for renewal, promotion, and tenure, ongoing support will be needed to transform investments in interdisciplinary graduate education into new knowledge and new modes of inquiry. While many of these barriers to successful interdisciplinary research and education programs are organizational, we should not neglect the cognitive and developmental processes that shape how people learn, think, and innovate. The challenge is to understand when in a scholar’s professional and intellectual maturation it is safest and most effective to dive into the pools of interdisciplinarity. One does not take this plunge empty-handed, but carries along disciplinary knowledge, skills, and concepts—interdisciplinary research is done among disciplines, not without them—and one must be confident that the collaborative work of the group will produce a question, a goal, and the means to produce a plausible answer and argue its merits. As Pauli cautioned Heisenberg, “lack of knowledge is no guarantee of success.”

Conclusion

We opened this article by positing that the momentum of successful scientific/intellectual movements, combined with escalating societal challenges and rising institutional investments in interdisciplinary programming, would ensure the continuing acceleration of integrative, interdisciplinary graduate education and research. But science is changing in ways that are easier to sense than to describe with much precision or certainty. Without committing to the causes and detailed characterizations of “post-normal” science (Funtowicz and Ravetz 1993), “Mode 2” science (Gibbons et al. 1994), or other alternative visions of the future of science, we can acknowledge that such widely shared perceptions may become real in their consequences and ask if IGERT programs, as currently conceptualized, would prepare “post-normal” scientists capable of bringing knowledge and rigorous methods from a spectrum of disciplines to bear on the high-stakes, high-uncertainty problems characteristic of “post-normal science”—climate change, for example—that are upon us today? Would scientists trained in today’s IGERT programs be prepared for a career in the multidisciplinary, multivalent, multivocal world of Mode 2 science? We think not.

The IGERT initiative was intended to ‘catalyze a cultural change in graduate education’ and ‘produce creative agents for change,’ but it was conceived within the paradigm of normal science and designed in the context of traditional academia. On some campuses our interviews revealed teams of faculty and students institutionalizing integrative programs in bioinformatics, nanotechnology, and other interdisciplinary areas. Often the campus climate was predisposed to welcome interdisciplinary innovation and program elements were assembling before the IGERT proposal was written, so designing and managing an IGERT graduate program contributed to an ongoing, perhaps incipient, structural change. To be sure, the initiative has led students, scientists, and universities to take a giant step in the right direction, but its current ‘disciplinary plus’ model is much more of an educational program than it is a new learning experience. In most (but not all) instances, an IGERT program asks students to acquire a mixture of disciplinary educations from a variety of departments (or centers or institutes), with some attention to their integration, and to engagement with the wider world. We would argue that this focus on educational breadth and organizational interconnections should be reinforced through the creation of new pedagogical dynamics designed to help students metabolize domain expertise with interdisciplinary experience by addressing real and challenging problems in complex environments of competing values and contested ethics. The resulting metabolites would be new forms of expertise, embodied by the scientists, activated by research challenges, deployed in new forms of organization, and resulting in original knowledge and novel solutions.

Expertise in this sense is not a binary property that one may have or not have, but instead is a quality that may be characterized along many dimensions. For example, Hatano and Inagaki (1986) distinguish “routine expertise” from “adaptive expertise:” Whereas routine experts solve familiar problems efficiently, often by using specialized tools and techniques, adaptive experts solve novel problems by creatively transferring and transforming elements of diagnoses, interpretations, and solutions across contexts (Schwartz et al. 2005). They suggest that one must develop both expertise profiles, and that a substrate of routine expertise is essential to any future conceptual adaptive innovation. In comparison, Evans and Collins (2008) array specialist expertise on a continuum that ranges from superficial “beer-mat” and popular understanding through primary-source knowledge and interactional expertise (the ability to work with experts in such a manner that they recognize you as a fellow expert) to generative contributory expertise, which is the ability to advance original knowledge in an expert domain. Traditional graduate programs generally produce contributory experts of a narrow gauge who are strong in the routine uses of expertise but somewhat uneven or unskilled in the adaptive or innovative dimension. IGERT programs augment that profile with mid-range expertise in one or two additional fields, preparing students to interact with experts through exposure to diverse epistemic cultures, but in the larger view IGERT programs may not be much better than others at instilling contributory or adaptive expertise in their graduates.

The challenges ahead require more than this, and so the task for interdisciplinary graduate education is to endow students with an appropriate portfolio of expertises and the abilities, occasions, and resources to develop them. Accomplishing this is less a matter of balancing exposure to disciplinary departments and more about balancing experience in different pedagogical opportunities. To this end it may strengthen interdisciplinary graduate education if students first acquire disciplinary concepts, knowledge, methods, and epistemic standards, and then learn to recombine them into interdisciplinary patterns of inquiry and understanding. A strong undergraduate education in a discipline might establish this foundation, but in our experience few do so. An educational model that oscillates between disciplinary and interdisciplinary phases might work best, allowing the material to be learned and integrated by parts. One way to achieve this balance is to return the charrette to its original purposes and to deploy it as an educational opportunity rather than an experimental methodology.

It is difficult to anticipate the knowledge and technology challenges that lie ahead, but easy to assert that they are growing in magnitude, complexity, and consequence. Not only will scientists and engineers be expected to produce new knowledge and know-how, they will also be expected to direct their originality toward the most pressing needs of the nation and the world, and to do so with exquisite sensitivity for the ethical, legal, and social implications of their work. Meeting such challenges will require new ways of organizing the work of scientists and engineers, new ways of integrating professional and indigenous forms of knowledge, perhaps a new understanding of the nature of science and technology, and certainly innovative strategies for educating and socializing those who will produce, integrate, and deploy knowledge.