In recent years, federal agencies and science policy experts have been trying to foster interdisciplinary research (IDR)—which “integrates perspectives, information, data, techniques, tools, concepts, and/or theories from two or more disciplines” (National Academies of Science et al. 2005: 188)—because of its high potential to spark innovation and solve complex, real-world problems (Lyall et al. 2013; Rhoten and Parker 2004). This is evident in recent publications, including the National Academies of Sciences’ Facilitating Interdisciplinary Research (2005) and Convergence: Facilitating Transdisciplinary Integration of Life Sciences, Physical Sciences, Engineering, and Beyond (2014), which suggest ways to foster cross-cutting and potentially innovative research. The National Science Foundation has accelerated cross-cutting program initiatives like CREATIV, interdisciplinary programs like Science of Science and Innovation Policy (SciSIP), and interdisciplinary research centers. And in recent years, the White House Office for Science and Technology Policy was investigating how to best foster IDR.

Anecdotal evidence suggests that a number of universities are heeding this call, and trying to catalyze IDR on their campuses. Research administrators are promoting IDR in many ways. Some ways can be developed and modified with relative ease, such as using internal seed grants to foster cross-disciplinary teams, developing new administrative positions to foster IDR initiatives (e.g., Duke University’s Vice Provost for Interdisciplinary Studies), pursuing cluster hiring initiatives (e.g., Dartmouth, Purdue University and UC Riverside) and hiring interdisciplinary faculty (Duke University 2017; Flaherty 2016; Jaschik 2014; Sá 2008). Other ways require not only financial support, but a longer-term commitment and organizational restructuring, such as establishing and maintaining research centers and interdisciplinary departments. For example, the Boston University School of Medicine founded the Evans Center for Interdisciplinary Biomedical Research as well as an intra-departmental Section of Computational Biomedicine in order to facilitate IDR and innovation (Coleman et al. 2013). It is these more structural manifestations of IDR that interest us here.

Beyond anecdotal evidence, we know little about whether and how universities are committed to interdisciplinary research. To be sure, a number of scholars have proposed and developed ways to measure the interdisciplinarity of a given research paper (Mugabushaka et al. 2016; Porter et al. 2007), a scholar (Leahey et al. 2017), a topic (Light and Adams 2017), a lab (Jensen and Lutkouskaya 2014), and even a field (Brint et al. 2009). But how can we capture commitment to interdisciplinarity at the more aggregate and policy-relevant level of the university? In this paper, we build upon and extend ideas proposed by Brint et al. (2009) and Jacobs and Frickel (2009) to measure universities’ structural commitment to IDR. The number of research centers (Jacobs and Frickel 2009) and departments (NAS 2005) alone may indicate IDR, and the ratio of the two may be informative (Gumport and Snydman 2002; Jacobs and Frickel 2009). Traditional departments are indeed disciplinary (Lee 2007), but we agree with Jacobs and Frickel (2009) that it is unwise to assume that all departments are disciplinary units, and build on Brint et al. (2009) effort to distinguish interdisciplinary departments from others. And although most research centers were developed as boundary organizations (Guston 2000) at the interstices of traditional disciplines (Geiger 2013), not all of them may be interdisciplinary in nature. We thus interrogate traditional assumptions by distinguishing interdisciplinary research centers from other centers, and interdisciplinary departments from other departments. We use this more nuanced understanding to develop a measure of universities’ structural commitment to IDR.

It is important to examine universities’ commitment to IDR because over half of the basic research conducted in the United States takes place within a university setting (Geiger and Sa 2005). Universities also play an increasingly prominent role outside the academy, by collaborating with industry, helping found start-ups, and providing expertise to companies’ scientific advisory boards (Barringer and Slaughter 2016; Barringer and Riffe 2018; Mathies and Slaughter 2013; Slaughter et al. 2014; Stuart and Ding 2006). As university and industry models of research, development, and evaluation become more intertwined (Bercovitz and Feldman 2011; Owen-Smith 2003), it is critical to understand how and to what extent universities promote innovative knowledge production and commercialization through their pursuit of IDR. While previous research has found that research centers perform this function at a single university (Biancani et al. 2014), we investigate commitment to IDR among the entire population of top research universities—something that is missing from the literature to date (Sá 2008).

Developing a measure of universities’ structural commitment to interdisciplinary research is a critical and timely task that will advance scholarship and research policy. By examining interdisciplinarity across all fields, we expand the scope of previous investigations, which have tended to focus on specific fields like biochemistry (Chen et al. 2015), computer science (Chakraborty 2018), nanotechnology (Schummer 2004), and bionanoscience (Rafols and Meyer 2010), and often at a single university or center (Biancani et al. 2014; Gowanlock and Gazan 2013; Taşkın and Aydinoglu 2015). It allows us, via descriptive analyses presented herein, to gauge how this form of commitment to IDR varies across universities and university types. The measure and our findings may be useful to universities, their administrators, and science policy experts at the national level who continue to seek ways to promote interdisciplinary research, and thereby innovative and transformative science (National Research Council 2014).

Efforts to measure IDR

Given its timeliness and policy import, a number of scholars have developed measures of interdisciplinary research,Footnote 1 most of which are binary and measured at the scholar or paper level. At the scholar level, Jacobs and Frickel (2009) suggest that holding a joint appointment, or working in a discipline that differs from the discipline one was trained in, may distinguish interdisciplinary scholars from others. This measure has been used by Leahey and Blume (2017) to examine how interdisciplinary appointments influence patenting activity. At the paper level, at least three measures have been developed. Chen et al. (2015: 454) suggest that interdisciplinary papers are those that are cited by a discipline other than the discipline the author(s) represent. Larivière and Gingras (2010) assess the percentage of a paper’s references made to journals from other fields. Porter et al. (2007) develop a paper-level measure of what they call “integration” that uniquely considers the relatedness of the fields (e.g., whether a sociology paper references a cognate field like anthropology, or a more distant field like geology), and not just the variety of fields and the evenness of their distribution.Footnote 2 This measure has been used by Leahey et al. (2017) to understand the career consequences of engaging in interdisciplinary research.

For several reasons, we choose not to aggregate such scholar- and paper-level measures of IDR to the university level. First, the most appropriate method of aggregation is uncertain, but also consequential. To compare universities in the U.K., Rafols et al. (2012) pool references appearing in all articles published by all university members, and then calculate an overall, or global, measure of interdisciplinarity. Cassi et al. (2014) show that this captures something substantively different than first calculating IDR at the article level, and then averaging those scores to generate a university level measure.Footnote 3 Second, these scholar- and paper-level measures capture the extent of engagement in IDR rather than commitment to it, which is our focus. And third, the most nuanced measures of IDR (like Porter et al. (2007) integration score) rely on not only information about a focal paper, but also about a focal paper’s references, so the data demands are daunting for the large number of universities (n = 157) we study. We would need to extract and code detailed information from all publications (including their bibliographies) associated with the 157 universities in our sample. For these reasons, we aim to develop an organization-level measure.

A few unit-level measures have been proposed to date, but most of these are at the level of the project, department, or center, rather than the larger university of which they are a part. Because “it’s very difficult to define what an interdisciplinary institution is” (Cheng and Liu 2006: 139), most scholars have circumvented the task of assessing interdisciplinarity at the university level. Cassi et al. (2014) determine authors’ affiliations, and then average their article-level interdisciplinary scores to the university level. Birnbaum (1981) identifies interdisciplinary projects in part by seeing whether different bodies of knowledge are represented in the research group, but this yields little variation at the project level (s.d. = .08), and likely even less at the university level. Boardman and Corley (2008) characterize centers by the number of departments that affiliated faculty members represent, but co-existence of multiple disciplines is a necessary but not sufficient condition for the integration of disciplines that interdisciplinarity requires, according to the National Academies of Science (2005) definition. Jacobs and Frickel (2009: 59) identify interdisciplinary fields by examining the proportion of faculty with degrees in other disciplines. And because the vast majority of research centers “are ostensibly interdisciplinary, at least in name and self-presentation,” Jacobs and Frickel (2009: 54) imply that the number of research centers, or the ratio of centers to (traditional, disciplinary) departments, may indicate interdisciplinarity at the university level. We develop this idea by also distinguishing interdisciplinary units (centers and departments) from other units.

We conceptualize universities’ structural commitment to IDR as efforts to promote IDR within its extant research infrastructure of centers and departments. How many centers exist, and are they really interdisciplinary? How many departments exist, and are they really all mono-disciplinary? Answers to these questions should give us a glimpse of universities’ structural commitment to IDR. Before discussing centers and department in more depth, we make two points that are critical to the conceptualization and measurement of this construct. First, our interest in structural commitment intentionally excludes efforts that require planning and financial resources but leave the organizational structure intact, such as: internal grant mechanisms; cluster hiring initiatives; and interdisciplinary teaching.Footnote 4 Second, our interest in structural commitment is not intended to speak to the underlying motivations for departmental mergers and consolidations. It is the case that as competition for resources within the field of higher education organizations has increased (Barringer 2016; Taylor et al. 2013; Weisbrod et al. 2008), department consolidations may be pursued solely to enhance administrative efficiencies (Gumport and Snydman 2002). And though we do see this pattern among lower tier comprehensive universities (like University of Wisconsin—Stevens Point most recently), it is less common among the population of top research universities we study here. Moreover, even among top research universities, we find no empirical evidence that the universities most pressured to consolidate for efficiency’s sake (i.e., public universities and those with lower revenues) actually have fewer departments, or a greater relative number of interdisciplinary departments.Footnote 5 In sum, our focus is purely structural: we are not interested in other, non-structural ways of fostering IDR, and we are not interested in the underlying motivations for merging departments and the reasons why the organization is structured as it is. Indeed, as we have shown, motivations for efficiency’s sake are not relevant to the set of top universities we study, and even if they were, we suggest that a unit created for one reason (i.e., administrative efficiencies) can also have unexpected effects (i.e., demonstrating commitment to IDR).

Research centers

To date, most researchers have presumed that research centers are interdisciplinary. This is a reasonable assumption, given that: (1) most centers (especially those founded in recent years) were developed as interstitial entities to foster research that couldn’t be conducted within single departments (Boardman and Corley 2008; Brint et al. 2009; Sá 2008: 542), (2) most IDR that is conducted at research universities takes place within centers (Nickelhoff and Nyatepe-Coo 2012), and (3) the “vast majority” of research centers present themselves as interdisciplinary (Jacobs and Frickel 2009: 54). Indeed, cross-disciplinary interaction is the main justification for the development of research centers (Geiger 1990). Thus there is reason to believe that research centers—which at some universities number over 100—serve as bridges between disciplines and departments, providing a more flexible space to coordinate collaborative research (Biancani et al. 2014). As Jacobs (2013: 92) writes, “research centers may function—imperfectly to be sure—as intra-university ‘boundary organizations’ [see Guston (2000)] that help to bridge disciplinary divides.” Frickel and Gross (2005) note that centers may serve as micro-mobilization contexts for recruitment into emerging interdisciplinary fields. At minimum, centers represent an “organizational context for bringing together scholars from diverse backgrounds with shared interests” (Jacobs 2013: 92). Indeed, this guided Bozeman and Boardman (2013), who defined research center affiliation as affiliation with an institute that includes participants from at least two distinct departments.

However, some centers appear to have been developed without an interdisciplinary mission (Rhoten 2003; Rhoten 2005), and even those with such a mission did not always live up to expectations (Rhoten 2005; Stahler and Tash 1994). This seems to be particularly true among social science research centers (Ikenberry and Friedman 1972). Indeed, research centers are sometimes established to meet other goals: to facilitate the recruitment of high-status faculty and help to retain them once hired; to allow researchers with diverse skill sets to work together; to provide access to specialized (and expensive) instruments and other types of support for research that may or may not be interdisciplinary (Kaplan et al. 2017); to be more flexible than departments so they can respond quickly to changing environments and funding opportunities (Mallon 2006); and to connect academe and industry and aid technology transfer (for example, the Industry-University Collaborate Research Centers (IUCRCs) studied by Leahey et al. (2017)).

With this in mind, we pursue a more nuanced and conservative approach. It may very well be the case that universities that house more research centers (relative to traditional, presumably disciplinary departments) manifest greater commitment to IDR than universities that house fewer centers. But we interrogate the presumed link between centers and IDR by using state-of-the-art natural language processing techniques to distinguish interdisciplinary centers from others, and thereby more closely tap universities’ structural commitment to IDR.

Departments

Even though departments are closely associated with disciplines (Abbott 2001; Abbott 1999; Lee 2007), research also documents the porous and interdisciplinary nature of some departments, suggesting that the sheer number of departments may indicate a structural commitment to interdisciplinarity. For example, Jacobs and Frickel (2009) note that departments hire faculty trained in other disciplines, and that ideas regularly cross disciplinary bounds. In his recent book, Jacobs (2013) shows that even among the traditional disciplines, there is much variation: fields like education readily and consistently engage in interdisciplinary scholarship, in contrast to more homogeneous fields like economics. And many of the fields and subfields that have emerged in recent decades are highly interdisciplinary in nature. Genetic toxicology, for example, developed out of and synthesized research in genetics, toxicology, public health, and environmental science and politics (Frickel 2004). Behavioral genetics is another new and highly interdisciplinary field (Panofsky 2014). Comparative studies, like Moody’s (2004), reveal that some disciplines, like sociology, are much more integrative and perhaps even interdisciplinary than other more insular fields, like economics. Adams and Light (2014) also found variation in the degree of interdisciplinarity across fields as well as subfields and topics like Demography and HIV/AIDS research.

Given this variability, we build on the approach pursued by Brint et al. (2009) to distinguish more and less interdisciplinary fields. By reviewing and manually coding the content of course catalogs that were available for almost 300 American colleges and universities, they distinguished fields typically organized as (disciplinary) departments from fields typically organized as interdisciplinary programs. To qualify as interdisciplinary, a field had to meet two criteria: (1) it had to draw faculty from two or more departments, and (2) it had to be classified as interdisciplinary by more than two-thirds of the universities in their sample. Examples of interdisciplinary programs include Asian studies, Race and Ethnic Studies, International/Global Studies, Brain and Biomedical Sciences, and Women’s Studies. Examples of traditional, disciplinary departments include Economics, Chemistry, and English. Their approach reveals that not all departments are mono-disciplinary, and that the names of departments themselves may well be indicative of an interdisciplinary focus. We build on Brint’s work by using natural language processing techniques to distinguish interdisciplinary departments from others.

In sum, the primary goal of our paper is to construct and validate a measure of universities’ structural commitment to IDR. We begin with measures suggested or implied by extant research, including the number of departments and centers, and the ratio of (presumably interdisciplinary) centers to (presumably disciplinary) departments. We build on this in several ways. First, we question the content validity (i.e., coverage) of previously proposed measures that rely only on a comparison of the sheer number of research centers and departments. Quantity isn’t always the best gauge of commitment, especially given that “interdisciplinary initiatives can lead just as easily to the multiplication of academic units rather than their consolidation” (Jacobs and Frickel 2009: 60).Footnote 6 Second, we interrogate the assumption that research centers are inherently interdisciplinary (e.g., Jacobs and Frickel 2009), and departments are inherently mono-disciplinary (Lee 2007). Instead, we assess this empirically using state-of-the-art methods for large-scale textual analysis, which allow us to identify (rather than presume) the interdisciplinary nature of units. Third, we develop a single continuous measure of universities’ structural commitment to IDR that incorporates information about the prevalence and nature of units that typically house research activity: centers and departments.

Data and methods

Sample

To provide the first large-scale and systematic analysis of interdisciplinarity at the university level, we study the population of top research universities across the country. Like Brint et al. (2009: 162) we focus on large and prestigious research universities because they “have the wealth and organizational capacity to respond to new developments” like shifts in science policy initiatives to foster IDR. Furthermore, research universities are an ideal site to examine commitment to IDR given their research-oriented mission and their support of research activity in a wide variety of fields. We define research universities as those schools included in the “very high research activity” category of colleges and universities listed in the 2010 or 2005 Basic Carnegie Classifications and those classified as research extensive institutions in the 2000 Basic Carnegie Classifications (N = 157). This approach encompasses longstanding core research universities, as well as more peripheral members of this group.

Given recent investments and interest in IDR (Jacobs 2013; National Academies of Science 2005; National Research Council 2014), we focus on a recent academic year (2012–2013) for which all the necessary data were available. We rely on two main sources to obtain data on the organizational units in which most research is conducted: centers and departments.

Data

Research Center Data We identified each university’s set of research centers (RCs) from the 2012 edition of the Gale Research Center Directory. Though underutilized in the scholarly literature, a comparison of the Gale data and data obtained via manual web-searches of all 157 universities led us to believe that the Gale data were more valid for two reasons. First, Gale uses a stricter definition of research center, whereas searches of university web pages tended to return a broad array of entities including not only centers and institutes, but also labs, museums, and libraries. Second, the Gale data were better vetted and curated, and were less likely to include duplicates of the same center with slight name variations, relative to university websites. In short, compared to web searches, Gale provided a more conservative count of research centers at universities. Moreover, it was reassuring to find no systematic differences in the disciplinary home or the apparent interdisciplinarity of research centers across the two sources.Footnote 7 Relying on the Gale Research Center Directory, we retrieved information on a total of 9,211 centers housed at the 157 universities we study. The text we analyze includes the name of the research center, the subject headings available from the Gale Research Center Directory (e.g., the center “The Florida-Mexico Institute” with subjects “Mexico” and “U.S.-Mexican relations”), and the “Multidisciplinary” field code which Gale assigned to some centers.

Department Data We identified each university’s set of departments from university websites, which are a standard source of data for research on higher education organizations (Barringer and Slaughter 2016; Barringer and Riffe 2018; Harris 2010; Morphew and Hartley 2006). This entailed visiting each university’s website, and obtaining either its master list of departments (when available), or visiting each constituent school or college’s website and identifying member departments. When determining what ‘counts’ as a department at some universities, professional schools presented a special case. Our perusal of websites revealed that a subset of professional schools—Law, Nursing, Music, Journalism/Communications, Pharmacy, and Architecture—operate effectively as departments, so we counted them as such. Other professional schools (including Engineering, Education, Business, Medical, and Agriculture) tend to have departments within them; if so, we counted each department individually, but if not, we counted them as a single department. We excluded honors, graduate, continuing education, distance colleges and schools, and administrative departments from our counts. We included programs only when they were listed under the heading “departments” on a university’s website, and thus presumably functioning as departments. In total, we retrieved information on 12,323 departments housed at the 157 universities we study. The text we analyze is the department name itself, which is surprisingly informative.Footnote 8

Analytic approach

Overview

As we describe in detail below, we combine a variety of methods to convert these textual data on centers and departments into quantitative indicators of university structural commitment to IDR. We use semi-supervised machine learning techniques in the Python programming language to distinguish interdisciplinary units (centers and departments) from other units. We then aggregate these binary classifications to the university level by calculating the fraction of centers that are interdisciplinary and the fraction of departments that are interdisciplinary, and these continuous variables serve as indicators in confirmatory factor analysis (CFA) models that we specify. From the best fitting CFA, a continuous factor score is derived, allowing us to identify how universities compare in terms of their structural commitment to IDR. To our knowledge, this combination of techniques has not been used to date.

Coding and classification

Our first aim is to empirically gauge whether each center and department is interdisciplinary. Our approach to this measurement task is more akin to qualitative content analysis than traditional bibliometric measures, which fail to grasp complexity (Cassi et al. 2014: 1873) and “leave considerable gaps in understanding” (Wagner et al. 2011). To distinguish interdisciplinary units from others, we rely on semi-supervised machine learning, a guided form of natural language processing that translates text into “socially relevant data” (Evans and Aceves 2016: 29). It is useful for classifying large amounts of textual data, which is the case here: across the 157 universities under study, we collated information about 9,211 centers and 12,323 departments. The use of machine learning allows us to overcome the main limitation of analyzing the content of textual data: it is time consuming and not conducive to large-scale inquires and systematic comparisons (Cassi et al. 2014: 1873). Machine learning is becoming more common in the social sciences (Evans and Aceves 2016), and has been used by others to identify interdisciplinary text (Evans 2016).

Semi-supervised machine learning is a two-stage process, each with multiple steps.

Stage 1 To build, train, and test a classifier requires not only programming skills but also thoughtful human input from which the machine can learn to identify and classify pieces of text. We built the classifier in Python using the SciKitLearn package (in our case, a curvi-linearSVC was ideal). The classifier was trained (i.e., the machine ‘learns’) from two related forms of human input: (1) a list of features that we deem indicative of interdisciplinarity and therefore encourage (but do not require) the classifier to assign an interdisciplinary classification; and (2) example pieces of text that we manually coded as interdisciplinary or not (i.e., the labeled dataset).

Both forms of human input are based on an elaborate coding scheme that we developed (see abbreviated form in “Appendix”). We based the coding scheme on approaches that others have used to identify interdisciplinary articles (Evans 2016) and units (Brint et al. 2009), on extant definitions of interdisciplinarity (Wagner et al. 2011), and on disciplinary classifications made by NSF and the Classification of Instructional Programs (CIP). The coding scheme delineates indications of interdisciplinarity that tend to appear in the name of departments and research centers. These include direct reference to the term ‘interdisciplinarity’ or related terms like ‘integration’ and ‘synthesis,’ reference to two or more disciplines or disciplinary stems (like ‘Bio’ and ‘Chem,’ ‘Geophysics,’ and ‘Department of Sociology and Anthropology’), and reference to one of the area/global/ethnic studies fields identified as interdisciplinary by Brint et al. (2009).

In addition to being programmed directly into the classifier as suggestive ‘features,’ the coding scheme guides our manual classification of example pieces of text and the resulting labeled dataset. These example pieces of text are only used to train and then test the classifier (which is then used to classify the real data of interest); they therefore cannot be a subset of the real textual data of interest. To construct a labeled datafile for training and testing purposes, we gathered the same textual data (i.e., research center names and subjects, and department names) for six universities just outside our population of interest in terms of research capacity.Footnote 9 Research team members labeled (i.e., coded) these pieces of text manually, and we reconciled differences jointly. Inter-coder reliability for manual coding of RCs was 88%. The remaining 12% of cases on which we did not agree did not help train the classifier, as they were inherently too ambiguous to serve as good examples.

Following protocol, the labeled dataset was divided so the data the machine was ‘trained’ on is not identical to the data used to ‘test’ it (i.e., assess its accuracy). Three quarters of the labeled dataset was used as input to help train a ‘machine’ (i.e., computer) to ‘learn’ what is and is not an interdisciplinary piece of text. The remaining quarter of the labeled dataset was used to test the accuracy of the classifier. This step produced information about precision rates: for example, of the 135 research centers and departments that we (manually) deemed interdisciplinary, the computer tagged 122 as interdisciplinary. This resulted in an excellent 90% accuracy rate for the machine classifier.

Stage 2. With an accurate classifier in hand, the labeled dataset was ignored, and the second stage was implemented: we ran the classifier on the textual data (9211 centers and 12,323 departments) from the 157 universities under study—i.e., our real data. The output from this second stage is a list of all of the pieces of text (either RC names + subjects, or department names) of interest to us, and a binary variable indicating whether the classifier deemed it interdisciplinary or not.

We validate these classifications by making comparisons to other measures. To validate research center classifications, we build upon Boardman and Corley (2008) and Bozeman and Boardman (2013) who characterize centers by the number of departments that affiliated faculty members represent. We selected a small set of well-known interdisciplinary centers that are often discussed in the literature (e.g., the Broad Institute at MIT), and for each we selected a comparable center at the same university that was not classified as interdisciplinary and had sufficient information available online, with an eye toward representing different disciplines. For each center, we determined the department affiliation of each affiliated faculty member (see Table 1). At the centers that our classifier deemed interdisciplinary, there are at least 16 departments represented, and no more than 30% of affiliated faculty hail from the modal department. In contrast, among centers that our classifier did not tag as interdisciplinary, there are only 1–3 departments represented, and more than three-quarters of affiliated faculty hail from the modal department. To validate department classifications, we draw on Jacobs and Frickel’s (2009) approach to identify interdisciplinary departments by looking at the fields in which member faculty received their Ph.D. For one of our own (large, public, R1) universities, we closely researched a few fields that Brint et al. (2009) and our classifier tagged as interdisciplinary and not (see Table 2). We find that interdisciplinary departments, like Gender and Women’s Studies, have faculty with Ph.D.s from more than a handful (5–10) of fields, and in only one case were more than half trained in the same modal field (27–55%). In contrast, departments that were not tagged as interdisciplinary, such as Physics and Economics have faculty with Ph.D.s from less than a handful (1–5) of fields, and almost all of them (81–100%) were trained in the same modal field. As an additional validation, we ran the list of 39 fields deemed interdisciplinary by Brint et al. (2009) through our classifier, and 92.3% were classified as interdisciplinary. Combined, these comparisons confirm the accuracy of our classifier and attest to the validity of our classifications of both research centers and departments.

Table 1 Research center classifications and faculty affiliates’ fields
Full size table
Table 2 Department classifications and faculty Ph.D. fields (University of Arizona)
Full size table

Because our ultimate aim is not to classify a specific RC or department, but to obtain the aggregate fraction of each university’s centers (and departments) that are interdisciplinary, we applied the correction developed by Hopkins and King (2010). Hopkins and King point out that, when machine learning has adequately high precision rates, any single document is fairly likely to be classified correctly; however, if the classifier is more likely to make mistakes when classifying certain types of documents (for example, it systematically underestimates interdisciplinarity), the aggregate-level ratio will be biased. The correction they developed adjusts for this, and produces bias-free aggregate-level ratios. We implement the correction in Stata 14 for each university in our sample, separately for each type of text.Footnote 10

With corrected estimates of the number of interdisciplinary RCs and Departments, we developed the primary indicators of interest (see Table 3). Because new departments often reflect a “blending of previously distinct fields” and yet the previously distinct fields rarely disappear, the sheer number of departments can indicate interdisciplinarity (National Academies of Science 2005: 19). We therefore include the number of departments housed at each university (#Depts) as an indicator of universities’ structural commitment to IDR. We also include the ratio of research centers to departments (#RCs/Depts), a measure we develop as a logical extension of Jacob and Frickel’s (2009) work. Additional measures move beyond sheer prevalence to consider the nature of the units—whether they are interdisciplinary or not, which was assessed with the machine learning classifier. For example, we calculated the ratio of interdisciplinary research centers to departments (#idRCs/Depts). We also calculated the fraction of RCs that are interdisciplinary (#idRCs/RCs) and the fraction of departments that are interdisciplinary (#idDepts/Depts).

Table 3 Indicators of universities’ structural commitment to IDR for 2012–2013
Full size table

Confirmatory factor analysis

The indicators shown in Table 3 serve as the basis for our main analysis: a confirmatory factor analysis (CFA) model. Given that university commitment to IDR has not been measured before, and that multiple indicators are available, we conceive of it as a latent (unobserved) variable that is best tapped via confirmatory factor analysis (Bollen 1989). Because our measurement approach is driven by theory and prior literature on the topic, and because we designed our data collection efforts to obtain these specific indicators, we rely on confirmatory rather than exploratory factor analysis. Confirmatory Factor Analysis has been used successfully by others in the field of scientometrics to develop new measures (Horta and Santos 2016). We estimate and compare a variety of CFA models to identify the best way to measure structural commitment to IDR. Predictions from the best fitting model allow us to calculate a factor score for each university—essentially a single value revealing its structural commitment to IDR relative to other universities.

Descriptive comparisons

The last part of our analysis examines how university commitment to IDR varies across university characteristics. This allows us to determine if, as Brint et al.’ work (2009) suggests, interdisciplinary departments are more prevalent at larger, arts and science-oriented universities on either the east or west coast and at wealthier universities. We compile such characteristics from the Integrated Postsecondary Education Data System (IPEDS) and the Higher Education Research and Development Survey (HERD) for the 2012–2013 academic year. We construct binary variables indicating five regions of the United States, urbanicity, land grant status, university type (public or private), research capacity based on 2010 Carnegie Classification (“very high research activity” vs. “high research activity”), affiliated medical school or hospital, and technical institution (i.e., universities with ‘tech’ in their name). We use continuous measures of total spending on R&D (in raw dollars), total revenues (in raw dollars), and size of the student body (the number of undergrad and graduate FTE students). We then examine bivariate patterns (i.e., t tests and correlations) between these university characteristics and the factor score representing university commitment to IDR.

Results

Descriptively, we find that universities vary widely across our indicators of structural commitment to IDR (see Table 4). The average number of departments is 79, but values range from a low of 8 to a high of 163. When we look at the ratio of research centers to departments, we find that on average, universities have 81 research centers for every 100 departments on campus (mean #RCs/#DEPTS = .81); but the median number of 64 centers per 100 departments indicates excessive skewness. Indeed, one university (Rockefeller University) has over 600 research centers per 100 departments. When we examine the ratio of interdisciplinary research centers (classified via machine learning) to departments, we see almost the same amount of variation but lower mean (0.52) and median (0.42) values. But these two ratios are correlated at 0.96, indicating that the universities that have a high ratio of centers to departments also tend to have high ratios of interdisciplinary centers to departments.

Table 4 Descriptive statistics for indicators of universities’ structural commitment to IDR (n = 157 universities)
Full size table

Contrary to assumptions in the literature, not all research centers are interdisciplinary, and—as Jacobs (2013) and Brint et al. (2009) have made clear—not all departments are monodisciplinary. Results from the semi-supervised machine learning effort, combined with the Hopkins and King (2010) correction, reveal that while at some universities, all centers are interdisciplinary, at others, as few as one-third are interdisciplinary (see Table 4). On average, only about two-thirds of research centers at a university are interdisciplinary (mean = 66%). With regard to departments, only about half are monodisciplinary and half are interdisciplinary, on average (mean = 49%), with a range between 32 and 85%. Descriptive univariate statistics also reveal a couple of outliers and (relatedly) some excessive kurtosis; however, excluding the culprit observations does not change the multivariate results we present below.

The ratio of centers to departments, a logical extension of Jacobs and Frickel’s (2009) work, does not appear to be a critical component of university commitment to IDR. This is evident from the initial CFA models we specified (see Table 5). Recall that universities’ structural commitment to IDR is the latent, exogenous variable of interest, and we are trying to identify the best way to measure this concept. Following standard practice, we scaled the latent variable by setting the factor loading for one indicator (#Depts) to equal 1 in all models. In Model 1, we include two indicators: the ratio of centers to departments (#RCs/Depts) and the sheer number of departments (#Depts). Although relevant model fit statistics are adequate, the coefficient for #RCs/Depts is small, negative, and fails to reach statistical significance. In Model 2, we use the refined ratio of the number of interdisciplinary RCs to the number of departments, and we see slight improvement in model fit (the AIC and BIC decline slightly) but the coefficient still fails to reach statistical significance, and remains negative. The ratio of centers to departments, even when restricted to interdisciplinary centers, is not a good indicator of structural commitment to IDR.

Table 5 Results from confirmatory factor analysis (CFA) model to measure universities’ structural commitment to IDR: factor loadings (b) and standard errors (SE)
Full size table

As we expected, the nature (and not merely the relative prevalence) of both types of units is critical to adequately gauging universities’ structural commitment to IDR. This is evident empirically in Table 5, Model 3, which adds the two ratios, #idRCs/RCs and #idDepts/Depts, that we derived from the machine learning effort to identify interdisciplinary units. This model is depicted graphically in Fig. 1. Compared to previous models, this model fits the data better, as assessed by the AIC and BIC, which become smaller (more negative) and by the 1-RMSEA, TLI and CFI, which all reach or come close to their maximum value of 1 (perfect fit). The insignificant Chi square test statistic (T) also indicates a close-to-perfectly fitting model. All coefficients reach traditional levels of statistical significance. The ratio of centers to departments remains negative, suggesting that higher levels of structural commitment to IDR are associated with lower ratios of centers to departments; likely because the sheer number of departments is so indicative (as many new departments are interdisciplinary). Our more refined indicators, generated with the help of machine learning to identify interdisciplinary centers and departments, are positive and reach traditional levels of statistical significance. As expected, the fraction of RCs that are interdisciplinary and the fraction of departments that are interdisciplinary serve as important indicators of universities’ structural commitment to IDR. This is especially the case for the latter (#idDepts/Depts); the R2 value for this ‘equation’ (recall that each indicator is an outcome variable in CFA) is 0.45, compared to only 0.03 for (#idRCs/RCs).

Fig. 1
figure 1

Best-fitting confirmatory factor analysis (CFA) model (Table 5, Model 3)

Full size image

Results from the multivariate confirmatory factor analysis model reveal that there is wide variation in universities’ structural commitment to IDR. From the best-fitting model (Table 5, Model 3), we compute a continuous factor score, which is a numerical value that indicates each university’s relative standing on the latent variable “Structural Commitment to IDR.” It is important to note that the value of a factor score is uninformative; it simply reveals the relative positioning of universities in terms of their structural commitment to IDR. The distribution of factor scores is displayed in Fig. 2.

Fig. 2
figure 2

Distribution of university commitment to IDR (factor score derived from Table 5, Model 3)

Full size image

We are intrigued that structural commitment to IDR does not align perfectly with common perceptions of which universities are more committed to IDR. Table 6 reveals the composite factor score as well as values on all indicators for 10 universities that are often highlighted in the literature. Research by Biancani et al. (2014) led us to believe that Stanford was committing many resources to interdisciplinary endeavors, but its factor score is at the lower end of the distribution, as is MIT’s. Duke University was the “first university to embrace interdisciplinary research and teaching as an explicit strategy of intellectual advance” (Brint et al. 2009: 177) but this strategy has not translated into structural changes; Duke has a good number of departments, but less than half of them are interdisciplinary in nature. Arizona State University, whose conversion of departments into schools is upheld as a model for interdiscipinarity, falls almost exactly in the middle of the distribution, suggesting an average structural commitment to IDR. Harvard is near the top in terms of its structural commitment to IDR, and this appears to be driven by its large number of departments, the majority of which are interdisciplinary (75%). University of California at Riverside, which has pursued cluster hiring extensively, also appears to have a strong structural commitment to IDR. These results provide support for Turner et al.’ (2015) finding that the strategies pursued depend on the particular political-academic context of each university. We note that the distribution of factor scores is largely defined by the fraction of departments that are interdisciplinary (as indicated by the high R2 value for this indicator in Model 3, and the fact that its values are ranked in almost the exact same order as the composite factor score), attesting to the foundational importance of Brint et al.’ (2009) work.

Table 6 Universities’ structural commitment to IDR (and its components) compared across select cases
Full size table

What kinds of universities are most committed to IDR?

The universities most structurally committed to IDR tend to be the most prestigious, top-tier universities: those that are members of the AAU, and those that have high revenues as well as spending on research and development (see Table 7). This is perhaps not surprising because the structural commitment we examine here requires resources to establish and maintain interdisciplinary departments and centers; in contrast, a more cultural form of commitment to IDR, manifested in written documents like strategic plans (Harris 2010), likely requires fewer material resources. The bivariate relationships presented in Table 7 also reveal that universities with medical schools and affiliated hospitals (characteristics that tend to be correlated positively with revenues, expenditures, and status) are also likely to be more structurally committed to IDR. Although Jacobs (2013) finds that universities with medical schools typically have more RCs, and we find this in our data too, these RCs are not particularly interdisciplinary;Footnote 11 it appears to be the sheer number and interdisciplinary nature of the departments that boosts their structural commitment to IDR (recall the relative R2 values from Table 5). A comparison of factor scores also reveals that technical universities (e.g., Georgia Tech, MIT, CalTech, RIT) also appear to be less structurally committed to IDR; this may be attributable to the greater amount of industry-sponsored contract research that takes place at such institutions, where the focus is more on spanning domains of research (academic and commercial) than disciplines per se (e.g., Barringer and Slaughter 2016; Mathies and Slaughter 2013).Footnote 12 We also find that universities in the southern United States tend to be less committed to IDR. Compared to the other five regions, universities in the south have fewer RCs per department overall. Further investigation reveals that the distribution of RCs at southern universities is also distinct, with these universities having a higher percentage of RCs in computers and mathematics, engineering and technology, business and economics, and government, but fewer RCs in agriculture, biology and environment, astronomy and space, social sciences, and humanities—precisely where interdisciplinary themes like Environmental Sciences, American Indian Studies, and Gender Studies are common.

Table 7 Comparing structural commitment to IDRa across university characteristics
Full size table

Discussion

In this paper we used vast amounts of textual data and uniquely combined cutting-edge machine learning techniques with factor analytic methods to develop the first measure of universities’ structural commitment to IDR. Our focus on structural commitments, which pertain to units that house research activity, led us to collect textual data on each university’s constituent departments and research centers (i.e., their names and keyword descriptors). Previous research suggested that the number of units and their relative prevalence could serve as indicators of universities’ structural commitment to IDR.Footnote 13 Moving beyond prevalence to examine the nature of such units, we used semi-supervised machine learning techniques in the Python programming language to develop a classifier that distinguished interdisciplinary units (centers and departments) from other units. We aggregate these binary classifications to the university level by calculating the fraction of centers that are interdisciplinary and the fraction of departments that are interdisciplinary, and these continuous variables serve as indicators in the confirmatory factor analysis models that we specified and compared.

The results indicate that the best way to measure universities’ structural commitment to IDR is to incorporate information about not only the prevalence but also the nature of academic units: whether they are interdisciplinary or not. This is because, as our analyses show, not all centers are interdisciplinary and not all departments are monodisciplinary. Rather, the best fitting model reveals the critical importance of the ratios (#idRCs/RCs and #idDepts/Depts) that we develop and apply broadly with the assistance of machine learning. Universities whose centers and especially departments tend to be interdisciplinary show the greatest commitment to IDR in structural terms. Given the costs associated with developing, maintaining, and modifying organizational structures to serve interdisciplinary research efforts (Harris 2010; Harris and Holley 2008), it is not surprising that we find that it is the more well financed, research-oriented, and prestigious universities that show the greatest structural commitment to IDR.

These findings have implications for scholarship and for research policy. The science of science scholarship will ideally steer away from a focus on the sheer quantity of academic units (e.g., absolute and relative numbers of centers and departments) and toward a greater understanding of their qualitative nature. As the first author argued in a recent policy report to NSF’s SciSIP program (Leahey 2018), the scholarship on research centers tends to focus on a single case or single type of center, rather than examine the types and qualities of those initiatives. While interdisciplinarity is the quality that interests us here, other characteristics likely matter too. For example, a unit’s size, location on campus, and design (Downey et al. 2016; Kabo et al. 2014) likely influence multifaceted collaboration (Jha and Welch 2010), productivity (Sabharwal and Qian 2013), and interdisciplinary graduate work (Millar 2013). A more nuanced understanding of the nature of academic units will permit finer grained analyses and specific insights to inform and tailor research policy, which currently invests in all kinds of centers without attending to the type and degree of integration that is expected to result.

In this paper we restricted our focus to universities’ structural commitment to IDR, but future research should explore whether and how this is related to other forms of commitment. The distinction between our findings about structural commitment and lay perceptions of (perhaps more) cultural forms of commitment paves the way for future studies that could compare the two forms, and attests to the widespread belief that some universities are symbolically (but not substantively or structurally) committed to IDR. We also suspect that commitment to IDR as manifested in the nature of departments and centers is quite different from commitment to IDR as manifested in faculty hires (e.g., cluster hires and interdisciplinary lines) and internal grant mechanisms designed to foster cross-college interaction. We suggest, and will examine in subsequent papers, that different decision-making bodies are likely shaping different manifestations of commitment to IDR (e.g., administrators may have more influence over department and centers, whereas faculty members have more say over hires), and that they are not always committed to the same degree. It may very well be that manifestations of commitment to IDR occur at one level or another, but not both simultaneously. Perhaps commitment to IDR needs to start somewhere, and then takes time to diffuse to other levels and decision-making bodies.

After developing the measure of structural commitment to IDR, we assessed which types of universities are more committed to IDR, but this is just an initial and exploratory step. Given its relevance to science policy and research investments, we are eager to use this measure of universities’ structural commitment to IDR in several follow-up studies that examine both the determinants and consequences of universities’ structural commitment to IDR. Gaining a better understanding of how university level resources and internal dynamics influence structural commitment to IDR would help university administrators and leaders select the best mechanisms for fostering IDR. An examination of whether and how university commitment to IDR, as measured here, affects critical outcomes like research productivity, patenting, funding levels, and prestige would help university administrators, science policy makers, and grant agencies know what can be reasonably expected from IDR.