Since Aristotle (350 B.C.E) differentiated “liberal” from “illiberal” education more than 2,000 years ago, liberal arts educationFootnote 1 has been widely praised as a model form of education, especially in the United States. Numerous publications describe the virtues of liberal arts education, including the renowned Yale Report in 1828 (see Turner 1996), which determined that a traditional liberal arts curriculum is the best means to prepare for a changing society. The more contemporary Association of American Colleges & Universities [AAC&U] (2002) report, Greater Expectations: A New Vision for Learning as a Nation Goes to College, named liberal education as the best means to prepare students for the multiple demands of life in the 21st century. Advocates of liberal arts education claim that it produces “broad and deep learning…” allowing students to acquire “…a rich fund of meaningful knowledge” (AALE 2003, p. 7).

However, as society and our system of higher education evolve, educators struggle to determine not only the purpose of a college education but in what manner that purpose is best achieved. Many argue for liberal education as opposed to practical education, defining the distinction as “the college way versus the university way, tradition or sentiment against size and money, the finishing school and the trade school” (Matthews 1997, p. 106). Others advocate a liberal education emphasizing practical skills, regardless of field of study or intended career (AAC&U 2002). The recent report of the Spelling’s Commission (U.S. Department of Education 2006) focused national attention on college access and quality of learning, without consideration of pedagogy, curriculum, or specific educational practices. The popularity of the U.S. News and World Report college rankings have sparked public discourse on what makes a given college or university education valuable (Farrell and Van Der Werf 2007).

These varied opinions and public debates demonstrate the need to find objective criteria to assess the value of specific types of education as they relate to desired learning and developmental outcomes. The elaborate rhetoric and anecdotal support, long used to advance liberal arts education as the premier type of education with value for all, is no longer sufficient. The institutional practices and conditions that lead to outcomes associated with a liberally educated student remain an empirical black box. The purpose of the present study is to examine the extent to which an institutional ethos, that values student–student and student–faculty interaction within a supportive environment characterized by high expectations for developing the intellectual arts, manifests in the lived experiences of students and thus predicts the development of the intellectual arts and skills necessary for a life of substance and achievement (Center of Inquiry in the Liberal Arts 2006). Specifically, we aim to shine a light into the black box of liberal arts education to better understand what institutional practices and conditions lead to the development of liberal arts outcomes.

Review of the Literature

Liberal Arts Colleges

Despite the long-held status and assumed benefits of liberal arts education, little research has carefully examined the impact of either the structure or the practices until recently. Pascarella et al. (2005) published an extensive analysis examining the institutional effects of attending a liberal arts college on student learning. They found that attending a liberal arts college, compared to a research university or regional institution, yielded mixed effects on student learning outcomes (i.e., positive effects on openness to diversity/challenge and learning for its own sake but negative effects on mathematics and science reasoning). Controlling for a host of demographic, precollege, and college characteristics, including a pretest on every outcome, attending a liberal arts college promoted the development of some outcomes, inhibited the development of others, and largely had no significant effect on the development of many of the learning outcomes under examination.

Liberal Arts Ethos and Practices

As part of their study of liberal arts education, Pascarella et al. (2005) also analyzed the effects of a combination of teaching practices and institutional conditions that capture the basic environmental elements of many liberal arts colleges on student learning. The Center of Inquiry in the Liberal Arts working definition of a liberal arts education informed the selection of the particular combination of institutional practices and conditions. The Center of Inquiry’s working definition holds that a liberal arts education is characterized by an institutional ethos that values: (a) the development of a set of intellectual arts (e.g., intellectual openness to inquire and discover; and the ability and desire to adopt a critical perspective of one’s and other’s beliefs) more than professional or vocational skills; (b) curricular and environmental structures that work in combination to create a coherent integrity to students’ intellectual experience; and (c) an institutional tradition of student–student and student–faculty interaction both in and out of the classroom (Blaich et al. 2004).

Although institutional type had inconsistent effects on the learning outcomes under investigation, the practices and conditions students experienced positively affected their development on a wide range of learning outcomes, from valuing literacy and learning for its own sake to scientific reasoning and critical thinking (Pascarella et al. 2005). These practices and conditions also acted in a compensatory manner, resulting in an even greater positive influence on student learning for academically at-risk students, women, and students of color (Pascarella et al.). Moreover, this combination of practices and conditions effectively promoted these outcomes regardless of the kind of institution students attended. In other words, student learning flourished where students experienced these powerful practices and conditions, but faltered in their absence.

Although these findings confirm the assertions made by AAC&U (1998, 2002) that a liberal arts education is not institution specific, or even discipline specific, Pascarella and colleagues found that students who attended liberal arts colleges were more likely to experience these powerful practices and conditions than their peers at other types of institutions. In three separate analyses, the practices and conditions under examination strongly differentiated liberal arts colleges from their research and regional counterparts (Pascarella et al. 2005).

Because the practices and conditions under examination displayed adequate discriminant validity by identifying known groups, Pascarella and his colleagues (2005) defined the practices and conditions as those that appeared to empirically characterize liberal arts education and then created commensurate scales for measurement purposes. The liberal arts emphasis (i.e., a scale aggregated at the institutional level) and the liberal arts experiences (i.e., a scale at the individual student level) scales were developed to measure the practices and conditions characteristic of liberal arts education. These variables included student self-reports of the following institutional practices and conditions: scholarly/intellectual emphasis of campus; number of essay exams; cumulative credit hours taken; extracurricular involvement; ratio of liberal arts courses to vocational courses; quality of non-classroom interactions with faculty; faculty interest in teaching and student development; instructional skill/clarity; academic effort/involvement; supportive relationships with students, staff, and faculty; quality of interactions with students; integration of ideas; course challenge/effort; and instructional organization and preparation.

The liberal arts emphasis and liberal arts experience scales built on Chickering and Gamson’s (1987, 1991, 1999) “good practices in undergraduate education” and the National Survey of Student Engagement’s benchmarks (Kuh 2001, 2003). Substantial research has found these good practices and benchmarks associate positively with student learning (Chickering and Reisser 1993; Cruce et al. 2006; Kuh et al. 2005; Pascarella and Terenzini 1991, 2005). However, this research has focused largely on the individual impact of one good practice or benchmark on student learning. Rather than separate indicators, the liberal arts emphasis and liberal arts experiences variables in the Pascarella et al. (2005) study captured the seamlessness of the collegiate learning experience. This is especially beneficial given that “the sources of influence on student development are themselves holistic” (Terenzini et al. 1996, p. 149) and that multiple, diverse, interdependent, and reinforcing experiences or conditions influence change (Pascarella and Terenzini 2005; Terenzini et al. 1996). Thus, it is the seamless or holistic interconnectedness of the liberal arts experiences that conceptually separates them from the more general “good practices.” In addition, the label of liberal arts experiences is appropriate, given research showing that students in liberal arts colleges and liberal arts disciplines are exposed to “good practices” more frequently than students in other colleges and disciplines (Astin 2000; Pascarella et al. 2004, 2005).

The notion of a seamless learning environment is not new to higher education. In light of the historical liberal arts tradition of student-focused education dating back to Socrates and Plato, one can argue that the best practices in undergraduate education actually stem from a liberal arts context. More recently, a host of scholars have extolled the benefits of an environment in which those areas once believed to be separate and distinct (e.g., in-class versus out-of-class; curricular versus cocurricular) are interwoven into a fluid and continuous whole (Kuh 1996; Kuh et al. 1991, 2005). The liberal arts emphasis and experience scales attempt to measure the practices and conditions that contribute to a seamless institutional ethos, which values student–student and student–faculty interaction within a supportive environment characterized by high expectations for developing a set of intellectual arts and habits of mind more than professional or vocational skills.

Liberal Arts Educational Outcomes

The Wabash National Study of Liberal Arts Education seeks to extend the findings of Pascarella et al. (2005) by enriching the range of measured outcomes, institutional conditions, and teaching practices through the use of multiple methods of inquiry (Center of Inquiry in the Liberal Arts 2006). The liberal arts emphasis and experience scales examined by Pascarella et al., while positively associated with the development of numerous student learning outcomes, were not tested in terms of predicting outcomes specifically associated with liberal arts education. The educational outcomes particularly tied to the liberal arts were determined through an extensive literature review and analysis (see King et al. in press). As expected, the liberal arts outcomes under investigation overlap somewhat with many general educational outcomes. However, King et al. explain that the distinctiveness of the liberal arts outcomes lies in their integrated connections with the other outcomes, as well as their holistic characteristics that span the cognitive, intrapersonal, and interpersonal domains. Collectively, the liberal arts outcomes embody “a cultivation of the whole human being for the functions of citizenship and life generally” (Nussbaum 1997, p. 9).

The lack of research on liberal arts outcomes noted by Pascarella et al. (2005) served as the foundation for the basic research question that continues to guide our work: after controlling for an array of background characteristics and institution type, to what extent, if any, do students’ liberal arts experiences influence liberal arts outcomes?

Methods

Sample

The sample consisted of students from the four institutions participating in the pilot phase of the Wabash National Study of Liberal Arts Education. The institutions represented three states, differed in Carnegie classification, institutional control, and selectivity; we selected them because of their willingness to participate in piloting the data collection processes. We included a research university, a regional institution with limited graduate programs, a liberal arts college, and a community college.

We randomly sampled and invited students from each institution to participate in the study and sought approximately 200 students from each institution evenly distributed across years in school. In the event students of color did not comprise 10% of the institution’s undergraduate student body, we oversampled students of color. Students received a $50 cash stipend for participating. In all, we invited 3,820 students to participate and 909 registered (a 23.8% response rate). We developed a sample weighting algorithm to adjust for sample bias by gender and year in school for each institution.

Data Collection

We collected data in three phases. In the first phase, students completed a registration form with basic demographic information either online or by mail. After registration, students then received a paper copy of the college experiences questionnaire, which measured a range of in- and out-of-classroom experiences as well as the openness to diversity and positive attitude toward literacy scales. Among those who registered, 723 completed the questionnaires. Finally, students attended a monitored session in which they completed one of two assessment batteries. Based on a matrix sample, we randomly assigned students to an assessment group. Assessment Group A completed the Reasoning and Current Issues [RCI] test (Wood et al. 2002), the Intercultural Developmental Inventory [IDI] (Hammer and Bennett 2001), and the Scales of Psychological Well-being (Ryff 1989; Ryff and Keyes 1995), while Assessment Group B completed the Defining Issues Test-2 [DIT-2] (Rest et al. 1999), the Intercultural Developmental Inventory (Hammer and Bennett), Need for Cognition (Cacioppo et al. 1984), and the Socially Responsible Leadership Scale (Tyree 1998). A total of 601 students completed the matrix of assessment instruments (285 students in Assessment Group A and 316 in Assessment Group B). Because of attrition between completing the questionnaire and the assessments, the matrix sampling procedure, and the different models estimated, the samples for these analyses vary from 708 to 279. We specify the sample size for each outcome in the tables that follow.

Variables

The main independent variable of interest in these analyses was the “liberal arts experiences” variable. Our conceptually-based scale draws upon the work of Pascarella et al. (2005) but differs in some elements. We present the differences between the operational definitions of the two liberal arts experiences scales in Table 1. We derived the scale for the present study from student reports of the following college experiences that we believed to be most closely aligned with the Center of Inquiry’s definition of a liberal arts education (Blaich et al. 2004): positive and influential student–faculty contact; faculty interest in teaching and student development; instructional clarity, organization, and preparation; academic effort and challenge; degree to which the institution is supportive; positive influence of interactions and relationships with peers; integration of ideas through class activities and assignments; challenging classroom environment characterized by high expectations; instructor feedback to students; emphasis on higher-order examinations and assignments; frequency of engaging in cooperative learning activities; frequency of faculty contact; frequency of student affairs contact; overall diversity experiences and interactions; academically meaningful out-of-class experiences; involvement with active learning; diversity courses; and out-of-class research with faculty member. The α reliability of the liberal arts experience scale was .842.

Table 1 Operationalizations of liberal arts experiences scale
Full size table

Because we were interested in estimating the net effect of liberal arts experiences on liberal arts outcomes, we included students’ age, race (a dichotomous variable for White versus student of color),Footnote 2 gender (with female as the reference category), parents’ education and household income, high school GPA, a scaled measure of high school involvement, precollege academic ability, educational aspirations, and the racial composition of high school to serve as controls for student background characteristics. We also added a series of dichotomous variables representing type of institution attended, with the liberal arts college serving as the reference category. We added students’ political attitude as a control to two of our analyses, because of its effect on the development of intercultural effectiveness.

Drawing from the breadth of literature on liberal arts education, we selected a number of outcomes we believe conceptually and theoretically relate to liberal arts education (Center of Inquiry in the Liberal Arts 2006; King et al. in press). Based on our review of the literature, we estimated the effects of students’ liberal arts experiences on the following liberal arts outcomes: moral reasoning, effective reasoning and problem solving, intercultural effectiveness, inclination to inquire and lifelong learning, well-being, and leadership. These outcomes are theoretically and conceptually-related to liberal arts education and as such required separate operationalization for their appropriate measurement.

We measured the liberal arts outcomes using a host of quantitative instruments. We measured students’ moral reasoning by using the Defining Issues Test, version 2 [DIT-2] (Rest et al. 1999). We used the Reasoning and Current Issues [RCI] test (Wood et al. 2002) to measure students’ effective reasoning and problem solving. We measured the outcome intercultural effectiveness through the use of two measures: the Intercultural Developmental Inventory [IDI] (Hammer and Bennett 2001) and the openness to diversity/challenge scale (Pascarella et al. 2005). We also used two measures to gauge students’ inclination to inquire and lifelong learning: the Need for Cognition (Cacioppo et al. 1984) and the positive attitude toward literacy scale (Pascarella et al.). We measured well-being using the six scales of psychological well-being identified by Ryff (1989) and colleagues (Ryff and Keyes 1995). Finally, we measured leadership by using the eight scales developed by Tyree (1998). The eight critical values of the social change model of leadership development (Higher Education Research Institute 1996) serves as the basis for the Socially Responsible Leadership Scale [SRLS] (Tyree). We present detailed descriptive information for variables in the analyses in Table 2.

Table 2 Descriptive information for all variables in modelsa
Full size table

Analyses

In an effort to maximize our statistical power, we retained the largest sample for each dependent measure. Therefore, we have a different sample size for each of our analyses, which we recognize can make interpretation of the findings across different analytic samples challenging. In order to account for any possible significant variation in the demographic, precollege, and institutional variables across the different models, we compared the descriptive statistics of these variables between the largest models (n = 708) and the smaller models (n = 279). We note differences in the analytic samples on the demographic, precollege, and college experience variables, where appropriate, in Table 2, as well as the sample size for each dependent variable.

We used Ordinary Least Squares (OLS) regression to conduct the analyses. Using weighted data, we estimated total and direct general effects models predicting either overall measures or sub-scales, depending on the properties of the instrument. In the total effects model, we regressed the liberal arts outcomes on the background characteristics and the dichotomous institutional variables. The direct effects model was similar to the total effects model but we added the liberal arts experience variable into the regression specification. This allowed us to examine the unique effect of students’ liberal arts experiences on the various liberal arts outcomes.Footnote 3 Our results report the direct effects standardized regression coefficient (β) of student reports of their liberal arts experiences on the outcomes. Thus, β represents the standard deviation change of the dependent variable that is associated with a standard deviation increase in the independent variable (Cohen et al. 2003). We present the results of the direct effects models in Table 3.

Table 3 Estimated weighted standardized regression coefficients for the effects of liberal arts experiences on liberal arts outcomesa
Full size table

Results

Overall, net of an extensive battery of student background characteristics and institution attended, we found students’ liberal arts experiences positively affected four of the six liberal arts outcomes. Adding the liberal arts experience variable significantly changed the amount of explained variation in a host of the liberal arts outcomes’ measures from slightly more than 1% to more than 14%. The magnitude of the statistically significant effects of liberal arts experiences on liberal arts outcomes ranged from .13 to .43 of a standard deviation, as measured by β.

One can consider the liberal arts outcomes in terms of their cognitive and affective orientation, although one may argue that each of these outcomes has multiple components. The more cognitively-oriented liberal arts outcomes include moral reasoning, effective reasoning and problem solving, and the inclination to inquire and lifelong learning. Intercultural effectiveness, well-being, and leadership reflect affective outcomes. Among the cognitively-oriented liberal arts outcomes, we found that liberal arts experiences had no significant effect on our measures of moral reasoning or effective reasoning and problem solving. Specifically, the liberal arts experience scale was not related to students’ postconventional moral reasoning as measured by the DIT-2 or reflective judgment as measured by the RCI. On the other hand, we found significant positive effects of liberal arts experiences on both measures of the inclination to inquire and lifelong learning. Net of confounding influences, students’ liberal arts experiences affected their need for cognition and positive attitude toward literacy by .24 and .26 SD, respectively.

Students’ liberal arts experiences consistently predicted all of the affective liberal arts outcomes. We found positive effects of students’ liberal arts experiences on both measures of intercultural effectiveness, but to differing degrees. Controlling for all other factors, students’ liberal arts experiences had a positive influence on students’ openness to diversity/challenge by .43 SD where the effect of liberal arts experiences on the developmental score of the Intercultural Development Inventory was .14 SD. Students’ liberal arts experiences had a positive effect on all of the dimensions of psychological well-being, with the effects having the greatest magnitude for the personal growth (.24 SD) and life purpose (.24 SD) scales. Finally, we found positive effects of liberal arts experiences on the eight scales of socially responsible leadership (Tyree 1998). The liberal arts experiences variable had the strongest relationship to the scales for citizenship and civility with increases of .39 and .32 of a standard deviation, respectively, controlling for student background characteristics and institution attended.

Discussion and Implications

Pascarella et al. (2005) found mere attendance at a liberal arts college did not consistently influence student learning and development and where attendance did influence student learning, the effect was not always positive. In contrast, they found a consistent, positive relationship between students’ liberal arts experiences and several measures of student learning. In the current study, we further tested the construct and predictive validity of the liberal arts experiences variable by examining its relationship with outcomes theoretically and conceptually-associated with liberal arts education. Given that we found significant positive relationships between four of the six liberal arts outcomes (consisting of 18 out of 20 separate measures), our results suggest the liberal arts experience variable is a valid construct in measuring liberal arts education as defined by Blaich et al. (2004).

It is interesting that the liberal arts experience variable failed to predict either postconventional moral reasoning or reflective judgment. We posit several reasons for our non-significant findings. First, it is possible that fostering moral reasoning in college students requires different liberal arts experiences and practices than those included in the current operationalization of the liberal arts experiences variable. Second, with regard to the reflective judgment measure from the Reasoning and Current Issues test, its low reliability made finding any statistically significant relationships difficult. The low reliability (α = .41) could have been an artifact of the data collection conditions or this particular sample.Footnote 4

For centuries, advocates of liberal arts education have asserted that it is the best means of education. The current study provided empirical evidence for the benefits of liberal arts education and an operational model for measuring it. Based on our findings, the practices and conditions embodied in the liberal arts experiences variable are indeed those that promoted the development of students’ intercultural effectiveness, inclination to inquire and learn for a lifetime, psychological well-being, and leadership. Consequently, colleges and universities are more likely to foster the development of these liberal arts outcomes in students by creating an ethos that (a) values the intellectual arts rather than professional or vocational skills; (b) integrates curricular and environmental structures coherently; and (c) cultivates a culture that values student–student and student–faculty interactions both in and out of the classroom (Blaich et al. 2004).

It is rare for a student to experience aspects of the campus environment like interaction with faculty and peers or class challenge in isolated segments. These dimensions of the environment overlap and blend together. We suggest this holistic “overlap and blending” is a key feature of the array of experiences, practices, and conditions that characterizes liberal arts education. Conceptually speaking, what sets the liberal arts experience variable apart from other good practices (Chickering and Gamson 1987, 1991) and student engagement benchmarks (Kuh 2001, 2003) is that it attempts to capture the holistic and seamless nature of this learning environment within a single scale.

We believe that our results are good news for colleges and universities. Despite the preliminary nature of the findings, the connection between liberal arts experiences and liberal arts outcomes is noteworthy because although it may be virtually impossible for a college or university to change its institutional type, any institution can implement the practices that foster rich and integrated learning environments. Like Pascarella and colleagues (2005), we found students’ liberal arts experiences influenced learning outcome development, net of the institution attended. Our findings demonstrate that liberal arts experiences and outcomes are not the exclusive domain of small liberal arts colleges. Consistent with previous research (Astin 1993; Chickering and Gamson 1987, 1991; Kuh et al. 2005, Pascarella and Terenzini 1991, 2005), our results lend further support to the evidence that an institutional focus on good teaching and student engagement in an active, collaborative, and supportive environment positively affects student learning and development. Our evidence suggests that any institution, possessing the organizational will to place student learning at the center, can create a culture that maximizes liberal arts experiences and thus, the development of liberal arts outcomes for all students.

This study may be limited in several important ways. First, it may be limited due to our operationalization of the liberal arts experience variable. It is possible that different operationalizations of the independent measure would have yielded different findings. This is particularly possible given the non-significant findings with regard to the measure of postconventional moral reasoning. Additionally, the external validity of this study may be limited in that our sample represented college students from four institutions. Although we made an effort to diversify our institutional sample by Carnegie classification, selectivity, and region of the country, the institutions in this study likely do not represent the vastness of the U.S. higher education landscape. Finally, the cross-sectional nature of the findings from the pilot phase of the Wabash National Study of Liberal Arts Education may limit the study’s interpretation. Students, who report higher levels of liberal arts experiences, may also be those who are predisposed to higher scores on the liberal arts outcome measures before entering college (Pascarella 2006). Without a pretest, we are not able to take into account this confounded predisposition. We hope to have a more robust test of the liberal arts experiences variable and its effects on liberal arts outcomes in the panel portion of the Wabash National Study of Liberal Arts Education. We believe this next phase of research will enhance the present findings because it will use a longitudinal pretest–posttest design, with a national sample that will follow 4,500 students from 19 institutions throughout their college career.