Background

In the mid-1990s Schroeder and Mable (1994) criticized the way in which residence halls support student learning stating that “residence halls have lacked educational planning, strong internal directions and a set of educational objectives connected to the goals of undergraduate education” (p. 13). While these criticisms were not new then, since the early 1990s work, such as that by Schroeder and Jackson (1987), Pike (2002), and Pike, Schroeder, and Berry (1997), has consistently sought to expand the emphasis and support of the academic components of undergraduate education for residential facilities. Further, Kezar (2006) suggests that the addition of “structure” within larger universities can aid in meeting institutional goals such as enhancing learning, community and engagement among students. This restructuring can manifest itself in numerous forms, including increases in technology, the development of institutional mission statements and, pertinent to the present study, the addition or expansion of residential facilities. These expansions come in the form of living-learning communities, freshman interest groups, themed residence halls (Johnson and Cavins, 1996; Warner and Noftsinger, 1994), and an overarching goal to connect and build bridges between out-of-classroom experiences and in-class experiences (Kuh, Douglas, Lund, and Ramin-Gyurnek, 1994).

At the same time that research called for heightened attention to the role played by residential facilities and the purposeful programming provided within these facilities, colleges and universities have for the past 20 years increasingly relied upon enhanced services, upgraded facilities, and customized living arrangements to both attract and retain students. Most recently, a number of initiatives such as these were described by Gordon and Raymond (2005) profiling substantial changes students could expect at a variety of institutions, all of which were designed to not just foster satisfaction, but to improve student outcomes as well. In many respects initiatives such as these rest on components of the person-environment premise, as described more fully by Strange and Banning (2001). Person-environment literature advances the notion that students seek an appealing and supportive environment that matches their internal needs and preferences, and that this environment can indirectly support development and growth.

The initiatives featured by Gordon and Raymond (2005) document the heightened expectations that students have today regarding living conditions. Simply put: no longer will traditional configurations do—students expect facilities to be state of the art; meaning that facilities must include space customized to accommodate a range of activities and must be encapsulated with high-technology options. Not only can residential facilities assist to recruit, but as research has shown, these facilities can support and foster student success and learning (Strange and Banning, 2001), and institutions “are beginning to leverage their physical facilities as major assets” (Hill, 2004, p. 25).

Residential investments, such as those described above, have costly implications; therefore, research should seek to explore the relationship of residential expansion and development with respect to student engagement. Research suggests that engagement should increase based on an increase in “maximized opportunities” (Pascarella and Terenzini, 2005), and in this case even more so when coupled with an institutional emphasis on fostering purposeful activities and campus life. The purpose of the present study was to explore the impact that on-campus student living has on dimensions of student engagement for both on-campus and off-campus first-year students, as measured by the National Survey of Student Engagement (NSSE) in academic years 2003–2004 and 2004–2005. While the connection between on-campus residence and engagement is more clear, a shift in campus life brought about by a significant increase in on-campus living could potentially alter the experience of off-campus students, as well. Through potentially increased institutional expectations, expanded opportunities and a heightened culture of participation, off-campus students might find themselves influenced to “engage” more. The previous research documenting differences between on-campus and off-campus students does not necessarily suggest this, but the “natural experiment” provided by this study offers a unique opportunity to investigate the premise.

The Benefits of On-campus Residence

A substantial body of research supports the contention that residence halls and on-campus living have a positive, although often indirect, effect on student growth and development (Pascarella and Terenzini, 1991, 2005). Blimling’s (1989, 1993) thorough reviews draw the conclusion that living on campus, as opposed to commuting to campus, is associated with significant student gains in a variety of domain areas—all that are seemingly the result of “maximize[d] opportunities for social, cultural, and extracurricular involvement” (Pascarella, Terenzini, and Blimling, 1994, p. 25). Previous studies show positive associations and effects of campus residence on student involvement, overall satisfaction, degree attainment, and personal growth and development. Studies suggest that on-campus students experience greater “aesthetic, cultural, and intellectual value” (Pascarella and Terenzini, 2005, p. 603), and greater openness to diversity (Pike, 2002). To summarize findings, Pascarella and Terenzini (1991) noted that “living on campus is perhaps the single most consistent within-college determinant of impact” (p. 611).

Despite the preponderance of studies finding that on-campus residence is associated positively with desired outcomes, much of the research suggests that these associations are at best indirect. Pascarella (1984), in a study examining the effect of on-campus living on students’ educational aspirations, persistence, overall satisfaction, and progress toward degree, found no direct effect of campus residence, but instead a small impact that was mediated through involvement with the campus community. More recently, Pascarella and Terenzini (2005) similarly found that “living on campus, however, appears to foster change indirectly, by maximizing the opportunities for social, cultural, and extracurricular engagement” (p. 603). In a rather short, ten year period the focus of discussion shifted from involvement to engagement, but regardless of the construct, campus residence seems to provide the greatest opportunity for either, and therefore greater potential benefit in terms of student learning, growth and development.

Student Engagement

At their best, Kuh claims that institutions that contribute significant value to students share “an unshakeable focus on student learning” (Sanoff, 2005). Student engagement has increasingly come to be recognized not as a direct measure of student learning, but as a measurement of the activities and patterns of participation in educationally purposeful activities that lead to development and growth. As Kuh states further:

“The engagement premise is deceptively simple, even self-evident: The more students study a subject, the more they learn about it. Likewise, the more students practice and get feedback on their writing, analyzing, or problem solving, the more adept they become” (Kuh, 2003, 25).

As institutions seek to foster student learning and support students striving to accomplish their educational goals, the NSSE instrument proves useful because the guiding conceptual basis is, in fact, the Seven Principles of Good Practice in Undergraduate Education (Chickering and Gamson, 1987; Kuh, 2000)—at the core, the seven principles are also “deceptively simple” practices. Overall, these notions reveal that engagement has two key components. “The first, is the amount of time and effort students put into their studies and other activities that lead to the experiences and outcomes that constitute student success. The second is the ways an institution allocates its human and other resources and organizes learning opportunities and services to encourage students to participate in and benefit from such activities” (Kuh, Kinzie, Schuh, Whitt and Associates, 2005, p. 9).

The NSSE instrument offers institutions data regarding institutional performance based on five benchmarks related to student engagement. These benchmarks were developed based on past research in the areas of student learning and student engagement. More specifically, these benchmarks address the areas of academic challenge, active and collaborative learning, student-faculty interaction, supportive campus environments and enriching educational experiences. Residence halls are increasingly recognized as prime venues for the expanded “learning opportunities” that are an essential component of student engagement, and are a mechanism that institutions may use to grow the size of student enrollment. As noted by Kezar (2006), “size is deemed one of the most important conditions in the organizational theory literature and is particularly relevant to the issue of engagement (p. 92). Furthermore, Kezar has documented that institutional size plays a pivotal role in four areas of student engagement, including: student faculty interaction, academic challenge, active and collaborative learning, and supportive educational environment. Differences are noted between large and small institutions in how they most often successfully promote and foster these four forms of engagement.

The present study sought to further explore the role played by size, as defined by an increase in the number of on-campus residents, and answer three primary questions as a means of exploring the impact of on-campus residential growth on student engagement for both on- and off-campus students. The following questions guided this study:

  1. 1.

    Does the addition of a new residence hall significantly increase student engagement for on-campus and off-campus students?

  2. 2.

    How do on-campus and off-campus students vary with respect to dimensions of student engagement?

  3. 3.

    Are dimensions of student engagement differentially related to student success for on-campus and off-campus students?

An understanding of these relationships is important to both researchers and practitioners alike. The push to expand or revise and update campus residential facilities is strong and often justified as being “educationally purposeful.” Answers to these questions may support assessment of these claims and isolate the overall impact.

Methodology

Data Source

Findings in this study are based on analyses of the NSSE results for first-time freshmen at a public doctoral research intensive university in the Midwest. The university’s first-time freshmen enrollment is nearing 1,000. Approximately 60% of the first-time freshmen are female, 23% are underrepresented minorities and 40% lived on-campus during their first year at the university. The data for this study were limited to include only the responses from first-time freshmen from the Spring 2004 and Spring 2005 survey administrations. Construction of a state-of-the-art $22 million, 132,000 sq. ft., 561 bed facility began in July of 2003, with a planned opening of Fall 2004 in between NSSE administrations.

In addition to 561 beds, configured into single and double occupancy options, the facility offered kitchens, a high-tech classroom, music practice rooms, eleven community areas on the living floors for social and academic events, presentation space, and a courtyard for social activities. As a guiding principle, campus administrators and architects designed the hall to encourage and support the “educationally purposeful” activities emphasized by Chickering and Gamson’s (1987) Seven Principles of Good Practice in Undergraduate Education. Additionally, student affairs staff worked in conjunction with academic units and programs to establish living-learning communities in the facility and to explore options to better tie residential life to academic and social activities on-campus. With the opening of the residence hall, the university realized a more than 50% growth in the number of students residing on-campus.

The total number of cases in 2004 was 375 representing a 48% response rate, and in 2005 was 356 representing a 38% response rate. In 2005, approximately 90% of the first-time freshman living on-campus resided in the new facility. Data from each administration were collected exclusively through the online web survey format. These data provide a representative sample of the first-time freshman populations in each year, and were merged with internal institutional data which then provided a comprehensive data source on the first-year experiences and outcomes of first-time freshman.

Data Analysis

This study initially sought to rely upon the 5 Benchmarks of Student Engagement as defined by researchers at NSSE. Ultimately, however, the “benchmarks” provided a jumping off point from which subsequent analysis was based. The five measures were inspected for multi-colinearity, and although each was significantly correlated with each other, none exceeded a commonly accepted threshold of .70 (Pedhazur, 1982). Upon further inspection, however, it became clear that a number of items used to produce the benchmark scores asked students about their participation and/or their intentions to participate in various activities. Many responses on these items indicated that upwards of 35% of students reported their intention. On at least one, almost 80% of students reported that they “planned to” participate in a culminating experience. On the surface these items seem to be intended to achieve meaning for both freshman and senior respondents. While these items are likely and logically related to student engagement and best practice activities, the inclusion of these items appeared particularly questionable for the population of first-year students for whom intentions, such as those associated with a culminating senior experience or study abroad, seemed quite distant. Additionally, because of the nature of these items, the impact of a student’s intentions appeared to be less meaningful to the research team than did actual activities a student participated in that were deemed more likely to play a direct role to the outcomes under consideration.

To further assess this decision, a factor analysis of the items used to produce the benchmarks was conducted by forcing the extraction of five factors based on the existence of NSSE’s Five Benchmarks of Student Engagement. However, the results were conceptually ambiguous, as many items loaded together into dimensions in a way that could not be interpreted. Therefore, the items used to produce the benchmarks (excluding those asking students to project participation in certain activities) were reduced using principal components analysis. This analysis identified nine dimensions of student engagement; however, the ninth dimension was dropped from subsequent analysis based on un-interpretable item loadings and less than desirable reliability (α < .40). The remaining eight dimensions of student engagement are presented in Table 1, and each possesses an acceptable reliability coefficient (Chronbach’s α) and contributes to explained variance following rotation (Carmines and Zeller, 1979). These dimensions were then used in subsequent analyses.

Table 1. Factor Loadings for the Eight Dimensions of Student Engagement
Full size table

To address the first two research questions, t-tests and analysis of variance procedures were used to identify significant differences between student engagement measures for on-campus and off-campus students, as well as differences between those measures in the 2004 and 2005 administrations—following the opening of the new residence facility.

In order to examine the final research question regarding the differential impact of the dimensions of student engagement, two linear regressions models were constructed with the student’s cumulative grade point average (GPA) at the end of the first year and the student’s overall evaluation of his or her experience as dependent measures. Results of frequency inspection led to the transformation of the cumulative GPA, using a square root transformation, to reduce skewness and improve normality (Tabachnick and Fidell, 2001). Dimensions of student engagement identified in the principle components analysis, residence, and year of administration provided the independent variables in the model. Background demographic variables, including age, gender, and ethnicity served as control variables, along with a student’s composite ACT score serving as a control for entering aptitude and academic qualifications.

Results

Analysis of the NSSE Benchmarks and Eight Dimensions of Student Engagement

As described previously, the study’s initial analysis relied upon the five benchmarks to address our first research question, which looked to determine whether the addition of a new residence hall would be related to an increase in student engagement for on- and off-campus students. Table 2 presents the results of t-tests to evaluate the difference between 2004 and 2005 results for on-campus residents. These results show that, for on-campus students, Enriching Educational Experiences increased significantly in 2005 (p = .012), but decreased significantly for Supportive Campus Environment (p = .028). No significant differences were noted for off-campus students.

Table 2. Comparison of 2004 and 2005 NSSE Benchmark Scores for On-Campus, First-Time Freshman (including individual items that show difference)
Full size table

The noted differences however suggest only general “domains” through the benchmark names (i.e. Supportive Campus Environment), so further analyses were required to identify significant individual item differences between the 2004 administration and the 2005 administration in order to make true meaning of the finding, or provide the foundation for changing practice. For this reason Table 2 also identifies individual items used to produce the benchmarks that were identified as significantly different for on-campus students.

Most startling among these items was the decline in quality of relationship between administration years for the items assessing students’ relationships with other students (p = .014), faculty (p = .011), and administrators (p = .000). Identifying that a “group” of items within Supportive Campus Environment were quite different, the research team began to view potentially unique sub-components of the benchmark scores as problematic, because without the individual item analysis these differences could go unrevealed. Additionally, and most important, this finding called for a more close inspection of the components of each benchmark. At this point in the data analysis, the conceptual issues surrounding the items used to build the five benchmarks became obvious and the factor structure of the benchmarks were explored further.

The results of the factor analysis, as described above and presented in Table 1, were used to look for differences between administration years. Because the factors identified were each independent and relatively consistent, these additional comparisons were intended to investigate changes in more discrete domain areas than those provided by the NSSE benchmarks and to provide a comparison to the individual NSSE benchmarks that appear to group some of these separate domains together.

Results showed that, similar to the previous analyses, no statistically significant differences were identified for off-campus students, but two of the eight dimensions of student engagement were significant for on-campus students. First, on-campus students saw an increase in participation in co-curricular activities (t = −2.037, p = .042). Secondly, and most significant, however, was the dramatic drop in the “overall relationships” dimension for these students (t = 4.506, p = .000). Again, findings revealed that students living on-campus reported significantly lower quality of relationships with fellow students, faculty members, and administrators. To estimate the effect size, Cohen’s d (Carver, 1984) was calculated for the co-curricular activity and overall relationships differences, and these effect sizes were .21 and −.46, respectively. This represents a small effect in the case of co-curricular activities, but a moderate effect in the case of overall quality of relationships (Cohen, 1977; Carver, 1984).

Variance in the Dimensions of Student Engagement

Analysis thus far only documented significant differences in students’ engagement between 2004 and 2005 treating on- and off-campus students as independent groups and not controlling for other student-level characteristics, such as academic preparation. Therefore, to assess our second research question regarding how student engagement varied between on-campus and off-campus students, the eight dimensions of student engagement were used as dependent measures in analysis of variance procedures where student aptitude, or composite ACT score, was entered as a covariate to control for differential effects of entering academic qualifications. Groupings for this analysis included residency and year of administration. Results indicated that four dimensions saw significant changes, after adjusting for the effects of the other factors and the effect of a student’s entering aptitude, which was significant in all four models (p of at least <.05). These results are summarized in Tables 3, 4, 5, 6.

Table 3. Relationship of Residency and Academic Year to Co-Curricular Activity
Full size table
Table 4. Relationship of Residency and Academic Year to Diverse Interactions
Full size table
Table 5. Relationship of Residency and Academic Year to Student Effort
Full size table
Table 6. Relationship of Residency and Academic Year to Quality of Relationships
Full size table

These analyses identified only one model where the academic year played a significant role: the relationship of campus residency and academic year to the quality of relationships. Consistent with the previously documented relationships based on NSSE and the researchers’ domains, students reported that the quality of relationships (with students, faculty, and administrators) was significantly lower in 2005 than in 2004 (F = 15.897, p = .001), even after controlling for the effect of residency and student aptitude (p = .048).

Differential Impact of Student Engagement on Student Outcomes

To address our final research question, the eight dimensions of student engagement were regressed against two separate dependent measures: end of the first year cumulative grade point average and the student’s self assessment of the entire first-year experience, with demographics and entering aptitude held constant. In contrast to the previous results, these analyses have been included to document the relative impact that the eight dimensions have on two desired student outcomes: student performance and satisfaction. Tables 7 and 8 summarize these results, respectively, and show that several of the dimensions are quite useful in explaining student outcomes.

Table 7. Relationships among Demographic Characteristics, College Preparation, Academic Year, Residency Status, and Dimensions of Student Engagement to Cumulative Gpa at the End of the First Year
Full size table
Table 8. Relationships among Demographic Characteristics, College Preparation, Academic Year, Residency Status, and Dimensions of Student Engagement to Student’s Evaluation of the Entire Experience at the End of the First Year
Full size table

For all first-time freshman, the performance model (GPA) was significant and explained almost 29% of the variance in cumulative GPA (R 2 = .289, p = .000). Inclusion of a hypothesized interaction of gender and ethnicity was not significant and did not contribute to improvement of explained variance, thus was not included in subsequent analysis. Table 7 reveals that neither residency nor the year of administration contributed significantly to student performance, as measured by grade point average.

Findings show that for this population several dimensions of student engagement were significantly, and positively associated with student performance. Learning strategies, or coursework emphasizing critical thinking and application skills; academic interactions and opportunities to interact with faculty members; a student’s effort and the institution’s emphasis on studying, and finally the quality of relationships were all significantly related to improved student performance, as measured by GPA—even after controlling for the consistently significant effect of entering student aptitude (p = .000). The net effects (based on partial coefficients of determination) indicate that each of these items, although significant, explains only a small portion of the variance in GPA not associated with the other items.

A regression model with the same independent and control variables was then fit against students’ evaluation of the entire experience, or satisfaction. Results, presented in Table 8, indicated that residency was a significant factor (p = .006); therefore, individual models are presented for on-campus and off-campus students. Several interesting differences and similarities are presented in Table 8. An interesting distinction obvious between the two models was the difference in explained variance for on- and off-campus students—.334 and .407 respectively.

Although not as drastic as the difference seen in the GPA model, these values document a greater explanatory value of “engagement” for on-campus populations. Also different was the variation in effect of ACT, or background aptitude. Unlike the case of cumulative GPA, where student ACT score played a consistently significant effect, when considering evaluation of the entire experience, ACT is only significant for off-campus students (p = .001). This finding reinforces the substantial body of literature investigating the impact of on-campus residence; living on-campus tends to provide access and exposure to a variety of indirect influences that offer the potential to overcome or depress the effects of background preparations, or lack thereof.

Despite the slight differences in explained variances, a number of dimensions of student engagement were quite similar for on- and off-campus students. The quality of relationships with students, faculty and administrators; the institutional emphasis and support they perceived, the learning strategies they experienced in their coursework emphasizing critical thinking and application of material; and the academically purposeful interactions they had were all positively and significantly associated with increased satisfaction for both on- and off-campus students.

For students living on-campus, the effort a student devoted to studying and the institutional emphasis they perceived on the importance of studying was positively associated with the evaluation of the entire first-year (p = .000). As on-campus students devoted more effort, regardless of entering ability, their satisfaction with the overall experience was higher, but there was no significant relationship identified for off-campus students. Effort does not appear to exert the same impact, however, for off-campus students, although it should be remembered that the initial analyses did identify significant differences in overall effort between on- and off-campus students.

In contrast to the regression looking at GPA where no single variable tended to stand out in terms of its individual contribution, these results indicated that both Overall Relationships and Institutional Emphasis explained a fair amount of variance (>20% each) not associated with the other items—indicating that these had the strongest association with a student’s evaluation of the entire first-year experience.

Discussion

Based on these analyses, results show little positive evidence that student engagement was substantially improved simply through the dramatic growth in students living on-campus in 2005 when compared to the student engagement reported by students in 2004. This study did, however, confirm the findings of numerous other studies that document the positive influences of on-campus residence versus living off-campus or commuting. Consistent with prior research, on-campus students were significantly more likely to engage in co-curricular activity (F = 18.251, p = .001), even after controlling for academic year which was not significant. Similarly, diverse interactions were more likely to occur for students living on-campus (F = 14.047, p = .001), although there was no effect played by the academic year. Student effort, a critical dimension of student learning (Kuh et al., 1994), was higher for students living on-campus (F = 19.901, p = .001) even after controlling for both student aptitude and academic year; however, again, academic year played no significant role.

Also relevant from these findings, in terms of what was not significant, was that neither co-curricular activity nor diverse interactions were significantly related to either cumulative GPA or to a student’s overall evaluation of the first-year experience. These findings are especially noteworthy because this study began by first identifying significant differences between the administration years, and indeed, both co-curricular activity and diverse interactions were among the three domains identified as being significantly improved through living on-campus. Unfortunately, in relation to this study’s broader purpose, the lack of relationship between these dimensions to the outcome measures, coupled with the lack of significant change after campus expansion, tends to diminish this initial finding.

Therefore, the results of this study do not support the contention that student engagement would be substantially improved simply through the increase of “structure,” which has been operationalized as an increase in on-campus residents in an educationally purposeful living environment. In fact, the only consistent finding related to the comparison between the academic years revealed that students’ perceived quality of relationships, for both on- and off-campus students, declined significantly between 2004 and 2005.

This central finding has important implications for researchers and practitioners alike. These findings document that simply doubling the number of on-campus residents does not equate to twice as much “engagement.” Although not entirely analogous, there are remarkable similarities between a campus racial climate and an institution’s attempts to promote student engagement, or an engagement climate. Overall, this study’s findings are consistent with research which notes that increases in “structural diversity,” such as simply adding students of color to an institution’s enrollment, cannot be an adequate method of increasing acceptance of diverse perspectives, shifting campus racial climate in a positive manner, or improving the success of students of color, if efforts are not constantly felt and integrated at all levels of an institution (Hurtado, Milem, Clayton-Pedersen, and Allen, 1998, 1999).

Similarly, though the addition of an educationally purposeful residential facility offers the unique potential to increase engagement on campuses, and is certainly accepted as a route to improve engagement because of the well documented benefits of on-campus residence, these results suggest that where efforts to affect engagement for students are in their infancy, it may not be realistic for institutions to expect change in such a short time frame. Simply adding students to the on-campus living environment alone does not appear to promote additional engagement; a finding consistent with the impact of diversification of the student body alone on climate and student success. Like acceptance of diverse perspectives, engagement requires a broad-based commitment from an entire institution (Kuh et al., 2005) working to shift expectations, culture, and the internal environment.

In this study, the one consistent and substantial dimension of student engagement identified as differential over the “treatment” period for all students was the quality of relationships experienced between respondents and their fellow students, the faculty, and administrators—and that dimension unfortunately went down for both on-campus and off-campus populations. When considering the additional finding that the quality of relationships was one of the most influential dimension of student engagement with respect to both dependent measures (see Tables 7 and 8) and the only significant variable for off-campus students in relation to academic performance, results suggest that administrators should seriously examine and evaluate this domain when major initiatives are being implemented, such as the expansion of residential facilities. Additionally, events on campus, programming, staff development, customer service training, campus-wide philosophy, and faculty attitudes should be assessed, especially where the initiative represents a departure from previously established modes of operation or established institutional culture.

Additionally, the results of this study further suggest that beyond a certain point of on-campus living, campuses must perhaps behave and perform differently to address student needs and potentially different student expectations brought about or introduced through growth. Because students reported a 360-degree decline in relations (with peers, staff, and faculty), it seems reasonable to suggest that the drop encountered represented an institution-wide issue. Kezar’s (2006) recent efforts to document distinctive approaches to student engagement among institutions of varied sizes perhaps is useful to aid with the interpretation of these findings. The theoretical premise driving Kezar’s study was grounded in the business literature examining the role played by size on effectiveness, and when reduced can be summarized simply that “structure needs to be modified as institutions grow to remain effective” (p. 106). This study may reinforce that contention; structure may simply not have been adequately adjusted to accommodate changed student needs or changing needs may simply not have been anticipated by the organization in advance, or alternatively the institutional environment was simply too slow to accommodate the rapid increase in on-campus residents. Re-evaluation of progress in subsequent years may help to assess these “organizational accommodation” hypotheses.

Finally, the other major implication of these findings concerns the overall applicability of the engagement “domains” when attempting to explain student performance for off-campus students. As shown in Tables 7 and 8, the models focused on GPA as the outcome explains just over twice the variance in end-of-first-year GPA for on-campus students than for their off-campus counterparts. Similar results, though less dramatic, were documented for the models focused on student satisfaction or evaluation of the first-year. Additionally, in contrast to the results for on-campus students, only one variable was identified as significant in relation to GPA for off-campus students: Overall Relationships. These each help to suggest that there is quite a bit missing or not yet understood in the engagement question of off-campus students.

An additional issue that may be problematic is revealed when comparing the results of Tables 7 and 8. Whereas Table 7 was noted for the high degree of difference between on-campus and off-campus students when the outcome was student performance, there is virtually no difference between the results when the outcome under consideration is the student’s evaluation of the entire first-year experience, or overall satisfaction. The explained variance is quite similar and with only one exception, the variables that contribute to the model match between the groups (effort being the sole exception). Further complicating this observation is the fact that a student’s evaluation of the first year experience is related differentially to student performance for on- and off-campus students. The simple correlation between satisfaction and academic performance at the end of the first-year is slightly stronger for off-campus students than for on-campus students, r = .274 and r = .211 respectively. Although these associations are each only moderate, it is nonetheless still stronger for off-campus students, meaning that an off-campus students’ satisfaction is more related to his or her learning (assuming the GPA is an accurate proxy) than for on-campus peers.

As campuses continue to expand, building more on campus facilities and residence halls to encourage on-campus living, the competitive nature of the postsecondary environment reinforces this drive toward a common approach where on-campus amenities and living are the preference (Hughes, 2006). Yet simultaneously, higher education is criticized to address persistent access issues (Department of Education, 2006)—and these are not access to residential hall issues. Residential halls and on-campus living are the more known variables in the complex equation of student learning and development; and clearly these results document that “student engagement” as defined in this study explains more for this population. Therefore, the results of this study must be viewed as additional evidence of the need for researchers to continue to explore and identify what promotes engagement and learning for off-campus students.

Limitations

The results of this study are somewhat limited in that they are based on the unique population of a single metropolitan university, and may not be generalizable to other institutional types. Most certainly, the use of the eight dimensions of student engagement generated for this study may not prove as beneficial to examine engagement on other campuses. However, the fact that the principal components analysis conducted by the present authors resulted in an interpretable structure indicates that this method of analyzing the NSSE items may be useful in future research, in addition to conducting analyses based solely on the five NSSE benchmarks. Of note in this area is the fact that, internally, among different campus constituencies, these dimensions have proven slightly more useful for discussion purposes, in part because of their increased specificity and apparent relation to practice and programming. These findings appear to be consistent with Pike’s (2006) recent recommendations encouraging institutions to use NSSE data customized to suit their campuses. There researchers should attempt to replicate these findings in other institutional settings.

Recommendations

Despite these limitations, described above, this study further highlights the need for institutions to validate their results and explore the potential need to construct items most appropriate to their institutional data, to reiterate Pike (2006). Additionally, these results suggest that institutions should closely inspect the individual items that comprise their benchmark scores. As institutions grapple with the challenge of promoting increased learning and face heightened calls to document an institution’s “value-added,” they are faced with an additional challenge of focusing their attention. For example, an institution that scores below average on any benchmark score may not necessarily concentrate increased efforts in the correct domains to bring about change. Does a lower than desired Enriching Educational Experiences benchmark score suggest that increased diversity and exposure to diverse perspectives is needed or do students at the institution need greater access to co-curricular activities? The benchmarks alone do not permit the easy answer of this question. The only way to adequately assess and answer this question is to conduct a thorough analysis at the item level. The eight dimension structure identified by this study may provide viable method to do this because the dimensions appear to be more unique and “actionable” as compared to the NSSE benchmark scores.

Conclusion

This study sought to examine the impact of a residence hall that substantially increased the proportion of on-campus residents on student engagement. The residence hall itself cannot necessarily be considered a “treatment” on this population because of the multiple influences on students in a dynamic environment such as a college campus. The residence hall itself did, however, serve as the most concrete example of the institution’s efforts to shift campus culture and change overall campus life. Although, these findings are entirely consistent with previous research noting the indirect role played by on-campus residence (Pascarella and Terenzini, 2005) and the consistent positive effects of on-campus residence in comparison to commuting to campus (Pascarella and Terenzini, 1991; 2005; Blimling, 1993, 1989), these results do not suggest that size itself either increases or decreases student engagement.

While this may leave the impression that the lack of difference can be attributed to the residence hall itself, as Kuh et al. (2005) have consistently repeated, institutions that excel in the area of student engagement bring to bear the entire campus community to focus on these goals. The consistent negative difference identified from both on-campus and off-campus student populations between NSSE administrations focused on the quality of relationships throughout the institution—documenting that perhaps the structures, attitudes and commitment to student engagement were not yet in place or not substantial enough to support the 50% growth in on-campus residents. This finding, more so than anything else appears to support the Kuh et al. mantra that the campus as a whole must organize to promote engagement, and documents the need for institutions to adequately assess their level of preparation beyond facility roll-out when expanding campus residential living—clearly equal attention must be focused on the programs, structures, and staff that will support residents once they are on-campus.