Introduction

Accreditation is important for ensuring and sustaining the quality of an educational program (Germaine and Spencer 2016). Institutions failing to meet the quality standards may put their students’ academic and professional future at risk. For example, employers often consider whether an applicant attended an accredited program (Ceccucci and White 2008). However, the accreditation process for higher education (HE) institutions can be challenging. HE institutions rely on other interrelated entities to assist with accreditation process. One of the entities playing a crucial role is faculty. Unfortunately, sometimes there is a lack of buy-in and involvement from faculty when assisting with accreditation-related tasks (Germaine and Spencer 2016) due to a shortage of time (Beld et al. 2009; Cummings et al. 2008), lack of perceived benefit to teaching and learning, insufficient expertise (Beld et al. 2009), and inadequate incentives for tenure and promotion (Andrade 2011).

Studies examining faculty buy-in and involvement in accreditation are limited (Germaine and Spencer 2016; Grunwald and Peterson 2003; Hutchings 2010). For instance, previous studies examining engineering faculty’s buy-in and involvement in program accreditation, such as the Accreditation Board for Engineering and Technology (ABET) process, usually discuss recommendations to overcome the challenges (Al-Yahya and Abdel-Halim 2013; Damaj et al. 2017; Irizarry and Cesaní 2014; Koehn 2006; McGourty et al. 2002) rather than utilizing the systems thinking perspective to analyze the issues holistically and track down the root causes. The aforementioned current gap of literature motivated us to conduct a cause-analysis study in an engineering department to examine the interrelated factors influencing faculty’s buy-in and involvement in undertaking program accreditation-related tasks.

Higher-Education Institutions as Open Systems

A system consists of interrelated objects working together to perform a function (Hall and Fagen 1975; Richey et al. 2011). Additionally, a system is ordered and hierarchical; it consists of interrelated components that can operate as subsystems, which also consist of interrelated components (Richey et al. 2011). There are two types of systems—closed and open. According to Churchman, closed systems are self-reliant and can endure changes occurring in the environment (as cited in Richey et al. 2011). However, open systems evolve continuously through the effects of interconnected components within the environment, and adapt to inputs, processes, outputs, and feedback that can either improve or worsen the environment (Swanson 1999).

Systems thinking focuses on the understanding that all key components are interrelated and comprehending the relationships among the components is crucial to ensure that the desired function takes place (Furst-Bowe 2011). Perceiving HE institutions as open systems is fundamental before approaching the challenges regarding the participation in accreditation activities. HE institutions continuously evolve because of the interrelationships among the entities or subsystems within the environment (Davidson-Shiver et al. 2018; Richey et al. 2011). The entities within a HE institution comprise of internal (e.g., administrators, faculty, staff, students, facilities, policies, and procedures) and external components (e.g., parents, accrediting agencies, legislatures, and taxpayers) (Muljana and Luo 2019; Davidson-Shiver et al. 2018).

In the context of this study, HE institutions rely on interrelated entities, such as faculty, to assist with accreditation tasks. Additionally, the program accreditation process can be perceived as a subsystem within the institutional system. As a subsystem, the successful program accreditation relies on faculty’s buy-in and involvement in performing the accreditation activities. Their buy-in and involvement can influence the inputs and processes entailed in accreditation, gradually impacting the outputs of accreditation, and eventually shaping the program’s and institution’s image.

Cause Analysis and Systems Thinking

Cause analysis, defined as “the process of determining the root cause(s) of past, present, or future performance gaps,” (Rosenberg 1996a, p. 79) is the bridge to finding suitable interventions (Rosenberg 1996a) and assists with determining why performance gaps exist (Van Tiem et al. 2004). The underlying causes originate from multiple factors that entail deficiencies of knowledge, skills, and training, as well as management deficiencies (Rosenberg 1996b; Van Tiem et al. 2004) such as lack of information and incentives, insufficient tools and resources, and a flawed working environment and process (Rossett 1999).

The description of cause analysis resonates with systems thinking, in which the term system refers to a set of interrelated parts that work together or perform a function together (Hall and Fagen 1975; Richey et al. 2011). Notably, the systems thinking approach necessitates the consideration of multiple factors holistically rather than simply an investigation of the parts. Echoing Thompson (2010), exploring what is occurring in the whole system can lead to an understanding of individual behavior. The systems thinking characteristic in the cause analysis process can help examine the interrelated, underlying factors that determine faculty’s buy-in and involvement. Additionally, the systems thinking characteristic can assist in revealing the areas of highest leverage, where actions and interventions will yield the most possible impactful outcomes, which sometimes are not obvious (Davidson 2005; Senge 1990).

Behavior Engineering Model for Conducting Cause Analysis

Gilbert’s Behavior Engineering Model (BEM) (Gilbert 1978), which provides the most influential list of factors impacting performance gap (Van Tiem et al. 2004), is often used as a framework for conducting cause analysis. BEM (Gilbert 1978) is deeply rooted the development in von Bertalanffy’s (1950) general systems theory (Wooderson et al. 2016), incorporating both employees’ individual factors and environmental supports to understand the causes of performance issues at the workplace (Wooderson et al. 2016), as well as identification of suitable interventions to improve performance productivity.

BEM has been utilized in educational settings, including HE. For example, King (2013) conducted an exploration of the use of BEM to identify barriers to technology integration in public schools. Lion (2011) examined the efficacy of performance support provided to faculty teaching online courses in a HE setting. Both studies provide examples of how BEM can be adapted to an educational setting and is suitable for the context of the engineering department analyzed in this study.

Chevalier (2003) updated the original BEM “to provide a more efficient method for troubleshooting performance and for discovering the most important opportunities for improving individual performance” (p.9). Chevalier intended the updated model to be used as a diagnostic tool, serving as a framework for better revealing underlying causes (see Table 1 for the updated BEM). Particularly, Chevalier (2006, 2008) also expanded Gilbert’s probing questions (Gilbert 1982, 1999) to promote the use of the updated BEM and assist with the discovery of more in-depth information within each factor. The updated BEM provides a more visible, straightforward process for determining the causes that impact a performance gap (Chevalier 2003), allowing a smooth adoption of the model. Based on this rationale, this study adopted the updated BEM to guide the development of a measurement tool for revealing the root causes that influence faculty’s buy-in and involvement in ABET-related tasks.

Table 1 The Updated Behavior Engineering Model
Full size table

Guided by Chevalier’s (2003) updated BEM, the interrelated factors influencing faculty’s buy-in and involvement are categorized into environmental and individual factors. Within the environmental factors, there are: (1) information, such as the importance of the accreditation-related tasks, results of accreditation-related assessments to show weaknesses and strengths of the program, the depth of discussion about the assessment results with faculty in the department, and sufficient guidance on helping faculty understand their role in the accreditation process; (2) resources for helping faculty improve their courses and the use of data collected from the assessments for supporting program self-improvement; and (3) incentives, such as funds for those who intend to learn about the accreditation process through conferences or workshops and culture of ownership regarding accreditation-related tasks.

The individual factors consist of: (1) knowledge, such as faculty’s awareness about assessments used in program accreditation and learning outcomes specified in the accreditation standards; (2) capacity, such as the time constraints of faculty and any anxiety or stress felt by the faculty related to the accreditation process; and (3) motives, such as faculty motivation and willingness to assess and become more involved with the tasks.

Since environmental factors have a significant impact on the way people perform their tasks and on the outcomes desired at the workplace (Rummler and Brache 1988), variables explaining faculty buy-in and involvement may not be solely in the individual factors. Other factors in the environment, such as the management, organization, and process can also account for faculty buy-in and involvement. As posited by Chevalier (2003), the information component within the environmental factors holds the highest impact; a change in this area can make a significant impact on other factors and desired results. Figure 1 illustrates the interconnections of each component and potential for the greatest leverage within environmental and faculty’s individual factors that impact their buy-in and involvement.

Fig. 1
figure 1

Environmental and Faculty’s Personal Factors that influence their buy-in and involvement in undertaking the departmental accreditation related tasks. Adapted from “Updating the Behavior Engineering Model,” by Chevalier 2003, Performance Improvement, 42(5), p. 10. Copyright 2003 by the International Society for Performance Improvement

Full size image

Cause-Analysis Goal and Questions

The overarching goal of this cause-analysis study was to investigate the root causes of low buy-in and involvement of faculty in carrying out departmental accreditation tasks. Using systems thinking as the foundation, the following questions guided this study:

  1. 1.

    What were the current perceptions of faculty in an engineering department regarding the ABET-related process?

  2. 2.

    Which environmental and individual factors influenced the faculty’s buy-in and involvement in ABET-related tasks?

  3. 3.

    What recommendations did faculty offer to increase their buy-in and involvement?

Methods

Context

The present cause-analysis study took place in a large engineering department at a U.S. public university. Like many engineering programs in the U.S., the department is accredited by the Accreditation Board for Engineering and Technology (ABET). ABET accredits college and university programs in applied and natural science, computing, engineering, and engineering technology (Accreditation Board for Engineering and Technology n.d.). With few exceptions, ABET accreditation is necessary for an engineering program (and the degrees conferred by that program) to have good standing. The ABET accreditation cycle is a six-year process in which an engineering program gathers data through various assessments and uses that information to undergo continual improvement. At the end of the six-year cycle, the program is visited on-site by ABET program evaluators who determine whether the program should retain accreditation for the next six years.

For the assessment cycle 2017–2023, a faculty member was appointed to lead the department’s accreditation process. The department’s other 24 tenure-track faculty members were asked to assist with the ABET-related tasks for improving their program and to prepare for the on-site visit. Conducting a cause-analysis in this context allowed us to identify which factors contribute to faculty resistance to participate in accreditation tasks. Additionally, the adoption of the systems thinking approach in this study guided us to explore holistically the environmental and individual factors influencing faculty’s buy-in and involvement in accreditation-related tasks. Therefore, the determined interventions would not merely address the symptoms of the issues, but rather the root causes.

Participants

Participants were 23 tenured and tenure-track faculty members (N = 23) of a large engineering department at a primarily undergraduate U.S. public university. Table 2 displays the participants’ basic contextual information. Sixty-five percent of the faculty participants were full professors, with 52% having worked in the department for more than ten years. Almost three-quarters of the participants had either been on the department’s assessment committee in the past or were currently on the committee.

Table 2 Participants’ Basic Contextual Information
Full size table

Data Collection and Analyses

A self-crafted paper-based survey was administered to participants during a departmental meeting. The survey consisted of three components, including a) items related to basic contextual information of faculty participants, b) five-point Likert-scale items to probe faculty’s current perception of the ABET accreditation process as a whole and the individual tasks involved (three items), and insights on the environmental supports and individual factors influencing their buy-in and engagement in the ABET-related tasks (19 items), and c) open-ended questions (four items).

The 19-item scale was adapted from Chevalier’s BEM probing questions and separated into six categories: information, asking whether role expectation, clear information, and guidance were communicated (six items); (b) resources, asking whether sufficient materials, tools, and procedures were provided (two items); (c) incentives, asking whether pursuing learning opportunities about ABET-related assessments are available and a positive environment exists (two items); (d) knowledge, asking faculty how familiar and knowledgeable they were regarding the ABET accreditation process (two items); (e) capacity, asking faculty about limitations hindering their participation and negative feelings toward the ABET accreditation process (two items); and (f) motives, intended to reveal current motivation to be more involved (two items). The open-ended items asked about faculty members’ current perceptions regarding ABET-related tasks and recommendations that would assist the department in identifying suitable interventions.

Prior to survey dissemination, two senior colleagues from the department reviewed the survey items to ensure content validity, language clarity, and suitability with the departmental context. A faculty member who is knowledgeable about BEM and systems thinking also reviewed the survey items to ensure the alignment with the updated BEM. We performed descriptive statistical analysis on the survey responses while using an open-coding technique for open-ended responses helped discover emerging themes.

Results

Question 1: What Were the Current Perceptions of Faculty in an Engineering Department Regarding the ABET-Related Process?

The Likert-scale items inquiring about faculty’s current perceptions revealed mixed results (see Table 3). Forty-eight percent agreed that ABET was an effective means for ensuring the self-improvement of the departmental program, and 68% agreed that conducting the ABET-related tasks was a significant time burden. However, most believed that the assessment tools used by the department are “very useful” or “extremely useful” for providing meaningful data for program improvement.

Table 3 Current perception of faculty regarding ABET-accreditation and individual tasks
Full size table

Regarding their current perception, nine faculty attested to the usefulness of ABET accreditation process for the self-improvement of the departmental program. One faculty member noted, “We need [ABET accreditation] otherwise we do not know if our courses are comparable to our peers’.” However, three faculty members who favored the usefulness of ABET additionally brought up the heavy workload in the implementation process. One faculty member lamented that the “constant changing of [student] outcomes every cycle hinders long-range efforts and makes for tons of useless work.” ABET occasionally revises student outcomes, which are skills that students should obtain at the time of graduation and are assessed as part of the accreditation process.

Question 2: Which Environmental and Individual Factors Influenced the Faculty’s Buy-in and Involvement in ABET-Related Tasks?

Table 4 displays the descriptive analysis for the Likert-scale survey items exploring the factors influencing the faculty’s buy-in and involvement.

Table 4 Measure of central tendency and spread of the Likert-scale questions
Full size table

Environmental Factors

The information items revealed that the department could improve its communication regarding ABET-related information. Only 41% of faculty “strongly agree” or “somewhat agree” the department provided sufficient information to identify which areas of the program require improvement (M = 3.32). Fifty-seven percent “strongly agree” or “somewhat agree” that the level of discussion about the results of ABET-related tasks was adequate to promote the continual self-improvement of the program (M = 3.52).

For the resources items, 50% “strongly agree” or “somewhat agree” that most data collected for ABET-related purposes were useful for self-improving the program (M = 3.41).

Responses related to the incentive items displayed mixed responses. Most faculty “strongly agree” or “somewhat agree” that funds are available for attending ABET-related workshops and conferences to learn about the accreditation process (M = 3.96). However, only 43% “strongly agree” and “somewhat agree” that a culture of ownership regarding ABET-related tasks was encouraged (M = 3.43).

Individual Factors

The faculty’s knowledge about ABET-related assessment tools and the overall process was mixed. About 74% of the faculty rated themselves as “extremely familiar and “very familiar” with the seven student outcomes in the current ABET standards (M = 4.00). When questioned about their awareness of the 6-year cycle process (M = 4.09), 78% admitted “extremely familiar and “very familiar.” However, unfamiliarity with a few assessments used for ABET was found, such as with employer survey (M = 2.78) and alumni survey (M = 3.17).

Within the faculty’s capacity, time constraints could be an issue. Forty-eight percent of faculty “strongly agree” and “somewhat agree” that time limitations hindered their involvement in the ABET accreditation process (M = 3.43). Responses were evenly split about whether faculty experienced anxiety related to the ABET accreditation process (M = 3.00).

The faculty’s motives seemed positive as 83% “strongly agree” and “somewhat agree” to assist with the ABET accreditation process (M = 4.17), and 83% “strongly agree” and “somewhat agree” that they were willing to be more involved with the ABET accreditation process (M = 4.04).

Question 3: What Recommendations Did Faculty Offer to Increase their Buy-in and Involvement?

Thirteen faculty members responded to two open-ended questions that solicited recommendations for improving the accreditation process. Clear communication emerged as a top theme. Faculty recommended that the department can “keep the faculty informed and then provide feedback,” solicit the input “from experienced faculty who understand our students and our university’s limitations,” and communicate the “survey results and how those are being used (if at all).”

The need to improve the resources emerged as well. For example, the department can consider the use of a tool to facilitate assessment such as:

“an electronic system which faculty can input this effort toward continuous improvement and a database in the system could assess the level of continuous improvement. The loop must be fed into the program every semester and at the end of year. Most departments do not have a system in place.”

Discussion and Implications

This cause-analysis study was aimed to identify the root causes of low buy-in and involvement of faculty in carrying out departmental accreditation tasks. Specifically, faculty’s perceptions about the ABET accreditation process and related tasks, factors affecting their buy-in and involvement toward ABET-related tasks, and their recommendations to promote their own buy-in and involvement were examined. The systems thinking approach incorporated in this study successfully guided us through analyzing and tracking down the root causes holistically. This study provides insights and implications on managing changes and strategies to promote faculty’s buy-in and involvement.

Managing Changes

Implementing interventions may cause disruptions, confusion, and even further anxiety (Bolman and Deal 1999; Wheatley 2005). Therefore, a strategic way to manage a system change can be performed through a participatory planning process (Furst-Bowe 2011), such as by allowing faculty to participate in identifying potential interventions and implementation strategies (Andrade 2011). In this study, we sought faculty’s insights and recommendations for interventions to promote their own buy-in and involvement.

Making changes through improving environmental supports is more cost-effective rather than attempting to change individual behaviors (Chevalier 2008; King 2013; Rummler and Brache 1988). Since some environmental factors were rated relatively poorly on the survey, improving environmental factors—information, resources, incentive—should be given the highest priority. Given the onerous nature of implementing changes and interventions, we present the following strategies to promote faculty buy-in and involvement which are supported by data in this study.

Information

Results indicate the need to discuss ABET assessments results more frequently and in-depth with the faculty. Some assessment data and assessment instruments are available to the faculty on a departmental Blackboard organization website, but the information is not updated frequently and a large fraction of assessment data is not placed on the website. We recommend updating the website at regular intervals, making available as much data as possible, and emailing faculty when new data are available to review. This would allow the department to identify specific weaknesses and strengths of the program in a timely manner; faculty who are more aware of assessment results can engage in more meaningful dialogue about how to improve the program. Departmental meetings would be a natural venue to discuss the assessment results and propose improvements more frequently, which would help demonstrate importance and the usefulness of the results (Andrade 2011).

Resources

Responses related to the resources factor indicate that some faculty do not believe some of the assessment data is used effectively for continual self-improvement of the program, echoing what Andrade (2011) reported. More frequent communication of assessment results may remedy this issue, as well as soliciting more feedback about the assessment tools from the entire department instead of just the department’s assessment committee. An additional type of communication can be considered, such as a departmental website or newsletter that can regularly highlight the success stories of the department and how they may be a result of ABET process (Andrade 2011).

Faculty mention time constraints and time-consuming tasks related to the accreditation process, similar to findings from earlier studies (Beld et al. 2009; Cummings et al. 2008; Hutchings 2010). Unsurprisingly, faculty recommended a tool like an electronic system to enter data or information related to the ABET criteria and assist in assessing the level of continuous improvement. The electronic system that faculty recommended is still rare, but workaround tools can be provided to reduce the work time, such as templates for planning the tasks, reporting assessment results, setting timelines, and establishing a systematic data collection and review (Andrade 2011). Another way is by providing time accommodation (e.g., time release) for taking upon ABET-related tasks (Andrade 2011; Germaine and Spencer 2016).

Incentive

Faculty disagree or are unsure whether the department encourages a culture of ownership regarding ABET-related tasks. This may additionally underlie the low buy-in and involvement of faculty with the tasks. As posited by Chevalier (2003), when the work environment is positive (in the context of this study, this can be represented by promoting a culture of task ownership), it will encourage people to undertake the tasks.

Currently, most of the gathering and organizing of assessment data is handled by the assessment committee chair, with other committee members providing input on assessment tools and interpretation of results. If committee members became more involved in gathering and organizing data, it may lead to a greater sense of ownership. Another way is by signifying the alignment of the ABET-related tasks with individual interests, either the teaching or scholarship activities. For instance, assessment already resides within the teaching realm (Hutchings 2010), suggesting that the accreditation-related tasks can be built around the regular, ongoing work of teaching activities. Another example is by reframing the ABET-related tasks as scholarship (Hutchings 2010), such as by providing opportunities and supports for faculty to pursue assessment-related research (Haviland 2008; Palomba and Banta 1999) that can count toward tenure and promotion requirements.

Conclusion and Future Work

Undertaking the accreditation process in HE can be challenging. There may be resistance from the crucial entities like faculty. This study has revealed several root causes and confirmed the appropriateness of the systems thinking approach and the updated BEM to address the issues in a HE setting. While our findings are directly related to the engineering departmental context, this study can raise awareness about the critical role of systems thinking in analyzing issues and the root causes. Additionally, this study can provide an example to other departments on how to tackle similar challenges in promoting participation in accreditation-related activities.

There are opportunities for future work. Within this department, we plan on conducting a follow-up study next academic year to evaluate the progress of interventions guided by the current results. Additionally, since we used a survey to obtain insights from faculty, a future investigation may utilize various data collection techniques such as document analysis and interviews. These data collection techniques can assist in discovering why some faculty members are more motivated to participate in the tasks and other factors that need to be addressed systemically. Insights from other entities within the system, such as the department chair and other stakeholders who also play a role in the accreditation process, deserve an in-depth analysis. A more holistic or macro-level analysis is further necessary so that the alignment of multiple components such as goals, procedures, and management for the program accreditation process is ensured at all levels.