Business Process Management Maturity and Process Performance - A Longitudinal Study

Abstract

Longitudinal Business Process Management (BPM) studies are rare. BPM maturity and process performance can be used to quantify an organization's BPM evolution. This research aims to examine the growth of BPM maturity over time and its impact on process performance inside an organization in continuous transformation. Over a seven-year period, BPM maturity and process performance were measured annually at a Dutch university. During this time, the organization has undergone an organizational restructuring with a focus on process management and has temporarily switched completely to digital education propelled by the Covid-19 crisis. Based on a repeated cross-sectional study (N = 921), the results present key BPM maturity features that are critical during disruptive organizational transformations. Furthermore, we found that BPM maturity is positively related to process performance throughout organizational changes during the period of our research.

1 Introduction

BPM refers to the process-oriented structuring of organizational activities in order to optimize and integrate business processes, obtain a competitive advantage, and create and distribute value [1]. While BPM seems to have a strong positive impact on organizational performance in terms of efficiency and effectiveness [2], it also creates tensions when responding to contingencies [3].

BPM maturity models have been developed to measure the extent to which an organization has implemented BPM capabilities and is able to apply BPM effectively. The outcomes of these models should assist organizations in determining which BPM capabilities to deploy and how to boost performance [4]. However, most BPM maturity assessments only measure process maturity in a descriptive manner and the practical relevance of these models is unclear due to a lack of empirical evidence [5].

It has long been recommended to analyze how BPM maturity affects organizational process performance over time, for example, to show or even forecast which BPM capabilities should be taken into account to increase performance or to react to organizational changes [6, 7]. However, no long-term effect measurement has been reported so far.

Despite the observation that educational managers score a higher level of perceived BPM maturity than employees such as lecturers [8], there have been few initiatives in higher education to apply business process management [9, 10] and a number of sector-wide process initiatives have failed [11]. This could be the result of organizational dynamics [11] and less structured processes in service organizations [12], which also result in a lower BPM maturity score for service organizations compared to product organizations [13]. This sparked our interest in the use of BPM maturity models in higher education.

As a result, we aim to close this gap in BPM literature, by applying a longitudinal research design to investigate how stable BPM maturity is in relation to organizational dynamics. In particular, we aim to answer the following research question: How do changes in BPM maturity affect higher education process performance over time?

The next part of this article provides the theoretical foundation that focuses on the key elements of this research, BPM, BPM maturity and process performance and longitudinal research. After this section, the quantitative research approach with PLS-SEM is specified. This is followed by a description of the empirical findings in the results section. The results are discussed, and the limitations of this study are stated in the discussion section and the article ends with the conclusion and recommendations for further research.

2 Theoretical Background and Research Model

2.1 Business Process Management

BPM is defined as “a holistic organizational management practice focused on the identification, definition, analysis, continuous improvement, execution, measurement, monitoring, and analysis of intra- and inter-organizational business processes” [14]. This management practice combines business management methods like business process redesign, quality management methods like total quality management, and process-oriented digital innovations like workflow management and enterprise resource planning software [15, 16]. It differs from traditional hierarchical management in that the emphasis is on continual efficiency and effectiveness improvement of process performance through the use of BPM lifecycles and digital components [17]. Therefore, BPM is a method to manage change through business process improvement, embracing the full process life cycle, from analysis and design to implementation, automation, and execution of business processes in order to improve process performance [18].

Additionally, Grisold, et al. [19] show that BPM also can aid in process innovation and Brocke, et al. [20] take into account that organizations and their environments are always changing. Their viewpoint has ensured that BPM evolves into a broader process science approach. However, empirical research establishing dependent and independent factors, or determining whether improvements such as digital innovations support and even increase BPM and hence process performance, is limited [19, 21].

2.2 BPM Maturity and Process Performance

BPM maturity models are used to quantify and communicate an organization's ability to manage its business processes. Humphrey's capacity maturity model is one of the first in the history of process maturity models [22]. There are currently numerous BPM maturity models that are utilized for various reasons, such as descriptive, prescriptive, and comparative analyses [5, 6]. Most BPM maturity models are descriptive [5]. In our study we use the BPM maturity scan of Ravesteijn et al. [23]. This scan is inspired by the research of Rosemann, de Bruin and Hueffner [14, 24] and was first used in 2010 to measure BPM maturity of organizations in the Netherlands [25]. Subsequently a regular benchmarking research was done to examine various views on the relationship between BPM maturity and process performance [13, 23, 26]. The studies conducted show that maturity of BPM improves process performance. These justifications, however, are based on snapshots in a specific context (e.g. place) and time. We do not know whether the effect lasts over a longer period of time because these samples have never been sequenced previously. As a result, we developed our main hypothesis:

H1: BPM maturity has a long-term positive impact on process performance.

Several academics have previously operationalized BPM maturity [27]. Ravesteijn, Zoet, Spekschoor, and Loggen [23] operationalized BPM maturity in seven dimensions: Process Awareness, Process Description, Process Measurement, Process Control, Process Improvement, Resources and Knowledge and Information Technology.

The five items of the dimension process awareness assess higher management's recognition of the value of a process-oriented organization and inclusion in the organization's strategy. The extent to which processes and related information are recorded inside the organization is used to assess process description (7 items). Six process measurement items encapsulate the degree to which an organizational structure to monitor and manage processes is in place to improve processes. The six process control items are concerned with whether process owners are designated inside the company who are responsible for managing processes. The seven process improvement items describe how far the organization seeks to continuously improve processes and if a structure is in place to support this. Five items analyze if the organization has adequate resources (such as individuals with process knowledge) to build a “culture of process orientation” in the penultimate dimension resources and knowledge. The eight information technology items analyze the organization's ability to use IT to develop, model, and execute processes, as well as offer real-time measurement data (key performance indicators).

Based on Hüffner [28] and Rudden [29], the construct process performance is added as a dependent construct to the BPM maturity assessment. The variables that make up this construct are: Costs, Traceability, Efficiency, Lead-time, Customer focus, Continuous improvement, Quality, Measurability, Employee satisfaction, Competitive advantage, Flexibility and Comprehensibility. This leads to the conceptual model shown in Fig. 1.

Fig. 1.

Conceptual model.

2.3 Longitudinal Research on BPM

There is an apparent scarcity of longitudinal research on BPM maturity. Only a few studies in the field of BPM maturity that used a longitudinal method were found. Larsen and Bjørn-Andersen [30] used a four-wave longitudinal case study approach, which resulted in a spiral of BPM activities. They revealed this finding through a longitudinal evaluation of BPM activities at a Danish manufacturing firm. Benner and Tushman [31] investigated the photography and paint industries from a larger perspective. Their findings indicate that initiatives aimed at enhancing process management outperform technology breakthroughs in this business. More recent longitudinal maturity assessments have concentrated on specific BPM events such as Business Process Outsourcing [32], Lean Management [33], and Business Process Orientation [34]. However, all of these studies have a limited scope of BPM, are primarily focused on product organizations with relatively stable processes, and do not explore the link between BPM maturity and organizational process performance.

3 Method

Although complex longitudinal data may be easily collected and analyzed thanks to modern technology [35], there are surprisingly few studies utilizing longitudinal research. This is especially true for PLS-SEM research [36].

In this study, such a design is used in accordance with Roemer's criteria [36]. In addition, the understanding of longitudinal by Ployhart and Vandenberg is followed [37]. This means that at least three data waves are required for the exact same construct. In this study, data from six waves is employed, which provides adequate data to grasp the natural oscillation of the concept of interest in this study [38].

3.1 Data Collection and Setting

In general, the Dutch public sector has transitioned from a vertical, one-way type of accountability to a process-oriented, decentralized form of accountability [39]. This trend is also seen in higher education in the Netherlands.

The Ministry of Education, Culture and Science (ECS) formulated several performance agreements with the universities, and in accordance with the ministry's strategic agenda, universities and the ministry have chosen which quality agreements will be in place to improve higher education [40,41,42].

As part of the transformation initiative the support processes of our case university were restructured. Additionally, the task of standardizing the business processes and promoting process-oriented functioning more extensively inside the university was given to a group of process consultants. The main objective of this initiative was to better equip the university for difficulties related to digitalization and future rapid changes. This was the perfect opportunity to answer our research question. Therefor a six-wave longitudinal field investigation is conducted at a Dutch university of applied sciences as part of its transformation to a more process-oriented organization.

Employees (researchers, teachers, and support staff) were invited to complete a digital questionnaire on an annual basis from 2015 to 2021. Figure 2 presents the timeline of our study including the primary changes and the number of respondents each year.

Fig. 2.

Timeline of our study

3.2 Selection of Respondents

Every year in December, all workers having a link to organizational process-related duties (crossing education and curricula) were invited to participate in the survey. Throughout the first four years, the tasks that went beyond education and curricula were mostly carried out by, educational managers, members of the examination board, researchers, professors, and support services. However, there has been more collaboration between curricula in recent years, which has increased the number of invitations. By 2020, it no longer seems necessary to differentiate, so all employees have been invited to complete the survey moving forward. This resulted in a dataset of 921 completed questionnaires.

3.3 Measurements

The constructs BPM maturity and process performance are measured using the items described in paragraph 2.2. Process performance (measured with 12 items) is used as a proxy for actual performance, as has been done in other studies (e.g., [43]. Each of the BPM maturity dimensions as well as the items for process performance are scored on a 5-point Likert scale ranging from “totally disagree” to “absolutely agree”.

3.4 Analysis Techniques

Since the same indicators were measured at several points in time with different samples, this study is referred to as a repeated cross-sectional study [44]. To investigate differences in BPM maturity across time, an independent t-test was conducted on the construct score. This provides an answer to the first research question. In addition, to answer our second research question a multigroup analysis was conducted to test changes in path coefficients over time [36]. To conduct our multigroup analysis, we relied on partial least squares (PLS) path modeling in SmartPLS 4 [45]. We followed the recommendations on how to conduct multigroup analyses in PLS path modeling by Hwa, Ramayah, Memon, Chuah, and Ting [46].

4 Findings

4.1 Evaluation of Measurement Models

We first focus on evaluating our measurement models before we continue with investigating the development of BPM maturity over time. We modelled the BPM maturity dimensions and performance as a reflective construct. Typically, reflective constructs are evaluated by the internal consistency reliability, convergent validity, and discriminant validity. First, reliability is assessed by Cronbach's Alpha and composite reliability. Since the values range between 0.70 and 0.95 (Table 1) these are considered “satisfactory to good” [47]. Second, convergent validity is assessed. The metric used for evaluating a construct’s convergent validity is the average variance extracted (AVE) for all items on each construct. Table 1 presents acceptable AVE’s as these are above the 0.5 threshold. Thirdly, discriminant validity is assessed by means of the heterotrait-monotrait ratio of correlations (HTMT). Recent literature shows that this criterion outperforms the Fornell-Larcker criterion and the examination of cross-loadings [48]. If the HTMT value is below 0.90, discriminant validity has been established between two reflective constructs. Table 1 presents values below this threshold. Hence, values fulfill all quality criteria.

Table 1. Quality criteria for reliability and validity of the reflective measures

In line with prior research, we modelled BPM maturity as a second-order formative construct. Following the guidelines of Cenfetelli and Bassellier [49] we first examined potential collinearity issues by assessing the variance inflation factor (VIF). Table 2 presents the VIF values of the measures used, which show satisfactory values below the threshold of 5 [47]. We then assessed the measures weights and respective significance level. The weights of four BPM maturity dimensions present satisfactory significance levels while three were found to be non-significant. However, if an indicator weight is not significant, it is not necessarily interpreted as evidence of poor measurement model quality [49]. Instead, the indicator’s absolute contribution to the construct is then considered. This contribution is reflected by its loadings. Hair, Hult, Ringle, and Sarstedt [50] suggests one should consider deleting the indicator when loadings show a value below 0.50, when the weight is non-significant. Since the loadings of the indicators are above this threshold, we can conclude that this measure considerably contributes to the construct. We therefore deemed that it would be prudent not to remove any of the dimensions.

Table 2. Quality criteria of the formative measures

In addition to the above quality criteria, we furthermore need to assess measurement invariance of composite models (MICOM) over time as recommended by multigroup analyses [51]. MICOM entails a three-step process: (1) the configurational invariance assessment, (2) the establishment of compositional invariance assessment and (3) the assessment of equal means (a) and variances (b). Table 3 shows the summary of the results of the permutation tests for all the constructs. The correlation between the composite scores using the weights obtained from the various subsequent years are close to 1 and above the 5% quantile with one exception between 2017 (t₂) and 2018 (t₃) on process improvement. Thus, it can be concluded that there is compositional invariance for the most established. The differences of mean values (step 3a) and variance (step 3b) for each construct score were estimated for the years. The results revealed that for most comparisons, zero is included in the mean and variance difference confidence intervals indicating the establishment of partial measurement invariance.

Table 3. Results of invariance measurement testing using permutation

4.2 Evaluation of the Changes in Constructs over Time

Like other researchers (e.g., [52]), we assess the trajectory of our model (Fig. 3) to see whether other non-linear growth models might also be suitable. The BPM maturity trajectory suggests that a linear model might be prevalent since we observe no – or a negative – growth between t₁ and t₃ and a linear growth between t₃ and t₆. Process performance follows a similar line with the exception between t₄ and t₅ which shows a negative growth. The growth and decline between t₀ and t₁, and t₁ and t₂ are significant for both BPM maturity (t = −1.415, p < .10; t = 1.488, p < .10) and process performance (t = −2.174, p < .10; t = 1.586, p < .10). Similarly, the BPM maturity presents substantial growth between the last two years (t = −1.682, p < .05) as well as process performance (t = −1.530, p < .10). An exception to this harmonious relationship is between t₃ and t₄, which presents only a significant growth of process performance (t = −3.318, p < .01).

Fig. 3.

The trajectory of the mean of BPM maturity and process performance over time

Additionally, independent samples t-tests are used to examine the changes in constructs over time of each of the different underlying BPM maturity dimensions. Table 4 presents the results of these calculations. The results show three significant differences. First, the dimension process description showed a significant decline between t₁ and t₂ (t = 2.143, p < .05). Second, the dimension process improvement significantly augmented in the last year (t = −1.981, p < .05) meaning that continues betterment of the processes takes a leap when other dimensions are in stable. Third, information technology has a significant improvement between t₄ and t₅ (t = −2.303, p < .05).

Table 4. Results of the changes in levels of the constructs

4.3 Evaluation of the Effects Over Time

In the case of repeated cross-sectional data, PLS path models need to be created separately. Thus, one model is created for each sample in time [36]. The tested structural models with path coefficients and its respective significance are shown in Table 5. The results present that the proposed hypothesis is significant at every timestamp. Therefore, the hypothesis is supported. This means that an increased BPM maturity leads to an improvement in process performance.

Table 5. Results of the test of significance of the direct effects

To test the significance of changes in the effects over time, this study conducted multigroup analysis with the six sets and compared the path coefficients between these six time points. Table 6 presents three significant differences in path coefficients. The effect of BPM maturity on process performance starts to fluctuate significantly after four years of slow increasement. After t₃, the relation between BPM maturity and process performance descended significantly after which it increased significantly the year after. Although less than before, in the final year (t₅–t₆) the path coefficient declined again.

Table 6. Results of the multigroup analysis

5 Discussion

5.1 Implications to Theory and Practice

The results of our study provide several important implications to both theory and practice. Overall, we observed a significant drop in BPM maturity between t₁ and t₂. This result coincided with the organizational transformation in which the hierarchical structure was changed into a more process-oriented organization. Moreover, the results suggest an effect on the enacted BPM practices by digitalization. BPM maturity and process performance have increased substantially since the start of Covid-19 (after t₄ effects measured in t₅), when the organization was forced to make all courses available online. Although scholars stipulate the importance of context in successful BPM implementation [53], the context of organizational dynamics that unfold over time are often neglected. The results in this study call on recognizing temporality as a contextual factor as the complexity of an organization and its dynamics affect the dynamics of business processes that are performed.

In more detail we observed that the dimension Process description decreased significantly during organizational restructuring (between t₁-t₂). This could indicate that the restructuring resulted in an unclear division of labor, which resulted in a call for new working arrangements in the form of process descriptions. The BPM maturity dimensions also indicate what was required in the second disruptive event, t₄-t₆ (Covid-19). There was an immediate need for improving IT resources during this period. This seems to provide empirical evidence that the dynamics emerging from digitalization defies the established logics of BPM [54]. Although it is impossible to anticipate how a process will be performed in the future [55], it does mean that management faces an ongoing gap about how the process is doing over time [56]. This ongoing interaction between digitalization and BPM also strengthens the importance to implement an organizational culture that fosters the continuous exploration of innovation opportunities [19] as innovations can be used to improve BPM maturity. Though this doesn’t mean that the performance will increase likewise.

Finally, we observed that over the last two years the demand for process improvement has significantly increased (Table 4). This might be because of the increasing amount of digitization, fueled by the digital transition related to Covid-19, which caused the as-is processes to be no longer deemed fit the changing organizational context (e.g. online teaching and working from home). This is in line with most well-known BPM lifecycles [57].

So, all of the aforementioned significant findings can be connected to crucial events that had an organizational strategy-level impact. To determine if the emphasis is or has been on the appropriate BPM capabilities, related to organizational dynamics, the BPM maturity model is thus appropriate for usage at this level within an organization. Because of how we have used it, the BPM maturity model is less suitable for use at the operational level. This is primarily due to the fact that we did not discriminate between the many procedures that our respondents are participating in. We observe that the number of respondents rises annually. This makes it possible to use more differentiation in the future, which might also provide better understanding of how specific processes change over time.

5.2 Limitations and Further Research

This study is not without limitations. First, a single case organization serves as the foundation for the research findings. Since there are not many differences between BPM in a university and the management of business processes in an organization, we do not expect BPM maturity development to be substantially different in these two contexts. However, one should be cautious about generalizing the findings to other settings. Future research should examine whether our findings hold across different types of organizations.

Second, although this study suggests an influence of the digital organizational evolution on organizational practices and business processes, this is not empirically examined in our study. Although, recent studies present a direct relationship between BPM and digital innovation (e.g., [58]), hitherto there is a lack of longitudinal research that takes into account the complexity and (digital) dynamics of an organization over time. Hence, an interesting avenue of research is to empirically investigate the role of digital transformation and its relation to BPM for a longer period.

Third, related to the used method, the design of a repeated cross-sectional study implies that responses at the different points in time cannot be traced back to the individual employee. It is thus unclear whether one employee has taken part in multiple surveys and what his/her responses have been. An analysis at the individual level, which can be taken up by future studies, could provide more rich insights in the individual evolution of the employees over time.

Fourth, we are aware that qualitative information may aid in a more accurate interpretation of the findings. As a result, we have spoken with support staff, researchers and teachers and have gotten thoughtful responses to our request to complete the questionnaire. These responses, while acknowledging our findings, were not documented or analyzed. Therefore, any qualitative information is excluded. It could be of use to collect these data in the future so that the findings would also be beneficial.

6 Conclusion

In this study we addressed the research question: How do changes in BPM maturity affect higher education process performance over time? The findings indicate that BPM maturity is affected strongly by organizational dynamics. BPM maturity diminishes with organization restructuring, but the differences in average BPM maturity scores are not significant over this period. In greater detail, one of the dimensions of BPM maturity (process description) has been significantly reduced. This is consistent with what happened, because organizational restructuring results in new ways of working, rendering old descriptions obsolete.

The discrepancies in BPM maturity over the last three years are significant. During this phase, BPM maturity increases. That was also the moment when Covid-19 forced the organization to completely switch to digital education. During this time, the element of information technology was significantly improved first, followed by the element of process innovation the following year. On these grounds, it is concluded that BPM maturity grows more firmly in the context of digitalization than in the context of an organizational restructuring.

When examining how these changes in BPM maturity affect process performance, the samples reveal a positive association. Each year's samples demonstrate a significant positive relation between BPM maturity and process performance. Although the changes are minor (±0.13), the strength of this association varies from year to year, and the variances are significant when looking at the last three years. As a result, we conclude that BPM maturity has a positive effect on process performance in both the short and long term.

Given that no longitudinal qualitative research have been found in the direction of BPM maturity, these findings are complimentary within the field of BPM. The findings shed light on how organizational dynamics affect the development of BPM maturity within one organization. The latter is crucial for organizational management because they have to prioritize which BPM capabilities need attention.