Keywords

1 Introduction

A project of business processes modeling has been running at the Czech Technical University in Prague at Faculty of Electrical Engineering (CTU FEE) since 2009 (led by the Centre for Knowledge Management [1, 2]). Within this project more than 400 business processes in BPMN notation have described. Identical projects (also led by the Centre for Knowledge Management) has been implemented at CTU - Faculty of Mechanical Engineering, University Centre for Energy Efficient Buildings, University of West Bohemia and Škoda Praha. An increasing demand for the future can be expected for business process modeling not only in academic, but also in commercial environment It was empirically encountered various difficulties which stem from deficiencies BPMN during all mentioned projects [3]. These were mainly:

  • Widely varying levels of detail captured business processes among individual creators.

  • Changing participant’s role during the execution of one business process.

  • The high number of BPMN symbols (tasks, gateways, events, etc.) within a business process.

  • Multiplicity of same BPMN symbols.

  • The different levels of the distribution of one business process into multiple sub processes.

The above-mentioned shortcomings have often led to a redesign of the entire business process. Therefore, the time required for quality of design and process description was disproportionately increased, which should be simple, easy to understand, and above all to clearly demonstrate real execution, including all details.

The goal of this study is to provide a systematic search of the available literature and to find answers to the following questions:

  1. 1.

    Can the quality of business process models be measured using certain measures, indicators or other methods?

  2. 2.

    Do any measures, indicators or methods exist?

  3. 3.

    If so, are they used in common practice?

  4. 4.

    If so, are they part of any standards?

  5. 5.

    Is there any standard for presenting business process models?

The meaningfulness of mentioned questions confirms one publication [5] where the authors deal with similar problems of measuring the quality and effectiveness of ten thousand processes. For this reason, we decided to arrange for a systematic literature review in order to find and analyze primary studies about business process modeling and measures of business process design quality.

2 Research Method

2.1 Search Process

Systematic literature review was performed on the basis of [4]. To ensure quality, we sought sources in knowledgeable digital libraries. We used the following databases:

  • Web of Science

  • ACM Digital Library

  • EBSCO

  • IEEE Xplore

  • Scopus

  • SpringerLink.

The area in which we wanted to perform literature review was the one of process measures. When we were searching for relevant resources we started from general keywords such as “Process metrics” and “BPMN measures”. The granularity of the search was gradually improved by refinement of the keywords. The final form of keywords stuck at “Process quality metrics” and “Process complexity metrics”. Using these keywords we found set of relevant resources. In the last step of the search we were able to specify process metrics using these expressions:

  • Process coupling complexity.

  • Process cohesion complexity.

  • Control flow complexity.

We have also influenced the results of our research by following criteria:

  • Publications are available online, in full text.

  • Language of publications was limited to English.

  • Publications are cited in other publications.

  • Publications take the form of scientific work, books or conference publications.

The results of the search were generated in the period from 22nd January 2015 to 24th February 2015 and contains the literary sources published by this date.

3 Results

3.1 Search Results, Data Extraction and Synthesis

During literature reviews we found Thirty-three suitable scientific publications [6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37]. We read each of these publications and selected information regarding to measures of quality in business processes. Subsequently we found twenty-two measures of quality. The results of the study sources can be assessed as very relevant and of high-quality. We searched relevant publications only in the established digital libraries. We therefore conclude that the quality of the studies is high.

3.2 Measures of Quality

As mentioned above we found twenty-two measures of quality:

  1. 1.

    Number of activities (NOA, NOT) - [5, 8, 10, 11, 19, 20, 30, 32, 35].

  2. 2.

    Control-Flow Complexity (CFC) - [5, 7, 8, 10, 11, 14,15,16,17, 19,20,21,22, 25, 27, 29,30,31,32,33, 35, 37].

  3. 3.

    Max/mean nesting depth - [19, 20, 32].

  4. 4.

    Number of handles - [19].

  5. 5.

    Cognitive weight (Cognitive Complexity) - [9, 13, 19, 20, 32, 35].

  6. 6.

    BPM (Anti) Patterns - [19, 20].

  7. 7.

    Fan-in/Fan-out (Modularization) - [11, 19, 20, 35].

  8. 8.

    Coefficient of network complexity (CNC) - [5, 8, 10, 11, 14, 26, 30, 32].

  9. 9.

    Cyclomatic number - [14, 21, 26].

  10. 10.

    Complexity index (CI) - [5, 8, 14, 26].

  11. 11.

    Restrictiveness estimator (RT) - [8, 14, 26].

  12. 12.

    Number of trees in graph - [14, 26].

  13. 13.

    Process Cohesion (TPC, LPC) - [6, 18, 24, 33].

  14. 14.

    Process Coupling (CBO, RFC, MPC, ICP) - [18, 23, 24, 33].

  15. 15.

    Process coupling/cohesion ratio - [18, 24].

  16. 16.

    Halstead-based Process Complexity (HPC) - [8, 10, 11, 25, 32, 33, 35].

  17. 17.

    Interface Complexity (IC) - [8, 10, 11, 32,33,34,35].

  18. 18.

    Density - [36].

  19. 19.

    Cross-Connectivity (CC) - [10, 29, 36].

  20. 20.

    CP - [12].

  21. 21.

    GQM - [20].

  22. 22.

    Q0, Q1, Q3 - [28].

The frequency of occurrence of metrics in publications is shown in Fig. 1. The most widely accepted metrics are (1) the Control-Flow Complexity, (2) number of Activities and (3) coefficient of network Complexity.

Fig. 1.
figure 1

Frequency of occurrence of metrics in publications.

Full size image

In terms of time we found publications between 2001 and 2014. Only one publication has earlier publication date as specified in Fig. 2. Figure 2 shows that the interest in metrics increased in 2005. Most publications were appearing between 2006 and 2010.

Fig. 2.
figure 2

Number of publications per published year.

Full size image

3.3 Demographic Data

From a demographic point of view we found out that publications come mostly from Western European regions, which indicates an increased interest in this issue, particularly in countries like the Netherlands and Portugal (Fig. 3).

Fig. 3.
figure 3

Number of publications per state.

Full size image

3.4 Metrics Implementation

We developed “Metrics calculation software” [40] according to the presented results. The first version of this software was presented on the EOMAS 2016. From the year 2016 till end of year 2018 we updated the software calculation methods according to our research [41, 42]. This software is available for free. The software is running in the application server [41]. The approach for that is via an internet browser. It is not necessary to install any software into the computer.

4 Discussion

From the research it is obvious that similar problems are solved by other scientific teams. As we supposed, a lot of metrics are based on the BPM chart analysis. Typical examples can be: Number of activities, Control-Flow Complexity etc. The metrics based on the chart analysis have got deep background in the typical software metrics as: Cyclomatic number, etc. [14, 21, 26].

It is not easy to use these metrics in the real business at all, because the BPM is influenced by factors which cannot be found from the BPM chart only. These factors influence the final business process indirectly.

For example, these factors can be identified according to the actor type used in the BPM [38, 39]:

  • Exactly defined actor (for example students).

  • Fuzzy defined actor (for example study department – nobody knows who will serve study requests – it is ambiguous).

  • Black box actor type (for example another system which communicates by defined interface with the final business process).

Other factors can be identified according to the BPM development team skills:

  • Beginners (less than 50 models designed).

  • Intermediate (less than 500 models designed).

  • Excellent (more than 500 models designed).

And finally, other factors that can be identified according to the BPM requested company organization type [38, 39]:

  1. 1.

    Organic (organization with excellent knowledge sharing).

  2. 2.

    Semi-detached (organization with mixed quality knowledge sharing).

  3. 3.

    Embedded (organization with problematic knowledge sharing).

In fact the current BPM metrics do not take example from COCOMO [38] and other methodologies (Function points, Use Case points). These methodologies try to describe the implementation process more comprehensively. In the Constructive cost model COCOMO [38], the authors defined cost estimation for the software development. Although the business process modeling is not software development, we can recognize some parallels. The parallels are factors which we have defined above. These factors can influence process modeling significantly. We can supposed to find more factors influencing the BPM.

The COCOMO [38] defines function count by type, direct quotation:

“The unadjusted function counts should be counted by a lead technical person based on information in the software requirements and documents design. The number of each of the five user function types should be counted (Internal Logical File* (ILF), External Interface File (EIF), External Input (EI), External Output (EO), and External Inquiry (EQ)).” (Fig. 4), [38]. The COCOMO determines the complexity level for function counts, direct quotation:

“Classify each function count into Low, Average and High complexity levels depending upon the number of data element types contained and the number of file types referenced. Use the following scheme” [38].

Fig. 4.
figure 4

Determined function counts by type COCOMO example [38].

Full size image

Based on the COCOMO approaches we can try to understand the business process as a chain of activities (parallel with COCOMO functions).

5 Conclusion

From this point of view, it’s very useful to design more business process metrics based on the factors of realization. Not only based on the BPM chart analysis.

There are examples which we found during the research:

  • Actor role is changed twice during the process of applicant study to student. It should be solved by two actors – Applicant and Student. These roles may follow, but do not change from one to the other.

  • The fuzzy actor responsibility. The typical example is if some important artefact for the process (for example the invoice) is consumed by the exact actor (director of accounting department) or the fuzzy actor (accounting department).

  • The business process is joined with more processes that can be wrong or ineffective. In this case the business process can be designed perfectly and quality metrics can report very positive values, nevertheless the complete system will not be working effectively.

All these examples cannot be described by the metrics based on the BPM chart analysis. Similarly as the COCOMO uses attributes of function complexity, we should try to design new attributes for process complexity from the point of realization. Today we can suppose the process model designer skills are fundamental. Do we have high or low skills with modeling processes? These factors are described and used for software complexity prediction by COCOMO. Maybe it will be useful to define them for the BPM, too. This questions are still waiting for the answer. To start finding the answer for them, we developed software tool. All metrics presented in this paper are now covered via Metrics calculation software [40]. We worked on that from 2016 till now. We try to get interested person for some easy way to measure his/her BPM model. The volunteers are helping us to update implemented metrics. The problem is, the calculation algorithm that must be robust. The BPM cases are not generating identical output during the model saving. Thanks to that, we need to make a lot of updates at the software. And in need time.