A study for project risk management using an advanced MCDM-based DEMATEL-ANP approach

Abstract

Adopting the multidimensional balanced scorecard approach to improve project management and organizational performance can help reduce project risks and potential disasters in order to maximize the investment benefits and security of IT projects with high risks. We adopted the stimulus-organism-response (S-O-R) research context, whose stimulus, organism, and response refer to project risk, project management, and organizational performance, respectively. Furthermore, a questionnaire survey of project management experts was conducted. We then utilized the DANP model, which combines the Decision-Making Trial Evaluation Laboratory (DEMATEL) and the analytical network process, to explore the relationships among project risks, project management, and organizational performance. We observed the following: (a) effective project management can reduce project risk and improve organizational performance; (b) project risk is most influenced by the user; (c) senior manager support significantly influences project management; (d) organizational learning and growth has a significant impact on organizational performance; and (e) the criteria with the largest relative weight is generally considered to pose a risk, which indicates that the experts seriously consider that project risk. Our results can provide a valuable reference for project management to reduce risk and improve organizational performance.

1 Introduction

Information systems are used to obtain big data computing, convert and distribute information. These activities are meant to strengthen smart management decisions, improve organizational effectiveness, and ultimately increase the company’s profitability. Information systems are a combination of elements such as management, organization and technology. The management of information systems includes leadership, strategy and management. So, the information system has a very important influence in the enterprise organization. The information system is managed by the enterprise’s information technology (IT) department, and the IT department performance is closely related to the organizational performance of the business. In the enterprise organization performance evaluation mostly use the balanced scorecard (BSC), it provides a good practical guide. Kaplan and Norton (1992) proposed the BSC concept that has been widely used in organizational performance evaluation. The BSC can effectively translate the strategy into actions that help organizations to achieve their goals. The BSC consists of the following dimensions: financial, learning and growth, internal business process, and customers. These are also the key performance indicators (KPIs) which are tracked by the organization and its departments. IT departments traditionally develop and manage a number of information system projects within an organization. The process of developing an information system project relies heavily on human intelligence, and thus has a high degree of uncertainty. Information system projects embody various kinds of risks, including IT, human resources, availability, project teams, projects and organizations, and strategic and political risks (Gido and Clements 2012). In particular, project risk management affects project selection, project scope determination, actual schedules, and cost estimates (Schwalbe 2010). Reviewing previous studies about how IT departments can determine the relationship between project risks, project management, and organizational performance is not only interesting, but vital. This study primarily focuses on the dynamic relationship between IT organizational performance targets, project management, and project risks. The purposes of the study are to: (a) explore important variables in these three areas; (b) explore the relationships between these three areas; and (c) establish effective improvements in these three areas and propose feasible methods for carrying them out.

The rest of our article is divided into the following parts: In Sect. 2, we review previous studies related to organizational performance and project management and summarize our framework and methodology; the expert questionnaires and data analysis are described in Sect. 3; Sect. 4 consists of our research findings and managerial implications; and in Sect. 5, we offer conclusions and potential research ideas for the future.

2 Literature review

In this section, we introduce project risk and management, the balanced scorecard concept, and the DANP method.

2.1 Project risks and management

The definitions of risk and uncertainty differ significantly (Zimmermann 2000). Risk refers to not knowing the outcome of an event, but understanding the outcome of many different events that may occur. Meanwhile, uncertainty refers not only to not knowing the outcome of an event, but also not being able to judge it qualitatively or quantitatively from the available information (Zimmermann 2000). Risk management is widely recognized as a method for improving IT project performance (De Bakker et al. 2010), and even “safe” projects can still have risks.

Project risk refers to an uncertain event that can negatively impact a project (Gido and Clements 2012), and risk management (Odzaly et al. 2018) is defined as the understanding of such risk and the subsequent planning to handle it. The goal is to obtain the maximum security assurance with the minimum cost. The Project Management Knowledge System (PMBOK) (PMI 2008) includes risk management planning, risk identification, analysis, monitoring, and control all within project risk management. Identifying any risks that may negatively impact project objectives is risk identification, while the purpose of risk monitoring and control is to identify uncertainties in risk factors and alleviate the impact of such risks, as well as assist managers in adopting the appropriate actions to counter those risks. Previous studies have extensively investigated project risk management (Han and Huang 2007; Van Os et al. 2015; Yim et al. 2015; Liu et al. 2016; Zhang 2016) and have generally observed that the proper handling and management of risk can significantly improve a project’s success (Gido and Clements 2012). To further explore the risk of an information project, this study refers to Han and Huang (2007) for the six risk scales.

Project management (Vidal et al. 2013) is the adaptable and effective application and coordination of resources to meet project objectives and requirements. Kerzner (1984) suggests that project management consists of not only the planning, organization, and utilization of controllable resources to achieve specific objectives but also the allocation of the specific project tasks to various employees within the department. In recent studies on project management (Sangaiah and Thangavelu 2013; De Carvalho et al. 2015; Golini et al. 2015; Willems and Vanhoucke 2015), many of the authors concluded that key success factors included cost, performance, completion time, and range. Pinto and Slevin (1987) and Kearns (2007) all found that senior management’s support for projects is another key to success. Without senior management to support the financial, human, and material resources to implement IT projects, the entire project is more likely to fail. Furthermore, Jun et al. (2011) suggested that project planning and control, internal integration, and user participation may all considerably influence project performance. Based on the existing literature, this study relies on four project management variables (senior manager support, project planning and control, internal integration, and user participation) to perform our research. In the project risk management process, risk analysis is first used to identify risks. According to the literature, project risks can be technical, time, monetary, employment, market, or structural risks (Schwalbe 2010; Gido and Clements 2012).

2.2 Balanced scorecard

Balanced Scorecard (Bassen et al. 2006; Lee and Lai 2007), abbreviated BSC, is a performance management tool that transforms an enterprise’s strategic objectives into a number of specific performance appraisal indicators and evaluates the realization status of these indicators at various times in order to establish a reliable implementation basis for completing the enterprise’s strategic objectives. Kaplan and Norton (1992) indicated that financial experts are responsible for overseeing the performance evaluation system and that senior management is not necessary. BSC assists managers in transforming their strategies into quantifiable goals related to finances, customers, internal business processes, and learning and growth, as well as judging performance aspects in all areas. The true purpose of BSC is to identify strategies and goals through cross-section communication. BSC divides performance targets into the following categories: financial, customer, internal business, and learning and growth.

The BSC questionnaire in this study is based on the work of Wang et al. (2006), Jun et al. (2011), and Devine et al. (2010). Their research is relevant to the learning and growth perspectives in project development. Therefore, we refer to BSC-related literature to determine the following four organizational performance variables: customer performance, internal business process performance, financial performance, and learning and growth performance.

2.3 Methods and materials

2.3.1 DANP

This study has collected relevant literature from journal articles and developed a research framework according to the various performance targets. We designed a pairwise comparison-based questionnaire and distributed it to relevant professionals. We adopted the Decision-Making Trial and Evaluation Laboratory (DEMATEL)-based analytical network process (ANP) model to prepare the questionnaires to determine relationships and criteria weights since they could be used in a number of fields, including supplier evaluation (Liou et al. 2014), carbon system (Liou 2015), and life insurance industries (Shen et al. 2017). The research framework of this study is validated by the DANP model, and explore the relationship between dimensions and criteria. DANP does not require any presumptions; therefore, applying DANP is suitable for exploring dependencies in this study. The DANP model has two parts: (1) using DEMATEL build IRM, and (2) using ANP to find influential weights, as shown in Fig. 1.

Fig. 1

DANP model

2.3.2 DEMATEL

The Decision-Making Trial and Evaluation Laboratory (DEMATEL) method converts the causal relationships of an element into a visible structural model and is suitable for handling complex social issues (Gabus and Fontela 1972, 1973). DEMATEL has a number of practical advantages and has thus been adopted in several fields, including software development (Sangaiah et al. 2015), portfolios (Altuntas and Dereli 2015), supply chains (Supeekit et al. 2016; Wu et al. 2017), suppliers (Mirmousa and Dehnavi 2016), emergency management (Zhou et al. 2017), emergency system (Han and Deng 2018) and knowledge transfer (Sangaiah et al. 2017; Gopal et al. 2018), among others.

Steps 1–3 of the DANP model for creating an IRM using the DEMATEL technique are summarized below:

Step 1: Create an initial direct-relationship matrix The purpose of this step is to sort out the basic information of the collected expert questionnaires. The initial direct-relationship matrix Z is the matrix of n × n and obtains the influence degree of 1 to 4 between the elements.

Step 2: Standardize the initial relationship matrix to obtain the total-relationship matrixes The initial relationship matrix $X=\left[ {{x_{ij}}} \right]~$. is obtained by joining Eqs. (1) and (2) together.

$$Y=ma{x_{ij}}\left[ {\mathop {max}\limits_{{1 \leqslant i \leqslant n}} \mathop \sum \limits_{{j=1}}^{n} {z_{ij}},\;\mathop {max}\limits_{{1 \leqslant j \leqslant n}} \mathop \sum \limits_{{i=1}}^{n} {z_{ij}}} \right],$$

(1)

$$X=\frac{1}{Y}Z.$$

(2)

The total-relation matrix can be obtained with Eqs. (3) and (4).

$$T=X+{X^2}+ \cdots +{X^p}=X \times {\left( {I - X} \right)^{ - 1}}={\left[ {{X_{ij}}} \right]_{n \times n}},\;\;p \to \infty$$

(3)

$$T={\left[ {{t_{ij}}} \right]_{n \times n}},\quad i,j=1,2, \ldots ,n.$$

(4)

In Eq. (5), T_c is a super-matrix that explains the relationship between the criteria and the degree of influence.

(5)

From Eq. (6), we can then find the total dimensions relation matrix ${T_d}$.

$${T_d}=\left[ {\begin{array}{*{20}{c}} {t_{d}^{{11}}}& \cdots &{t_{d}^{{1j}}}& \cdots &{t_{d}^{{1m}}} \\ \vdots && \vdots && \vdots \\ {t_{d}^{{i1}}}& \cdots &{t_{d}^{{ij}}}& \cdots &{t_{d}^{{im}}} \\ \vdots && \vdots && \vdots \\ {t_{d}^{{m1}}}& \cdots &{t_{d}^{{mj}}}& \cdots &{t_{d}^{{mm}}} \end{array}} \right].$$

(6)

Step 3: Develop an impact relationship map The results of Eqs. (7) and (8) can obtain r and s, which are the sums of rows and the sums of columns (s), respectively.

$$r={\left[ {{r_i}} \right]_{n \times 1}}={\left[ {\mathop \sum \limits_{{j=1}}^{n} {t_{ij}}} \right]_{n \times 1}},$$

(7)

$$s={\left[ {{s_j}} \right]_{n \times 1}}={\left[ {\mathop \sum \limits_{{i=1}}^{n} {t_{ij}}} \right]_{n \times 1}}.$$

(8)

r_i is the criteria i (or dimensions) represents the cause; s_j is the criteria j (or dimensions) represents the result. If (r_i − s_j) is positive, the criteria is a “cause”. If (r_i − s_j) is negative, the criteria is a “result” (Wu 2008; Wu and Lee 2007). The quadrants are described as follows: quadrant I represents a high degree of correlation with a high degree of causality; quadrant II represents a low degree of correlation with a high degree of causality; quadrant III represents a low degree of correlation with a low degree of causality; and quadrant IV represents a high degree of correlation and low degree of causality.

2.3.3 ANP

To solve the problem of complex nonlinear network relationships, Saaty (1996) proposed the Analytical Network Process (ANP).

Step 4: Standardize the total criteria relationship matrix The operation of Eq. (9) allows the total criteria relationship matrix T_c to be calculated in order to obtain $T_{c}^{*}$.

$$d_{{ci}}^{{11}}=\mathop \sum \limits_{{j=1}}^{{{m_1}}} t_{{ij}}^{{11}},\quad i=1,2, \ldots ,{m_1},$$

$$\begin{aligned} T_{{c*}}^{{11}} & =\left[ {\begin{array}{*{20}{c}} {t_{{c11}}^{{11}}/d_{{c1}}^{{11}}}& \cdots &{t_{{c1j}}^{{11}}/d_{{c1}}^{{11}}}& \cdots &{t_{{c1{n_1}}}^{{11}}/d_{{c1}}^{{11}}} \\ \vdots &&&& \vdots \\ {t_{{ci1}}^{{11}}/d_{{ci}}^{{11}}}& \cdots &{t_{{cij}}^{{11}}/d_{{ci}}^{{11}}}& \cdots &{t_{{ci{n_1}}}^{{11}}/d_{{ci}}^{{11}}} \\ \vdots &&&& \vdots \\ {t_{{c{n_1}1}}^{{11}}/d_{{c{n_1}}}^{{11}}}& \cdots &{t_{{c{n_{1j}}}}^{{11}}/d_{{c{n_1}}}^{{11}}}& \cdots &{t_{{c{n_1}{n_1}}}^{{11}}/d_{{c{n_1}}}^{{11}}} \end{array}} \right]=\left[ {\begin{array}{*{20}{c}} {t_{{c11*}}^{{11}}}& \cdots &{t_{{c1j*}}^{{11}}}& \cdots &{t_{{c1{n_1}*}}^{{11}}} \\ \vdots &&&& \vdots \\ {t_{{ci1*}}^{{11}}}& \cdots &{t_{{cij*}}^{{11}}}& \cdots &{t_{{ci{n_1}*}}^{{11}}} \\ \vdots &&&& \vdots \\ {t_{{c{n_1}1*}}^{{11}}}& \cdots &{t_{{c{n_1}1*}}^{{11}}}& \cdots &{t_{{c{n_1}{n_1}*}}^{{11}}} \end{array}} \right] \\ {\text{and}}\quad T_{c}^{*} & =\left[ {\begin{array}{*{20}{c}} {T_{{c*}}^{{11}}}& \cdots &{T_{{c*}}^{{1j}}}& \cdots &{T_{{c*}}^{{1m}}} \\ \vdots && \vdots && \vdots \\ {T_{{c*}}^{{i1}}}& \cdots &{T_{{c*}}^{{ij}}}& \cdots &{T_{{c*}}^{{im}}} \\ \vdots && \vdots && \vdots \\ {T_{{c*}}^{{m1}}}& \cdots &{T_{{c*}}^{{mj}}}& \cdots &{T_{{c*}}^{{mm}}} \end{array}} \right]. \\ \end{aligned}$$

(9)

Step 5: Standardize the total dimensions relationship matrix Eq. (10) allows the total criteria relationship matrix T_d to be calculated in order to obtain $T_{d}^{*}$.

$$t_{d}^{i}=\mathop \sum \limits_{{j=1}}^{m} t_{d}^{{ij}},$$

$$T_{d}^{*}=\left[ {\begin{array}{*{20}{c}} {t_{d}^{{11}}/t_{d}^{1}}& \cdots &{t_{d}^{{1j}}/t_{d}^{1}}& \cdots &{t_{d}^{{1m}}/t_{d}^{1}} \\ \vdots && \vdots && \vdots \\ {t_{d}^{{i1}}/t_{d}^{i}}& \cdots &{t_{d}^{{ij}}/t_{d}^{i}}& \cdots &{t_{d}^{{im}}/t_{d}^{i}} \\ \vdots && \vdots && \vdots \\ {t_{d}^{{m1}}/t_{d}^{m}}& \cdots &{t_{d}^{{mj}}/t_{d}^{m}}& \cdots &{t_{d}^{{mm}}/t_{d}^{m}} \end{array}} \right]=\left[ {\begin{array}{*{20}{c}} {T_{{d*}}^{{11}}}& \cdots &{T_{{d*}}^{{1j}}}& \cdots &{T_{{d*}}^{{1m}}} \\ \vdots && \vdots && \vdots \\ {T_{{d*}}^{{i1}}}& \cdots &{T_{{d*}}^{{ij}}}& \cdots &{T_{{d*}}^{{im}}} \\ \vdots && \vdots && \vdots \\ {T_{{d*}}^{{m1}}}& \cdots &{T_{{d*}}^{{mj}}}& \cdots &{T_{{d*}}^{{mm}}} \end{array}} \right].$$

(10)

Step 6: Develop the weighted super-matrix and obtain the influential weights of the elements The original weighted super-matrix S can be obtained by multiplying the normalized $T_{c}^{*}$ and $T_{d}^{*}$ through Eq. (11).

$$S=\left[ {\begin{array}{*{20}{c}} {T_{{c*}}^{{11}} \times T_{{d*}}^{{11}}}& \cdots &{T_{{c*}}^{{1j}} \times T_{{d*}}^{{1j}}}& \cdots &{T_{{c*}}^{{1m}} \times T_{{d*}}^{{1m}}} \\ \vdots && \vdots && \vdots \\ {T_{{c*}}^{{i1}} \times T_{{d*}}^{{i1}}}& \cdots &{T_{{c*}}^{{ij}} \times T_{d}^{{ij}}}& \cdots &{T_{{c*}}^{{im}} \times T_{{d*}}^{{im}}} \\ \vdots && \vdots && \vdots \\ {T_{{c*}}^{{m1}} \times T_{{d*}}^{{m1}}}& \cdots &{T_{{c*}}^{{mj}} \times T_{{d*}}^{{mj}}}& \cdots &{T_{{c*}}^{{mm}} \times T_{{d*}}^{{mm}}} \end{array}} \right].$$

(11)

The S matrix can be transposed by Eq. (12) to obtain ${S^*}$.

$${S^*}=\left[ {\begin{array}{*{20}{c}} {T_{{c*}}^{{11}} \times T_{{d*}}^{{11}}}& \cdots &{T_{{c*}}^{{i1}} \times T_{{d*}}^{{i1}}}& \cdots &{T_{{c*}}^{{m1}} \times T_{{d*}}^{{m1}}} \\ \vdots && \vdots && \vdots \\ {T_{{c*}}^{{1j}} \times T_{{d*}}^{{1j}}}& \cdots &{T_{{c*}}^{{ij}} \times T_{d}^{{ij}}}& \cdots &{T_{{c*}}^{{mj}} \times T_{{d*}}^{{mj}}} \\ \vdots && \vdots && \vdots \\ {T_{{c*}}^{{1m}} \times T_{{d*}}^{{1m}}}& \cdots &{T_{{c*}}^{{im}} \times T_{{d*}}^{{im}}}& \cdots &{T_{{c*}}^{{mm}} \times T_{{d*}}^{{mm}}} \end{array}} \right].$$

(12)

After the ${S^*}$ matrix is multiplied by itself (${\text{li}}{{\text{m}}_{\varphi \to \infty }}{({S^*})^\varphi }$), the weight of each criteria can be obtained.

3 Research methodology

In this subsection, we summarize our research process, including its framework and design, as well as the data collection.

3.1 Research framework

According to relevant literature and the S-O-R model, environmental factors act as stimuli that can influence an individual’s response and behavior (Mehrabian and Russell 1974). We illustrate the dimensions and criteria of the research framework shown in Fig. 2.

Fig. 2

Study research framework

3.2 Research design

To analyze the relationships among project risk, project management, and organizational performance on the BSC, we sent questionnaires to experts with considerable experience in information systems project management. The returned questionnaires were then analyzed using both a DANP model and hybrid multi-criteria decision-making (MCDM) (Abdel-Basset et al. 2017; Goyal et al. 2018) framework-based DEMATEL, as well as ANP techniques so that we could better understand the relationship between various dimensions and the criteria. The proposed method for this study is shown in Fig. 3.

Fig. 3

Flowchart of the proposed method

3.3 Data collection

This section contains a description of the expert survey process, from the introduction to the research participants to the topics, dimensions, and criteria.

• 1. Research participants We recruited 18 experts with background experience in IT departments and years of project management experience. According to the findings of Northcutt and McCoy (2004), a focus group should include 12–20 experts that meet the following conditions: (a) have a wealth of experience in research topics; (b) able to express their ideas; (c) actively participate in research; (d) homogeneous group regarding distance and power; and (e) able to demonstrate excellent team spirit by not being overly assertive or too afraid to share their opinions.

• 2. Research topic Experts with rich experience in project management were invited to participate in the group consensus, and the DANP method was used to study the relationship between variables (see Fig. 2).

• 3. Dimensions and criteria In accordance with our literature review, the dynamic relationship between the dimensions and criteria are shown in Table 1.

Table 1 Dimensions and criteria

3.4 Analysis

3.4.1 Expert respondents

We recruited senior employees from IT departments to respond to our questionnaire after determining the ideal number of participants according to the information saturation principle. The demographic information of the surveyed experts is shown in Table 2.

Table 2 Background information of experts

3.4.2 Establishing IRMs using DEMATEL technique

Tables 3 and 4 show the group consensus from the experts concerning the impact of each criterion. The error rate in Table 4 is 4.294%, which indicates a significant confidence level of 95%. Therefore, the initial direct-relationship matrix Z can be obtained from Table 14, and this data can be used for DEMATEL operations.

Table 3 Interview with 17 experts’ group consensus on the impact of criteria

Table 4 Interview with 18 experts’ group consensus on the impact of criteria

We subsequently carried out steps 1–3 previously mentioned in subsection 2.3.2. (1) Step 1: The initial direct-relationship matrix Z was normalized to obtain matrix X. All elements x_ij in X must satisfy 0 ≤ x_ij ≤ 1, and the principal diagonal element must be equal to 0. The results of the normalized initial direct-relation matrix are shown in Table 5. (2) Step 2: The total criteria relation matrix T_c and total dimensions relation matrix T_d were then found through matrix X. The total criteria relationship matrix and total dimensions relationship matrix are presented in Tables 6 and 7, respectively. (3) Step 3: Lastly, we determined the degree to which each criteria and dimension influenced and was influenced by all others, as shown in Table 8.

Table 5 Normalized initial direct-relationship matrix

Table 6 The relationship matrix of T_c

Table 7 The relationship matrix of T_d

Table 8 The degree of influence of dimensions and criteria

After performing steps 1–3, we used the results to produce 4 IRMs, which are shown in Fig. 4, 5, 6 and 7.

Fig. 4

IRM of dimensions

3.4.3 Criteria weights of ANP

We adopted the ANP technique as follows in order to perform steps 4–6: (1) Steps 4 and 5: Find the original weighted super-matrix. Tables 9 and 10 show the results of the standardized total criteria relationship matrix and the normalized total dimensions relationship matrix, respectively. (2) Step 6: Make ${S^*}$ satisfy the column-stochastic principle. The weighted super-matrix is shown in Table 11. Next, ${S^*}$ was multiplied by itself (${\text{li}}{{\text{m}}_{\varphi \to \infty }}{({S^*})^\varphi }$) to create the converged stable limited matrix W. The limit of the weighted super-matrix is presented in Table 12. As for our experimental results, the results of the criteria weights, the relationship matrix of organizational performance and project risk, and the relationship matrix of organizational performance and project management are shown in Tables 13, 14 and 15, respectively.

Table 9 The relationship matrix of $T_{c}^{*}$

Table 10 The relationship matrix of $T_{d}^{*}$

Table 11 Calculation of ${S^*}$

Table 12 Calculation of W

Table 13 The list of weight ranks for each dimension and criterion

Table 14 The relationship matrix of Z and X

Table 15 The relationship matrix of Z and Y

4 Results and discussion

With the work experiences of senior employees from IT departments and the application of the MCDM-based DANP model, we were able to determine the impact relationships and importance of the dimensions examined in this study.

4.1 Findings

We report the following findings based on the analytical DANP model results.

• 1. Dimensions impact relationship The impact relationship of the three dimensions are valid, as shown in Fig. 4. The degree of influence of these elements is understood. Since the matrix does not contain null values, it truly reflects a dynamic relationship of the real-world organization. Experts believe that Dimension Z (BSC) is the most affected by other criteria.

• 2. Impact relationship of criteria under dimension X User risk is greatly affected by other criteria, as shown in the analysis results in Fig. 5. Project risk has a high impact, if users view the project as unfavorable.

• 3. Impact relationship of criteria under dimension Y Support from senior managers is a high-impact factor, as shown in Fig. 6. Therefore, senior managers’ support for a project will have a significant impact on project management.

• 4. Impact relationship of criteria under dimension Z “Learning and growth performance” is highly influential, meaning that organizational learning and growth has a high impact on organizational performance in information project activities, as shown in the dynamic relationship in Fig. 7.

• 5. Weights of each dimension and criteria The influential weights of the criteria and dimensions are shown in Table 13. According to the analysis results, the experts highly valued project risk. Risk planning and control was ranked number 1, while two criteria (learning and growth performance, financial performance) had the least impact compared to the other criteria.

• 6. The impact relationships of X, Y, and Z We have learned that risk planning and control are valued for organizational performance. Furthermore, project planning and control are critical to organizational performance in the project management dimension, as shown in the analysis results of Tables 14 and 15.

• 7. Necessary improvements for each dimension and criteria In accordance with the causal relationship between each dimension and criteria, we listed the dimensions that were studied in order of priority with regard to requiring improvement: project management, project risk, and BSC (i.e., organizational performance). Improving risk planning and control and project planning and control should be particularly emphasized. Through project management technology, project managers reduce project risks and improve organizational performance in information systems.

Fig. 5

IRM of criteria under dimension X

Fig. 6

IRM of criteria under dimension Y

Fig. 7

IRM of criteria under dimension Z

4.2 Managerial implications

The research results show that information system project managers should make improvements related to the following criteria: (a) risk planning and control, for which the project manager should accurately assess resources and strictly manage the progress of a project; (b) project planning and control, for which the critical path method should be used to improve and review project milestones; and (c) project complexity risk, for which unfamiliar techniques should be avoided.

5 Conclusion

In this study, we applied the MCDM evaluation method to obtain valuable smart defense risk management results. Using the DEMATEL method, this study found that project management (Y) had the greatest impact, which BSC (Z) had the least impact was. In Dimension X’s analysis, user risk had the greatest degree of impact, while project complexity risk had the least impact. In Dimension Y’s analysis, senior manager support had the greaters degree of impact, while project planning and control had the least impact. In Dimension Z’s analysis, learning and growth performance had the greatest impact, while the internal business processes had the least impact. We adopted IRM and DANP in order to understand the causality and weights between each dimension and criterion. Our empirical results show that the key to the success of an information system project depends project risk management. When manpower, time, and budget are limited, project managers need to be able to adjust resources at any time to enhance project resource integration.

This study has two major limitations. First of all, we can only verify the situation at the specific point in time, so we can only prove the current situation at a specific point in time and consider that point in a status study or a status survey. We recommend that future follow-up researchers extend the data collection period through longitudinal studies. Secondly, the evaluation criteria of this study came from historical documents. Through in-depth interviews, we can find out other possible criteria that can be used as an extension of follow-up research. Furthermore, this research mainly focuses on information system project experts and discusses the relationship between the information system project and the organizational performance of the IT department. Therefore, in the future, differences between of information system projects and non-information system projects can be explored. In follow-up studies, researchers may expand the scope of research sampling so that the research results can be applied to projects in other areas.