Keywords

1 Introduction

With the rapid development of information technology, the use of various software products in the field of industrial control is becoming more and more popular, and their status is becoming more and more important. As the nerve center of nuclear power plant, the Digital Control System (DCS) plays an important role in ensuring the safe, reliable and stable operation of nuclear power plant [1]. Especially in the process of localization of DCS in nuclear power plants, the design and development of DCS software products are characterized by heavy tasks, high accuracy and reliability requirement. The quality of DCS software directly affects the stable, safe and efficient operation of nuclear power plant [2]. Quality control in the process of localization is the premise of ensuring the quality of nuclear software [3].Therefore, it is urgent to evaluate the software quality of instrument and control system independently developed in nuclear power plant objectively, scientifically and efficiently. According to ISO/IEC 25010 software quality evaluation standard, a new quantitative evaluation model based on analytic hierarchy process is proposed in this paper [4].

2 Software Quality Evaluation Index

Software quality evaluation, that is, the detection and measurement of software product quality characteristics, is based on the criteria of software quality evaluation, run through the software life cycle process, and implement in parallel with the software development process. It measures the software quality throughout and continuously in the software development process, and reveals the current state of software quality, and estimates the follow-up trend of software quality. It provides a powerful means of software quality quantitative management for the demander, developer and evaluator. Evaluators provide powerful means of quantitative software quality management to accurately control the quality of software products [5].

Software quality evaluation is a complex project. Because of the rapid development and updating of software technology, there is no unified evaluation standard at present. In recent years, McCall standard, Boehm standard, FURPS standard, Dromey standard, ISO 9126 standard and ISO/IEC 25010 standard are common software quality evaluation systems. Among them, the ISO/IEC 25010 standard is made progress from ISO 9126 standard. The quality attributes of software products are divided into eight characteristics, each of which is composed of a set of related sub-characteristics. The definition and division of these characteristics are detailed in Table 1.

Table 1. Software quality characteristics and sub-characteristics
Full size table

Because of the particularity of the instrumentation and control system software in nuclear power plants, the software of different security levels must conform to the national and international standards. Especially its security and reliability must strictly comply with the requirements for IEC 60880-2006, IEC 62138-2004, IEC 61513-2011 and NUREG/CR 6101-1993 [6]. So in the process of studying DCS software quality evaluation model, security and reliability must be considered as key factors. In order to explain the process of quality assessment, this paper only establishes the software quality evaluation model of Instrument and Control System according to commercial software.

3 Software Quality Evaluation Model Based on Analytic Hierarchy Process

When the software quality evaluation system is established, the qualitative indicators are basically established. The establishment of quantitative indicators requires a scientific and rational distribution of the weight of each evaluation index. Common weight allocation methods include expert estimation method, analytic hierarchy process, fuzzy inverse equation method, ring ratio method, entropy method, etc. This paper uses analytic hierarchy process to determine the weight of evaluation index.

3.1 Establishing Hierarchical Model

According to the analytic hierarchy process, the ISO/IEC 25010 quality evaluation system standard is divided into three levels: total quality layer A, quality characteristic layer B and sub-feature layer C. Among them, the total quality layer is the final evaluation result; the quality characteristic layer is divided into eight characteristics: functional applicability, performance efficiency, compatibility, easy to use, reliability, security, maintainability and portability; the sub-feature layer includes 31 sub-features in Table 1. The hierarchical model is shown in Fig. 1.

Fig. 1.
figure 1

Hierarchical Structure Model of Software Quality Evaluation

Full size image

3.2 Constructing Judgment Matrix

In determining the weights of factors at different levels, if only the qualitative results, it is often not easy to be accepted by others, so Saaty et al. put forward the consistent matrix method, that is, not to put all factors together, but to compare two factors [7]. At this time, the relative scale is used to reduce the difficulties of comparing many factors of different nature, so as to improve the accuracy. For example, each element of the quality characteristic layer is compared in two ways and graded according to its relative importance. To quantify the results of the importance comparison among the elements, Table 2 lists nine importance levels given by Saaty and their assignments.

Table 2. Scaling and meaning of judgment matrix
Full size table

A matrix consisting of two or two comparisons is called a judgment matrix \({\text{A}} = \left( {a_{{ij}} } \right)_{{k \times {\text{k}}}}\), k is the number of indicators, as Formula 1. The matrix has the following characteristics:

  1. I)

    \(a_{{ij}} > 0\), specifically, if index i is more important than index j, then \(a_{{ij}} > 1\), whereas \(0 < a_{{ij}} < 1\);

  2. II)

    \(a_{{ij}} = 1/a_{{ji}}\), if the relative importance of index i and index j is \(a_{{ij}}\), then the relative importance of index i and index j is \(1/a_{{ij}}\);

  3. III)

    \(a_{{ii}} = 1\), the diagonal line of the matrix is 1.

    $$ {\text{A}} = \left[ {\begin{array}{*{20}c} {a_{{11}} } & {a_{{12}} } & \cdots & {a_{{1k}} } \\ {a_{{21}} } & {a_{{22}} } & \cdots & {a_{{2k}} } \\ \vdots & \vdots & \vdots & \vdots \\ {a_{{k1}} } & {a_{{k2}} } & \cdots & {a_{{kk}} } \\ \end{array} } \right] $$
    (1)

Considering the importance of security and reliability in the operation and maintenance of DCS software, it is necessary to add a restriction condition to construct the judgment matrix:\(a_{{pq}} > 5\), index p \(\subset\) [Security, Reliability], index q \(\subset\) [Functional applicability, Performance efficiency, Compatibility, Easy to use, Maintainability, Portability].

3.3 Computing Weight Vector

The judgment matrix is obtained through comprehensive evaluation and decision-making by professionals and relevant experts according to the set principles. After the judgment matrix is established, the weight vector is calculated. Firstly, the \(P_{i}\) that means the element productof each line in matrix A is calculated:

$$ ~P_{i} = \prod\nolimits_{{j = 1}}^{k} {a_{{ij}} } $$
(2)

\({\text{i}} = 1,2, \cdots ,{\text{k}}\);

Calculate the extraction of a root, \(v_{i}\) is obtained:

$$ v_{i} = \sqrt[k]{{P_{i} }} $$
(3)

\({\text{i}} = 1,2, \cdots ,{\text{k}}\);

By normalizing \(v_{i}\), \(w_{i}\) is obtained:

$$ w_{i} = v_{i} /\sum\nolimits_{{i = 1}}^{k} {v_{i} } $$
(4)

\({\text{i}} = 1,2, \cdots ,{\text{k}}\);

\({\text{W}} = \left[ {w_{1} ,w_{2} , \cdots ,w_{k} } \right]^{T}\) Vector is the matrix weight vector required.

3.4 Checking consistency

Since the judgment matrix is based on the expert's independent judgment and is obtained by comparing the two indexes in the hierarchy, it is necessary to check the consistency of the judgment matrix. If the consistency requirement is not satisfied, the judgment matrix needs to be regenerated. The inspection steps are as follows:

  1. (1)

    Calculate the \({\text{S}}_{{\text{i}}}\) that means the sum of each column of elements in matrix A:

    $$ S_{j} = \sum\nolimits_{{i = 1}}^{k} {a_{{ij}} } $$
    (5)

    \({\text{j}} = 1,2, \cdots ,{\text{k}}\);

  2. (2)

    Calculate the \(\lambda _{{{\text{max}}}}\) that means the maximum eigenvalue of matrix A:

    $$ \lambda _{{max}} = \sum\nolimits_{{i = j = 1}}^{k} {w_{i} S_{j} } $$
    (6)
  3. (3)

    Calculate the Computational consistency test index \({\text{CI}}\):

    $$ {\text{CI}} = (\lambda _{{max}} - k)/\left( {k - 1} \right) $$
    (7)
  4. (4)

    Calculate the Computing Random Consistency Ratio \({\text{CR}}\):

    $$ {\text{CR}} = {\text{CI}}/{\text{RI~}} $$
    (8)

RI is called the mean random consistency index. The specific values are shown in Table 3 [8]. When \({\text{CR}} < 0.1\), then the judgment matrix A has satisfactory consistency; otherwise, thejudgment matrix needs to be modified until it has satisfactory consistency.

Table 3. Average random consistency index number table
Full size table

3.5 Computing Comprehensive Score

According to the above steps, the weight vectors \({\text{W}}^{\prime}\) are calculated for eight indicators, such as functional applicability and compatibility, in the quality characteristic layer B.

$$ {\text{W}}^{\prime} = \left[ {w_{1} ,w_{2} , \cdots ,w_{8} } \right]^{T} $$
(9)

Then, in the sub-feature layer C, the weight vector \(W_{m}\) is calculated for each sub-feature index subordinate to eight indexes, and each sub-feature is scored one by one, and the scoring vector is obtained, as shown in Table 4.

$$ W_{m} = \left[ {w_{{m1}} ,w_{{m2}} , \cdots ,w_{{mq}} } \right]^{T} $$
(10)
$$ G_{m} = \left[ {g_{{m1}} ,g_{{m2}} , \cdots ,g_{{mq}} } \right] $$
(11)

\(\sum\nolimits_{{i = 1}}^{q} {g_{{mi}} } = 100\), \({\text{m}} = 1,2, \cdots ,8\), q is the number of sub-properties under a certain mass characteristic.

Table 4. Weights and scores of indicators in layer B and C
Full size table

So far, we can construct the scoring vector T of eight indicators in layer B, and calculate the final comprehensive score F of quality evaluation.

$$ {\text{T}} = \left[ {t_{1} ,t_{2} , \cdots ,t_{8} } \right] $$
(12)
$$ {\text{F}} = {\text{T}} \cdot {\text{W}}^{\prime} $$
(13)

4 Concluding remarks

In order to evaluate the software quality of DCS system in nuclear power plant scientifically and effectively, a new quantitative evaluation method and model based on analytic hierarchy process (AHP) is proposed according to ISO/IEC 25010 standard. The model constructs a judgment matrix of relative importance for each evaluation index, and assigns two high important values of security and reliability. Through consistency test, the weight vectors of each index are obtained. The lowest indexes are scored by experts one by one, and the weighted sum is made from bottom to top. Finally, the software quality score of DCS system is calculated. The quality evaluation model proposed in this paper draws lessons from the traditional analytic hierarchy process and restricts the relative important value of some evaluation indexes. It conforms to the two high performance standards of DCS system for securityand reliability, and plays a guiding role in the software quality evaluation activities of instrument and control system in nuclear power plants.