Software Reliability Models and Multi-attribute Utility Function Based Strategic Decision for Release Time Optimization

Abstract

The software industry is working hard to keep up with these rapid changes by devising methods to increase the pace of their work without compromising software quality and reliability. Various factors, such as the testing environment, testing strategy, and resource allocation, can influence the optimal release time. The choice of whether or not to release a software product would become much more complicated and significant. When a software developer, clients, or end-users face significant potential financial losses, a decision has strategic significance. A software release decision is a trade-off between early release to take advantage of an earlier market launch and product release deferral to ensure reliability. If a software product is released too soon, the software developer must pay for post-release costs to correct bugs. To decide the best software release time, two attributes, reliability and cost, must be combined. This study discusses a realistic approach to determining when to stop software testing that considers reliability and cost. A multi-attribute utility theory-based proposed decision model is analyzed on various separate weighted combinations of utility functions.

1 Introduction

Software technologies are the most prevalent human-made technology that impacts our daily lives due to the importance of software applications in recent years. In the last two decades, the penetration of software-based technologies into people’s everyday lives has been remarkable. Everything we see around us is dependent on software or has some connection to software systems. There is a requirement for highly dependable, secure, and high-quality software development since our social structure has become increasingly dependent on software-based technologies [31]. Reliability can only be accomplished by thoroughly testing the program before it is made available to the public. Program errors are found, identified, and fixed throughout the testing process, improving software reliability [13]. Reliability is an essential statistic for evaluating commercial software quality in the testing and operational phases. Software reliability may be defined as the likelihood of error-free software execution in a particular environment over a predetermined duration [17, 23]. Non-homogeneous Poisson process (NHPP) based growth models are frequently used in software systems to describe stochastic failure behavior and measure growth reliability [3, 5, 7, 9, 33].

NHPP models have also been extensively used in the cost-control analysis, software time-to-market analysis, and resource allocation issues [1, 10, 12, 26, 32]. The correctness and security of a software system can only be improved with sufficient testing time and effort, such as CPU hours and qualified testing specialists [10, 29]. In general, software testing uses around half of the resources for software development. Continuous software testing for a more extended period may obstruct the timely delivery of the software system. Furthermore, it will quickly result in significant development expenses. Simultaneously, shorter testing combined with an insufficient debugging procedure would cause customer disappointment, potentially affecting the growth of the software as well as the software firm’s goodwill. In today’s market, a software testing budget should be prioritized over its development budget [11, 20]. As a result, software reliability engineering provides a cost-effective compromise between client needs for dependability, accessibility, delivery time, and life cycle [16, 19]. SRGMs are used to optimize testing techniques for increased organizational competitiveness, estimate the amount of required resources, and calculate the overall cost of the development process [22, 34, 39, 40].

The software reliability may be predicted using appropriate software reliability growth models (SRGMs) based on the fault count data obtained during the testing process [21, 36]. The testing phase is the most significant since it is at this step that the fault detection and removal procedure takes place, which is critical for the dependability and quality of any software system. A critical decision point for management is when to end testing and release the software system to the user [30]. This is referred to as the “Software Release Time Problem”. Before being released, the software is subjected to a rigorous testing procedure in order to identify flaws that might have devastating effects if not corrected. Several methods of software testing are now in use with the goal of eliminating faults. It’s possible that many bugs went undiscovered because of the short testing time and the sudden release [37]. The choice to release software is a complicated one, and there are significant dangers involved with a release agreement that is either too rapid or too delayed [18, 24]. One of the most common applications of SRGMs is to assist developers in determining the optimal timing to deploy software [2, 6, 22, 35].

The main contribution of this work is as follows:

• 1. Proposed new SRGMs with log-logistic and Burr Type XII distribution as a fault detection rate.

• 2. This study suggests the multi-attribute utility theory based optimal release time.

2 Software Reliability Modeling

The NHPP based SRGM is used in this work. The NHPP is a method of calculating the total number of faults found throughout the testing procedure. In this technique, SRGMs such as exponential [8], delayed S-Shaped [41], inflected delayed S-shaped [27], and power function have been used to anticipate potential bugs laying latent in the program. Let N(t) be the total number of defects discovered at time t, and m(t) be the expected number of faults. The failure intensity $\lambda (t)$ is therefore linked as follows:

$$\begin{aligned} m(t) = E[N(t)] = \int _{0}^{t}\lambda (s)ds \;, \end{aligned}$$

(1)

where N(t) has a Poisson probability mass function with parameter m(t), which is as follows:

$$\begin{aligned} Pr\{N(t)=n\} = \dfrac{m(t)^{n}.e^{-m(t)}}{n!}\;, \quad x=0,1,2,... \end{aligned}$$

(2)

Various time-dependent models that describe the stochastic failure process of an NHPP have been published in the literature. The failure intensity function $\lambda (t)$ differs across these models, and therefore m(t) . In the case of finite failure NHPP models, let “$\varLambda$” indicate the estimated total number of faults that would be identified given infinite testing time.

One of the main goals of testing is to identify software faults to fix them. Once the software code has been written, testing can begin. Before the software is released to the public, the software testing team thoroughly tests it to ensure that the software contains the least number of bugs. Despite the fact is that it is almost impossible to eliminate all the software bugs. As a result, when the testing team tests the software, there’s a probability they’ll only find a finite number of problems in the code (less than the total number of faults).

2.1 Assumption

• i. NHPP models the failure observation/fault removal phenomenon.

• ii. The software system is susceptible to failure at any time due to errors that have remained in the system.

• iii. There are a finite number of bugs present in the software.

• iv. When a failure occurs, it is instantly removed.

• v. The severity level of all faults is the same.

• vi. The perfect debugging environment is taken into account.

• vii. All remaining software faults have an equal impact on the failure rate.

• viii. The number of defects discovered throughout the testing process is directly proportional to the number of faults still present in the software.

• ix. With a probability distribution function, each occurrence of failure is distributed independently and identically across the software life-cycle.

As a result, finite numbers of bugs are perfectly eradicated, with the mathematical equation. The finite failure NHPP models’ differential equation formulated based on the modeling assumption and it expressed as:

$$\begin{aligned} \dfrac{dm(t)}{dt} = r(t)[\Lambda -m(t)] \end{aligned}$$

(3)

When Eq. (3) is solved for the initial condition $m(0)= 0$, the MVF can be calculated as follows:

$$\begin{aligned} m(t) = \varLambda [1-e^{-\int _{0}^{t}r(v)dv}] = \varLambda [1-e^{-B(t)}] \end{aligned}$$

(4)

$$\begin{aligned} m(t) = \varLambda .F(t)\;. \end{aligned}$$

(5)

where F(t) is a distribution function.

Various researches assume that fault detection is constant throughout the testing process, but it is not possible in practical behavior. For the detection rate, we know that it is low at the initial stage, and in the mid-stage, it’s on the peak; in the later stage, it’s again low. So, the FDR is modeled through the specific distribution handling the situation. Therefore, this study proposed the SRGMs with the two most applicable distribution functions for B(t), i.e., Log-logistic and Burr type XII distribution functions.

2.2 Fault Detection Rate

2.2.1 Log-Logistic Distribution

The logistic distribution and the log-logistic distribution are closely linked. A probability distribution whose logarithm has a logistic distribution is known as a log-logistic distribution. Log(x) is distributed logistically with mean and standard deviation if x is distributed loglogistically with parameters $\mu$ and $\sigma$. The log-logistic distribution is a good replacement for the Weibull distribution. It’s a hybrid of the Gompertz and Gamma distributions, with the mean and variance values equal to one. The log-logistic distribution has its own status as a life testing model; it is an increasing failure rate model as well as a weighted exponential distribution. The generalized log-logistic distribution refers to several distinct distributions that include the log-logistic as a particular instance. The Burr Type XII distribution and the Dagum distribution, both of which have a second shape parameter, are examples. It’s a flexible distribution family that may represent a wide range of distribution types that are shown in Fig. 1. In survival analysis, this distribution is frequently used to simulate events that have an initial rate increase followed by a rate decrease.

Fig. 1

Log-logistic distributions with effect of a shape and b scale parameter values

The log-logistic distribution with positive scale parameter $\gamma$ and shape parameter $\alpha$ is described as follows:

$$\begin{aligned} B(t) = \dfrac{(\frac{t}{\gamma })^{\alpha }}{1+(\frac{t}{\gamma })^{\alpha }}\;, \end{aligned}$$

(6)

and the density function is:

$$\begin{aligned} b(t) = \frac{(\frac{\alpha }{\gamma })(\frac{t}{\gamma })^{\alpha -1}}{[1+(\frac{t}{\gamma })^{\alpha }]^{2}} \end{aligned}$$

(7)

2.2.2 Burr Type XII Distribution

The Burr distribution can fit a wide range of empirical data. The parameters’ various values span a wide range of skewness and kurtosis. As a result, it is used to represent a range of data types in diverse disciplines such as finance, hydrology, and reliability. The Burr type XII distribution generalizes Burr distribution with additional scale parameters. It is a three-parameter family of positive real-line distributions. It’s a versatile distribution family that may represent a variety of different distribution forms that are shown in Fig. 2. Many widely used distributions, such as gamma, log-normal, log-logistic, bell-shaped, and J-shaped beta distributions, are included, overlapped, or have the Burr distribution as a limiting case (but not U-shaped). The Burr distribution is also found in several compound distributions. A Burr distribution is created by compounding a Weibull distribution with a gamma distribution for the scale parameter. There are two asymptotic limiting cases for the Burr distribution: Weibull and Pareto Type I.

Fig. 2

Burr Type XII distributions with effect of a shape, b shape and c scale parameters

The Burr distribution’s cumulative distribution function (cdf) is:

$$\begin{aligned} B(t) = 1- \dfrac{1}{\left[ 1+\left( \frac{t}{\gamma }\right) ^{\alpha }\right] ^{\beta }}\;, \end{aligned}$$

(8)

$$\begin{aligned} b(t) = \dfrac{\frac{\alpha \beta }{\gamma }(\frac{t}{\gamma })^{\alpha -1}}{[1-(\frac{t}{\gamma })^{\alpha }]^{\beta +1}} \end{aligned}$$

(9)

2.3 Software Reliability Growth Models

$$\begin{aligned} m(t) = \varLambda [1-e^{-B(t)}] \end{aligned}$$

(10)

2.3.1 Model-1

The MVF of Log-logistic FDR based SRGM is defined as:

$$\begin{aligned} m(t) = \varLambda \left[ 1-e^{-\dfrac{(\frac{t}{\gamma })^{\alpha }}{1+(\frac{t}{\gamma })^{\alpha }}}\right] \;. \end{aligned}$$

(11)

2.3.2 Model-2

The MVF of Burr type XII FDR based SRGM is defined as:

$$\begin{aligned} m(t) = \varLambda \left[ 1-e^{-\left( 1- \dfrac{1}{\left[ 1+\left( \frac{t}{\gamma }\right) ^{\alpha }\right] ^{\beta }}\right) }\right] \;. \end{aligned}$$

(12)

One of the most common applications of SRGMs is to assist developers in determining the optimal timing to deploy software. This study formulated a cost model to estimate the best software release timing in the latter portion of this paper. This field of study is strongly connected to the wider software reliability research.

3 Numerical Illustration

The practical applicability of the suggested problem is demonstrated in this section using historical fault discovery data as an example. The fault count data set was used for the numerical illustration. The non-linear least square estimation (LSE) method is used to estimate model parameters. The estimated model parameter findings for detected faults throughout the testing period are shown in Table 1.

Table 1 Datasets from the existing literature

Table 2 Estimated values of SRGMs parameters for all six datasets

Table 3 Performance comparison of SRGMs for all six datasets

The behavior of actual defects data for software release is observed in the graph and most of them are in S-shaped form. This is further supported by the usage of the log-logistic and Burr Type XII FDR function to detect software faults. As evidenced by the values of several comparison criteria, model-1 and model-2 provide a perfect fit. Table 2 present the estimated values of proposed and existing models for DS-1 to DS-6. Table 3 present a comparative analysis of the proposed and existing models. The comparison criteria used here are the sum of square error (SSE), coefficient of determination $(R^{2})$ and Adjusted R-square $(R^{2}_{adj})$ Fig. 3 illustrate a graphical representation of estimated vs. real cumulative failures over time for a better understanding. Based on these findings, we can conclude that the proposed model-2 produces good performance and is more realistic when it comes to forecasting the growth behavior of application-based software systems.

Fig. 3

a–f The fitting results of SRGMs comparison with actual failure data for DS-1-DS-6

In the next section we discuss about the optimal release policy.

4 Optimal Release Policy

With increasing competition in the software industry, continually changing client expectations, and the usual challenges involved with software maintenance, the timing of a new software release has become increasingly critical for a software vendor’s success in the market [15]. Given the fierce competition in the market, deploying software on time has become a vital aspect in deciding the software development team’s success. The dynamic release problem in software testing processes is discussed in this work [4]. The process of choosing between alternative courses of action in order to achieve goals and objectives is known as decision-making. Software release time, for example, estimating when it should be completed. Other managerial functions rely substantially on decision-making, such as organizing, implementing, and controlling [14].

If the testing period is extended in the software development process, the developed software will presumably be more reliable, but the testing cost will escalate. If we end testing too soon, the program may have too many flaws, resulting in too many failures during operation and significant losses owing to failure penalties or customer discontent. We may incur a considerable testing expense if we spend too much time testing. If the testing period is too short, the software may not be error-free. As a result, software testing and release are mutually exclusive. The testing procedure should determine the release timing dynamically. As a result, our goal is to come up with an appropriate release policy that reduces the cost and time of software testing while increasing the system’s reliability. The ideal release time based on the cost-reliability criterion has been described and evaluated.

4.1 Cost and Reliability Modeling

4.1.1 Cost Modeling

• 1. Testing cost per unit testing time: The effort necessary to perform and execute the testing procedure is included in the testing cost. The cost of testing rises linearly with the time of the test. If $C_{1}$ is the testing cost per unit time, then the total testing cost is as follows:

  $$\begin{aligned} C_{TCPU} = C_{1}.T \;. \end{aligned}$$

  (13)

• 2. Debugging cost during testing-phase: This cost includes the testing team’s effort to handle failures. The expected number of bugs identified during this time is assumed linearly in software reliability literature. So, in the testing phase, the error-debugging cost is:

  $$\begin{aligned} C_{DCDT} = C_{2}.m(T) \;. \end{aligned}$$

  (14)

• 3. Debugging cost during operational-field: In the operational phase, it is believed that Debugging cost during operational-field $C_{2}(T)$ is proportional to the number of software faults that were removed. Thus,

  $$\begin{aligned} C_{DCDO} = C_{3}.[m(T_{LC})-m(T)] \;. \end{aligned}$$

  (15)

  Because $C_{2}$ represents the deterministic cost to remove each fault per unit time during testing, and $C_{3}(T)$ represents the cost of eliminating a fault during the operational phase, $C_{3}$ is typically more than $C_{2}$, i.e., $C_{3}>C_{2}$.

They presented a three-part software cost model structure: testing cost per unit time, debugging cost in the testing phase, and debugging cost in the operational phase. The mathematical version of the overall cost model is:

$$\begin{aligned} C(T)= & {} C_{DCDT} + C_{DCDT} + C_{DCDO} \;. \end{aligned}$$

(16)

$$\begin{aligned}= & {} C_{1} T + C_{2} m(T) + C_{3} [m(T_{LC})-m(T)] \;. \end{aligned}$$

(17)

4.1.2 Reliability Modeling

$$\begin{aligned} R(\Delta t/ T) = e^{-[m(T+\Delta t)-m(T)]} \end{aligned}$$

(18)

Cost and reliability analysis with time is shown in Fig. 4.

Fig. 4

Cost versus testing time and reliability vs testing time for DS-2 and DS-5

4.1.3 Release Time Problem Using MAUT

When a sequence of possibilities is presented, the goal is to obtain a conjoint measure indicating how desirable one conclusion is in comparison to the others. It is a classical multi-objective optimization technique that addresses the optimization problem by applying weights and utility functions to determine which objectives should be prioritized [30]. The following is the formula for the multi-attribute utility function (MAUF), a weighted sum of single utility functions. It is defined as follows:

$$\begin{aligned} U(x_{1}, x_{2}, ... x_{n}) = f(u(x_{1}), u(x_{2}), ... u(x_{n})) = \sum _{i=1}^{n} \theta _{i} u_{i}(x_{i}) \end{aligned}$$

(19)

This work uses MAUT to construct a new decision model for software release schedule determination that trades off two conflicting objectives at the same time.

The process of determining the utility value consists of four steps.

• 1. Selection of Attributes.

• 2. Evaluate the utility function for a single attribute.

• 3. Allocation of credit and preference for trade-offs.

• 4. Single attribute to multi attribute utility function transformation.

• 1. Attribute selection Reliability is a necessary attribute that influences optimal software time-to-market and testing length selections. As a result, the proposed optimization problem’s first attribute is reliability (R) . The second attribute is overall software development cost (C) , because no company wants to spend more than it can afford. We take the R and C as two attributes in this study. Our initial goal is to strike a compromise between these two goals by maximizing reliability while minimizing total software development costs:

  $$\begin{aligned} max: \quad R(T) = e^{-[m(T+\Delta t)-m(T)]} \;, \end{aligned}$$

  (20)
  $$\begin{aligned} min: \quad C(T) = \left[ \dfrac{C(T)}{C_{b}}\right] , \quad \dfrac{C(T)}{C_{b}} \le 1\;. \end{aligned}$$

  (21)

  The total budget available to the testing team is denoted by $C_{b}$.

• 2. Single attribute utility function

  Each attribute’s aim is represented by a utility function applied to each attribute. The single-attribute utility theory (SAUF) expresses the level of satisfaction of management concerning each of the attributes. There are many different functional forms of the utility function, such as linear, exponential, and so on. The utility function of two qualities, namely, reliability and cost function, is used in this study. The linear (additive) form $u(x) = y_{1}+y_{2}x$ should be employed if they are equivalent to each other because management is risk-neutral. The proposed framework is illustrated as follows:

  $$\begin{aligned} u(R) = l_{r} R(T) + k_{r} \;, \end{aligned}$$

  (22)
  $$\begin{aligned} u(C) = l_{c} C(T) +k_{c} \;. \end{aligned}$$

  (23)

  where, $k_{r},$ $l_{r}$, $k_{c}$, $l_{c}$ are constants.

• 3. Weight parameter estimation

  The management decision determines the relative value of each attribute. In this study, we perform various weight combinations values for each attribute. The weight parameter has a value between 0 and 1, with a value closer to 1 denoting greater significance. Furthermore, the sum of the weight parameters must equal 1, i.e.,

  $$\begin{aligned} w_{r}+w_{c}=1 \;. \end{aligned}$$

  (24)

  where $w_{r}$ and $w_{c}$ are weight for the reliability and cost respectively.

• 4. Formulation of MAUT

  The MAUT function is created by multiplying all of the single utility functions by their corresponding weights. The MAUT function with the maximizing objective for the given problem is:

  $$\begin{aligned} Max: \quad U(R, C) = w_{r}.u(R)-w_{c}.u(C) \;. \end{aligned}$$

  (25)

  where

  $$\begin{aligned} u(R) = 2R - 1\;, \end{aligned}$$

  (26)
  $$\begin{aligned} u(C) = 2C - 1 \;. \end{aligned}$$

  (27)

  U(R, C) is a max function that has been written in terms of R and C. From the manager’s perspective, R should be maximized while C should be minimized.

  where, $T_{LC} = 1000$, $C_{1}= 100$, $C_{2} = 10$, $C_{2} = 50$. For DS-2, $C_{b} = 8500 \$$, $\varDelta t = 0.025$ and for DS-5, $C_{b} = 8500 \$$, $\varDelta t = 0.4$. With the different combination of weights to reliability and cost based optimal release time is shown in Table 4.

Table 4 Optimal release time by MAUT for DS-2 and DS-5

5 Conclusions

It is also possible to optimize software release and testing times by maximizing utility. The results show that a corporation should publish software early to achieve a competitive edge. The solution to the problem can also assist software firms design efficient release and testing procedures. In this work, we propose an effort-based optimum decision model that takes into account the cost of detection during testing and operational phases separately using MAUT. SRGMs provide a statistical foundation for determining optimal software testing release time. A decision model based on MAUT is suggested to make wise decisions on optimal test runs before software release. This study optimizes cost and reliability using multi-attribute utility theory and gets optimal release time. These models may help the software industry anticipate software system dependability and release time.