1 Introduction

It is well-known that software testing located in the software development process is resource-consuming process. The testing-resource means CPU time, man-month, the number of test cases, and so forth. Generally, software testing consists of the module, integration and system testing. Especially in the module testing, a lot of testing-resource is needed to enhance software reliability of each software module because the software modules are tested independently. Therefore, it is important for software development managers to allocate the testing-resource in the module testing efficiently. The optimal testing-resource allocation problem (Yamada and Ohtera 1990; Nishiwaki et al. 1995; Yamada et al. 1995; Ichimori et al. 2002) is known as one of the interesting problems for software development management. The optimal testing-resource allocation problems discuss how to allocate the testing-resource for each module testing to conduct debugging activities more efficiently. As for most of existing discussions on optimal testing-resource allocation problem, the optimal testing-resource allocation is decided by minimizing the total number of remaining faults in the all software modules under certain constraint of the total testing-resource for conducting the whole module testing.

However, from the point of view of actual software development management, it must be better to consider several evaluation criteria, not only the constraint of the total testing-resource, for deciding the testing-resource allocation in the module testing. As one of the approaches, we discuss optimal testing-resource allocation problems based on the multi-attribute utility theory. The multi-attribute utility theory has been often applied in discussion of software development management. Especially, Kapur et al. (2012) applied the multi-attribute utility theory for developing an optimal software release problem and discussed the estimation procedure for estimating optimal software release time maximizing the utility of the software development manager. We discuss an application of multi-attribute utility theory to the testing-resource allocation problem, which is one of the interesting issues on software development management. Compared with the existing discussion on the optimal testing-resource allocation approach (Nishiwaki et al. 1995; Yamada et al. 1995; Ichimori et al. 2002), our approach gives the another aspects in the software develop management by proposing another optimal problem for testing-resource allocation, such as multi-attribute maximization problem on estimating the optimal testing-resource allocation in the module testing. Further, we show examples of the applications of our approaches by using actual data, in which the multi-attribute utility is formulated from the aspects of the reliability, testing-resource, and testing-cost. And we give some considerations on our results and the importance to develop testing management strategy in the module testing.

2 Section Heading

We introduce an existing approach for optimal testing-resource allocation problem (Nishiwaki et al. 1995; Yamada et al. 1995; Ichimori et al. 2002). In this approach, the software reliability growth process for each module is described by using the testing-effort dependent software reliability growth model (Yamada 2014; Yamada and Ohtera 1990). Let us denote the mean value function of the nonhomogeneous Poisson process (Pham 2000) or the expected number of faults detected up to testing time t by Z(t). The testing-effort dependent software reliability growth model is given as

$$\begin{aligned} Z(t)=a(1-\exp [-r\cdot TE(t)]), \end{aligned}$$
(1)

which is the mean value function of the non-homogeneous Poisson process. In Eq. (1), TE(t) is the total testing-effort expended up to time t, a is the initial faults content, and r is the software fault detection rate per expended testing-effort. Regarding the function TE(t) in Eq. (1), we have

$$\begin{aligned} TE(t)=\int ^{t}_{0} te(t)\mathrm{d}t, \end{aligned}$$
(2)

where te(t) is the testing-effort expenditure expended at testing-time t. And the expected number of remaining faults, n(t), is given as

$$\begin{aligned} n(t) = a-Z(t) = a\cdot \exp [-r\cdot TE(t)]. \end{aligned}$$
(3)

The optimal testing-recourse allocation problem is discussed within the following situation: (1) a software system consists of M independent software modules, (2) the number of remaining faults in each module can be estimated by the testing-effort dependent software reliability growth model, (3) the software development managers have to allocate the testing-resource expenditures to each software module testing efficiently, so that the total number of remaining faults in the software system may be minimized.

Let \(d_{i}\) denote the amount of testing-resource expenditure spent for testing software module, i \((i=1, 2, \cdots , M)\). From Eq. (3), the expected number of remaining faults in the software module i, which is denoted by \(n_{i}\) is given as

$$\begin{aligned} n_{i}=a_{i}\cdot \exp [-r_{i}d_{i}], \end{aligned}$$
(4)

where \(a_{i}\) is the initial fault content for the software module i and \(r_{i}\) represents the fault detection rate per expended testing-resource for the software module i \((0<r_{i}<1)\). Then, the total expected number of remaining faults in the software system is

$$\begin{aligned} N = \sum _{i=1}^{M} n_{i}. \end{aligned}$$
(5)

Then, the software testing-resource allocation problem is formulated as

$$\begin{aligned} \left. \begin{array}{l} \displaystyle {\text{ min: } \sum _{i=1}^{M} \delta _{i} a_{i} \cdot \exp [-r_{i}d_{i}]} \\ \displaystyle {\text{ subject } \text{ to } \quad \sum _{i=1}^{M}d_{i}\le W_{P}, \quad d_{i} \ge 0}, \end{array} \right\} , \end{aligned}$$
(6)

where \(W_{P}\) is the planned total amount of the testing-resource and \(\delta _{i}\) is the weight representing the importance or complexity of the software module i. We should note that the optimal testing-resource allocation problem in Eq. (6) is defined as the minimization problem on the number of remaining faults in the whole software modules under the constraint of the planned total amount of the testing-resource. For solving the Eq. (6), the Lagrange multipliers method is generally applied. Then, we have

$$\begin{aligned} L = \sum _{i=1}^{M} \delta _{i} a_{i} \cdot \exp [-r_{i}d_{i}] + \lambda \left( \sum _{i=1}^{M} d_{i} - W_{P} \right) , \end{aligned}$$
(7)

where \(\lambda \) is the Lagrange multiplier. The necessary and sufficient conditions for obtaining the optimal solutions are

$$\begin{aligned} \left. \begin{array}{l} \displaystyle { \frac{\partial L}{\partial d_{i}} = -\delta _{i} a_{i} r_{i} \cdot \exp [-r_{i}d_{i}] + \lambda } = 0 \\ \displaystyle { \frac{\partial L}{\partial \lambda } = \sum _{i=1}^{M} d_{i} - W_{P} \ge 0} \\ \displaystyle { \lambda \left( \sum _{i=1}^{M} d_{i} - W_{P} \right) = 0 }, \qquad \displaystyle {\lambda \ge 0} \\ \end{array} \right\} . \end{aligned}$$
(8)

Further, we assume \(A_{1} \ge A_{2} \ge \cdots \ge A_{k-1} \ge \lambda \ge A_{k} \ge A_{k+1} \ge \cdots \ge A_{M}\), where \(A_{i}=\delta _{i} a_{i} r_{i}\) \((i = 1, ,2, \cdots , M)\). Then, the optimal testing-resource allocation, \(d_{i}^{*}\), is derived as

$$\begin{aligned} \left. \begin{array}{l} \displaystyle {d_{i}^{*} = - \frac{1}{r_{i}} (\ln A_{i} - \ln \lambda ) \qquad (i= 1, 2, \ldots , k-1),}\\ \displaystyle {d_{i}^{*} = 0 \qquad (i = k, k+1, \ldots , M)} \end{array} \right\} , \end{aligned}$$
(9)

where \(\ln \lambda \) is given as

$$\begin{aligned} \ln \lambda = \frac{\sum _{i=1}^{M} \frac{1}{r_{i}} \ln A_{i} -W_{P}}{\sum _{i=1}^{M}\frac{1}{r_{i}}}. \end{aligned}$$
(10)

Consequently, the optimal testing-resource allocation is obtained as

$$\begin{aligned} d_{i}^{*} = \text{ max }\left\{ 0, -\frac{1}{r_{i}} (\ln A_{i}-\ln \lambda ) \right\} \qquad (i=1, 2, \ldots , M). \end{aligned}$$
(11)

3 Proposed Approach

We propose another approaches for estimating optimal testing-resource allocation with multi-attribute utility of software development manager. Now, we consider the following situation:

  1. (1)

    A software system consists of M independent software modules.

  2. (2)

    The number of remaining faults for each software module are estimated by the testing-effort dependent software reliability growth model.

  3. (3)

    The software development managers allocate the testing-resource to testing for each software module for maximizing their utility, which consists of several attributes being related to the software development management.

We now consider that the following attributes: the testing-resource and the software reliability attributes, respectively. Regarding the testing-resource attribute, we formulate

$$\begin{aligned} \text{ min }: E = \frac{\sum _{i=1}^{M}d_{i}}{W_{P}}=\frac{W}{W_{P}}, \end{aligned}$$
(12)

since the software managers prefer to spend the testing-resource less than the planned total amount of testing-resource. In Eq. (12), W is the testing-resource expenditure spent up to the end of the module testing. Further, we give the software reliability attribute as

$$\begin{aligned} \text{ max }: R = 1 - \frac{\sum _{i=1}^{M} a_{i}\cdot \exp [-r_{i}d_{i}]}{\sum _{i=1}^{M} a_{i}} = 1 - \frac{N}{\sum _{i=1}^{M} a_{i}}. \end{aligned}$$
(13)

Here we develop the utility function for each attribute based on the following certain situation on the software development management strategy: (1) for the testing-resource attribute, at least 60\(\%\) of the planned total amount of testing-resources must be consumed, (2) for the software reliability attribute, at least 80\(\%\) of software faults should be removed, (3) the software development managers take the risk neutral position for each attribute. Then, we set the lowest and highest consumptions for the testing-resource attribute are \(E^{L}=0.6\) and \(E^{H}=1.0\), respectively. And the lowest and highest requirements for the software reliability attribute are \(R^{L}=0.8\) and \(R^{H}=1.0\), respectively. Then, we obtain the following utility functions for each attribute based on the notion of the risk neutral position:

$$\begin{aligned} \left. \begin{array}{l} u(E) = 2.5E - 1.5 \\ u(R) = 5R - 4 \end{array} \right\} . \end{aligned}$$
(14)

From Eq. (14), we define the following optimal testing-resource allocation problem with the additive multi-attribute utility function under the testing management strategy:

$$\begin{aligned} \left. \begin{array}{ll} \text{ max }: u(E, R) &{}= \delta _{R}\cdot u(R) - \delta _{E} \cdot u(E)\\ &{} = \delta _{R} \cdot (5R-4) - \delta _{E} \cdot (2.5E - 1.5) \\ \text{ subject } \text{ to }&{} \delta _{R} + \delta _{E} = 1, \quad d_{i}\ge 0 \end{array} \right\} , \end{aligned}$$
(15)

where \(\delta _{R}\) and \(\delta _{E}\) are the weight parameters for the attributes R and E, respectively. Consequently, we obtain the optimal testing-resource allocation, \(d_{i}^{*}\) \((i = 1, 2, \ldots , M)\), by maximizing the multi-attribute utility function in Eq. (15). From Eq. (15), the optimal testing-resource allocation, \(d_{i}^{*}\), is obtained by

$$\begin{aligned} \left. \begin{array}{ll} d_{i}^{*} &{} = \displaystyle { -\frac{1}{r_{i}} \ln \frac{2.5 \delta _{E} \sum _{i=1}^{M} a_{i}}{5 \delta _{R} a_{i} r_{i} W_{P}} } \qquad (i = 1, 2, \cdots , k-1) \\ d_{i}^{*} &{} = 0 \qquad (i = k, k+1, \ldots , M) \end{array} \right\} , \end{aligned}$$
(16)

Then, we have

$$\begin{aligned} d_{i}^{*} = \text{ max } \left\{ 0, -\frac{1}{r_{i}} \ln \frac{2.5 \delta _{E} \sum _{i=1}^{M} a_{i}}{5 \delta _{R} a_{i} r_{i} W_{P}} \right\} \qquad (i = 1, 2, \cdots , M). \end{aligned}$$
(17)

In Eq. (17), we should note that

$$\begin{aligned} B_{1} \ge B_{2} \ge \cdots \ge B_{k-1} \ge \frac{2.5 \delta _{E} \sum _{i=1}^{M} a_{i}}{5 \delta _{R} W_{P}} \ge B_{k} \ge B_{k+1} \ge \cdots \ge B_{M}, \end{aligned}$$
(18)

where \(B_{i} = a_{i} r_{i}\) \((i=1, 2, \ldots , M)\).

Furthermore, we add the cost attribute for treating more practical situation. For formulating the cost attribute, we set the following cost parameters:

  • \(c_{1}\): the debugging cost per fault discovered in the module testing,

  • \(c_{2}\): the debugging cost per fault undiscovered during the module testing (\(c_{1}<c_{2}\))

  • \(c_{3}\): the cost per unit of the testing-resource for the module testing.

Then, the cost function is given as

$$\begin{aligned} V = c_{1} \sum _{i=1}^{M} a_{i} (1-\exp [-r_{i}d_{i}]) + c_{2} \sum _{i=1}^{M} a_{i} \exp [-r_{i}d_{i}] + c_{3} \sum _{i=1}^{M} d_{i}, \end{aligned}$$
(19)

by following the notion of the testing-effort dependent software reliability growth model. From Eq. (19), the cost attribute is given as

$$\begin{aligned} \text{ min: }\quad C=\frac{V}{C_{P}}, \end{aligned}$$
(20)

where \(C_{P}\) represents the planned budget for software testing. And we add the following test management strategy: (4) for the cost attribute, at least 50\(\%\) of the budget must be consumed in the module testing. The lowest and highest consumptions for the cost attribute are \(C^{L}=0.5\) and \(C^{H}=1.0\). Further, we assume the following utility function on the cost attribute: \(u(C)=2C-1\), which is just one of the examples.

When we consider the three attributes, such as the testing-resource, software reliability, and cost attributes, the optimal testing-resource allocation problem with the three attributes is defined as

$$\begin{aligned} \left. \begin{array}{ll} \text{ max }: u(E, R, C) &{}= \delta _{R}\cdot u(R) - \delta _{E} \cdot u(E) - \delta _{C} \cdot u(C)\\ &{} = \delta _{R} \cdot (5R-4) - \delta _{E} \cdot (2.5E - 1.5) - \delta _{C}\cdot (2C-1)\\ \text{ subject } \text{ to }&{} \delta _{R} + \delta _{E} + \delta _{C} = 1, \quad d_{i}\ge 0 \end{array} \right\} , \end{aligned}$$
(21)

where \(\delta _{C}\) is the weight parameter for the cost attribute. From Eq. (21), the optimal testing-resource allocation, \(d_{i}^{*}\), is obtained by

$$\begin{aligned} \left. \begin{array}{ll} d_{i}^{*} &{}= \displaystyle { -\frac{1}{r_{i}} \ln \frac{ \frac{2.5\delta _{E}}{W_{P}} + \frac{ 2\delta _{C} c_{3}}{C_{P}} }{a_{i}r_{i} \left\{ \frac{5\delta _{R}}{\sum _{i=1}^{M}a_{i}} + \frac{2(c_{2}-c_{1})\delta _{C}}{C_{P}} \right\} }} \qquad (i = 1, 2, \ldots , k-1) \\ d_{i}^{*} &{}= 0 \qquad (i = k, k+1, \ldots , M) \end{array} \right\} . \end{aligned}$$
(22)

Then, we have

$$\begin{aligned} d_{i}^{*} = \text{ max } \left\{ 0, \displaystyle { -\frac{1}{r_{i}} \ln \frac{ \frac{2.5\delta _{E}}{W_{P}} + \frac{ 2\delta _{C} c_{3}}{C_{P}} }{a_{i}r_{i} \left\{ \frac{5\delta _{R}}{\sum _{i=1}^{M}a_{i}} + \frac{2(c_{2}-c_{1})\delta _{C}}{C_{P}} \right\} }} \right\} \qquad (i = 1, 2, \ldots , M). \end{aligned}$$
(23)

In Eq. (23), we should note

$$\begin{aligned} C_{1} \ge C_{2} \ge \cdots \ge C_{k-1} \ge \displaystyle { \frac{ \frac{2.5\delta _{E}}{W_{P}} + \frac{ 2\delta _{C} c_{3}}{C_{P}} }{ \frac{5\delta _{R}}{\sum _{i=1}^{M}a_{i}} + \frac{2(c_{2}-c_{1})\delta _{C}}{C_{P}} }} \ge C_{k} \ge C_{k+1} \ge \cdots \ge C_{M}. \end{aligned}$$
(24)

4 Numerical Examples

We show numerical examples for our proposed approach by using actual data Yamada and Ohtera (1990). The data consists of 10 modules and 251 faults still remain through the module testing. Table 1 shows the estimated values of \(a_{i}\) and \(r_{i}\), which have been estimated by following the testing-effort dependent software reliability growth model. “M” means module, then “M1” means the module 1. We set the cost parameters as \(c_{1}=1\), \(c_{2}=2\), and \(c_{3}=5\). And we also set the planned total amount of testing-resource and the budget for the module testing as \(W_{P}=1.0\times 10^{6}\) and \(C_{P}=1.0\times 10^{6}\), respectively.

Table 1 Estimated expected number of remaining faults (2 attributes)
Full size table
Table 2 Estimated optimal testing-resource allocation (2 attributes)
Full size table
Table 3 Estimated optimal testing-resource allocation (3 attributes)
Full size table
Table 4 Estimated expected number of remaining faults (3 attributes)
Full size table

Tables 2 and 3 show the estimated optimal testing-resource allocation for each module, total amount of optimal testing-resource, and utility along with the several weight patterns for the 2 and 3 attributes, respectively. In Table 2, the optimal total amount of testing-resource increases as the \(\delta _{R}\) is increasing and the \(\delta _{E}\) is decreasing. And we can say that the utility is ordered as \(\text{ P1 }>\text{ P5 }>\text{ P2 }>\text{ P4 }>\text{ P3 }\), i.e., the utility is getting higher as the difference between \(\delta _{R}\) and \(\delta _{E}\) is increased. In Table 3, the optimal total amount of testing-resource is increased as \(\delta _{R}\) is increasing and the \(\delta _{R}\) and \(\delta _{E}\) are decreasing. The order on the utility is \(\text{ P6 }>\text{ P1 }>\text{ P5 }>\text{ P2 }>\text{ P3 }>\text{ P4 }\). From Table 3, we can see the differences among the weights must influence the utility. That is, we can say the testing strategy for the module testing influences on the value of the utility. Further, Tables 1 and 4 show the estimated expected number of remaining faults based on the estimated optimal testing allocation for each module for the cases of 2 and 3 attributes, respectively. Focusing on the column of P5 in Table 1, the number of remaining faults can be reduced to 1.5933 from 251 faults due to the testing-strategy, in which the software development manager set the higher weight for the software reliability attributes. We can obtain the same investigation in Table 4.

5 Concluding Remarks

We discussed approaches for estimating optimal testing-resource allocation based on the multi-attribute utility theory in the module testing of software system. Concretely, applying the software reliability estimated the testing-effort dependent software reliability growth model, testing-resource, and testing-cost attributes for developing the additive linear form multi-attribute function, we formulated a testing-resource allocation problem, which enables us to estimate the optimal testing-resource allocation maximizing the utility of the software development manager under the certain testing-strategy. Further, we showed examples of application of our approaches by using actual data. Then, we discussed the behavior of the utility, optimal testing-resource allocation, and the number of remaining faults, which depend on the weight parameters, such as \(\delta _{R}\), \(\delta _{E}\) and \(\delta _{C}\). From the examples of application of our approaches, we investigated that the developing the certain strategy for conducting the differentiated injections to the software reliability, testing-resource, and testing-cost leads to obtaining the higher utility of software developing manager. However, we need to conduct more investigations for our approaches and to figure out the relationship between the testing-strategy and utility by using other actual data and module testing situation.