1 Introduction

Software product has been rapidly expanding to a wide array of various real-industry and service-based application. The increasing dependence of our society on software-driven system has led the development of software product becomes very competitive and time-consuming. As the software development moves further away from the rigid and monolithic model, the importance of software multiple release is brought to the vanguard (Saliu and Ruhe 2005). Most of the software organizations release the initial version with sufficient functionalities to meet the customer requirements and occupy a certain portion of market share at first. However, it is unlikely to deliver all features that customers wanted in the single release because of the limited budget, unavailable resource, estimated risk and constrained schedules.

Staying competitive in the market and keep profitable for a software product unlikely happen in this increasing-innovational society if only has a single release especially when rival has a new release carrying more attractive features and satisfying more customer requirements (Saliu and Ruhe 2005). In this point of view, multiple-release planning not only makes software organization easily balance the competing stakeholder’s demands and benefits according to the available resources, but lowers the risk of not satisfying customer requirements (Ruhe and Momoh 2005; Svahnberg et al. 2010).

On the other hand, large software system continually needs to align with the changing customer requirements for the sake of market share. In order to get the feedback earlier and figure out what customer really wants, and assigning a lower software development cost, with a certain portion of increments on requirement for multiple release product is essential for the growth of an organization (Maurice et al. 2006; Greer and Ruhe 2004; Missbauer 2002; Mehlawat 2013). Thus, it is plausible for software company to modify the parts of the existing modules to extend the current functionality, usability, and understandability by adding new features and correcting the issues from previous release (Al-Emran and Pfahl 2007; Gorschek and Davis 2008).

Since multi-release is critical for modern software product, release planning is becoming a popular research topic in the past few years. Release planning is a very complex problem. It has to take into account the consequence of feedback and update from customers, the demands of potential customers, market feedback, defects from the previous release and other technical and non-technical constraints (Al-Emran and Pfahl 2007; Ruhe and Momoh 2005). Many researchers have studied software release planning problems (Saliu and Ruhe 2005; Maurice et al. 2006; Al-Emran and Pfahl 2007; Ruhe and Saliu 2005; Svahnberg et al. 2010; Carlshamre et al. 2001; Ho and Ruhe 2013). For instance, Saliu and Ruhe (2005) presented a more comprehensive approach to characterize the relative impact of integrating features into an existing system. They extended the former solution approach EVOLVE* and proposed S-EVOLVE* method by taking into account the characteristics of the target system for feature implementation. Maurice et al. (2006) proposed F-EVOLVE* approach to determine the desired features based on the financial contribution, which is also the results of extending EVOLVE* approach. Moreover, Ruhe and Saliu (2005) described the principle of “art of release planning” and “science of release planning”, respectively. Human intuition, communication and capabilities to negotiate between conflicting objectives and constraints referred to the art of releasing planning; while the science of release planning indicated the emphasizing formalization of the problem and algorithm application optimization.

It is generally considered reliability as a key factor in software quality measurement owing to the fact that it qualifies software failures and misbehaviors (Febrero et al. 2016). As the growth of software size and complexity, how to improve software quality and reliability, control the total cost, and carry adequate and attractive features are critical issues faced by software organization. Therefore, in the past three decades, a great number of Software Reliability Growth Models (SRGMs) has been proposed to quantify the quality of software system. Non-homogeneous Poisson Process (NHPP) is considered as one of the most effective models to study software reliability (Goel and Okumoto 1979; Musa 1975; Huang and Kuo 2002; Kapur et al. 2011; Yamada et al. 1986, 1993; Jeske and Zhang 2005; Zhu and Pham 2016). Nevertheless, most of them only can be applied on a single release. How to model software reliability based on a multiple release perspective just starts gaining researcher’s attention not very long.

In this paper, we take into account two types of software faults for developing the next release: (1) Fault from previous release, i.e. remaining faults from previous release since it is unlikely to detect and remove all faults within limited resources; (2) New introduced-faults, i.e. new features are added in the next release, which also brings new software faults into the next release. We also assume that the detection of software fault for the next release’s development depends on the detection of the remaining faults from previous release and the new introduced-faults. To the extent of our knowledge, we haven’t seen any research focus on remaining faults from previous release, new introduced-faults and dependent fault detection process in multi-release software reliability modeling.

This paper is organized as follows. In Sect. 2, a literature review is presented. Section 3 outlines the proposed multi-release software reliability modeling. The behaviors of software reliability function illustrated by a theoretical fashion are also discussed. Parameter estimation by the use of Least Square Estimation (LSE) and the comparison criteria are discussed in Sect. 4. Section 5 demonstrates the proposed multi-release software reliability modeling with two practical Open Source Software (OSS) datasets from Apache. Finally, conclusions and future research are described in Sect. 6.

2 Literature review

Most software products are not introduced into the market with full capacities at their initial release, only with sufficient functionalities. New features will be added, and existing features will be enhanced in the next release. A lot of researches have been done for the single version software system for the past few decades. Modeling and predicting software failure behavior are also investigated in those researches. However, most of the existing models developed for single version software product cannot apply on the multi-release software product due to different assumptions and applications. Only a few research studied multi-release software reliability and introduced some models to illustrate software detection process and fault removal process for multi-release software products. It is thus necessary to investigate software measurement metrics for multi-release software product.

The optimization for software version planning and release has been studied by many researchers (Szöke 2011; Naciri et al. 2015; Etgar et al. 2017; Li et al. 2014. For example, Szöke (2011) proposed a theoretically method for agile release planning. The proposed staged-delivery global optimized model gives the main parameters of the typical agile planning space. Moreover, Naciri et al. (2015) aimed to tackle software release planning problem for a Third Party Application Maintenance (TPM) context. A strategic release planning model of software defects based on Third Party Application Maintenance (TPM) constraints and challenges was proposed to produce an effective release planning of software maintenance in outsourcing context. Etgar et al. (2017) explored several optimization approaches to determine the content and release date for each release in order to provide optimal net present value (NPV). Li et al. (2014) and Mehlawat (2013) proposed a multi-objective multi-choice optimization technique to optimize the requirement choice for the three main objectives, cost, revenue, and uncertainty for robust next release problem.

Given the fact that there was lack of attempt to create a release history database of a large number of projects in the Open source ecosystem, Tsay et al. (2011) created a software release history database including the tools, techniques and pitfalls, to provide insightful and sound information for the future researchers and industrial release engineering practices.

In the real application, Leszak (2005) provided a study of several processes on an industrial large-scale multi-release software system. Sukhwani et al. (2016) performed the software reliability analysis of the flight software of a recent launches space mission. They also linked the activities in the major releases with the problems encountered during the development of those releases. Mahimkar (2016) focused on the detection of software upgrades on smartphones and analyzing their service performance impact, for example, smartphone-centric, or network-centric impact.

In terms of software reliability study, Garmabaki et al. (2011) incorporated different severities level used to describe the difficulty of correcting faults in the upgrade process to develop a multi up-gradation software reliability model. Faults are classified into two categories, simple fault and hard fault. The fault removal for the development of the new release depends on the fault from previous release and fault generated in that release. Hu et al. (2011) considered the effect of multiple releases on the fault detection process in software development. They assumed that there is no gap between the release of previous version and the development of next version. Optimal release time for each version is also present in this paper.

Kapur et al. (2012) introduced the combined effect of schedule pressure and resource limitations by the use of Cobb–Douglas production function in software reliability modeling. The Cobb–Douglas function illustrates the total production output can be obtained by the amount of labor input, capital input, and total factor productivity. An optimal release planning problem is formulated in this paper for software with multiple releases with the solution obtained by applying genetic algorithm method.

Pachauri et al. (2015) proposed a software reliability growth model by considering fault reduction factors (FRFs) and extended this idea to multi-release software systems. FRFs is defined as the ratio of the total number of reduced faults to the total number of failures, which can be affected by other factors, such as resources allocation, debugging time lag and imperfect debugging.

Yang et al. (2016) incorporated fault detection and fault correction process in multi-release software reliability modeling. There is a time delay in fault repair after detecting faults. The time delay function is explained by an exponential function or a gamma function. They also assumed the faults in a new version including both undetected faults from last version and new introduced faults during the development process of the new version.

Ahmadi et al. (2016) incorporated bugs removed from pre-commit test and bugs reported by parallel debugging test based on software lifecycle development process (SDLC) proposed by Jørgensen (2001). Additionally, the fault removal of the new release depends on the reported faults from previous release and the faults generated by the new functionalities.

However, most literature aimed to develop multi-release software reliability model only by optimizing software cost model to determine software release time except Yang et al. (2016). Some of them have also incorporated the fault removal of the next release depends on the reported faults from previous release and the faults generated by the new adding functionalities, like Ahmadi et al. (2016), but they didn’t consider the dependent fault detection process in the next release. Hence, our research focuses on the dependent fault detection process for next release’s development along with the consideration of the remaining faults from previous release and the new introduced-faults (from newly added features); in other words, the detection of the new faults for the development of next release depends on the detection of remaining faults from previous release and the new introduced-faults.

3 The multi-release software reliability modeling framework

3.1 Multi-release software reliability modeling

It is unlikely to get bug-free software product within limited resources and tightened schedules. Software detection process still follows an NHPP process for developing the next release. The cumulative number of detected faults \(N\left( t \right) \) follows Poisson Process.

$$\begin{aligned} \Pr \left\{ {N\left( t \right) =n} \right\} =\frac{\left( {m\left( t \right) } \right) ^{n}\exp \left( {-m\left( t \right) } \right) }{n!},\quad \hbox {for } n=0,1,2,\ldots \end{aligned}$$

where \(m\left( t \right) \) is the mean value function of the counting process \(N\left( t \right) \).

Two types of software faults will be addressed in this paper. Remaining faults from previous release (Part I) and new introduced-faults (Part II) will be both incorporated with the aim of developing the next release. Fault detection is a dependent process. We assume the detection of a software fault depends on the fault detected from Part I and Part II.

Thus, the multi-release software reliability modeling can be formulated as follows:

$$\begin{aligned} \frac{dm\left( t \right) }{dt}=d\left( t \right) \left[ {a\left( t \right) -m\left( t \right) } \right] \left[ {b\left( t \right) -m\left( t \right) } \right] m\left( t \right) \end{aligned}$$
(1)

where \(m\left( t \right) \) represents the expected number of software failures by time \(t,d\left( t \right) \) denotes the fault detection rate function, a(t) and b(t) represent the total remaining faults from previous release and the total fault content of the current release, respectively. In this paper, we assume that

$$\begin{aligned} a\left( t \right) =a,b\left( t \right) =b \end{aligned}$$
(2)

Substitute (2) into (1), we can obtain a general solution for the mean value function m(t) by solving the following equation:

$$\begin{aligned} e^{dt+C_0 }=m\left( t \right) ^{\frac{1}{ab}}\left( {m\left( t \right) -a} \right) ^{\frac{1}{a\left( {a-b} \right) }}\left( {m\left( t \right) -b} \right) ^{-\frac{1}{b\left( {a-b} \right) }} \end{aligned}$$
(3)

where \(C_0\) is a constant. In this study, we consider that the initial solution of the function m(t) is as follows:

$$\begin{aligned} m\left( {t=0} \right) =m_0 \end{aligned}$$
(4)

where \(\hbox {m}_{0} \ge 0\) is unknown. At time \(t=0\), the expected number of initial software failures is \(m_0 \). Since multiple software releases are considered in this study, the expected number of software failures at the beginning of next release should be less than or equal to the expected number of failures at the end of previous release. Additionally, it is unlikely to remove all the software faults for each release due to the limitation of all available resource, including software programmer’s domain knowledge and other environmental factors, as seen in reference Al-Emran and Pfahl (2007), Ruhe and Momoh (2005), Gorschek and Davis (2008), Kapur et al. (2012), Yang et al. (2016), Pachauri et al. (2015), Zhu et al. (2015).

Substitute (4) into (3), we obtain

$$\begin{aligned} e^{C_0 }=m_0 ^{\frac{1}{ab}}\left( {m_0 -a} \right) ^{\frac{1}{a\left( {a-b} \right) }}\left( {m_0 -b} \right) ^{-\frac{1}{b\left( {a-b} \right) }} \end{aligned}$$
(5)

Thus, the solution for the function m(t) from the Eq. (1) can be obtained by solving the following equation:

$$\begin{aligned} e^{dt}=\left( \frac{m\left( t \right) }{m_0 }\right) ^{\frac{1}{ab}}\left( {\frac{m\left( t \right) -a}{m_0 -a}} \right) ^{\frac{1}{a\left( {a-b} \right) }}\left( {\frac{m\left( t \right) -b}{m_0 -b}} \right) ^{-\frac{1}{b\left( {a-b} \right) }} \end{aligned}$$
(6)

3.2 Multi-release software reliability function discussion

Let

$$\begin{aligned} g\left( t \right) =e^{dt+C_0 } \end{aligned}$$
(7)

and

$$\begin{aligned} f\left( x \right) =x^{\frac{1}{ab}}\left( {x-a} \right) ^{\frac{1}{a\left( {a-b} \right) }}\left( {x-b} \right) ^{-\frac{1}{b\left( {a-b} \right) }} \end{aligned}$$
(8)

We now present the following main results.

Lemma 1

The solution m(t) of Eq.  (3) can be obtained by solving the following function

$$\begin{aligned} g\left( t \right) =f\left( {m\left( t \right) } \right) \end{aligned}$$
(9)

and,

  • If \(g\left( 0 \right) >\max \left\{ {f\left( {m\left( t \right) } \right) } \right\} \) then there exists no solution, as shown in Fig.  1a.

  • Otherwise, there exists at least one solution for the function m(t), as shown in Fig.  1b.

where the function g(t) and \(f\left( x \right) \) are given in Eqs.  (7) and (8), respectively.

The proof of Lemma  1 is in the Appendix.

Fig. 1
figure 1

Illustration of solutions for convex function

Full size image

4 Parameter estimation and comparison criteria

4.1 Parameter estimation

Most SRGMs use the least square estimate (LSE) or maximum likelihood estimate (MLE) to estimate the parameters carried in the model. For example, minimizing the Eq. (10) or maximizing the Eq. (11).

$$\begin{aligned}&\displaystyle f\left( t \right) =\mathop \sum \limits _{i=1}^n \left( {m\left( {t_i } \right) -y_i } \right) ^{2} \end{aligned}$$
(10)
$$\begin{aligned}&\displaystyle LLF=\mathop \sum \limits _{i=1}^n \left( {y_i -y_{i-1} } \right) \hbox {log}\left[ {m\left( {t_i } \right) -m\left( {t_{i-1} } \right) } \right] -m\left( {t_n } \right) -\mathop \sum \limits _{i=1}^n \log \left( {y_i -y_{i-1} } \right) ! \end{aligned}$$
(11)

In this paper, we apply LSE to minimize Eq. (12) to estimate the parameters. Since \(g\left( t \right) =f\left( {m\left( t \right) } \right) \), indeed, \(log\left[ g\left( t \right) \right] =log \left[ f\left( {m\left( t \right) } \right) \right] \). The optimization function is given by

$$\begin{aligned} \hbox {min } S\left( {a,b,d} \right) =\mathop \sum \nolimits _{a,b,d} (\log \left[ {f\left( {y_i } \right) } \right] -\hbox {log}\left[ {\left[ {g\left( {t_i } \right) } \right] } \right) ^{2} \end{aligned}$$
(12)

where \(y_i \) is the observed number of failures at time \(t_i \), \(g\left( {t_i } \right) =e^{dt_i +C_0 }\). The Lemma 1 presented in Sect. 3 is to show that there exists the solutions and explain the behaviors of the proposed model. The Genetic Algorithm (GA) is employed in order to solve the optimization function as given in Eq. 12. The schematic diagram of the algorithm (Kachitvichyanukul 2012) is described in Fig. 2. We use Matlab Optimization Toolbox to solve the optimization function and estimate parameters.

Fig. 2
figure 2

Schematic diagram of Generic Algorithm

Full size image
Table 1 Open source software Juddi project data
Full size table

4.2 Comparison criteria

  1. (1)

    Mean squared error

    $$\begin{aligned} \textit{MSE}=\frac{\mathop \sum \nolimits _{i=1}^n \left( {\hat{m} \left( {t_i } \right) -y_i } \right) ^{2}}{n-N} \end{aligned}$$
    (13)

    The mean squared error (MSE) measures the distance of a model estimate from the observed data where n is the total number of observations and N represents the number of unknown parameters in each model.

  2. (2)

    Predictive-ratio risk and predictive power (Pham 2014)

    $$\begin{aligned} \textit{PRR}= & {} \mathop \sum \limits _{i=1}^n \left( {\frac{\hat{m} \left( {t_i } \right) -y_i }{\hat{m} \left( {t_i } \right) }} \right) ^{2} \end{aligned}$$
    (14)
    $$\begin{aligned} \textit{PP}= & {} \mathop \sum \limits _{i=1}^n \left( {\frac{\hat{m} \left( {t_i } \right) -y_i }{y_i }} \right) ^{2} \end{aligned}$$
    (15)

    The predictive-ratio risk (PRR) and the predictive power (PP) are calculated to compare the power of different models. PRR measures the distance of the model estimates from the actual data against the model estimates; while predictive power (PP) measures the distance of the model estimates from the actual data against the actual data.

  3. (3)

    Variation The Variation is defined as (Huang and Kuo 2002):

    $$\begin{aligned} \textit{Variation}= & {} \sqrt{\frac{1}{n-1}\mathop \sum \limits _{i=1}^n \left( {y_i -\hat{m} \left( {t_i } \right) -Bias} \right) ^{2}} \end{aligned}$$
    (16)

    where

    $$\begin{aligned} \textit{Bias}=\frac{1}{n}\mathop \sum \limits _{i=1}^n \left( {\hat{m} \left( {t_i } \right) -y_i } \right) \end{aligned}$$
    (17)

    The smaller the variation, the better of the model should be.

Table 2 Parameter estimate and model comparison for Juddi dataset
Full size table

5 Numerical example

In this section, two numerical applications are given to validate the multi-release software reliability model. We employ two datasets both collected from Open Source software (OSS) project. Open Source Software (OSS) is a new way to build a global-based large software system, which differs in many perspectives with the traditional software engineering (Raymond 2001). The evolution process of OSS is much faster than the traditional close source software. Widespread OSS projects bring in a great change in terms of software development paradigms and software architectures (Li et al. 2011; Raymond 2001; Li et al. 2006).

5.1 Juddi project data

Juddi Open Source Software project data is shown in Table 1. Failure dataset from week 1 to week 31 are considered as Release 1; failure dataset from week 32 to week 49 are considered as Release 2; failure dataset from week 50 to week 61 are considered as Release 3. We use Release 2 in this paper to validate our proposed model. The parameter estimate and model comparison are described in Table 2.

As shown in Table 2, we can see that the proposed model has the best performance in terms of all criteria present here. Figure 3 illustrates the comparison between model prediction and observed failure data. All models showing in Table 2 only consider single-release software product except proposed model. In other words, they didn’t consider the remaining faults from the previous release since most of models assume all software faults will be removed before software company release the product.

Fig. 3
figure 3

Comparison between proposed model and other models for Juddi dataset

Full size image
Table 3 Open source software APACHE 2.0
Full size table
Table 4 Parameter estimate and model comparison for APACHE dataset
Full size table

5.2 Apache 2.0

Apache 2.0Footnote 1 is available on the website since 2002. The first two releases are employed to verify the proposed model, as shown in Table 3. The failure data from day 1 to day 18 is taken into account as Release 1; failure data from day 19 to day 164 is considered as Release 2.

Fig. 4
figure 4

Comparison between proposed model and other models for Apache 2.0 dataset

Full size image

We can see that the proposed model provides the smallest MSE, PP, and Variation as shown in Table 4. The PRR value even though is not the smaller one, however, 0.155 is just slightly higher than 0.117. It is thus considering the proposed model presents the best performance to model this dataset. Figure 4 also plots the comparison between the predicted values and the observed values.

In summary, the proposed model has considered a dependent fault detection process. Specifically, the newly detected faults depend on the detection of remaining faults from previous release and new introduced faults. In order to detect a new fault, we need to detect corresponded faults from remaining faults from previous release and new introduced faults first. Therefore, there is only a small portion of software faults detected for developing the next release.

6 Conclusion remark and future research

It is unlikely to deliver all the features in a single release for the modern software products. The proposed software reliability model provides a new paradigm to integrate the dependent fault detection process and different types of software faults in multiple software release. Due to the resource limitation, there also exists a portion of undetected software faults for the current release. Thus, how to incorporate the faults from previous release into the development for next release becomes an important issue.

As an effort to reflect the development of multi-release software, remaining faults from previous release, new introduced-faults, and dependent fault detection process are discussed in this paper. In order to accurately illustrate the performance of the proposed model, we employ two datasets both collected from Open Source Software (OSS) project to validate the usage of the model. The behavior of software reliability function is studied as well. We are currently investigating the new features adding in the next release and the remaining faults from previous release as fixed numbers in this study, which can be extended as a random number or as a time-dependent function corresponding to its optimal profit and release time for the organization. The impact of environmental factors (Zhu et al. 2015) during the software development process can be considered into the future research as well.