Abstract
In this paper, first- and second-order discrete-time semi-Markov systems are considered with their finite state space divided into three subsets as perfect functioning states, imperfect functioning states and failure states, respectively. The counting processes for one-step increasing transitions, one-step equivalent transitions and one-step decreasing transitions in working/failure periods are defined and investigated in detail. Formulas for related distributions, joint distributions, expectations, generating functions and joint generation functions are derived in their Z-transforms. Numerical examples are presented to illustrate the results established. Extended discussions on related reliability measures are also considered. Finally, some concluding remarks and discussions are presented. Applications of the results presented here can be found in different fields such as seismology, reliability, biology and finance.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
1 Introduction
In recent years, multi-state systems are attracting more and more attention since many real-world systems are composed of multi-state components that have various effects on the performance of the entire system with different performance levels and failure modes. As mentioned by Lisnianski et al. (2010), there are many multi-state systems in practice, such as a power supply system consisting of generating and transmitting facilities (each generating unit can function at different levels of capacity), a wireless communication system consisting of transmission stations (each station can have a different number of subsequent stations covered in its range), a task processing system (for example, a control system, a data processing system or a manufacturing system) and many others. Tools or methods that contribute to modeling of multi-state systems include, but are not limited to, stochastic processes such as Markov and semi-Markov processes, statistical analysis, universal generating function methods, combined methods, coherent system theory, multidimensional spectra and multi-state signature, and many different kinds of algorithms (Lisnianski et al. 2010; Lisnianski and Frenkel 2011; Natvig 2011). Recently, research on multi-state systems is being carried out extensively as can be seen in Yi and Cui (2017), Yi et al. (2018), Barbu and Vergne (2019), Wu et al.(2019).
For multi-state systems, the concept of state aggregation was first presented by Burke and Rosenblatt (1957) based on the assumption that an experimenter can only observe a derived process Y (n) = f(X(n)) rather than the underlying stochastic process X(n). (Note that the derived process may no longer be Markov or semi-Markov processes after the state aggregation.) The problem of state aggregation subsequently received great attention in the area of single ion channel in biology. As we know, there are plenty of channels on cell membrane for materials like ions to get through. Biologists are always interested in the behaviors (open, closed) of those ion channels, especially single ones. However, experimental data can only show current levels of electricity of a single ion channel, which is not sufficient for describing the underlying mechanism since behaviors with the same current level are not distinguishable from the data. Colquhoun and Hawkes (1977, 1981, 1982) proposed a general aggregated Markov process model, which changed the way biologists dealt with experimental data on single ion channels. Compared with traditional Markov process, state space of the aggregated Markov process is divided into different subsets according to the performance levels of the system, which make the model more general by taking indistinguishable states into consideration. For systems with no indistinguishable state, the aggregated Markov processes will degenerate to traditional Markov processes. Since then, many generalizations and extensions have been discussed in the area of single ion channel (Hawkes et al. 1990; Ball 1997; The and Timmer 2006).
Aggregated stochastic processes also have important applications and developments in the context of reliability. Rubino and Sericola (1989) first discussed the reliability of an aggregated stochastic process and also presented an application of a fault-tolerant system with several buffer states. Cui et al. (2007) investigated aggregated Markov processes by considering history-dependent states (changeable states whose performances depend on the immediately preceding state of the system evolution process), and derived reliability indices such as availability. Hawkes et al. (2011) studied the evolution of aggregated Markov processes under alternative environments and obtained results for the system availability and probability distributions for uptimes. Liu et al. (2014) developed an aggregated Markov process model with repair time omission, and derived reliability indices such as availability, interval reliability and interval unreliability. Cui et al. (2016) considered aggregated stochastic processes with cyclic multiple mission periods, and obtained closed-form solutions for some reliability indices and sojourn time distributions. Recently, aggregated stochastic processes have also been used in the study of new reliability indices (Yi et al. 2018), phase-type models (Wu et al. 2020) and balanced systems (Fang and Cui 2021).
Most of existing research on aggregated stochastic processes are for continuous time systems, but there are also plenty of discrete-time systems in practice; for example, data obtained from laboratory experiments are often discrete-time ones. The discussions on discrete-time aggregated stochastic processes are far few as compared to those on continuous ones. For example, Yi et al. (2018) developed a discrete-time semi-Markov aggregated process, and discussed some stochastic properties and reliability measures. Yi and Cui (2017) considered a discrete-time second-order aggregated semi-Markov system, and derived some distributions and availabilities of the system. Actually, the calculation issue is much more complex for continuous-time aggregated stochastic processes when it comes to the essential step of inverse Laplace transform. In this paper, we consider discrete-time aggregated semi-Markov systems since semi-Markov processes break the memoryless requirement of Markov processes. That is quite reasonable in practice; see Limnios and Oprisan (2003) for a review of semi-Markov processes and to Janssen and Manca (2006), Barbu and Limnios(2008, 2004) for additional related discussions. Also, second-order aggregated semi-Markov systems are considered since evolutions of many real-world systems depend not only on their present states, but also on some previous states; see (D’Amico et al. 2013; 2015a, b) for more discussions on higher-order Markov and semi-Markov models and related applications to wind speed prediction.
Some basic assumptions in this paper are similar to those in Yi and Cui (2017) and Yi et al. (2018); for example, states of the system are divided into three subsets \({\mathscr{A}},{\mathscr{B}},{\mathscr{D}}\) corresponding to perfect functioning states, imperfect functioning states and failure states, respectively; a working period is defined as a duration that the system is in functioning states (including perfect functioning states and imperfect functioning states), which starts from the moment of transition to a perfect functioning state from a failure state directly or through some imperfect functioning states, and ends at the moment of transition from a perfect functioning state to a failure state directly or through some imperfect functioning states; and a failure period is the duration of time between two consecutive working periods. As mentioned above, in this paper, we define the split of state space and working/failure periods in a traditional way so that any misunderstanding caused by the complexity of modeling can be avoided to some extent and that the method developed can also be discussed in clear terms. For practical systems whose state space is divided into more subsets and whose working periods/failure periods are defined in a different way, the results developed here can be generalized without much technical difficulty, and so the related discussions are omitted in this paper for brevity.
In the field of reliability, counting processes are often used to describe the arrivals and repairs of different types of shocks, which do impact greatly on the performance of the system in different ways. Moreover, properties of counting processes are attracting increasing attention both theoretically and practically (Howard 1964, 1971; Mode and Peckins 1988), and possible applications can be found in such diverse areas as queuing theory, wireless communication systems, software error detection, health science and many others (Badía and Sangüesa 2017; Michael and Eutichia 2020; Nuel 2019). For a discrete-time semi-Markov process with finite state space, Csenki (1995) derived a closed-form expression for the cumulative distribution function of the number of visits to a subset of its state space, and explained that this number served as an useful tool for analyzing dependability characteristics. Based on a continuous-time semi-Markov model with finite state space, Votsi et al. (2012) estimated the expected number of earthquake occurrences with a real data set from Northern Aegean Sea in Greece. In their work, model states were classified by earthquake magnitude, and fault orientation was also taken into account. For general time-homogeneous Markov processes with general state space, Landriault et al. (2017) presented an unified approach for the first passage time of their drawdown/drawup processes.
For continuous-time Markov process, Cui et al. (2021) considered the split of state space into m ordered subsets and then studied counting processes of one-step increasing/equivalent/decreasing transitions with durations longer/shorter than a pre-specified threshold. In the present work, we consider first- and second-order discrete-time semi-Markov processes with finite state space \({\Omega } = {\mathscr{A}} \cup {\mathscr{B}} \cup {\mathscr{D}}\). Transitions from states in \({\mathscr{D}}\) to states in \({\mathscr{A}} \cup {{\mathscr{B}}}\) or from states in \({\mathscr{B}}\) to states in \({\mathscr{A}}\) are called one-step increasing transitions as they correspond to repairs of the system. Transitions from states in \({\mathscr{A}}\) to states in \({\mathscr{B}} \cup {\mathscr{D}}\) or from states in \({\mathscr{B}}\) to states in \({\mathscr{D}}\) are called one-step decreasing transitions as they correspond to degradations of the system. Also, other transitions from subset \({\mathscr{A}}\) or \({{\mathscr{B}}}\) or \({\mathscr{D}}\) to themselves are called one-step equivalent transitions. Here, we are interested in all three different kinds of transitions in working periods and failure periods, which leads to the following counting processes:
-
(1)
Let \({{N_{I}^{W}}}(n),{N_ E^{W}}(n),{N_ D^{W}}(n)\) be the number of one-step increasing transitions, one-step equivalent transitions and one-step decreasing transitions in working periods by time n, respectively;
-
(2)
Let \({{N_{I}^{F}}}(n),{N_ E^{F}}(n),{N_ D^{F}}(n)\) be the number of one-step increasing transitions, one-step equivalent transitions and one-step decreasing transitions in failure periods by time n, respectively;
-
(3)
Let NW(n),NF(n),N(n) be the number of all one-step transitions in working periods, in failure periods and in all of them by time n, respectively. This means \(N^{W}(n)={{N_{I}^{W}}}(n)+{N_ E^{W}}(n)+{N_ D^{W}}(n),N^{F}(n)={{N_{I}^{F}}}(n)+{N_ E^{F}}(n)+{N_ D^{F}}(n)\) and N(n) = NW(n) + NF(n).
It is not easy to investigate counting processes defined above since whether a system is in a working period or a failure period depends not only on its present state, but also on history of its previous states. However, it is of great interest to consider this problem since transitions, including ones-step increasing/equivalent/decreasing ones, have different costs and benefits in working periods and failure periods. Applications of this paper can be found in many fields, including seismology, reliability, biology, finance and many others. For example, as in Votsi et al. (2012), the occurrences of earthquake can be modeled by a first-order semi-Markov model, and states of the system can be divided into three subsets according to magnitudes (eg. \({\mathscr{A}}\) for [0,4.4], \({\mathscr{B}}\) for [4.5,5.9] and \({\mathscr{D}}\) for [6.0,10.0]). Consider the fact that earthquakes with magnitudes greater than a given threshold (4.4, say) will lead to some financial loss and earthquakes with magnitudes greater than another given threshold (5.9, say) will lead to serious casualties, we can use the concepts of working period and failure period introduced before to distinguish whether there is risk of life for human beings in different periods. Then, we are interested in the counting processes for one-step increasing/equivalent/decreasing in the two types of periods, respectively, since losses in them are not estimated in the same degree of importance and we need different post-earthquake reconstruction strategies for them.
Take a generating unit as another example, as in Billinton and Allan (1996), in which performance rates (namely, the power generating capacities) can be 50 MW (perfect functioning), 30 MW (imperfect functioning) and 0 MW (failure), respectively, which means the state space can be divided into subsets \({\mathscr{A}}\), \({\mathscr{B}}\) and \({\mathscr{D}}\) mentioned above. Considering the fact that each sojourn in the failure subset \({\mathscr{D}}\) will cause serious economic losses, machine damage and long shutdown period, the generating unit system is regarded to be in good condition only when it functions perfectly or imperfectly without any breakdown into failure states. Then, we can use the concepts of working period and failure period introduced before to distinguish whether the system is in a good condition or not. Besides, state changes cause different results when the system is in different conditions, and we are therefore interested in the counting processes mentioned before to evaluate possible maintenance costs and machine damage losses in different system conditions.
The rest of this paper is organized as follows. In Sections 2 and 3, stochastic properties of related counting processes are discussed for first- and second-order discrete-time aggregated semi-Markov processes, respectively; formulas for some distributions, joint distributions, expectations, generating functions and joint generation functions are presented by their Z-transforms. Illustrative examples are presented in Section 4 to explain the results established in preceding sections and to give some intuitive understanding of the results. Some extended discussions on related reliability measures are considered in Section 5. Finally, some conclusions and related discussions are made in Section 6.
2 First-order Discrete-Time Semi-Markov System
Consider the time-homogenous first-order semi-Markov chain {Z(n),n = 0,1,…}, as in Yi et al. (2018), with discrete time and finite state space Ω = {1,…,kΩ}. The state that the system jumps to at the k th transition time Sk is denoted by Jk (k = 1,2,…), and the initial state at time S0 = 0 is denoted by J0. Let X0 = 0, and the sojourn time in state Jk− 1 between the (k − 1)th and k th transitions be expressed as Xk = Sk − Sk− 1 (k = 1,2,…). Then, the semi-Markov kernel of the related Markov renewal chain can be defined as follows:
With the state space Ω divided into three subsets \({\mathscr{A}},{\mathscr{B}}\) and \({\mathscr{D}}\), the semi-Markov kernel can be divided correspondingly as follows:
Here, the subsets \({\mathscr{A}},{\mathscr{B}}\) and \({\mathscr{D}}\) consist of perfect functioning states, imperfect functioning states and failure states, respectively. Besides, there are \({k_{{\mathscr{A}}}}\), \({k_{{\mathscr{B}}}}\) and \({k_{{\mathscr{D}}}}\) states in the subsets \({\mathscr{A}},{\mathscr{B}}\) and \({\mathscr{D}}\), respectively, namely, \({\mathscr{A}} = \{ 1, {\ldots } ,{k_{{\mathscr{A}}}}\} ,{\mathscr{B}} = \{ {k_{{\mathscr{A}}}} + 1, {\ldots } ,{k_{{\mathscr{A}}}} + {k_{{\mathscr{B}}}}\}\) and \({\mathscr{D}} = \{ {k_{{\mathscr{A}}}} + {k_{{\mathscr{B}}}} + 1, {\ldots } ,{k_{{\mathscr{A}}}} + {k_{{\mathscr{B}}}} + {k_{{\mathscr{D}}}}\} \).
As in the discussions in Yi et al. (2018), sojourns in subset \({\mathscr{B}}\) contribute to working periods if and only if the system is in \({\mathscr{A}}\) right before it enters \({\mathscr{B}}\) and right after it leaves \({\mathscr{B}}\), which implies that they contribute to failure periods otherwise. Obviously, sojourns in subset \({\mathscr{A}}\) contribute to working periods and sojourns in subset \({\mathscr{D}}\) contribute to failure periods. For studying properties of the semi-Markov system, a new stochastic process {Y (n),n = 0,1,…} is defined as follows:
Note that Y (n) is a right-continuous function, which means its value at a transition time depends on whether the sojourn after transition contributes to a working period or not. Besides, a transition is said to be in a working period if the system is in a working period when the transition starts, and in a failure period if the system is in a failure period when the transition starts.
Denote the conditional probability that the system stays in subset X in time interval [0,n) and jumps to state j in another subset Y at time n, given the initial state i in subset X at time 0, by \(g_{ij}^{XY}(n)\), where \((X,Y) \in \{ ({\mathscr{A}},{\mathscr{B}}),({\mathscr{A}},{\mathscr{D}}),({\mathscr{B}},{\mathscr{A}}),({\mathscr{B}},{\mathscr{D}}),({\mathscr{D}},{\mathscr{A}}),({\mathscr{D}},{\mathscr{B}})\} \); that is,
where no self-transition is allowed.
In matrix form, let us denote
where \({{\boldsymbol {G}}_{XY}}(n) = {\left ({g_{ij}^{XY}(n),i \in X,j \in Y} \right )_{{k_{X}} \times {k_{Y}}}}\), kX and kY denote the numbers of states in subsets X and Y, respectively, and 0 is a zero matrix of suitable dimension, whose elements are all 0. Define I as an unit matrix of suitable dimension, and the Z-transform of GXY(n) can be represented as \({\boldsymbol {G}}_{XY}^{*}(z) = {[{ {I}} - {\boldsymbol {Q}}_{XX}^{*}(z)]^{- 1}}{\boldsymbol {Q}}_{XY}^{*}(z)\); see Yi et al. (2018) for the proof.
For this first-order semi-Markov system, there are 9 different types of transitions to be considered for Z(n), namely, transitions \({\mathscr{A}} \to {\mathscr{A}},{\mathscr{A}} \to {\mathscr{B}},{\mathscr{A}} \to {\mathscr{D}},{\mathscr{B}} \to {\mathscr{A}},{\mathscr{B}} \to {\mathscr{B}},{\mathscr{B}} \to {\mathscr{D}},{\mathscr{D}} \to {\mathscr{A}},{\mathscr{D}} \to {\mathscr{B}}\) and \({\mathscr{D}} \to {\mathscr{D}}\). Obviously, transitions from \({\mathscr{B}}\) to \({\mathscr{A}}\) (\({\mathscr{B}} \to {\mathscr{A}}\)) are one-step increasing transitions in working periods if and only if the system is in subset \({\mathscr{A}}\) before it enters subset \({\mathscr{B}}\), and they are one-step increasing transitions in failure periods if and only if the system is in subset \({\mathscr{D}}\) before it enters subset \({\mathscr{B}}\). To identify the two different cases of transitions, namely, to identify whether a transition is in a working period or in a failure period when it is from subset \({\mathscr{B}}\), we define a new semi-Markov chain {Z0(n),n = 0,1,…} with state space \({{\Omega }_{0}} = {\mathscr{A}} \cup {{\mathscr{B}}_{1}} \cup {{\mathscr{B}}_{2}} \cup {{\mathscr{D}}}\), where
Note that subsets \({{\mathscr{B}}_{1}}\) and \({{\mathscr{B}}_{2}}\) can be regarded as a “copy” of subset \({\mathscr{B}}\). The most important property of Z0(n) is that Z0(n) = Z(n) for \(Z(n) \in {\mathscr{A}}\) or \(Z(n) \in {{\mathscr{B}}}\) with the system being in subset \( {\mathscr{A}}\) beforing entering subset \( {\mathscr{B}}\), and \({Z_{0}}(n) = Z(n) + {k_{{\mathscr{B}}}}\) for \(Z(n) \in {\mathscr{D}}\) or \(Z(n) \in {{\mathscr{B}}}\) with the system being in subset \( {\mathscr{D}}\) beforing entering subset \( {\mathscr{B}}\), which makes it possible to identify whether the system is in a working period or in a failure period when the system state Z(n) is in subset \({\mathscr{B}}\). Then, the new semi-Markov kernel and its Z-transform can be presented as follows:
Assume that the initial probability vector of Z0(n) can be denoted as \({ {\pi }} = ({ {\pi }}_{{\mathscr{A}}}, { {\pi }}_{{\mathscr{B}}_{1}}, { {\pi }}_{{\mathscr{B}}_{2}}\), \({ {\pi }}_{{\mathscr{D}}})\) \(= ({\pi }_{1}, {\ldots } , {\pi }_{{k_{{\mathscr{A}}}} + {2 k_{{\mathscr{B}}}}+ {k_{{\mathscr{D}}}}})\), where πi = P{Z0(0) = i}, for \(i = 1, \ldots , {k_{{\mathscr{A}}}} + {2k_{{\mathscr{B}}}}+ {k_{{\mathscr{D}}}}\), is the probability that the initial state of Z0(n) is state i. To investigate the transition rules, some notations for one-step increasing/equivalent/decreasing transitions in working/failure periods are defined as follows:
Note that \({{\boldsymbol {P}}_ I^{W}}(z) + {{\boldsymbol {P}}_ E^{W}}(z) + {{\boldsymbol {P}}_ D^{W}}(z) + {{\boldsymbol {P}}_ I^{F}}(z) + {{\boldsymbol {P}}_ E^{F}}(z) + {{\boldsymbol {P}}_ D^{F}}(z) = {{\boldsymbol {D}}^{*}}(z)\). In addition, let
Further, let \({{ {P}}^{W}}(z) = {{ {P}}_ I^{W}}(z) + {{ {P}}_ E^{W}}(z) + {{ {P}}_ D^{W}}(z)\), \({{ {P}}^{F}}(z) = {{ {P}}_ I^{F}}(z) + {{ {P}}_ E^{F}}(z) + {{ {P}}_ D^{F}}(z)\) and
where \({{ {u}}_{{\mathscr{A}}}},{{ {u}}_{{\mathscr{B}}}},{{ {u}}_{{\mathscr{D}}}}\) are column vectors of dimension \({k_{{\mathscr{A}}}},{k_{{\mathscr{B}}}},{k_{{\mathscr{D}}}}\), respectively, whose elements are all ones.
Then, the distributions of counting processes \({N_ I^{W}}(n),{N_ E^{W}}(n),{N_ D^{W}}(n)\) are presented in Theorem 2.1 for the one-step increasing/equivalent/decreasing transitions in working periods together with the distribution of their summation NW(n).
Theorem 2.1
Denote the distributions of counting processes \({N_ X^{W}}(n),X \in \{I,E,D\}\) and NW(n) in working periods by
for k = 0,1,…. Then, they can be given by their Z-transforms as follows:
Proof
The proof can be presented for \({N_ I^{W}}(n),{N_ E^{W}}(n),{N_ D^{W}}(n),{N^{W}}(n)\), respectively, in (1)-(4) as follows. (1) For counting process \({N_ I^{W}}(n)\), denote the conditional probability \({\phi _{I, i}}(n,k): = P\{ {N_ I^{W}}(n) = k\left | {{Z_{0}}(0) = i} \right .\} \) by \({\phi _{I, i}^{Y}}(n,k)\) when i ∈ Y, namely, when the initial state i of semi-Markov process Z0(n) is in subset \(Y \in \{{\mathscr{A}}, {\mathscr{B}}_{1}, {\mathscr{B}}_{2},{\mathscr{D}}\}\). Specifically, for \(i \in {\mathscr{A}}\), there are four cases to consider: □
Case 1: The system stays in state i in interval [0,n], which means the first transition from state i happens at time u (u = n + 1,n + 2,…);
Case 2: The system jumps to another state j (j≠i) in subset \({\mathscr{A}}\) at time u (u = 1,…,n) from the initial state \(i \in {\mathscr{A}}\), which is a one-step equivalent transition in a working period that will not contribute to \({N_ I^{W}}(n)\);
Case 3: The system jumps to state j in subset \({\mathscr{B}}\) at time u (u = 1,…,n) from the initial state \(i \in {\mathscr{A}}\), which is a one-step decreasing transition in a working period that will not contribute to \({N_ I^{W}}(n)\);
Case 4: The system jumps to state j in subset \({\mathscr{D}}\) at time u (u = 1,…,n) from the initial state \(i \in {\mathscr{A}}\), which is also a one-step decreasing transition in a working period that will not contribute to \({N_ I^{W}}(n)\).
Then, due to the conditional independence of events, for \(i \in {\mathscr{A}}\), we have
Upon taking the Z-transform, we have
In matrix form, these formulas can be expressed as
As in the above discussion, for \(i \in {{\mathscr{B}}_{1}}\), we also have four cases to consider, and the only difference is that the transition from \(i \in {{\mathscr{B}}_{1}}\) to \(j \in {\mathscr{A}}\) will contribute to \({N_ I^{W}}(n)\). This means
Then, we obviously have
Similarly, for \(i \in {{\mathscr{B}}_{2}}\), we have
and for \(i \in {\mathscr{D}}\), we have
From all the above expressions, with \( {{\varPhi } }_{I}^{W}(z,k) = \left ({\begin {array}{*{20}{c}} { {{\varPhi } }_{I}^{{\mathscr{A}}}(z,k)}\\ { {{\varPhi } }_{I}^{{{\mathscr{B}}_{1}}}(z,k)}\\ { {{\varPhi } }_{I}^{{{\mathscr{B}}_{2}}}(z,k)}\\ { {{\varPhi } }_{I}^{{\mathscr{D}}}(z,k)} \end {array}} \right )\), we can conclude that
This means that the distribution \({\phi _ I^{W}}(n,k),k=0,1,\ldots ,\) of counting process \({{N_{I}^{W}}(n)}\) can be given by its Z-transform as
(2) The detailed discussions and associated notations for \({N_{E}^{W}}(n)\) are similar to (1), and are therefore omitted. For \(i \in {\mathscr{A}},\) based on the conditional independence of events, we have
for \(i \in {{\mathscr{B}}_{1}}\), we have
for \(i \in {{\mathscr{B}}_{2}}\), we have
for \(i \in {\mathscr{D}}\), we have
Then, similar to (1), the distribution \({\phi _ E^{W}}(n,k),k=0,1,\ldots ,\) of counting process \({{N_{E}^{W}}(n)}\) can be given by its Z-transform as
(3) The detailed discussions and associated notations for \({N_{D}^{W}}(n)\) are similar to (1), and are therefore omitted. For \(i \in {\mathscr{A}}\), based on the conditional independence of events, we have
for \(i \in {{\mathscr{B}}_{1}}\), we have
for \(i \in {{\mathscr{B}}_{2}}\), we have
for \(i \in {\mathscr{D}}\), we have
Then, similar to (1), the distribution \({\phi _ D^{W}}(n,k),k=0,1,\ldots ,\) of counting process \({{N_{D}^{W}}(n)}\) can be given by its Z-transform as
(4) The detailed discussions and associated notations for NW(n) are similar to (1), and are therefore omitted. For \(i \in {\mathscr{A}}\), based on the conditional independence of events, we have
for \(i \in {{\mathscr{B}}_{1}}\), we have
for \(i \in {{\mathscr{B}}_{2}}\), we have
for \(i \in {\mathscr{D}}\), we have
Then, similar to (1), the distribution ϕW(n,k),k = 0,1,…, of counting process NW(n) can be given by its Z-transform as
As in Theorem 2.1, the distributions of counting processes \({N_ I^{F}}(n),{N_ E^{F}}(n),{N_ D^{F}}(n)\) are presented in Theorem 2.2 for the one-step increasing/equivalent/decreasing transitions in failure periods together with the distribution of their summation NF(n).
Theorem 2.2
Denote the distributions of counting processes \({N_ X^{F}}(n),X \in \{I,E,D\}\), and NF(n) in failure periods by
for k = 0,1,…. Then, they can be given by Z-transforms as follows:
Proof
The proof is similar to Theorem 2.1, and is therefore not presented here for conciseness. □
With the results presented in Theorems 2.1 and 2.2, expectations and generating functions of these derived counting processes are derived in Theorem 2.3.
Theorem 2.3
(1) The Z-transforms of expectations of \({N_ X^{W}}(n), {N_ X^{F}}(n), X \in \{ I , E , D \}\), and NW(n),NF(n) are as follows:
where \({{ {u}}_{{{\Omega }_{0}}}} = {(\underbrace {1, {\ldots } ,1}_{{k_{{\mathscr{A}}}} + 2{k_{{\mathscr{B}}}} + {k_{{\mathscr{D}}}}})^{T}}\); (2) The Z-transforms of the generating functions of \({N_ X^{W}}(n), {N_ X^{F}}(n), X \in \{ I , E , D \}\), and NW(n),NF(n) are as follows:
Proof
We only consider \(N_{I}^{W}(n)\) since the results for other cases can be established similarly.
(1) Expectation of \(N_{I}^{W}(n)\) is \(E[N_{I}^{W}(n)] = { {\pi } {E}}[N_{I}^{W}(n)],\) where \({ {E}}[N_{I}^{W}(n)]= \sum \limits _{k = 1}^{\infty } {k {{\varPhi } }_{I}^{W}(n,k)}\), can be given by its Z-transform as follows:
(2) Generating function of \(N_{I}^{W}(n)\) is \({h_ I ^W}(n,x) = { {\pi }}{{ {h}}_ I ^W}(n,x),\) where \({{ {h}}_ I ^W}(n,x) = \sum \limits _{k = 0}^{\infty }{ {{\varPhi } }_{I}^{W}(n,k){x^k}}\), can be given by its Z-transform as follows:
Then, some joint distributions of \(N_{I}^{W}(n),{N_ E ^W}(n),{N_ D ^W}(n)\) are presented in Theorem 2.4, and similarly some joint distributions of \({N_ I ^F}(n),{N_ E ^F}(n),{N_ D ^F}(n)\) are presented in Theorem 2.5. □
Theorem 2.4
Denote the joint distribution of \(N_{I}^{W}(n),{N_ E ^W}(n),{N_ D ^W}(n)\) and the joint distributions of \({N_ X ^W}(n),{N_ Y ^W}(n), (X,Y) \in \{(I,E),(I,D),(E,D)\}\), by
for k1,k2,k3 ∈{0,1,…}. Then, they can be given by their Z-transforms as
where, for ΦW(z,k1,k2,k3), we have
and for \( {{\varPhi } }_{XY}^W(z,{k_1},{k_2})\), we have
Proof
Take ϕW(n,k1,k2,k3) as an example. Then, as in Theorem 2.1, we have
which completes the proof. □
Theorem 2.5
Denote the joint distribution of \({N_ I ^F}(n),{N_ E ^F}(n),{N_ D ^F}(n)\) and the joint distributions of \({N_ X ^F}(n),{N_ Y ^F}(n), (X,Y) \in \{(I,E),(I,D),(E,D)\}\), by
for k1,k2,k3 ∈{0,1,…}. Then, they can be given by their Z-transforms as
where, for ΦF(z,k1,k2,k3), we have
and for \( {{\varPhi } }_{XY}^F(z,{k_1},{k_2})\), we have
Proof
The proof is similar to that of Theorem 2.4, and is therefore not presented here for the sake of conciseness. □
Based on the results in Theorems 2.4 and 2.5, the joint generating functions of \(N_{I}^{W}(n)\), \({N_ E ^W}(n),{N_ D ^W}(n)\) and \({N_ I ^F}(n),{N_ E ^F}(n),{N_ D ^F}(n)\) are presented next in Theorem 2.6.
Theorem 2.6
-
(1)
The joint generating function of \(N_{I}^{W}(n),{N_ E ^W}(n),{N_ D ^W}(n)\) can be given by its Z-transform as gW∗(z,x1,x2,x3) = πgW∗(z,x1,x2,x3), where
$${{{g}}^{W*}}(z,{x_{1}},{x_{2}},{x_{3}}) = {[{{I}} - {{\boldsymbol{P}}^{F}}(z) - {{\boldsymbol{P}}_ I^{W}}(z){x_{1}} - {{\boldsymbol{P}}_ E^{W}}(z){x_{2}} - {{\boldsymbol{P}}_ D^{W} }(z){x_{3}}]^{- 1}}{\boldsymbol{P}}(z);$$ -
(2)
The joint generating function of \({N_X^W}(n),{N_Y^W}(n),(X,Y) \in \{ (I , E ),(I , D ),(E , D )\}\), can be given by its Z-transform as \({g_{XY}^{W*}}(z,{x_1},{x_2})= { {\pi }}{{ {g}}_{XY}^{W*}}(z,{x_1},{x_2})\), where
$${{g}}_{XY}^{W*}(z,{x_{1}},{x_{2}}) = {[{{I}} - {{{D}}_{XY}^{W}}(z) - {{\boldsymbol{P}}_{X}^{W}}(z){x_{1}} - {{\boldsymbol{P}}_{Y}^{W}}(z){x_{2}}]^{- 1}}{\boldsymbol{P}}(z);$$ -
(3)
The joint generating function of \({N_ I ^F}(n),{N_ E ^F}(n),{N_ D ^F}(n)\) can be given by its Z-transform as gF∗(z,x1, x2,x3) = πgF∗(z,x1,x2,x3), where
$${{{g}}^{F*}}(z,{x_{1}},{x_{2}},{x_{3}}) = {[{{I}} - {{\boldsymbol{P}}^{W}}(z) - {{\boldsymbol{P}}_ I^{F}}(z){x_{1}} - {{\boldsymbol{P}}_ E^{F}}(z){x_{2}} - {{\boldsymbol{P}}_ D^{F} }(z){x_{3}}]^{- 1}}{\boldsymbol{P}}(z);$$ -
(4)
The joint generating function of \({N_X^F}(n),{N_Y^F}(n),(X,Y) \in \{ (I , E ),(I , D ),(E , D )\}\), can be given by its Z-transform as \(g_{XY}^{F*}(z,{x_1},{x_2}) = { {\pi }}{ {g}}_{XY}^{F*}(z,{x_1},{x_2})\), where
$${{g}}_{XY}^{F*}(z,{x_{1}},{x_{2}}) = {[{{I}} - {{{D}}_{XY}^{F}}(z) - {{\boldsymbol{P}}_{X}^{F}}(z){x_{1}} - {{\boldsymbol{P}}_{Y}^{F}}(z){x_{2}}]^{- 1}}{\boldsymbol{P}}(z).$$
Proof
Take (1) as an example. Then, as in Theorem 2.3, we have
□
3 Second-Order Discrete-Time Semi-Markov System
Consider the time-homogenous second-order semi-Markov chain {Z(n),n = 0,1,…}, as in Yi and Cui (2017), with discrete time and finite state space Ω = {1,…,kΩ}. The state that the system jumps to at the k th transition time Sk is denoted by Jk (k = 1,2,…), and the system enters the initial recorded state J0 at time S0 = 0 from a known state J− 1. Let X0 = 0, and the sojourn time in state Jk− 1 between the (k − 1)th and k th transitions can be expressed as Xk = Sk − Sk− 1 (k = 1,2,…). Then, the second-order (in state) semi-Markov kernel of the related second-order (in state) semi-Markov chain can be defined as \({{ {Q}}}(n): = {({Q_{{i^{\text {*}}}j^{\text {*}}}}(n))_{{k_{{{\Omega }^2}}} \times {k_{{{\Omega }^2}}}}}\), where Ω2 = {(i,j) : i,j ∈Ω} is the state space whose elements are state pairs (Jk− 1,Jk) = (i,j) of the original chain \(\{ {J_k},k = 0,1, {\ldots } ,\infty \} \) and
With the state space Ω divided into three subsets \({\mathscr{A}},{\mathscr{B}}\) and \({\mathscr{D}}\), the second-order (in state) semi-Markov kernel can be divided correspondingly as follows:
Here again, the subsets \({\mathscr{A}},{\mathscr{B}}\) and \({\mathscr{D}}\) consist of perfect functioning states, imperfect functioning states and failure states, respectively. Besides, there are \({k_{{\mathscr{A}}}},{k_{{\mathscr{B}}}}\) and \({k_{{\mathscr{D}}}}\) states in the subsets \({\mathscr{A}},{\mathscr{B}}\) and \({\mathscr{D}}\), respectively, namely, \({\mathscr{A}} = \{ 1, {\ldots } ,{k_{{\mathscr{A}}}}\} ,{\mathscr{B}} = \{ {k_{{\mathscr{A}}}} + 1, {\ldots } ,{k_{{\mathscr{A}}}} + {k_{{\mathscr{B}}}}\} ,{\mathscr{D}} = \{ {k_{{\mathscr{A}}}} + {k_{{\mathscr{B}}}} + 1, {\ldots } ,{k_{{\mathscr{A}}}} + {k_{{\mathscr{B}}}} + {k_{{\mathscr{D}}}}\} \).
As in Section 2, sojourns in subset \({\mathscr{B}}\) contribute to working periods if and only if the system is in \({\mathscr{A}}\) right before it enters \({\mathscr{B}}\) and right after it leaves \({\mathscr{B}}\), which implies that they contribute to failure periods otherwise. Obviously, sojourns in subset \({\mathscr{A}}\) contribute to working periods and sojourns in subset \({\mathscr{D}}\) contribute to failure periods. For discussing the properties of the semi-Markov system, a new stochastic process {Y (n),n = 0,1,…} is defined as follows:
Note that Y (n) is a right-continuous function, which means its value at a transition time depends on whether the sojourn after transition contributes to a working period or not. Besides, a transition is said to be in a working period if the system is in a working period when the transition starts, and in a failure period if the system is in a failure period when transition starts.
Denote the conditional probability that the system stays in subset Y in time interval [0,n) and jumps to state j in another subset Z at time n, given the initial state i in subset X at time − 1 and state k in subset Y at time 0, where
by \(g_{ik,hj}^{XY,YZ}(n)\). This implies, for i ∈ X,k,h ∈ Y,j ∈ Z, we have
where no self-transition is allowed.
In matrix form, denote \({{{ {G}}}_{XY,YZ}}(n) = {\left ({ g_{ik,hj}^{XY,YZ}(n),i \in X,k,h \in Y,j \in Z} \right )_{{k_X}{k_Y} \times {k_Y}{k_Z}}}\), where kX,kY,kZ denote the numbers of states in subsets X,Y,Z, respectively, and denote 0 as a matrix whose elements are all 0 of suitable dimension. Then, as mentioned in Yi and Cui (2017), the Z-transform of GXY,Y Z(n) can be represented as
where I is an unit matrix of suitable dimension. As in the discussion in Section 2, we then have a new semi-Markov chain {Z0(n),n = 0,1,…} with state space \({ {\Omega } _0} = {\mathscr{A}} \cup {{\mathscr{B}}_1} \cup {{\mathscr{B}}_2} \cup {{\mathscr{D}}}\) and the semi-Markov kernel in Z-transform as
Assume that the initial probability vector of Z0(n) can be denoted by
where its element πi,k = P{Z0(− 1) = i,Z0(0) = k} is the probability that the initial state pair of Z0(n) is (i,k). To investigate the transition rules, some notations are defined for one-step increasing/equivalent/decreasing transitions in working/failure periods as follows:
Note that \({{\boldsymbol {P}}_ I ^W}(z) + {{\boldsymbol {P}}_ E ^W}(z) + {{\boldsymbol {P}}_ D ^W}(z) + {{\boldsymbol {P}}_ I ^F}(z) + {{\boldsymbol {P}}_ E ^F}(z) + {{\boldsymbol {P}}_ D ^F}(z) = {{ {D}}^{*}}(z)\). In addition, let
Further, let \({{\boldsymbol {P}}^W}(z) = {{\boldsymbol {P}}_ I ^W}(z) + {{\boldsymbol {P}}_ E ^W}(z) + {{\boldsymbol {P}}_ D ^W}(z)\), \({{\boldsymbol {P}}^F}(z) = {{\boldsymbol {P}}_ I ^F}(z) + {{\boldsymbol {P}}_ E ^F}(z) + {{\boldsymbol {P}}_ D ^F}(z)\) and
where \({{ {u}}_{XY}},X,Y \in \{ {\mathscr{A}},{\mathscr{B}},{\mathscr{D}}\},\) are column vectors of dimension kXkY whose elements are all ones.
Then, the distributions of counting processes \({ N_ I ^W}(n),{ N_ E ^W}(n),{ N_ D ^W}(n)\) are presented in Theorem 3.1 for the one-step increasing/equivalent/decreasing transitions in working periods together with the distribution of their summation NW(n).
Theorem 3.1
Denote the distributions of counting processes \({N_ X ^W}(n),X \in \{I,E,D\}\), and NW(n) in working periods by
for k = 0,1,…. Then, they can be given by their Z-transforms as follows:
Proof
Take \({\phi }_ I ^W (n,k)\) as an example. Then, the proof is similar to that of Theorem 2.1. Consider the initial state pair (Z0(− 1),Z0(0)) = (i,k), for \(i \in {\mathscr{A}}\) and \(k \in {\mathscr{A}}\). Then, we have
for \(i \in {\mathscr{A}}\) and \(k \in {{\mathscr{B}}_1},\)
for \(i \in {\mathscr{A}}\) and \(k \in {\mathscr{D}},\)
for \(i \in {{\mathscr{B}}_1}\) and \(k \in {\mathscr{A}},\)
for \(i \in {{\mathscr{B}}_1}\) and \(k \in {{\mathscr{B}}_1},\)
for \(i \in {{\mathscr{B}}_1}\) and \(k \in {\mathscr{D}},\)
for \(i \in {{\mathscr{B}}_2}\) and \(k \in {\mathscr{A}},\)
for \(i \in {{\mathscr{B}}_2}\) and \(k \in {{\mathscr{B}}_2},\)
for \(i \in {{\mathscr{B}}_2}\) and \(k \in {\mathscr{D}},\)
for \(i \in {\mathscr{D}}\) and \(k \in {\mathscr{A}},\)
for \(i \in {\mathscr{D}}\) and \(k \in {{\mathscr{B}}_2},\)
for \(i \in {\mathscr{D}}\) and \(k \in {\mathscr{D}},\)
From all the above expressions, as in the proof of Theorem 3.1, the distribution \({\phi _ I^W}(n,k),k=0,1,\ldots ,\) of counting process \({N_I ^W(n)}\) can be given by its Z-transform as
□
As in Theorem 3.1, the distributions of counting processes \({ N_ I ^F}(n),{ N_ E ^F}(n),{ N_ D ^F}(n)\) are presented in Theorem 3.2 for the one-step increasing/equivalent/decreasing transitions in failure periods together with the distribution of their summation NF(n).
Theorem 3.2
Denote the distributions of counting processes \({N_ X ^F}(n),X \in \{I,E,D\}\), and NF(n) in failure periods by
for k = 0,1,…. Then, they can be given by their Z-transforms as follows:
Proof
The proof is similar to that of Theorem 3.1, and is therefore not presented here for the sake of conciseness. □
Furthermore, other theorems in Section 2 can all be presented here in the same form, but are omitted for brevity.
4 Illustrative Eexamples
To illustrate and verify the results established in the preceding sections, two examples are presented here. In Example 4.1. we consider the case when a power generating system follows a time-homogenous first-order semi-Markov chain. To accommodate that future states of the system may depend not only on its present state, but also on its immediately preceding state, in Example 4.2, we consider the case when it follows a time-homogenous second-order semi-Markov chain.
Example 1
Consider a power generating system that follows a time-homogenous first-order semi-Markov chain {Z(n),n = 0,1,…} with state space \({\Omega } = {\mathscr{A}} \cup {\mathscr{B}} \cup {\mathscr{D}}\), where \({\mathscr{A}} = \{ 1\} \), \({\mathscr{B}} = \{ 2,3\} \) and \({\mathscr{D}} = \{ 4\} \) consist of a perfect functioning state (with a power generating capacity of 50 MW), two imperfect functioning states (with the power generating capacity of 30 MW and different functioning modes) and a failure state (with a power generating capacity of 0 MW), respectively. Assume that the system starts from state 1 at time 0 and the semi-Markov kernel can then be presented as follows:
For this system, a working period is defined as a period of time that the system is in perfect/imperfect functioning states which starts with a sojourn in the perfect functioning state and ends with a sojourn in the perfect functioning state, and a failure period is defined as a period of time between two consecutive working periods. Then, the expected numbers of one-step increasing transitions, one-step equivalent transitions, one-step decreasing transitions and all one-step transitions in working periods and in failure periods in time interval [0,n] are all plotted in Fig. 1, respectively, for n = 0,…,30. The comparisons of all one-step transitions in working periods and in failure periods are displayed in Fig. 2.
a \(N_{I}^{W}(n)\) (Curve 1), \({N_ E ^W}(n)\) (Curve 2), \({N_ D ^W}(n)\) (Curve 3), NW(n) (Curve 4) (Left); b \({N_ I ^F}(n)\) (Curve 1), \({N_ E ^F}(n)\) (Curve 2), \({N_ D ^F}(n)\) (Curve 3), NF(n) (Curve 4) (Right)
NW(n) (Curve 1), NF(n) (Curve 2), N(n) (Curve 3)
As seen in Fig. 1a, \(E[N_{I}^{W}(n)],E[{N_ E ^W}(n)],E[{N_ D ^W}(n)],E[N^W(n)]\) are all increasing functions of n. Besides, it is seen that
Similarly, in Fig. 1b, \(E[{N_ I ^F}(n)],E[{N_ E ^F}(n)],E[{N_ D ^F}(n)],E[N^F(n)]\) are also increasing functions of n, and it is seen that
In Fig. 2, E[N(n)] = E[NW(n)] + E[NF(n)] is almost a linear increasing function of n. Also, E[NW(n)] > E[NF(n)] for n = 1,…,11 and E[NW(n)] < E[NF(n)] for n = 12,13,…, which is reasonable since the system is in a working state at first.
Suppose an one-step increasing transition corresponds to 3 units of maintenance costs in a working period and 5 units of maintenance costs in a failure period, an one-step equivalent transition corresponds to no loss and no cost no matter which period it is in, and an one-step decreasing transition corresponds to 2 units of machine damage losses in a working period and 10 units of machine damage losses in a failure period. Then, the expected total costs needed in intervals [0,10), [10,20) and [20,30) can be evaluated as follows:
which means the expected total costs in interval [10,20) is a little more than in interval [20,30) and also more than in interval [0,10).
Example 2
Consider a power generating system that follows a time-homogenous second-order (in state) semi-Markov chain \(\{ Z(n),n = 0,1, \ldots , \infty \}\) with state space \( {\Omega } = {\mathscr{A}} \cup {\mathscr{B}} \cup {\mathscr{D}}\), where \({\mathscr{A}} = \{ 1\} \), \({\mathscr{B}} = \{ 2,3\} \) and \({\mathscr{D}} = \{ 4\} \) consist of a perfect functioning state (with a power generating capacity of 50 MW), two imperfect functioning states (with the power generating capacity of 30 MW and different functioning modes) and a failure state (with a power generating capacity of 0 MW), respectively. Assume that the system jumps to state 2 from state 1 at time 0 and the second-order (in state) semi-Markov kernel can then be presented as follows:
For this system, a working period is defined as a period of time that the system is in perfect/imperfect functioning states which starts with a sojourn in the perfect functioning state and ends with a sojourn in the perfect functioning state, and a failure period is defined as a period of time between two consecutive working periods. Then, the expected numbers of one-step increasing transitions, one-step equivalent transitions, one-step decreasing transitions and all one-step transitions in working periods and in failure periods in time interval [0,n] are plotted in Fig. 3, respectively, for n = 0,…,30. The comparisons of all one-step transitions in working periods and in failure periods are displayed in Fig. 4.
a \({ N_ I ^W}(n)\) (Curve 1), \({ N_ E ^W}(n)\) (Curve 2), \({ N_ D ^W}(n)\) (Curve 3), NW(n) (Curve 4) (Left); b \({ N_ I ^F}(n)\) (Curve 1), \({ N_ E ^F}(n)\) (Curve 2), \({ N_ D ^F}(n)\) (Curve 3), NF(n) (Curve 4) (Right)
NW(n) (Curve 1), NF(n) (Curve 2), N(n) (Curve 3)
As seen in Fig. 3, \(E[{ N_ I ^W}(n)],E[{ N_ E ^W}(n)],E[{ N_ D ^W}(n)],E[ N^W(n)]\) are all increasing functions of n, and \(E[{ N_ I ^F}(n)],E[{ N_ E ^F}(n)],E[{ N_ D ^F}(n)],E[ N^F(n)]\) are also increasing functions of n. Besides, it is seen that
In Fig. 4, E[N(n)] = E[NW(n)] + E[NF(n)] is almost a linear increasing function of n. Also, we have E[NW(n)] > E[NF(n)] for n = 1,…,16 and E[NW(n)] < E[NF(n)] for n = 17,18,…, which is reasonable since the system is in a working period in the beginning.
Suppose an one-step increasing transition corresponds to 3 units of maintenance costs in a working period and 5 units of maintenance costs in a failure period, an one-step equivalent transition corresponds to no loss and no cost no matter which period it is in, and an one-step decreasing transition corresponds to 2 units of machine damage losses in a working period and 10 units of machine damage losses in a failure period. Then, the expected total costs needed in intervals [0,10), [10,20) and [20,30) can be evaluated as follows:
which means the expected total costs needed in interval [20,30) is a little more than in interval [10,20) and also more than in interval [0,10).
5 Extended Discussions on Related Reliability Measures
The key step in Sections 2 and 3 is to define a new first/second-order semi-Markov chain {Z0(n),n = 0,1,…} based on the orginal first-/second-order semi-Markov chain {Z(n),n = 0,1,…} to distinguish whether the system is in a working period or a failure period. In this section, some extended discussions are presented to obtain reliability measures for the two semi-Markov systems in Sections 2 and 3.
As in Yi et al. (2018) and Yi and Cui (2017) , reliability measures such as reliability, point availability and interval availability can be definded for the first-/second-order semi-Markov system in Sections 2 and 3 by correspongding stochastic process {Y (n),n = 0,1,…} as follows:
-
(1)
Reliability: the probability that the system is always in working periods before time n, namely,
$$R(n) = P\{Y(1) = {\dots} =Y(n) = 1\};$$ -
(2)
Point availability: the probability that the system is in a working period at time n, namely,
$$A(n) = P\{Y(n) = 1\};$$ -
(3)
Interval availability: the probability that the system is always in working periods in time interval [a,b], namely,
$$A[a,b] = P\{Y(a) = {\dots} = Y(b)= 1\};$$
In existing research like Yi et al. (2018) and Yi and Cui (2017), explicit formulas of these reliability measures have been given by an analysis of possible transitions. Here, with the new first-/second-order semi-Markov chain {Z0(n),n = 0,1,…} defined in Sections 2 and 3, they can be presented in a new and simple way. Assume that the first-/second-order semi-Markov system is at the beginning of a working period at time 0. Then, smiliar to the discussions in Yi et al. (2018), reliability R(n) of the system can be given by its Z-transform as
with \({ {\pi }}, {{ {u}}_{{\Omega _0}}}, {\boldsymbol {P}}^W(z), {\boldsymbol {P}}^F(z)\) as defined in Sections 2 and 3 and
Point availability A(n) of the original system can be given by its Z-transform as
where \({ {G}}_{FW}^{*}(z) = {[{ {I}} - {\boldsymbol {P}}^F(z)]^{- 1}}{\boldsymbol {P}}^W(z)\) and
Interval availability A[a,b] of the original system can be given in a recursive way as A[a,b] = πA[a,b], where
and fWF(u) can be given by its Z-transform as \({ {f}}_{WF}^{*}(u) = { {G}}_{WF}^{*}(z){ {G}}_{FW}^{*}(z)\).
With the disucssions in this section and in Sections 2 and 3, some optimization problems can be considered to obtain best reliability performance under constraints like costs and benifits of transitions, which may be useful in practice. Note that this method can also be applied to other reliability measures in a similar way, and definitions of working/failure periods can also be considered in different ways by changing the split of state spaces or by taking issues such as time interval omission into consideration.
6 Conclusions
In this paper, we have defined several kinds of derived counting processes for first- and second-order discrete-time aggregated semi-Markov systems; for example, one-step increasing/equivalent/decreasing transitions in working/failure periods. Stochastic properties of these counting process are studied, and related distributions, joint distributions, expectations, generating functions and joint generation functions are derived and presented in their Z-transforms. Some numerical examples are finally presented to illustrate the established results. Extended discussions on related reliability measures have also been made. Applications of the results developed in this work can be found in seismology, reliability, biology, finance and others fields. In the future, some other indices and their properties can be considered for semi-Markov systems, and further results can be presented for new high-order models as well.
Data Availability
No datasets were generated or analysed during the current study.
References
Badía FG, Sangüesa C (2017) Negative ageing properties for counting processes arising in virtual age models. Appl Math Model 40:5271–5282
Ball F (1997) Empirical clustering of bursts of openings in Markov and semi-Markov models of single channel gating incorporating time interval omission. Adv Appl Probab 29:909–946
Barbu V, Boussemart M, Limnios N (2004) Discrete-time semi-Markov model for reliability and survival analysis. Commun Stat Simul Comput 33:2833–2868
Barbu VS, Limnios N (2008) Semi-Markov Chains and Hidden Semi-Markov Models towards Applications: Their Use in Reliability and DNA Analysis. Springer, New York
Barbu VS, Vergne N (2019) Reliability and survival analysis for drifting Markov models: modeling and estimation. Methodol Comput Appl Probab 21:1407–1429
Billinton R, Allan R (1996) Reliability evaluation of power systems. Plenum, New York
Burke CJ, Rosenblatt M (1957) A Markovian function of a Markov chain. Ann Math Stat 29:1112–1122
Colquhoun D, Hawkes AG (1977) Relaxation and fluctuations of membrane currents that flow through drug-operated channels. Proc R Soc Lond B 199:231–262
Colquhoun D, Hawkes AG (1981) On the stochastic properties of single ion channels. Proc R Soc Lond B 211:205–235
Colquhoun D, Hawkes AG (1982) On the stochastic properties of bursts of single ion channel openings and of clusters of bursts. Philos Trans R Soc Lond B 300:1–59
Cui LR, Li HJ, Li JL (2007) Markov repairable systems with history-dependent up and down states. Stoch Models 23:665–681
Cui LR, Li Y, Shen JY, Lin C (2016) Reliability for discrete state systems with cyclic missions period. Appl Math Model 40:10783–10799
Cui LR, Yi H, Balakrishnan N (2021) On derivative counting processes of a finite state Markov process, under review
Csenki A (1995) The number of visits to a subset of the state space by a discrete-parameter semi-Markov process. Stat Probab Lett 22:71–77
D’Amico G, Petroni F, Prattico F (2013) First and second order semi-Markov chains for wind speed modeling. Phys A 392:1194–1201
D’Amico G, Petroni F, Prattico F (2015a) Performance analysis of second order semi-Markov chains: an application to wind energy production. Methodol Comput Appl Probab 17:781–794
D’Amico G, Petroni F, Prattico F (2015b) Reliability measures for indexed semi-Markov chains applied to wind energy production. Reliab Eng Syst Saf 144:170–177
Fang C, Cui LR (2021) Balanced systems by considering multi-state competing risks under degradation processes. Reliab Eng Syst Saf 205:107252
Hawkes AG, Jalali A, Colquhoun D (1990) The distributions of the apparent open times and shut times in a single channel record when brief events cannot be detected. Phil Trans R Soc Lond A 332:511–538
Hawkes AG, Cui LR, Zheng ZH (2011) Modeling the evolution of system reliability performance under alternative environments. IIE Trans 43:761–772
Howard RA (1964) System analysis of semi-Markov processes. IEEE Trans Military Electron 8:114–124
Howard RA (1971) Dynamic Probabilistic Systems. Volume II: Semi-Markov and Decision Processes. Springer, New York
Janssen J, Manca R (2006) Applied Semi-Markov processes. Springer, New York
Landriault D, Li B, Zhang HZ (2017) A unified approach for drawdown (drawup) of time-homogeneous Markov processes. J Appl Prob 54:603–626
Limnios N, Oprisan G (2003) An introduction to semi-Markov processes with application to reliability. In: Balakrishnan N, Rao CR (eds) Handbook of Statistics, vol 21, Amsterdan, pp 515–556
Lisnianski A, Frenkel I, Ding Y (2010) Multi-state System Reliability Analysis and Optimization for Engineers and Industrial Managers. Springer, London
Lisnianski A, Frenkel I (2011) Recent Advances in System Reliability: Signatures, Multi-state Systems and Statistical Inference. Springer, London
Liu BL, Cui LR, Wen YQ (2014) Interval reliability for aggregated Markov repairable system with repair time omission. Ann Oper Res 212:169–183
Michael BV, Eutichia V (2020) On the distribution of the number of success runs in a continuous time Markov chain. Methodol Comput Appl Probab 22:969–993
Mode CJ, Pickens GT (1988) Computaional methods for renewal theory and semi-Markov processes with illustrative examples. Stat Comput 42:143–152
Natvig B (2011) Multistate systems reliability theory with applications. Wiley, Chichester
Nuel G (2019) Moments of the count of a regular expression in a heterogeneous random sequence. Methodol Comput Appl Probab 21:875–887
Rubino G, Sericola B (1989) Sojourn times in finite Markov processes. J Appl Probab 26:747–756
The YK, Timmer J (2006) Analysis of single ion channel data incorporating time-interval omission and sampling. J R Soc Interface 3:87–97
Votsi I, Limnios N, Tsaklidis G, Papadimitriou E (2012) Estimation of the expected number of earthquake occurrences based on semi-markov models. Methodol Comput Appl Probab 14:685–703
Wu B, Cui LR, Fang C (2019) Reliability analysis of semi-Markov systems with restriction on transition times. Reliab Eng Syst Saf 190:106516
Wu B, Cui LR, Fang C (2020) Generalized phase-type distributions based on multi-state systems. IISE Trans 52:104–109
Yi H, Cui LR (2017) Distribution and availability for aggregated second-order semi-Markov ternary system with working time omission. Reliab Eng Syst Saf 166:50–60
Yi H, Cui LR, Shen JY, Li Y (2018) Stochastic properties and reliability measures of discrete-time semi-Markovian systems. Reliab Eng Syst Saf 176:162–173
Yi H, Cui LR, Shen JY (2018) Multipoint and multi-interval covering availabilities. IEEE Trans Rel 67:666–677
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This work is supported by the National Natural Science Foundation of China (No. 72001016 and No. 71631001).
Rights and permissions
About this article
Cite this article
Yi, H., Cui, L. & Balakrishnan, N. On the Derivative Counting Processes of First- and Second-order Aggregated Semi-Markov Systems. Methodol Comput Appl Probab 24, 1849–1875 (2022). https://doi.org/10.1007/s11009-021-09896-0
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11009-021-09896-0