1 Introduction

Semi-Markov processes have received extensive attention recently in that it is revealed as a very efficient tool for many applied science and engineering problems. As an extension of Markov processes and renewal processes, the sojourn time of the semi-Markov process can be any distribution, which provides great flexibility in modelling the real problems. Many applications can be found on queuing theory problems (Afanasyeva et al. 2012), reliability and survival analysis (Barbu and Limnios 2008), biological sequences (Barbu and Limnios 2008; Garcia-Maya and Limnios 2019), finance and insurance (Petersson 2016) and so on. The present paper concentrates on the discrete-time semi-Markov process, i.e., the semi-Markov chain.

Since Anselone (1960) first introduced the semi-Markov chain and Howard (1971) further refined its description, more and more researchers have been attracted and dedicated to this topic. Stenflo (1996) presented the periodicity concept of the semi-Markov chain. Mode and Pickens (1998) investigated the computational methods for the renewal theory of the semi-Markov chain and applied it to biology. Barbu et al. (2004) presented an efficient algorithm for solving the Markov renewal equation which is the basic tool in semi-Markov chain theory and applications. Girardin and Limnios (2006, 2018) presented entropy rate calculation of semi-Markov chains. Barbu and Limnios (2006), Barbu and Limnios (2008) presented estimation results with asymptotic properties of semi-Markov chains, and the theory and algorithms of hidden semi-Markov chains. Properties of the maximum likelihood estimator of hidden semi-Markov chains were analyzed by Trevezas and Limnios (1979), and Malefaki et al. (2010). Limnios (2011) and Limnios and Swishchuk (2013), Limnios and Swishchuk (2020) presented the asymptotic theory of random evolution in semi-Markov chain media.

One of the most important characteristics in the ergodic processes is their stationary probability measure. It is a significant problem to estimate the stationary distribution of a semi-Markov chain, because in applied sciences and engineering, we are usually interested in the equilibrium behavior of a system or device, which can be reached after a more or less long run, depending on some spectral properties of the transition operators.

Hence, to estimate the stationary distribution of a semi-Markov chain and find an estimator with asymptotic properties is meaningful and essential. This is the problem we face in this paper, i.e., how to estimate the stationary probability given an observation that consists of a trajectory of a semi-Markov chain in a time interval. We propose an empirical estimator and obtain its asymptotic properties in a functional setting.

Nevertheless, the stationary distribution of a semi-Markov process, in discrete or continuous-time, cannot be defined as in the Markov case, that is, \(\pi A=0\), where \(\pi \) is the stationary probability of the Markov process and A its generator, since we do not have a semigroup. The stationary probability of a semi-Markov process is defined either as a limit of the transition function of the process, or as a marginal distribution of a coupled process, e.g., the semi-Markov process together with the backward recurrence time process.

The stationary distribution for continuous-time semi-Markov processes is studied in the case of finite-state space case in Limnios et al. (2005) and general-state space in Limnios (2006) respectively. However, the present paper is the first to consider the estimation of the stationary distribution for semi-Markov chains in the case of general state space. The proof method used here, for consistency and asymptotic normality, is based on the random evolution theory. The law of iterated logarithm is based on results presented in Herkenrath et al. (2003).

Next Sect. 2, presents the semi-Markov chain setting. Section 3 defines the proposed estimator and the asymptotic results. Section 4 presents the proofs, and finally Sect. 5 presents some concluding remarks.

2 Stationary probability of semi-Markov chains

Semi-Markov chain. Let us denote by \(\mathbb {N}:= \{0,1,2,...\}\) the set of natural numbers, and a complete probability space with countably generated \(\sigma \)-algebra, say \((\Omega , \mathcal{F}, \mathbf {P})\), on which we define all stochastic processes and random variables consider here. It is worth noticing here that \(k\in \mathbb {N}\) denotes the calendar time, while \(n\in \mathbb {N}\) denotes the number of jumps. Let us consider a semi-Markov chain \((Z_k, k\in \mathbb {N})\), with state space a measurable space \((E,\mathcal{E})\). Let us also consider the embedded Markov renewal chain \((J_n, S_n, n\in \mathbb {N})\), where \(J_n\) is the embedded Markov chain, with state space \((E,\mathcal{E})\), and \(0=S_0<S_1<...<S_n<...\) the successive integer valued jump times, and the semi-Markov kernel q(x, B, k), \(x\in E\), \(B\in \mathcal{E}\), \(k\in \mathbb {N}\), that is,

$$\begin{aligned} q(x,B,k):= \int _B P(x,dy)f(x,y,k),\quad k>0,\quad \hbox {and}\quad q(x,B,0)=0, \end{aligned}$$

where \(P(x,B):= \mathbf {P}(J_{n+1}\in B\mid J_n=x)\) is the transition kernel of the EMC J, and \(f(x,y,k):= \mathbf {P}(S_{n+1}-S_n=k\mid J_n=x,J_{n+1}=y)\) the conditional distribution of the sojourn time in state x when the next visited state is y.

The process \((N_k, k\in \mathbb {N})\), counts the jumps of the process up to time k, that is, \(N_k = \sup \{n : S_n\le k\}\). The processes \((Z_k, k\in \mathbb {N})\) and \((J_n, n\in \mathbb {N})\) are connected by the following relations:

$$\begin{aligned} Z_k = J_{N_k},\quad k\in \mathbb {N}, \qquad \hbox {and}\qquad J_n= Z_{S_n},\quad n\in \mathbb {N}. \end{aligned}$$

Let us define also the survival function of the time the SMC Z spends in a state \(x\in E\), \(\bar{F}_x(k) := 1 - \sum _{\ell \le k} q(x,E,\ell )\), and the mean sojourn time in state \(x\in E\), that is, \(m(x):= \sum _{k\ge 0}\bar{F}_x(k)\). Denote also by \(f_x(\ell ):= q(x,E,\ell )\) the law of unconditional sojourn time in state \(x\in E\).

Backward recurrence time process. Consider now the backward recurrence time \(U_k := k - S_{N_k}\), \(k\ge 0\), and the coupled process \((Z_k,U_k)\), which is a Markov chain, with transition kernel operator \(P^\sharp \), see (Limnios and Swishchuk 2020), defined by acting on measurable and bounded test functions \(\varphi (x,k)\), can be written as follows,

$$\begin{aligned} P^\sharp \varphi (x,k) = \varphi (x,k+1) + \lambda (x,k)[P\varphi (x,0) - \varphi (x,k+1)] \end{aligned}$$
(1)

where P is the transition kernel operator of the embedded Markov chain J, and \(\lambda (x,k):= \mathbf {P}_x(S_1 =k \mid S_1\ge k)\), the discrete time transition rate of the sojourn time in state \(x\in E\).

It is worth noticing that operator \(P^\sharp \), defined in (1), acting on test functions \(\varphi (x)\), can be written as

$$\begin{aligned} P^\sharp = I + \lambda (x,\cdot )(P-I) \end{aligned}$$

where I is the identity operator.

Markov Renewal Equation. The Markov renewal equation plays a crucial role, and it is defined as follows

$$\begin{aligned} u(x,k) = v(x,k) + \sum _{\ell =0}^k \int _E q(x,dy,\ell )u(y,k-\ell ), \quad x\in E, k\ge 0, \end{aligned}$$
(2)

where u is an unknown function, and v a given function, both defined on \(E\times \mathbb {N}\).

For a semi-Markov kernel q, we define the successive convolution powers, by induction, as follows

$$\begin{aligned} q^{(n+1)}(x,B,k)= \sum _{\ell =0}^k \int _E q(x,dy,\ell ) q^{(n)}(y,B,k-\ell ),\quad n\ge 1, \end{aligned}$$

and

$$\begin{aligned} q^{(1)}=q,\quad q^{(0)}(x,B,k)= \left\{ \begin{array}{lll} 0 &{} \hbox {if} &{} k\le 0\\ \mathbf{1}_B(x) &{} \hbox {if} &{} k>0 \end{array} \right. \end{aligned}$$

where \(\mathbf{1}_B\) is the indicator function of the set B.

Markov Renewal Theorem. First define the Markov renewal functions as follows

$$\begin{aligned} \psi (x,B,k):=\sum _{n=0}^k q^{(n)}(x,B,k). \end{aligned}$$

Let us give now the following important result, which is Shurenkov’s Markov renewal theorem (Shurenkov 1984) restated for discrete-time semi-Markov processes.

The following assumptions are needed in the sequel.

  • A1: The Markov chain \((J_n)\) is ergodic with stationary distribution \(\rho (B), B\in \mathcal{E}\).

  • A2: \(0< m < \infty \), with \(m := \int _E \rho (dx)m(x)\).

Theorem 1

(Markov Renewal Theorem). Under assumptions A1-A2, and the additional assumption that the function v(x, k) satisfies \(\int _E \rho (dx) \sum _{k\ge 0}\left| v(x,k) \right| < \infty \), the MRE (2) has a unique solution, given by \(u(x,k)=\psi * v(x,k)\), and

$$\begin{aligned} \lim _{k\rightarrow \infty } \psi * v(x,k) = \frac{1}{m} \int _E \rho (dx) \sum _{k\ge 0}v(x,k). \end{aligned}$$

Limiting and stationary probability. Let us apply this theorem for the transition function of the semi-Markov chain calculation. The transition function of the semi-Markov process is defined as follows

$$\begin{aligned} P_k(x,B) := \mathbf {P}(Z_k\in B \mid Z_0=x), \end{aligned}$$

and satisfies the following Markov renewal equation

$$\begin{aligned} P_k(x,B) = \mathbf{1}_B(x)\overline{F}_x(k) + \sum _{\ell =0}^k \int _E q(x,dy,\ell )P_{k-\ell }(y,B). \end{aligned}$$

Then, by the above Markov renewal theorem, we obtain the solution,

$$\begin{aligned} P_k(x,B)= (\psi * \mathbf{1}_B\overline{F})(x,k) \end{aligned}$$

and the limit

$$\begin{aligned} \lim _{k\rightarrow \infty }P_k(x,B) = \frac{1}{m}\int _B \rho (dx)m(x)=: \pi (B), \quad \hbox {for any}\quad x\in E, B\in \mathcal{E}. \end{aligned}$$

The limit distribution \(\pi (B)\) is said to be the stationary distribution of the semi-Markov chain \(Z_k\).

Another way to define it, is to consider \(\pi ^\sharp \) the stationary distribution of the Markov process \((Z_k,U_k)\), where \(U_k= k-S_{N_k}\), the backward recurrence time of the SMC, that is

$$\begin{aligned} \pi ^\sharp (dx\times \{\ell \})= \rho (dx)\overline{F}_x(\ell )/m. \end{aligned}$$

The stationary distribution \(\pi \) of Z, is now defined as the marginal distribution of \(\pi ^\sharp \), that is,

$$\begin{aligned} \pi (B) := \pi ^\sharp (B\times \mathbb {N})= \int _B\rho (dx)m(x)/m. \end{aligned}$$

3 Estimation of stationary probability

Let us observe the semi-Markov chain Z on the time interval \([0,k]\subset \mathbb {N}\), that is the censured trajectory \((Z_\ell (\omega ), 0\le \ell \le k)\). Based on these data, we propose the following empirical estimator for the stationary probability of the SMC, \(\pi \), as follows

$$\begin{aligned} \hat{\pi }_k(B) := \frac{1}{k}\sum _{\ell =1}^k \mathbf{1}_B(Z_\ell ), \quad k> 0, \end{aligned}$$
(3)

for any \(B\in \mathcal{E}\).

Convergence results considered here are in the weak functional sense in the space \(D_E[0,\infty )\) with Skorohod topology, denoted by \(\Rightarrow \). The following results are functional asymptotic properties of the estimator (3), namely, consistency, asymptotic normality, and convergence rate results.

In fact \(\hat{\pi }_{[t/\varepsilon ]}(B)\) and \(\hat{\pi }_{[t/\varepsilon ^2]}(B)\), \(t\ge 0\), \(\varepsilon >0\), where the [x] denotes the integer part of the real number x, are families of stochastic processes in continuous time \(t>0\).

Let us denote by \(L_0^2(\pi )\) the subspace of the centered second order elements of \(L^2(\pi )\), that is, \(v\in L_0^2(\pi )\) implies that \(\int _E v^2(x)\pi (dx) <\infty \) and \(\int _E v(x)\pi (dx) =0\).

Proposition 1

(Consistency). Under assumptions A1-A2, we have, for any fixed \(B\in \mathcal{E}\),

$$\begin{aligned} \hat{\pi }_{[t/\varepsilon ]}(B) \Longrightarrow \pi (B),\quad \varepsilon \rightarrow 0 \end{aligned}$$

for \(t > 0\). And, we can also state this result in the following convergence in probability,

$$\begin{aligned} \lim _{\varepsilon \downarrow 0}\sup _{0 < t\le T}\left| \hat{\pi }_{[t/\varepsilon ]}(B) - \pi (B) \right| =0, \end{aligned}$$

for any but fixed \(T\in \mathbb {R}_+\).

Let us now state the following assumption, (see, e.g., Limnios and Swishchuk 2013; Koroliuk and Limnios 2005).

  • C1: We assume moreover the following uniform integrability condition

    $$\begin{aligned} \lim _{M\rightarrow \infty }\sup _{x\in E} \sum _{k\ge M}k^2 {f}_x(k) = 0 \end{aligned}$$

    which, of course, implies

    $$\begin{aligned} \sup _{x\in E} \sum _{k\ge 1}k^2 {f}_x(k)< \infty . \end{aligned}$$

This assumption is needed for tightness of the semi-Markov chains.

For notational simplicity, let us define the process

$$\begin{aligned} \xi _t^\varepsilon := \varepsilon ^{-1}t[\hat{\pi }_t^\varepsilon (B) - \pi (B)]:= \varepsilon ^{-1}t[\hat{\pi }_{[t/\varepsilon ^2]}(B) - \pi (B)] \end{aligned}$$

for any but fixed \(B\in \mathcal{E}\) and \(\xi _0^\varepsilon =0\) for any \(\varepsilon >0\).

Proposition 2

(Asymptotic normality). Under assumptions A1-A2 and C1, we have

$$\begin{aligned} \xi _t^\varepsilon \Longrightarrow b B_t, \quad \varepsilon \rightarrow 0, \end{aligned}$$

where \(B_t\), \(t\ge 0\), is the standard Brownian motion, and

$$\begin{aligned} b^2:=b^2(B) := 2 \int _B \pi (dx)R_0(x,B) - \pi (B)+ \pi ^{2}(B)-2\pi (B) \int \limits _{E}n (dx)R_{0}(x,B), \end{aligned}$$

and \(0< b< \infty \), where \(R_0\) is the potential operator of the discrete generator \(P^\sharp -I\), and I is the identity operator.

Remark 3.1

In the weak convergence scheme, the initial value, \(\xi ^\varepsilon _0\), must satisfy the following two conditions, (see, e.g., Koroliuk and Limnios 2005, p. 93),

$$\begin{aligned} \xi ^\varepsilon _0 {\mathop {\longrightarrow }\limits ^{\mathbf {P}}} 0, \qquad \hbox {and}\quad \mathbf {E}\left| \xi _0^\varepsilon \right| \le C < \infty . \end{aligned}$$
(4)

In our case, both conditions are trivially satisfied. We could also consider \(\xi _0^\varepsilon = \varepsilon r\), for a constant r.

It is worth noticing that from the above Proposition 2, for an observation time of order \(\varepsilon ^{-2}\), we have approximately

$$\begin{aligned} \hat{\pi }_{1}^\varepsilon (B) \sim N(\pi (B) ; \varepsilon ^2b^2). \end{aligned}$$

Let us now define K, a subset of C[0, 1], the space of continuous functions on [0, 1], with sup-norm, \(\left\| \cdot \right\| \), as follows

$$\begin{aligned} K:= \{x\in C[0,1] : \, x\, \hbox {is absolutely continuous},\, x(0)=0, \int _0^1 (\dot{x}(t))^2 dt \le 1\} \end{aligned}$$

where \(\dot{x}(t)\) denotes the derivative of the function x with respect to t.

Let us suppose that the time \(k=0\) is not necessary a jump time, that is, we define \(S_0 \le 0< S_1<S_2< \cdots<S_n < \cdots \), with \(U_0= - S_0 \ge 0\). In that case, we have a slight modification of the definition of \(U_k\), for \(k< S_1\), as follows

$$\begin{aligned} U_k = \left\{ \begin{array}{ll} U_0 + k, &{} k < S_1\\ k- S_{N_k}, &{} k\ge S_1 \end{array} \right. \end{aligned}$$

Moreover we suppose that the process \((Z_k,U_k)\) is a stationary Markov chain, that is, its initial distribution is equal to its stationary distribution \(\pi ^\sharp (dx\times \{\ell \})= \rho (dx)\overline{F}_x(\ell )/m\), defined previously.

The following assumption is needed in the sequel.

  • C2: Denote \(\left\| \cdot \right\| _2\) the usual norm of the space \(L^2(\pi ^\sharp )\). We assume that

    $$\begin{aligned} \sum _{n\ge 1}\left\| (P^\sharp )^{n}v \right\| _2 < \infty , \quad v\in L^2_0(\pi ^\sharp ) \end{aligned}$$

    where the operator \(P^\sharp \) is defined in (1).

For example, for an one state semi-Markov chain (i.e., a renewal process in discrete-time, Barbu and Limnios 2008), the above condition is trivially satisfied since the first member of the relation is bounded by mv where m is here the mean value of the distribution f.

Proposition 3

(Law of the Iterated Logarithms). Under assumptions A1-A2 and C1-C2, and the additional condition that the process (Z, U) is stationary, the limit points of the following family of processes, as \(\varepsilon \downarrow 0\), \((0<\varepsilon <1)\),

$$\begin{aligned} \frac{t \hat{\pi }_{[t/\varepsilon ]}(B)}{ b \sqrt{2\varepsilon \log (-\log (\varepsilon ))}} \end{aligned}$$

are exactly the set K, \(\mathbf {P}\)-a.s.

As a divergence result, it means that the convergence of the estimator \(\hat{\pi }_{[t/\varepsilon ]}(B)\), in the worst case, is of order

$$\begin{aligned} O((2 \varepsilon \log (-\log (\varepsilon )))^{-1/2}). \end{aligned}$$

Proposition 4

(Rates of convergence). The rates of convergence, for consistency and asymptotic normality, are given by

$$\begin{aligned} \left\| \mathbf {E}\hat{\pi }_{[t/\varepsilon ^r]}(B) - \pi (B) \right\| \le \varepsilon C_r \end{aligned}$$

for \(0\le t \le T\), where \(C_r:= C_r(T, \left\| R_0 \right\| )\), with \(r=1,2\), are two positive constants.

The above result can be also stated, for both cases, as

$$\begin{aligned} \left\| \mathbf {E}\hat{\pi }_{[t/\varepsilon ^r]}(B) - \pi (B) \right\| = O(\varepsilon ). \end{aligned}$$

And, from the above, we can easily obtain that the bias of estimator (3), goes to zero with a rate O(1/k), that is, as \(k\rightarrow \infty \),

$$\begin{aligned} \left\| \mathbf {E}\hat{\pi }_{t}(B) - \pi (B) \right\| \le C_1/k. \end{aligned}$$

Remark 3.2

It is worth noticing here that all the above results can be also obtained by considering the stochastic functional \(\beta _\varepsilon (t) := \alpha _\varepsilon (t) + \varepsilon b^{-1}(t/\varepsilon ^2 - [t/\varepsilon ^2])a(Z_{[t/\varepsilon ^2]+1})\) in the space \(C[0,\infty )\) of the continuous functions equipped with the uniform topology, instead of \(\alpha _\varepsilon (t)\). In fact, since \(a(\cdot )\) is a bounded function, we have, as \(\varepsilon \rightarrow 0\), \(\varepsilon (t/\varepsilon ^2 - [t/\varepsilon ^2])a(Z_{[t/\varepsilon ^2]+1}) \rightarrow 0\), a.s.

4 Proofs

In this section we will prove the results presented in the previous section 3. In order to prove the results for Proposition 1, 2 and 4, we are using results in paper Limnios and Swishchuk (2013), in particular Result 4.2 and Corollary 5.1. For the asymptotic normality in the stationary case, see also (Herkenrath et al. 2003), Theorem 1. Here in fact we don’t need the process (Z, U) to be stationary, while in Proposition 3, we consider that this process is stationary and we use results from (Herkenrath et al. 2003). For Theorem 1, in Herkenrath et al. (2003), see also (Heyde and Scott 1973; Scott 1973). See also Anisimov (2008), Gikhman and Skorohod (1974), Girardin and Limnios (2018), Shurenkov (1984), Stroock and Varadhan (1979).

Let us now define the projection operator \(\Pi \) of transition operator \(P^\sharp \) by

$$\begin{aligned} \Pi \varphi (x,k) := \sum _{s\ge 0}\int _E \pi (dy) \bar{F}_y(s)\varphi (y,s)/m. \end{aligned}$$

Define also the potential operator \(R_0\) of \(P^\sharp -I\) by

$$\begin{aligned} R_0= (I- P^\sharp + \Pi )^{-1} - \Pi \end{aligned}$$

where \(P^\sharp \) is the transition kernel operator of the MC \((Z_k, U_k)\), as defined by (1).

Proof of Proposition 1

Consider an additive functional of the semi-Markov chain, defined as follows,

$$\begin{aligned} Y_{k} := \sum _{\ell =1}^k a(Z_{\ell }),\quad k\ge 0,\quad \hbox {and} \quad Y_0 = u, \end{aligned}$$

where the function \(a: E \longrightarrow \mathbb {R}\), is measurable and bounded.

Consider the additive functional in the series scheme, \(Y_t^\varepsilon \),

$$\begin{aligned} Y_t^\varepsilon = \varepsilon \sum _{\ell =1}^{[t/\varepsilon ]} a(Z_{\ell }),\quad t\ge 0, \end{aligned}$$

and \(Y_0^\varepsilon \) satisfying conditions (4), with limit say u, then we have the following averaging result, see (Limnios and Swishchuk 2013),

$$\begin{aligned} Y_t^\varepsilon \Rightarrow u + \hat{a}t,\quad \varepsilon \rightarrow 0, \end{aligned}$$

where \(\hat{a}= \int _E \pi (dx)a(x)\).

Now from the above result, for \(a:= \mathbf{1}_B\), we get directly the proof of the first part of Proposition 1.

The second part of this proposition is obtained directly since the above limit is a constant (see, e.g., Billingsley 1968). \(\square \)

Proof of Proposition 2

Consider now the continuous time processes \(\xi _t^\varepsilon \), defined as follows

$$\begin{aligned} \xi _t^\varepsilon = \varepsilon \sum _{\ell =1}^{[t/\varepsilon ^2]} a(Z_{\ell }),\quad t\ge 0, \end{aligned}$$

and \(\xi _0^\varepsilon \) satisfy conditions (4), with limit initial point u.

Then for \(a\in L_0(\pi )\), which is equivalent to the balance condition, \(\Pi a=0\), in Limnios and Swishchuk (2013), Result 4.2, we have,

$$\begin{aligned} \xi _t^\varepsilon \Rightarrow u + bB_t,\quad \varepsilon \rightarrow 0, \end{aligned}$$

where \(b^2 = 2\overline{a}_0 -\hat{a}_2 \), with

$$\begin{aligned} \overline{a}_0=\int _E \pi (dx)a(x)R_0a(x),\qquad \hat{a}_2= \int _E \pi (dx)a^2(x), \end{aligned}$$

and \(B_t, t\ge 0\), is a standard Brownian motion.

Now putting \(a(x):= \mathbf{1}_B(x) - \hat{a}\), \(x\in E\), which obviously satisfy the balance condition \(\Pi a =0\), we get the desired result. \(\square \)

Proof of Proposition 3

Law of the Iterated Logarithms.

Define

$$\begin{aligned} \sigma ^2_a := \lim _{k\rightarrow \infty } \frac{1}{k} \mathbf {E}(S_k^2) \end{aligned}$$

with \(a_k:= a(Z_k)\) and \(S_k:= a_1+...+a_k\).

Let us now consider the processes

$$\begin{aligned} \alpha _\varepsilon (t):= \varepsilon \sum _{\ell =1}^{[t/\varepsilon ^2]} a(Z_\ell ),\qquad a\in L_0^2(\pi ) \end{aligned}$$

By comparison of Theorem 1, in Herkenrath et al. (2003), and the above Proposition 2, we have that \(\sigma _a=b\). Under assumptions A1-A2, C1-C2 and \(\sigma ^2_a>0\), and from Herkenrath et al. (2003), Theorem 1, (see also Heyde and Scott 1973; Scott 1973), we have that the sequence

$$\begin{aligned} \frac{\alpha _\varepsilon (t)}{b\sqrt{2 \log (-\log \varepsilon )}}, \quad 0<\varepsilon <1 \end{aligned}$$

as a subset of C[0, 1] is \(\mathbf {P}_{\pi ^\sharp }-\)a.s. relative compact (in the uniform topology) and the set of its limit points is exactly K.

Now by taking \(a= \mathbf{1}_B - \hat{a}\), we conclude the proof of Proposition 3. \(\square \)

Proof of Proposition 4

This is an easy consequence of Corollary 5.1 in Limnios and Swishchuk (2013). \(\square \)

5 Concluding remarks

The finite or countable state spaces can be deduced directly as particular cases of the present one. For the stationary case, the estimator (3) can be considered in any interval, that is, for any fixed integer \(c\in \mathbb {N}\),

$$\begin{aligned} \hat{\pi }_k(B) := \frac{1}{k}\sum _{\ell = c+1}^{c+k} \mathbf{1}_B(Z_\ell ), \end{aligned}$$

its asymptotic properties remain still true. Here we have as initial distribution for the process Z at time c its limit distribution which satisfy conditions (4).

The semi-Markov chains are important for applications, and also for numerical calculus schemes of continuous-time processes. We can also use the estimator and its properties, proposed above, to estimate the stationary probability of a continuous-time semi-Markov process. If Q(t) is the semi-Markov kernel of a continuous-time semi-Markov process (see, e.g., Limnios and Oprişan 2001), we can consider it in the discrete-time scheme, with time step \(h>0\), as a SMC with semi-Markov kernel, q(k), given by the relations \(q(k)= Q(kh)-Q((k-1)h)\), \(k\ge 1\), and \(q(0)=0\).

Finally, it is worth noticing that all results presented in this paper are equally valuable when Z is an ergodic Markov chain with transition probability kernel P, since this process is a particular case of the semi-Markov chain with semi-Markov kernel \(Q(x,B,k):= P(x,B)[P(x,\{x\})]^{k-1}\), \(k\ge 1\).