Abstract
We consider inhomogeneous continuous-time Markov chains with vanishing perturbations. It is proved that, under some natural conditions, the limiting regimes of the initial and perturbed chains coincide. We obtain explicit estimates, which allow construction of the limiting regime of the perturbed chain, and show how these results can be used in the analysis of several known classes of queuing systems.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
1 INTRODUCTION
In this paper, we consider inhomogeneous Markov chains with intensities tending to prescribed values as \(t \to \infty \). More precisely, we assume that the infinitesimal matrix \(\bar {Q}(t)\) can be represented in the form \(\bar {Q}(t) = Q(t) + \hat {Q}(t)\), where \(\hat {Q}(t) \to 0\) as \(t \to \infty \). Thus, if the chains with intensity matrices \(Q(t)\) and \(\bar {Q}(t)\) are called original and perturbed, then we study the case of vanishing perturbations. Such models arise primarily in the case when the service and/or arrival rates asymptotically approach some “optimal” values. Such chains have been intensively studied since the 1970s. However, a wide range of problems remains open (see, e.g., [1–5], which present qualitative results). Below, we prove that, under some natural conditions, the limiting regimes of the original and perturbed chains coincide and, in contrast to previous works, obtain an explicit estimate for constructing the limiting regime of the perturbed chain. Additionally, we show how the results can be applied to several classes of queuing systems.
2 BASIC CONCEPTS
Let X(t), \(t \geqslant 0\), be an inhomogeneous continuous-time Markov chain with at most a countable state space \(\left\{ {0,1,~ \ldots ,S} \right\}\), \(0 < S \leqslant \infty .\) The transition probabilities for X(t) are denoted as \({{p}_{{ij}}}(s,t)\, = \,\Pr \{ X(t)\, = \,j{\text{|}}X(s)\) = i}, \(0 \leqslant i,j \leqslant S,~0 \leqslant s \leqslant t\). Let \({{p}_{i}}\left( t \right) = {\text{Pr}}\left\{ {X\left( t \right) = i} \right\}\) be the probability of the corresponding state of the chain and \(p(t) = {{({{p}_{0}}(t),{{p}_{1}}(t), \ldots ,{{p}_{S}}(t))}^{{\text{T}}}}\) be the vector of state probabilities. Assume that
where \({{q}_{{ij}}}\left( t \right)\) are locally integrable functions on the half-line \(\left[ {0,\infty } \right)\) (transition intensities), h is a “small” increment of time, and all \({{\alpha }_{i}}\left( {t,h} \right)\) are o(h) uniformly with respect to i, i.e., \({\text{su}}{{{\text{p}}}_{i}}\left| {{{\alpha }_{i}}\left( {t,h} \right)} \right| = o\left( h \right)\).
Setting \({{a}_{{ij}}}\left( t \right) = {{q}_{{ji}}}\left( t \right)~\) for \(j \ne i\) and aii(t) = \( - \mathop \sum \limits_{j \ne i} {{a}_{{ji}}}(t) = - \mathop \sum \limits_{j \ne i} {{q}_{{ij}}}(t)\), we consider the matrix A(t) = \(({{a}_{{ij}}}(t))\) made up of the functions \({{a}_{{ij}}}\left( t \right)\). Assuming that
for almost all \(~t \geqslant 0\) (i.e., except for a set of measure zero), we have a forward system of Kolmogorov differential equations, which can be written as a single vector equation, namely,
Note that \(A\left( t \right) = {{Q}^{{\text{T}}}}\left( t \right)~\) is the transposed infinitesimal matrix of the Markov chain X(t).
In what follows, let ||⋅|| (or ||⋅||1) denote the usual l1 norm, i.e., \({\text{||}}x{\text{||}} = \mathop \sum \limits_i \left| {{{x}_{i}}} \right|~\) for any vector x and ||B(t)|| = \({\text{su}}{{{\text{p}}}_{j}}\mathop \sum \limits_i \left| {{{b}_{{ij}}}(t)} \right|\) if \(B(t)\, = \,({{b}_{{ij}}}(t))\). Let \(\Omega \, = \,\{ x~:~{{x}_{i}}\, \geqslant \,0,\,~{\text{||}}~x{\text{||}}\, = \,1\} ,\) which is the set of all vectors with nonnegative coordinates and a unit \({{l}_{1}}\)-norm. Since
for almost all \(t \geqslant 0\), we can use the corresponding theory (see, e.g., [6]), treating (2) as an equation in the space l1. Specifically, the Cauchy problem for Eq. (2) has a unique solution for any initial condition, and if \(p(s) \in \Omega ,\) then \(p(t) \in \Omega \) for any \(0 \leqslant s \leqslant t\) and any initial condition p(s). Introducing z(t) = \({{({{p}_{1}}(t),{{p}_{2}}(t), \ldots ,{{p}_{S}}(t))}^{{\text{T}}}}\), we derive from (2) the equation
where \(~f(t) = {{\left( {{{a}_{{10}}}(t),{{a}_{{20}}}(t), \ldots ,{{a}_{{S0}}}(t)} \right)}^{{\text{T}}}}\) and
Now let \(\bar {X}\left( t \right)\) be a “perturbed” Markov chain with the same state space as X(t), state probabilities \({{\bar {p}}_{i}}\left( t \right)\), and the transposed infinitesimal matrix \(\bar {A}(t)\, = \,({{\bar {a}}_{{ij}}}(t))\). The deviations of the perturbed characteristics from the original one are denoted by \({{\hat {a}}_{{ij}}}\left( t \right)\) and \(\hat {A}\left( t \right)\), respectively.
Recall that the Markov chain X(t) is weakly ergodic if for any pair of vectors p*(t), \(p{\text{**}}\left( t \right)\) solving (2) with different initial conditions, it is true that \({\text{||}}p{\text{*}}(t)\, - \,p{\text{**}}(t){\text{||}}\, \to \,0\) as \(t \to \infty .\)
Let \(p{\text{*}}\left( t \right)\) and \(p{\text{**}}\left( t \right)\) be solutions of (2). Then, by the definition of the vector z(t), we have the inequalities
where z*(t) and z**(t) are vectors corresponding to p*(t) and p**(t).
In what follows, Eq. (3) is considered not only in l1, but also in the embedded subspace
(with a suitable linear operator D; for more details, see Section 3 below), which is denoted by \({{l}_{{1D}}}\), and the norm in it, by \(||\, \cdot \,|{{|}_{{1D}}}\). Then given some \(M > 0\), \(a > 0\), and any initial values p*(s), \(p{\text{**}}\left( s \right) \in {{l}_{1}}\), if the inequality
holds for all \(t \geqslant s \geqslant 0\), then such a Markov chain X(t) is called 1D-exponentially ergodic (see [7]). Note that, if X(t) has a stationary (i.e., time-independent) regime, then X(t) is 1D-exponentially ergodic. Ergodicity conditions, corresponding convergence estimates, and their relationship with perturbation bounds have been studied by numerous authors (see, e.g., [5, 7, 8, 10, 11, 13, 15, 16, 18, 19, 21, 22]).
3 GENERAL ESTIMATES
Standard approaches to the study of continuous time Markov chains are described in [7]. However, they cannot be applied directly to the case of vanishing perturbations. In this section, for the first time, we present an estimate for solutions of Eq. (3) that makes the subsequent study possible. For illustrative purposes, the results are stated in explicit form after the obtained estimates.
Consider the equation for the perturbed chain corresponding to Eq. (3):
which can be rewritten as
If \(V\left( {t,~s} \right)\) denotes the Cauchy operator of Eq. (3), then solutions of Eqs. (3) and (6) can be written as
and
Assuming \(1D\)-exponential ergodicity, we obtain \({\text{||}}V\left( {t,s} \right){\text{|}}{{{\text{|}}}_{{1D}}} \leqslant M{{e}^{{ - a\left( {t - s} \right)}}}\). Introducing the notation \(\hat {z}(t) = \bar {z}(t) - z(t)\), we have the following upper bound in the \({{l}_{{1D}}}\) norm:
Obviously, the first term tends to zero as \(t \to \infty \). To estimate the second and third terms, the following conditions are assumed to hold:
(А) \({\text{||}}~\hat {B}\left( t \right){\text{|}}{{{\text{|}}}_{{1D}}} \leqslant \chi \left( t \right),\) where the “perturbation” \(\chi \left( t \right) \to 0\) as \(t \to \infty \); moreover, without loss of generality, we may assume that χ is a continuous, bounded, and monotonically decreasing function;
(В) \({\text{||}}\hat {f}\left( t \right){\text{|}}{{{\text{|}}}_{{1D}}} \leqslant \chi \left( t \right).\)
Remark. Under the additional assumption on the monotonicity of perturbations, the convergence of the integral of the perturbation norm over the interval from zero to infinity (this case was studied, for example, in [5, 8]) is a stronger condition than tending to zero.
For notational brevity, we set \(\chi \left( 0 \right) = {{\varepsilon }_{0}}\). Without loss of generality, it can be assumed that \({{\varepsilon }_{0}} < a\); otherwise, we can use \({{t}_{0}} > 0\) as initial time.
To estimate the norm of the solution of the perturbed equation, the Cauchy operator of this equation is denoted by \(\bar {V}\left( {t,s} \right)\). Then, in the \(1D\) norm,
Next, writing the solution of Eq. (6) as
and assuming that \({\text{||}}f\left( t \right){\text{|}}{{{\text{|}}}_{{1D}}} \leqslant F\) (for some \(0 < F < \infty \)) for almost all \(t \geqslant 0\), we obtain, in the 1D norm,
We choose an arbitrary \(\varepsilon \in \left( {0,{{\varepsilon }_{0}}} \right)\) and an arbitrary moment of time \({{t}_{*}}\) such that \(\chi \left( {{{t}_{*}}} \right) = \varepsilon \). Since
the desired estimate follows from (7) and (8):
Since ε is arbitrary, inequality (9) guarantees that the norm of the perturbation tends to zero as \(t \to \infty \) and yields an estimate for this convergence.
Thus, the following result holds.
Theorem 1. Given a \(1D\)-exponentially ergodic Markov chain X(t) in a subspace \({{l}_{{1D}}} \subset {{l}_{1}}\), suppose that the perturbed chain \(\bar {X}\left( t \right)\) has a perturbation norm tending to zero as \(t \to \infty \) so that conditions (A) and (B) are satisfied. Then \(\bar {X}\left( t \right)\) is weakly ergodic, it has the same limiting regime, and estimate (9) holds.
Corollary 1. Under the conditions of Theorem 1, suppose that the intensities of the original chain X(t) are 1‑periodic. Then the limiting regime of the perturbed chain \(\bar {X}\left( t \right)\) is also 1-periodic.
Corollary 2. Under the conditions of Theorem 1, suppose that the intensities of the original chain X(t) are proportional, i.e., all \({{q}_{{ij}}}\left( t \right) = \vartheta \left( t \right){{q}_{{ij}}}\). Then the unperturbed and perturbed chains, X(t) and \(\bar {X}\left( t \right)\), are both strongly ergodic and have identical stationary distributions.
4 DERIVATION OF ESTIMATES FOR MARKOV MODELS OF QUEUING SYSTEMS
In the analysis of models of queuing systems, the main role in obtaining particular values involved in estimate (9) is played by inequality (4). A rather simple and convenient method is one based on the logarithmic norm of a linear operator function (see [6, 9, 10]). If the matrix of a linear system \(K\left( t \right) = ({{k}_{{i,j}}}\left( t \right)\)) is essentially nonnegative (i.e., its off-diagonal elements are all nonnegative), then its logarithmic norm \(\gamma \left( {K\left( t \right)} \right)\) is given by \(\gamma \left( {K\left( t \right)} \right) = {\text{su}}{{{\text{p}}}_{j}}\sum\nolimits_i {{{k}_{{i,j}}}\left( t \right)} \). Moreover, for the corresponding Cauchy operator, the estimate \({\text{||}}V\left( {t,s} \right){\text{||}} \leqslant {{e}^{{\mathop \smallint \limits_s^t \gamma \left( {K\left( \tau \right)} \right)d\tau }}}\) holds.
If the matrix K(t) is not essentially nonnegative, then the following approach is usually adopted. Consider a matrix of the form
and the homogeneous system corresponding to (3), namely,
Setting \(w\left( t \right) = Dz\left( t \right)\) yields the equation
where \(B{\text{*}}\left( t \right) = DB\left( t \right){{D}^{{ - 1}}}\), in contrast to B(t), can be made essentially nonnegative for a broad class of models of queuing systems by choosing a “weight” sequence \(\left\{ {{{d}_{i}},~i \geqslant 1} \right\}\) bounded away from zero (which guarantees that \({{l}_{{1D}}} \subset {{l}_{1}}\)). If B*(t) can be made essentially nonnegative, then the basic difficulty is associated with choosing a weight sequence that provides accurate convergence rate bounds (see [10]). Note that accurate convergence rate bounds correspond to the most accurate perturbation bounds [22].
Now we describe the constructions corresponding to some classical models.
We begin with the Erlang loss system \({{M}_{t}}{\text{/}}{{M}_{t}}{\text{/}}N{\text{/}}N\) (see [1, 2, 5, 7, 10]). In this case, the number X(t) of customers in the system is described by a birth-death process with a finite number of states, i.e., the transition intensities are given by \({{q}_{{ij}}}\left( t \right) = 0~\) for all \(t \geqslant 0\) if \(\left| {i - j} \right| > 1\) and \({{q}_{{i,i + 1}}}\left( t \right) = \lambda \left( t \right)\) and \({{q}_{{i + 1,i}}}\left( t \right) = i\mu \left( t \right)\), \(i = 0, \ldots ,N - 1.\) It is well known that the process X(t) is weakly ergodic if and only if \(\mathop \smallint \limits_0^\infty \left( {\lambda \left( t \right) + \mu \left( t \right)} \right)dt = + \infty \). For definiteness, consider the case of an essential service rate. In this case, the transformation matrix is finite. Put \({{d}_{i}} = 1\), \(~i \geqslant 1\). Then the corresponding logarithmic norm is equal to \( - \mu \left( t \right)\) and, hence, \({\text{||}}V\left( {t,s} \right){\text{|}}{{{\text{|}}}_{{1D}}} \leqslant {{e}^{{ - \mathop \smallint \limits_s^t \mu \left( \tau \right)d\tau }}}\). Specifically, if the service rate \(\mu \left( t \right)\) is a 1-periodic function of time, then the values of the unknown parameters on the right-hand side of (4) are equal to \(a = \mathop \smallint \limits_0^1 \mu \left( t \right)dt\); \(M \leqslant {{e}^{a}}\).
Now we consider a nonstationary queuing model with a unbounded waiting line and S servers \({{M}_{t}}{\text{/}}{{M}_{t}}{\text{/}}S\) (see, e.g., [12, 16]) with arrival and service rates being \({{\lambda }_{k}}(t)\, = \,\lambda (t)\) and \({{\mu }_{k}}(t)\, = \,\mu (t){\text{min}}(k,S)\), respectively. It is well known (see [16]) that the process describing the number X(t) of customers in the system is weakly ergodic if there exists d > 1 such that \(\mathop \smallint \limits_0^\infty (S\mu (t) - d\lambda (t))dt = + \infty \). Assume that S = 2 (in contrast to \(S = 1\), this case is more complicated; see, e.g., [9]). If we set \({{d}_{i}} = {{d}^{{i - 1}}}\) for \(i \geqslant 1,\) where \(d \in \left( {1,2} \right]\), in (11), then the logarithmic norm \(\gamma (B{\text{*}}(t))\) satisfies the estimate \(\gamma (B{\text{*}}(t)) \leqslant \left( {1 - \frac{1}{d}} \right)(2\mu (t)\) – dλ(t)). In the case of 1-periodic rates, estimate (4) holds for a = \(\left( {1 - \frac{1}{d}} \right)\mathop \smallint \limits_0^1 (2\mu (t) - d\lambda (t))dt\) and for the corresponding value of M.
Interestingly, the same weight sequence \(\{ {{d}_{i}}\, = \,{{d}^{{i - 1}}},\,\,~i\, \geqslant \,1\} ~\) can be used to investigate an entirely different queueing model with an extraordinary arrival flow controlling the length of the queue (see, e.g., [15, 17, 23]). Although the infinitesimal intensity matrix has a complex structure of the form
in the case of exponentially decaying probabilities \({{b}_{i}}\) of arriving a batch of customers of size i (i.e., for \({{b}_{i}} \leqslant C{{q}^{i}}\)), weak ergodicity is guaranteed with the logarithmic norm estimate \(\gamma \left( {B{\text{*}}\left( t \right)} \right) \leqslant \left( {1 - \frac{1}{d}} \right)\mu \left( t \right)\) for \(d \in \left( {1,\frac{1}{q}} \right)\). Then, for a 1-periodic service rate, estimate (4) holds for a = \(\left( {1 - \frac{1}{d}} \right)\mathop \smallint \limits_0^1 \mu \left( t \right)dt\) and the corresponding value of M.
Note that similar transformations (with a significantly more complicated choice of a weight sequence) and the logarithmic norm method can be used to obtain explicit convergence rate estimates and, hence, estimates in the case of vanishing perturbations for other classes of Markov nonstationary queueing systems, including for models of the type \(M_{t}^{x}{\text{/}}M_{t}^{x}{\text{/}}1\) [7, 20], queueing systems with catastrophes [9], systems with absorption at zero [13], and systems with bulk arrival and service and state-dependent control [9, 14].
REFERENCES
D. B. Gnedenko, “On a generalization of Erlang formulae,” Zastosow. Mat. 12, 239–242 (1971).
B. Gnedenko and A. Soloviev, “On the conditions of the existence of final probabilities for a Markov process,” Math. Oper. Stat. 4, 379–390 (1973).
J. Liu, P. E. Pare, A. Nedic, C. Y. Tang, C. L. Beck, and T. Basar, “Analysis and control of a continuous-time bi-virus model,” IEEE Trans. Autom. Control 64 (12), 4891–4906 (2019).
P. E. Pare, J. Liu, C. L. Beck, A. Nedic, and T. Basar, “Multi-competitive viruses over time-varying networks with mutations and human awareness,” Automatica 123, 109330 (2021).
A. I. Zeifman and D. L. Isaacson, “On strong ergodicity for nonhomogeneous continuous-time Markov chains,” Stochastic Processes Appl. 50 (2), 263–273 (1994).
Ju. L. Daleckiĭ and M. G. Kreĭn, Stability of Solutions of Differential Equations in Banach Space (Nauka, Moscow, 1970; Am. Math. Soc., Providence, 1974).
A. Zeifman, V. Korolev, and Y. Satin, “Two approaches to the construction of perturbation bounds for continuous-time Markov chains,” Mathematics 8 (2), 253 (2020).
V. Abramov and R. Liptser, “On existence of limiting distribution for time-nonhomogeneous countable Markov process,” Queueing Syst. 46 (3), 353–361 (2004).
A. Zeifman, Y. Satin, I. Kovalev, R. Razumchik, and V. Korolev, “Facilitating numerical solutions of inhomogeneous continuous time Markov chains using ergodicity bounds obtained with logarithmic norm method,” Mathematics 9 (1), 42 (2021). https://doi.org/10.3390/math9010042
E. van Doorn, A. I. Zeifman, and T. L. Panfilova, “Bounds and asymptotics for the rate of convergence of birth-death processes,” Theory Probab. Appl. 54 (1), 97–113 (2010).
C. Fricker, P. Robert, and D. Tibi, “On the rates of convergence of Erlang’s model,” J. Appl. Probab. 36, 1167–1184 (1999).
Ya. M. Agalarov, “Optimal threshold-based admission control in the M/M/s system with heterogeneous servers and a common queue,” Inf. Primen. 15 (1), 57–64 (2021).
H. Caswell, “Perturbation analysis of continuous-time absorbing Markov chains,” Numer. Linear Algebra Appl. 18 (6), 901–917 (2011).
A. Chen, X. Wu, and J. Zhang, “Markovian bulk-arrival and bulk-service queues with general state-dependent control,” Queueing Syst. 95, 331–378 (2020). https://doi.org/10.1007/s11134-020-09660-0
A. I. Zeifman, R. V. Razumchik, Y. A. Satin, and I. A. Kovalev, “Ergodicity bounds for the Markovian queue with time-varying transition intensities, batch arrivals and one queue skipping policy,” Appl. Math. Comput. 395, 125846 (2021).
A. I. Zeifman, “Upper and lower bounds on the rate of convergence for nonhomogeneous birth and death processes,” Stochastic Processes Their Appl. 59 (1), 157–173 (1995).
A. Marin and S. Rossi, “A queueing model that works only on the biggest jobs,” in European Workshop on Performance Engineering (Springer, Cham, 2019), pp. 118–132.
A. Yu. Veretennikov and M. A. Veretennikova, “On convergence rates for homogeneous Markov chains,” Dokl. Math. 101 (1), 12–15 (2020).
B. Heidergott, H. Leahu, A. Löpker, and G. Pflug, “Perturbation analysis of inhomogeneous finite Markov chains,” Adv. Appl. Probab. 48 (1), 255–273 (2016).
R. Nelson, D. Towsley, and A. N. Tantawi, “Performance analysis of parallel processing systems,” IEEE Trans. Software Eng. 14 (4), 532–540 (1988).
A. Y. Mitrophanov, “Sensitivity and convergence of uniformly ergodic Markov chains,” J. Appl. Probab. 42 (4), 1003–1014 (2005).
A. Y. Mitrophanov, “Connection between the rate of convergence to stationarity and stability to perturbations for stochastic and deterministic systems,” in Proceedings of the 38th International Conference Dynamics Days Europe (DDE 2018) (Loughborough, UK, 2018), pp. 3–7.
S. I. Matyushenko and R. V. Razumchik, “Stationary characteristics of discrete-time Geo/G/1 queue with batch arrivals and one queue skipping policy,” Inf. Primen. 14 (4), 25–32 (2020).
Funding
This work was supported by the Ministry of Science and Higher Education of the Russian Federation, project no. 075-15-2020-799.
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
The authors declare that they have no conflicts of interest.
Additional information
Translated by I. Ruzanova
Rights and permissions
About this article
Cite this article
Zeifman, A.I., Korolev, V.Y., Razumchik, R.V. et al. Limiting Characteristics of Queueing Systems with Vanishing Perturbations. Dokl. Math. 106, 375–379 (2022). https://doi.org/10.1134/S1064562422050209
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1134/S1064562422050209