Discrete-Time Markov Queueing Models

Abstract

Queueing models can often be described by finite or denumerable Markov chains. In these cases their study becomes rather simple. One of natural ways in which such models arise is the following. Suppose, that we describe a queueing system with the help of a random process z = {z(t)} _{t ≥0}, which is difficult to study. Imagine that we are lucky to discover a sequence of random times T = {T _k } _{k ≥0} such that r.v.’s ν _k = z(T _k ) comprise a Markov chain imbedded into the process z. From a practical point of view, this chain ought to be informative enough in order one can judge about z examining characteristics of ν = {ν _k } _{k ≥0}. Below, we give a few examples of such imbedded chains. Due to the fact of considering the initial process z only at some isolated times T we can not expect to obtain exhaustive information about z. In particular, let us pay attention to the following important fact. Suppose that the process z has a stationary distribution

$$Q(B) = \mathop {\lim }\limits_{t \to \infty } P(z(t) \in B).$$

.