Keywords

1 Introduction

This paper presents a survey of research results in the area of stochastic processes obtained by me and my collaborators during a long period, which began about 50 years ago. My first results have been published in papers [1, 2]. The corresponding complete bibliography of works on stochastic processes and related areas includes 11 books, more than 150 research papers and 15 editorial works. It can be found at the webpage [68].

The main areas of research cover limit theorems for Markov-type processes and randomly stopped stochastic processes, renewal theory and ergodic theorems for perturbed stochastic processes, quasi-stationary distributions for perturbed stochastic systems, methods of stochastic approximation for price processes, asymptotic expansions for nonlinearly perturbed semi-Markov processes, and their applications to queuing systems, reliability models, stochastic networks, bio-stochastic systems, perturbed risk processes, and American-type options. For convenience of readers, the works pointed in the references are ordered by years of their publication. This makes it possible to see all together works related to every research direction mentioned above.

I would also like to mention paper [61] and books [44, 49, 57, 58, 65], which contain comprehensive bibliographies of works in the above research areas, and the corresponding bibliographical remarks with historical and methodological comments.

About half of my works are written or co-edited together with more than 50 collaborators, including more than 20 of my former doctoral students. Their names can be found in the references given in this paper and the complete bibliography given at [68].

I would like to use this opportunity and to sincerely thank all my collaborators for the fruitful cooperation.

This survey was presented at the International Conference “Stochastic Processes and Algebraic Structures – From Theory Towards Applications” (SPAS 2017, https://spas2017blog.wordpress.com), which was organised on the occasion of my 70th birthday and held at Västerås – Stockholm, on 4–6 October 2017.

I am very grateful to the Organising and Scientific Committees, keynote speakers and other conference participants as well as to the Division of Applied Mathematics (School of Education, Culture and Communication, Mälardalen University) and the Department of Mathematics (Stockholm University) who have supported this conference.

2 Limit Theorems for Markov-Type Processes

The main objects of research studies in this area were limit theorems for sums and stepwise sum-processes of random variables defined on asymptotically ergodic and asymptotically recurrent random walks, Markov chains, and semi-Markov processes.

It is worth noting that limit theorems for sums of random variables defined on Markov chains are a very natural generalisation of classical limit theorems for sums of independent random variables. In the case of asymptotically ergodic Markov chains, the corresponding conditions of convergence are similar to well-known classical conditions of convergence for sums and sum-processes of i.i.d. random variables. Also, as in the above classical case, Lévy processes appear as limiting ones.

In the case of asymptotically recurrent Markov chains, the corresponding conditions of convergence and limiting processes take much more complex forms. Possible limiting processes have been described for the above sum-processes defined on Markov chains and semi-Markov processes, first, with countable and, then with general phase spaces. The corresponding limiting processes are generalised exceeding processes. Such processes are constructed with the use of two-dimensional, càdlàg  Lévy processes with the second nonnegative component, in the following way. The first component of the above Lévy process is randomly stopped at the moment of first exceeding a level t (which plays the role of time) by the second component. In addition, this stopping process can possibly be truncated by some exponentially distributed random variable (independent of the above two-dimensional Lévy process) taking into account asymptotic recurrence effects for the above mentioned Markov-type processes.

The corresponding asymptotic results have been obtained firstly in the form of usual limit theorems about weak convergence of finite-dimensional distributions for the corresponding sum-processes, and then in the more advanced form of functional limit theorems about convergence of these processes in the uniform U and Skorokhod J topologies.

Further, the above sum-processes, randomly stopped at different Markov moments, such as hitting times for the above asymptotically recurrent Markov-type processes, have been thoroughly studied and analogous asymptotic results have been obtained as well.

The results related to finite and countable Markov chains and semi-Markov-type processes are well presented in paper [4], dissertation [3], based on 14 research papers, and two books, [8, 14].

Later works in this area have been concentrated on limit theorems for Markov and semi-Markov-type models with general phase spaces, [13, 20, 22, 28], finding not only sufficient but also necessary conditions of convergence, [21, 24, 31], as well as generalisation of the above limit theorems to non-Markov models, [11, 17,18,19, 26].

The latest results in this area concern necessary and sufficient conditions of convergence for first-rare-event times and processes, [47, 60]. It is appropriate to note that these results yield, in particular, necessary and sufficient conditions for diffusion and stable approximations of ruin probabilities for classical risk processes.

3 Limit Theorems for Randomly Stopped Stochastic Processes

A natural research area connected with limit theorems for Markov-type processes is that of limit theorems for randomly stopped stochastic processes and for compositions of stochastic processes.

This model can appear in a number of natural ways, for example: when studying limit theorems for additive or extremal functionals of stochastic processes; in models connected with a random change of time, change point problems and problems related to optimal stopping of stochastic processes; and in different renewal models, particularly those which appear in applications to risk processes, queuing systems, etc.

This model also appears in statistical applications connected with studies of samples with a random sample size. Such sample models play an important role in sequential analysis. They also appear in sample survey models, or in statistical models, where sample variables are associated with stochastic flows. The latter models are typical for insurance, queuing and reliability applications, as well as many others.

There exists a huge bibliography of works devoted to limit theorems for models with independent or asymptotically independent external processes and stopping moments.

The aim of the author was to build a general theory of limit theorems for compositions of dependent càdlàg  external and internal non-negative and non-decreasing stopping processes.

Pre-limiting joint distributions of external and stopping processes usually have a complicated structure. The idea was to find conditions of convergence, where these processes would be involved together only in the simplest and most natural way, via the condition of their joint weak convergence. Also, conditions of compactness in Skorokhod J-topology should be required for external processes and internal stopping processes, in order to provide compactness in Skorokhod J-topology for their compositions. These conditions are standard ones. They were thoroughly studied for various classes of càdlàg  stochastic processes.

However, it turns out that the above three conditions do not provide convergence in J-topology for compositions. Some additional assumptions, which link discontinuity moments and values of the corresponding limiting processes at these moments, should be made. First, the probability that the limiting internal stopping process takes the same value at two moments of time \(t'< t''\) and this value hits the set of discontinuity moments for the limiting external process should be 0, for any \(0 \le t'< t'' < \infty \). Second, the probability that the intersection of the set of left and right limiting values for the limiting internal stopping process at its jump moments and the set of discontinuity moments for the limiting external process is non-empty should be 0.

The limiting processes usually have simpler structure than the corresponding pre-limiting processes. This permits one to check the above continuity conditions in various practically important cases. For example, the first continuity condition holds, if the limiting external process is a.s. continuous or the limiting internal stopping process is a.s. strongly monotonic. The above second continuity condition holds, if at least one of the limiting external or internal processes is a.s. continuous. Also, both conditions hold if the limiting external and internal processes are independent and the limiting external process is stochastically continuous. The above continuity conditions can not be omitted. If the first continuity condition does not hold for some \(t' < t''\), the compositions may not weakly converge at interval \([t', t'']\). If the second continuity condition does not hold, the compositions may not be compact in J-topology.

One of the main theorems proven by the author states that the five conditions listed above do imply J-convergence for compositions of càdlàg  processes. These conditions have a good balance that makes this theorem a flexible and effective tool for obtaining limit theorems for randomly stopped stochastic processes.

The main new results found by the author include general limit theorems about weak convergence of randomly stopped stochastic processes and compositions of dependent càdlàg  stochastic processes, functional limit theorems about convergence of compositions of càdlàg  stochastic processes in topologies U and J as well as applications of these theorems to random sums, extremes with random sample size, generalised exceeding processes, sum-processes with renewal stopping, accumulation processes, max-processes with renewal stopping, and shock processes.

Some of the most valuable works in this area are the following papers: [5, 6, 9, 36, 41,42,43], dissertation [7], based on 27 research papers, and book [8]. The final extended version of the theory developed by the author is presented in the book [44].

4 Renewal Theory and Ergodic Theorems for Perturbed Stochastic Processes

Another research area connected in a natural way with limit theorems for Markov-type processes is renewal theory and ergodic theorems for perturbed stochastic processes.

An important role is played in both limit and ergodic theorems by such random functionals as hitting times and their moments. Necessary and sufficient conditions of existence and the most general explicit recurrent upper bounds for power and exponential moments of hitting-time type functionals for semi-Markov processes have been given in book [14] and papers [34, 46].

Further, related recurrent computational algorithms based on various truncation and phase space reduction procedures are given for semi-Markov-type processes and networks in papers [16, 56, 64].

Ergodic theorems of the law of larger numbers type and related ergodic theorems for mean averages for accumulation processes and iterated functions systems are given in papers [15, 31, 37] and book [14].

Uniform asymptotic expansions for exponential moments of sums of random variables defined on exponentially ergodic Markov chains and distributions of hitting times for such Markov chains are given in paper [29].

As is well known, the most effective tool for getting so-called individual ergodic theorems for regenerative and Markov-type processes is the famous renewal theorem. In the case of perturbed processes, an effective generalisation of this important theorem to the model of perturbed renewal equation is required. Such a generalisation was given in paper [12].

These results and their applications to perturbed regenerative, semi-Markov and risk processes, and ergodic theorems for perturbed queuing systems and bio-stochastic systems are presented in papers [40, 55] and book [49]. Also, the recent paper, [66], presents results of a detailed analysis and classification of ergodic theorems for perturbed alternating regenerative processes.

In papers [23, 30], exact coupling algorithms have been composed for general regenerative processes and stochastic processes with semi-Markov modulation, and explicit estimates for the rate of convergence in related individual ergodic theorems for such processes were given. It is worth noting that, in the continuous time case, the algorithms of exact coupling do require construction of dependent coupling trajectories in the essentially more sophisticated way, if we are to compare them with the corresponding coupling algorithms for discrete time processes.

In addition, paper [53] can be mentioned, where the above coupling algorithms are applied for obtaining explicit estimates for the rate of convergence in the classical Cramér-Lundberg approximation for ruin probabilities.

5 Quasi-Stationary Phenomena for Perturbed Stochastic Systems

Quasi-stationary phenomena in stochastic systems describe the behaviour of stochastic systems with random lifetimes. The core of the quasi-stationary phenomenon is that one can observe something that resembles a stationary behaviour of the system before the lifetime goes to the end.

The objects of interest are the asymptotic behaviour of lifetimes in the forms of weak convergence and large deviation theorems, conditional ergodic theorems (describing the asymptotic behaviour, when \(t \rightarrow \infty \), for the conditional distribution of the corresponding stochastic process at moment t, under the condition that the lifetime takes a value larger than t), and the corresponding limiting, usually referred to as quasi-stationary, distributions.

In the model of perturbed stochastic processes of Markov-type, their transition characteristics depend on a small perturbation parameter \(\varepsilon \) and, moreover, they may admit some asymptotic expansions with respect to this parameter. A problem arises in constructing asymptotic expansions for distributions of lifetimes, conditional distributions for the underlying stochastic processes pointed out above, and the corresponding quasi-stationary distributions.

It is relevant to note that quasi-stationary distributions are essentially nonlinear functionals of transition characteristics for underlying Markov-type processes. They depend on so-called characteristic roots for distributions of return times for the above processes. This significantly complicates the problem of constriction of asymptotic expansions for quasi-stationary distributions, if we are to compare it with the analogous problem for ordinary stationary distributions of perturbed Markov-type processes.

It also turns out that the balance between the velocities with which \(\varepsilon \) tends to zero and time t tends to infinity (expressed in the form of asymptotic relation, \(t \varepsilon ^r \rightarrow \lambda _r \in [0, \infty ]\)) has a delicate influence on the quasi-stationary asymptotics. The above-mentioned expansions make it possible to perform the corresponding detailed asymptotic analysis.

New methods based on asymptotic expansions for solutions of perturbed renewal equations have been proposed in paper [32] and used in papers [38, 39, 55] for finding quasi-stationary asymptotics given in the form of asymptotic expansions, for perturbed regenerative processes, Markov chains, semi-Markov processes, and risk processes. Also, asymptotic expansions for discrete time nonlinearly perturbed renewal equations have been given in papers [35, 54] and for the renewal equation with nonlinear non-polynomial perturbations in paper [48].

The comprehensive book [49] contains a detailed presentation of the above mentioned methods for nonlinearly perturbed regenerative processes and finite semi-Markov processes with absorption. It also includes their applications to the analysis of quasi-stationary phenomena in nonlinearly perturbed highly reliable queuing systems, M/G queuing systems with quick service, and stochastic systems of birth-death type, including perturbed epidemic, population and meta-population models, and perturbed risk processes.

Also, the paper [66] presents new ergodic theorems for perturbed alternating regenerative processes obtained with the use of quasi-stationary ergodic theorems for perturbed regenerative processes.

6 Stochastic Approximation Methods for Price Processes and American-Type Options

American-type options are one of the most important financial instruments and, at the same time, one of the most interesting and popular objects for research studies in financial mathematics.

The main mathematical problems connected with such options relate to finding of the optimal expected option rewards, in particular, fair prices of options, as well as finding of optimal strategies for buyers of options that are optimal stopping times for execution of options.

In this way, the theory of American-type options is connected with optimal stopping problems for stochastic processes, which play an important role in the theory of stochastic processes and its applications.

As is well known, analytical solutions for American-type options are available only in some special cases and, even in such cases, the corresponding formulas are not easily computable. These difficulties dramatically increase in the case of multivariate log-price processes and non-standard pay-off functions.

Approximation methods are a reasonable alternative that can be used in cases where analytical solutions are not available. The main classes of approximation methods are: stochastic approximation methods based on approximation of the corresponding stochastic log-price processes by simpler processes, for which optimal expected rewards can be effectively computed; integro-differential approximation methods based on approximation of integro-differential equations that can be derived for optimal expected rewards by their difference analogues; and Monte Carlo methods based on simulation of the corresponding log-price processes.

Stochastic approximation methods have important advantages in comparison with the other two methods. They usually allow one to impose weaker smoothness conditions on transition probabilities and pay-off functions, in comparison with integro-differential approximation methods, and they are also computationally more effective, in comparison with Monte Carlo based methods.

Selected papers presenting the author and his collaborators’ results in the above area are [45, 50,51,52, 59]. Some of these results, as well as many new results, are presented in the author’s comprehensive two volume monograph, [57, 58].

This monograph gives a systematic presentation of stochastic approximation methods for models of American-type options with general pay-off functions for discrete (Volume 1) and continuous (Volume 2) time modulated Markov log-price processes. Advanced methods, combining backward recurrence algorithms for computing of option rewards for discrete time atomic Markov chains with transition probabilities concentrated on finite sets, and general results on convergence of the corresponding stochastic time-space skeleton and tree approximations for option rewards, are applied to a variety of models of multivariate modulated Markov log-price processes.

In the discrete time case, these are modulated autoregressive and autoregressive stochastic volatility log-price processes, log-price processes represented by modulated random walks, Markov Gaussian log-price processes with estimated parameters, multivariate modulated Markov Gaussian log-price processes and their binomial and trinomial approximations, and log-price processes represented by general modulated Markov chains.

In the continuous time case, these are multivariate Lévy log-price processes, multivariate diffusion log-price processes and their time-skeleton, martingale and trinomial approximations, and general continuous time multivariate modulated Markov log-price processes.

Further, some more advanced models of American-type options are treated; in particular, options with random pay-offs, reselling options and knockout options.

The principal novelty of results presented in the monograph [57, 58] is based on the consideration of multivariate modulated Markov log-price processes and general pay-off functions, which can depend not only on price but also on an additional stochastic modulating index component, and the use of minimal conditions of smoothness for transition probabilities and pay-off functions, compactness conditions for log-price processes and rate of growth conditions for pay-off functions.

7 Nonlinearly Perturbed Semi-Markov Processes

The models of perturbed Markov chains and semi-Markov processes attracted the attention of researchers in the middle of the twentieth century. Particular attention was given to the most difficult cases of perturbed processes with absorption and the so-called singularly perturbed processes. An interest in these models has been stimulated by applications to control and queuing systems, reliability models, information networks, and bio-stochastic systems.

Markov-type processes with singular perturbations appear as natural tools for mathematical analysis of multicomponent systems with weakly interactingcomponents. Asymptotics for moments of hitting-time type functionals and stationary distributions for corresponding perturbed processes play an important role in studies of such systems.

The role of perturbation parameters can be played by small probabilities or failure rates in queuing and reliability systems, or by small probabilities or intensities of mutation, extinction, or migration in biological systems. Perturbation parameters can also appear as artificial regularisation parameters for decomposed systems, for example, as so-called damping parameters in information networks, etc.

In many cases, transition characteristics of the corresponding perturbed semi-Markov processes, in particular transition probabilities (of embedded Markov chains) and power moments of transition times are nonlinear functions of a perturbation parameter, which admit asymptotic expansions with respect to this parameter.

The main results obtained so far in these ongoing studies are presented in papers [61,62,63] and the recent book [65].

These works present new methods of asymptotic analysis for nonlinearly perturbed semi-Markov processes with finite phase spaces. These methods are based on special time-space screening procedures for sequential reduction of phase spaces for semi-Markov processes combined with the systematic use of the operational calculus for Laurent asymptotic expansions.

Models with non-singular and singular perturbations are considered to be those where the phase space is one class of communicative states for the embedded Markov chains of pre-limiting perturbed semi-Markov processes, while it can possess an arbitrary communicative structure (i.e., can consist of one or several closed classes of communicative states and, possibly, a class of transient states) for the limiting embedded Markov chain.

Effective recurrent algorithms for the construction of Laurent asymptotic expansions for power moments of hitting times for nonlinearly perturbed semi-Markov processes are composed. These results are applied for obtaining asymptotic expansions for stationary and conditional quasi-stationary distributions for nonlinearly perturbed semi-Markov processes. Also, the detailed asymptotic analysis and the corresponding asymptotic expansions are given for semi-Markov birth-death-type processes, which play an important role in various applications.

Further, the recent paper [67], which presents applications of asymptotic expansions for semi-Markov birth-death-type processes to perturbed models of population dynamics, epidemic models and models of population genetics, should be mentioned.

It is worth noting that asymptotic expansions are a very effective instrument for studies of perturbed stochastic processes. The corresponding first terms in expansions give limiting values for properly normalised functionals of interest. The second terms let one estimate the sensitivity of models to small parameter perturbations. The subsequent terms in the corresponding expansions are usually neglected in standard linearisation procedures used in studies of perturbed models. This, however, cannot be acceptable in cases where values of perturbation parameters are not small enough. Asymptotic expansions let one take into account high-order terms in expansions, and in this way allow one to improve accuracy of the corresponding numerical procedures.

An important novelty of the results presented in works [61,62,63, 65] is that the corresponding asymptotic expansions are obtained with remainders given not only in the standard form of \(o(\varepsilon ^k)\), but, also, in the more advanced form, with explicit power-type upper bounds for remainders, \(|o(\varepsilon ^k)| \le G_k \varepsilon ^{k + \delta _k}\) , asymptotically uniform with respect to the perturbation parameter. The latter asymptotic expansions for nonlinearly perturbed semi-Markov processes were not known before.

The corresponding computational algorithms have a universal character. They can be applied to perturbed semi-Markov processes with an arbitrary asymptotic communicative structure of phase spaces and are computationally effective due to the recurrent character of computational procedures.

8 Conclusion

There exist a number of prospective directions for continuation of research and many interesting unsolved problems in the research areas listed above.

In the area of limit theorems for Markov-type processes, the methods developed so far, particularly general limit theorems for randomly stopped stochastic processes, allow, I believe, to obtain new, more advanced versions of these limit theorems and, moreover, to improve conditions of convergence for some of these theorems to the final necessary and sufficient form, without gaps between necessary and sufficient counterparts. One of the most recently published papers, [60], provides an example of such results.

The theory of limit theorems for randomly stopped stochastic processes can also be effectively used in such applied areas as asymptotic problems of statistical analysis. The book [44] contains some results related to samples with random size. However, this very prospective area of applications is still underdeveloped.

In the area of renewal theory and ergodic theorems for perturbed Markov-type processes, applications of the exact coupling method presented in papers [23, 30] can be essentially extended. For example, the results of the above-mentioned paper [53] show how this method can be effectively applied to risk processes.

The book [49] contains a survey of new potential directions of studies related to quasi-stationary phenomena for perturbed stochastic systems. I believe it will be possible to combine asymptotic quasi-stationary and coupling methods for perturbed renewal equations and to obtain explicit upper bounds for rates of convergence in the corresponding limit, large deviation and ergodic theorems. In this book, results on quasi-stationary asymptotics for perturbed regenerative processes are applied to models of nonlinearly perturbed Markov chains and semi-Markov processes with finite phase spaces. An analogous program of research studies can be realised for Markov chains and semi-Markov type processes with countable and general phase spaces, which also can be embedded in the class of regenerative processes using the well known method of artificial regeneration. Another direction for advancing the studies carried out in this book is connected with models of nonlinearly perturbed Markov chains and semi-Markov processes with asymptotically uncoupled phase spaces. One of the latest my paper [66] presents new results in this research direction.

Stochastic approximation methods are, as was pointed out above, one of the most effective instruments for studies of complex financial contracts. Time-space skeleton approximation methods developed in the books [57, 58] can be applied to American-type contracts of different types. In particular, the above-mentioned results for American-type options with estimated parameters, options with random pay-offs, reselling options and knockout options given in the above books can be essentially extended. Also, these methods can be effectively applied to European and Asian options, as well as Bermudian and other types of exotic options. Another prospective area is associated with the combination of the above stochastic approximation methods with Monte Carlo type algorithms.

The latest actual direction of research of the author and some of his collaborators are asymptotic expansions for nonlinearly and singularly perturbed semi-Markov processes. The method of sequential phase space reduction proposed in the works [61,62,63, 65] allows, as was mentioned above, for the attainment of asymptotic expansions for different moment functionals for non-singularly and singularly perturbed semi-Markov process in two forms, without and with explicit upper bounds for remainders.

Both the class of semi-Markov type processes and the class of moment functionals can be essentially extended. For example, asymptotic expansions for power and power-exponential moments of hitting times and quasi-stationary distributions can be obtained for singularly perturbed stochastic processes with semi-Markov modulation.

Prospective directions of research studies based on the above methods are asymptotic expansions for perturbed semi-Markov type processes with multivariate perturbation parameters and asymptotic expansions based on non-polynomial systems of infinitesimals.

Further, an unbounded area of applications is perturbed queuing and reliability models, stochastic networks and bio-stochastic systems.

In conclusion, I would also like to mention some books which reflect my interests in some other scientific areas thematically connected with stochastic processes and their applications, [10, 25, 27, 33].

My journey in the world of stochastic processes continues.