Abstract
In the present paper, the reward paths in non homogeneous semi-Markov systems in discrete time are examined with stochastic selection of the transition probabilities. The mean entrance probabilities and the mean rewards in the course of time are evaluated. Then the rate of the total reward for the homogeneous case is examined and the mean total reward is evaluated by means of p.g.f’s.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Y. Balcer and I. Sahin, “Pension accumulation as a semi-Markov reward process with applications to pension reform.” In Semi-Markov Models: Theory and Applications, pp. 181–199, Plenum Press: New York, 1986.
D. J. Bartholomew, Stochastic Models for Social Processes, Wiley: Chichester, 1982.
E. Cinlar, “Markov renewal theory,” Advances in Applied Probability vol. 1 pp. 123–187, 1969.
E. Cinlar, “Markov renewal theory: a survey,” Management Science vol. 21 pp. 727–752, 1975.
E. Cinlar, Introduction to Stochastic Processes, Prentice-Hall: Englewood Cliffs, NJ, 1975.
R. De Dominicis and R. Manca, “Some new results on the transient behaviour of semi-Markov reward processes,” X symposium on operations research, Part I, Sections 1-5 (Munich, 1985), Methods of Operations Research vol. 53 pp. 387–397, 1985.
R. A. Howard, Dynamic Probabilistic Systems, Wiley: Chichester, 1971.
A. Iosifescu - Manu, “Non homogeneous semi-Markov processes,” Studiisi Cercetuari Matematice vol. 24 pp. 529–533, 1972.
J. Janssen, “Semi-Markov models: theory and applications.” In J. Janssen (ed.), Plenum Press: New York, 1986.
J. Janssen and R. De Dominics, “Finite non homogeneous semi-Markov processes: theoretical and computational aspects,” Insurance: Mathematics and Economics vol. 3 pp. 157–165, 1984.
J. Janssen, R. Manca, and E. V. di Prignano, “Continuous time non homogeneous semi-Markov reward processes and multistate insurance application.” In 8th International Congress on Insurance: Mathematics and Economics, June 14–16, 2004, Rome 2004.
J. Janssen and N. Limnios, “Semi-Markov models and Applications.” In J. Janssen and N. Limnios (eds.), Kluwer: Dordrecht, 1999.
L. Jianyong and Z. Xiaobo, “On average reward semi-Markov decision processes with a general multichain structure,” Mathematics of Operations Research vol. 29(2) pp. 339–352, 2004.
J. Keilson, “On the matrix renewal function for Markov renewal processes,” Annals of Mathematical Statistics vol. 40 pp. 1901–1907, 1969.
J. Keilson, “A process with chain dependent growth rate. Markov Part II: the ruin and ergodic problems,” Advances in Applied Probability vol. 3 pp. 315–338, 1971.
N. Limnios and G. Oprisan, Semi-Markov Processes and Reliability, Birkhauser: Boston 2001.
Y. Masuda and U. Sumita, “A multivariate reward process defined on a semi-Markov process and its first-passage-time distributions,” Journal of Applied Probability vol. 28(2) pp. 360–373 1991.
Y. Masuda, “Partially observable semi-Markov reward processes,” Journal of Applied Probability vol. 30(3) pp. 548–560, 1993.
S. I. McClean, “A semi-Markovian model for a multigrade population,” Journal of Applied Probability vol. 17 pp. 846–852, 1980.
S. I. McClean, “Semi-Markov models for manpower planning.” In Semi-Markov Models: Theory and Applications, pp. 283–300, Plenum: New York, 1986.
R. A. Mclean and M. F. Neuts, “The integral of a step function defined on a semi-Markov process,” SIAM Journal on Applied Mathematics vol. 15 pp. 726–737, 1967.
A. A. Papadopoulou, “Counting transitions -Entrance probabilities in non homogeneous semi-Markov systems,” Applied Stochastic Models and Data Analysis vol. 13 pp.199–206, 1997.
A. A. Papadopoulou and P.-C. G. Vassiliou, “Asymptotic behavior of non homogeneous semi-Markov systems,” Linear Algebra and Its Applications vol. 210 pp. 153–198, 1994.
R. Pyke and R. A. Schaufele, “Limit theorem for Markov renewal process,” Annals of Mathematical Statistics vol. 55 pp. 1746–1764, 1964.
A. Reza Soltani and K. Khorshidian, “Reward processes for semi-Markov processes: asymptotic behaviour,” Journal of Applied Probability vol. 35 pp. 833–842, 1998.
S. M. Ross, Stochastic Processes, 2nd edn, Wiley: Chichester, 1996.
J. L. Teugels, “A bibliography on semi-Markov processes,” Journal of Computational and Applied Mathematics vol. 2 pp. 125–144, 1976.
P.-C. G. Vassiliou and A. A. Papadopoulou, “Non homogeneous semi-Markov systems and maintainability of the state sizes,” Journal of Applied Probability vol. 29 pp 519–534, 1992.
P.-C. G. Vassiliou, A. Georgiou, and N. Tsantas, “Control of asymptotic variability in non homogeneous Markov systems,” Journal of Applied Probability vol. 27 pp. 756–766, 1990.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Papadopoulou, A., Tsaklidis, G. Some Reward Paths in Semi-Markov Models with Stochastic Selection of the Transition Probabilities. Methodol Comput Appl Probab 9, 399–411 (2007). https://doi.org/10.1007/s11009-007-9027-5
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11009-007-9027-5