ArticleOriginal scientific text
Title
A generalization of Ueno's inequality for n-step transition probabilities
Authors 1
Affiliations
- Institute of Mathematics, Wrocław University of Technology, Wybrzeże Wyspiańskiego 27, 50-370 Wrocław, Poland
Abstract
We provide a generalization of Ueno's inequality for n-step transition probabilities of Markov chains in a general state space. Our result is relevant to the study of adaptive control problems and approximation problems in the theory of discrete-time Markov decision processes and stochastic games.
Keywords
adaptive control, transition probabilities, stochastic control, Markov chains
Bibliography
- R. B. Ash, Real Analysis and Probability, Academic Press, New York, 1972.
- J. P. Georgin, Contrôle de chaînes de Markov sur des espaces arbitraires, Ann. Inst. H. Poincaré Sér. B 14 (1978), 255-277.
- O. Hernandez-Lerma, Adaptive Markov Control Processes, Springer, New York, 1989.
- N. W. Kartashov, Criteria for uniform ergodicity and strong stability of Markov chains in general state space, Probab. Theory Math. Statist. 30 (1984), 65-81.
- G. B. Di Masi and Ł. Stettner, Bayesian ergodic adaptive control of discrete time Markov processes, Stochastics and Stochastics Reports 54 (1995), 301-316.
- A. S. Nowak and E. Altman, ε-Nash equilibria in stochastic games with uncountable state space and unbounded cost, Technical Report, Institute of Mathematics, Wrocław University of Technology, 1998.
- W. J. Runggaldier and Ł. Stettner, Approximations of Discrete Time Partially Observed Control Problems, Appl. Math. Monographs 6, C.N.R., Pisa, 1994.
- Ł. Stettner, On nearly self-optimizing strategies for a discrete-time uniformly ergodic adaptive model, Appl. Math. Optim. 27 (1993), 161-177.
- T. Ueno, Some limit theorems for temporally discrete Markov processes, J. Fac. Sci. Univ. Tokyo 7 (1957), 449-462.
Pages:
295-299
Main language of publication
English
Received
1996-12-09
Accepted
1997-12-15
Published
1998
Exact and natural sciences