A generalization of Ueno's inequality for n-step transition probabilities

• Autorzy: Andrzej S. Nowak
• Języki publikacji: EN

Abstrakty

EN

We provide a generalization of Ueno's inequality for n-step transition probabilities of Markov chains in a general state space. Our result is relevant to the study of adaptive control problems and approximation problems in the theory of discrete-time Markov decision processes and stochastic games.

Słowa kluczowe

EN

adaptive control; transition probabilities; stochastic control; Markov chains

• Wydawca: Institute of Mathematics Polish Academy of Sciences
• Czasopismo: Applicationes Mathematicae
• Rocznik: 1998-1999
• Tom: 25
• Numer: 3
• Strony: 295-299

Daty

wydano

1998

otrzymano

1996-12-09

poprawiono

1997-12-15

Twórcy

autor

Andrzej S. Nowak

• Institute of Mathematics, Wrocław University of Technology, Wybrzeże Wyspiańskiego 27, 50-370 Wrocław, Poland

Bibliografia

• [1] R. B. Ash, Real Analysis and Probability, Academic Press, New York, 1972.
• [2] J. P. Georgin, Contrôle de chaînes de Markov sur des espaces arbitraires, Ann. Inst. H. Poincaré Sér. B 14 (1978), 255-277.
• [3] O. Hernandez-Lerma, Adaptive Markov Control Processes, Springer, New York, 1989.
• [4] N. W. Kartashov, Criteria for uniform ergodicity and strong stability of Markov chains in general state space, Probab. Theory Math. Statist. 30 (1984), 65-81.
• [5] G. B. Di Masi and Ł. Stettner, Bayesian ergodic adaptive control of discrete time Markov processes, Stochastics and Stochastics Reports 54 (1995), 301-316.
• [6] A. S. Nowak and E. Altman, ε-Nash equilibria in stochastic games with uncountable state space and unbounded cost, Technical Report, Institute of Mathematics, Wrocław University of Technology, 1998.
• [7] W. J. Runggaldier and Ł. Stettner, Approximations of Discrete Time Partially Observed Control Problems, Appl. Math. Monographs 6, C.N.R., Pisa, 1994.
• [8] Ł. Stettner, On nearly self-optimizing strategies for a discrete-time uniformly ergodic adaptive model, Appl. Math. Optim. 27 (1993), 161-177.
• [9] T. Ueno, Some limit theorems for temporally discrete Markov processes, J. Fac. Sci. Univ. Tokyo 7 (1957), 449-462.