On adaptive control of Markov chains using nonparametric estimation

ArticleOriginal scientific text

Title

Authors ¹, ²

Faculty of Economics, University of Białystok, Warszawska 63, 15-062 Białystok, Poland,
Institute of Mathematics, Polish Academy of Sciences, Śniadeckich 8, 00-950 Warszawa, Poland

Two adaptive procedures for controlled Markov chains which are based on a nonparametric window estimation are shown.

controlled Markov chain, estimation, adaptive control

R. Agraval, The continuum-armed bandit problem, SIAM J. Control Optim. 33 (1995), 1926-1951.
V. S. Borkar, Recursive self-tuning of finite Markov chains, Appl. Math. (Warsaw) 24 (1996), 169-188.
E. Drabik, On nearly selfoptimizing strategies for multiarmed bandit problems with controlled arms, ibid. 23 (1996), 449-473.
T. Duncan, B. Pasik-Duncan and Ł. Stettner, Discretized maximum likelihood and almost optimal adaptive control of ergodic adaptive models, SIAM J. Control Optim. 36 (1998), 422-446.
T. Duncan, B. Pasik-Duncan and Ł. Stettner, Adaptive control of discrete Markov processes by the method of large deviations, in: Proc. 35th IEEE CDC, Kobe 1996, IEEE, 360-365.
O. Hernández-Lerma and R. Cavazos-Cadena, Density estimation and adaptive control of Markov processes; average and discounted criteria, Acta Appl. Math. 20 (1990), 285-307.
A. Nowak, A generalization of Ueno's inequality for n-step transition probabilities, Appl. Math. (Warsaw) 25 (1998), 295-299.