Adaptive control of discrete time Markov processes by the large deviations method

• Autorzy: T. E. Duncan , B. Pasik-Duncan , Łukasz Stettner
• Języki publikacji: EN

Abstrakty

EN

Some discrete time controlled Markov processes in a locally compact metric space whose transition operators depend on an unknown parameter are described. The adaptive controls are constructed using the large deviations of empirical distributions which are uniform in the parameter that takes values in a compact set. The adaptive procedure uses a finite family of continuous, almost optimal controls. Using the large deviations property it is shown that an adaptive control which is a fixed almost optimal control after a finite time is almost optimal with probability nearly 1.

Słowa kluczowe

EN

discrete time controlled Markov processes; adaptive control; large deviations

• Wydawca: Institute of Mathematics Polish Academy of Sciences
• Czasopismo: Applicationes Mathematicae
• Rocznik: 2000
• Tom: 27
• Numer: 3
• Strony: 265-285

Daty

wydano

2000

otrzymano

1999-01-19

Twórcy

autor

T. E. Duncan

• Department of Mathematics, University of Kansas, Lawrence, KS 66045, U.S.A.

autor

B. Pasik-Duncan

• Department of Mathematics, University of Kansas, Lawrence, KS 66045, U.S.A.

autor

Łukasz Stettner

• Institute of Mathematics, Polish Academy of Sciences, Śniadeckich 8, 00-950 Warszawa, Poland

Bibliografia

• [1] G. B. Di Masi and Ł. Stettner, Bayesian ergodic adaptive control of discrete time Markov processes, Stochastics Stochastics Rep. 54 (1995), 301-316.
• [2] M. D. Donsker and S. R. S. Varadhan, Asymptotic evaluation of certain Markov process expectations for large time, I, Comm. Pure Appl. Math. 28 (1975), 1-47.
• [3] M. D. Donsker and S. R. S. Varadhan, Asymptotic evaluation of certain Markov process expectations for large time--III, ibid. 29 (1976), 389-461.
• [4] M. Duflo, Formule de Chernoff pour des chaînes de Markov ( d'après Donsker et Varadhan), in: Grandes déviations et applications statistiques, Séminaire Orsay 1977-78, Astérisque 68 (1979), 99-124.
• [5] T. E. Duncan, B. Pasik-Duncan and Ł. Stettner, Discretized maximum likelihood and almost optimal control of ergodic Markov models, SIAM J. Control Optim. 36 (1998), 422-446.
• [6] O. Hernández-Lerma, Adaptive Markov Control Processes, Springer, 1976.
• [7] N. Maigret, Majorations de Chernoff pour des chaînes de Markov contrôlées, Z. Wahrsch. Verw. Gebiete 51 (1980), 133-151.
• [8] N. Maigret, Statistiques des chaînes controlés Felleriennes, in: Grandes déviations et applications statistiques, Séminaire Orsay, 1977-1978, Astérisque 68 (1979), 143-169.
• [9] Ł. Stettner, On nearly self-optimizing strategies for a discrete-time uniformly ergodic adaptive model, Appl. Math. Optim. 27 (1993), 161-177.