1. Introduction........................................................................................................ 5 2. Preliminary results and assumptions.................................................................. 7 3. Approximation of the invariant measure.............................................................. 14 4. Construction of nearly optimal control functions................................................ 24 4.1. Approximation of admissible control functions.................................. 24 4.2. State space discretization...................................................................... 25 4.3. Comments on further discretizations................................................... 31 5. Nearly optimal control values................................................................................. 31 6. An example................................................................................................................ 35 References....................................................................................................... 36
Institute of Mathematics, Polish Academy of Sciences, Śniadeckich 8, 00-950 Warszawa, Poland
Bibliografia
[1] A. Arapostatis, V. S. Borkar, E. Fernandez-Gaucherand, M. K. Ghosh, and S. Marcus, Discrete-time controlled Markov processes with average cost criterion, a survey, SIAM J. Control Optim. 31 (1993), 282-344.
[2] A. Arapostatis, E. Fernandez-Gaucherand and S. Marcus, Analysis of an adaptive partially observed controlled Markov chain, IEEE Trans. Automat. Control 38 (1993), 987-992.
[3] D. P. Bertsekas and S. E. Schreve, Stochastic Optimal Control: The Discrete Time Case, Academic Press, New York, 1978.
[4] P. Billingsley, Convergence of Probability Measures, Wiley, New York, 1968.
[5] G. Di Masi and Ł. Stettner, On adaptive control of a partially observed Markov chain, Applicationes Math. 22 (1994), 165-180.
[6] J. L. Doob, Stochastic Processes, Wiley, New York, 1953.
[7] O. Hernandez-Lerma, Adaptive Markov Control Processes, Springer, New York, 1989.
[8] M. Kurano, On the existence of an optimal stationary J-policy in non-discounted Markovian decision processes with incomplete state information, Bull. Math. Statist. 17 (1977), 77-81.
[9] H. J. Kushner and P. G. Dupuis, Numerical Methods for Stochastic Control Problems in Continuous Time, Springer, 1993.
[10] H. L. Royden, Real Analysis, MacMillan, New York, 1968.
[11] W. Runggaldier and Ł. Stettner, Nearly optimal controls for stochastic ergodic problems with partial observation, SIAM J. Control Optim. 31 (1993), 180-218.
[12] W. Runggaldier and Ł. Stettner, Partially observable control problems with compulsory shifts of the state, IIASA Working Paper 92-34, May 1992.
[13] W. Runggaldier and Ł. Stettner, Approximations of Discrete Time Partially Observed Control Problems, Appl. Math. Monographs CNR, in preparation.
[14] F. Schoen, Stochastic techniques for global optimizationg: a survey of recent advances, J. Global Optimization 1 (1991), 207-228.
[15] Ł. Stettner, On invariant measures of filtering processes, in: Proc. 4-th Bad Honnef Conf. on Stochastic Differential Systems, Lecture Notes in Control and Inform. Sci. 126, Springer, 1989, 279-292.
[16] Ł. Stettner, Ergodic control of partially observed Markov processes with equivalent transition probabilities, Applicationes Math. 22 (1993), 25-38.
[17] K. Wakuta, Semi-Markov decision processes with incomplete state observation-average cost criterion, J. Oper. Res. Soc. Japan 24 (1981), 95-108.
[18] R. Zieliński und P. Neumann, Stochastische Verfahren zur Suche nach dem Minimum einer Funktion, Akademie-Verlag, Berlin, 1983.