Average cost Markov control processes with weighted norms: value iteration

• Autorzy: Evgueni Gordienko , Onésimo Hernández-Lerma
• Języki publikacji: EN

Abstrakty

EN

This paper shows the convergence of the value iteration (or successive approximations) algorithm for average cost (AC) Markov control processes on Borel spaces, with possibly unbounded cost, under appropriate hypotheses on weighted norms for the cost function and the transition law. It is also shown that the aforementioned convergence implies strong forms of AC-optimality and the existence of forecast horizons.

Słowa kluczowe

EN

average cost optimality equation; strong average optimality; (discrete-time) Markov control processes; long-run average cost; weighted norms

• Wydawca: Institute of Mathematics Polish Academy of Sciences
• Czasopismo: Applicationes Mathematicae
• Rocznik: 1995-1996
• Tom: 23
• Numer: 2
• Strony: 219-237

Daty

wydano

1995

otrzymano

1994-10-02

Twórcy

autor

Evgueni Gordienko

• Departamento de Matemáticas, Universidad Autónoma Mettropolitana-I, Apartado Postal 55-534, 09340 México D.F., Mexico

autor

Onésimo Hernández-Lerma

• Departamento de Matemáticas, Cinvestav-IPN, Apartado Postal 14-740, 07000 México D.F., Mexico

Bibliografia

• [1] D. P. Bertsekas, Dynamic Programming : Deterministic and Stochastic Models, Prentice-Hall, Englewood Cliffs, N.J., 1987.
• [2] E. B. Dynkin and A. A. Yushkevich, Controlled Markov Processes, Springer, New York, 1979.
• [3] J. Flynn, On optimality criteria for dynamic programs with long finite horizons, J. Math. Anal. Appl. 76 (1980), 202-208.
• [4] E. Gordienko and O. Hernández-Lerma, Average cost Markov control processes with weighted norms: existence of canonical policies, this volume, 199-218.
• [5] O. Hernández-Lerma, Adaptive Markov Control Processes, Springer, New York, 1989.
• [6] O. Hernández-Lerma and J. B. Lasserre, A forecast horizon and a stopping rule for general Markov decision processes, J. Math. Anal. Appl. 132 (1988), 388-400.
• [7] O. Hernández-Lerma and J. B. Lasserre, Average cost optimal policies for Markov control processes with Borel state space and unbounded costs, Systems Control Lett. 15 (1990), 349-356.
• [8] O. Hernández-Lerma and J. B. Lasserre, Linear programming and average optimality of Markov control processes on Borel spaces-unbounded costs, SIAM J. Control Optim. 32 (1994), 480-500.
• [9] O. Hernández-Lerma and J. B. Lasserre, Discrete-Time Markov Control Processes, book in preparation.
• [10] G. P. Klimov, Existence of a final distribution for an irreducible Feller process with invariant measure, Math. Notes 37 (1985), 161-163.
• [11] R. Montes-de-Oca and O. Hernández-Lerma, Value iteration in average cost Markov control processes on Borel spaces, Acta Appl. Math., to appear.
• [12] E. Nummelin, General Irreducible Markov Chains and Non-Negative Operators, Cambridge University Press, Cambridge, 1984.
• [13] R. Rempała, Forecast horizon in a dynamic family of one-dimensional control problems, Dissertationes Math. 315 (1991).
• [14] H. L. Royden, Real Analysis, 2nd ed., Macmillan, New York, 1971.
• [15] M. Schäl, Conditions for optimality and for the limit of n-stage optimal policies to be optimal, Z. Wahrsch. Verw. Gebiete 32 (1975), 179-196.
• [16] L. I. Sennott, Value iteration in countable state average cost Markov decision processes with unbounded costs, Ann. Oper. Res. 28 (1991), 261-272.
• [17] D. J. White, Dynamic programming, Markov chains, and the method of successive approximations, J. Math. Anal. Appl. 6 (1963), 373-376.