Download PDF - Average cost Markov control processes with weighted norms: value iterationAll rights reserved. Copyright exceptions and limitations apply

ArticleOriginal scientific text

Title

Average cost Markov control processes with weighted norms: value iteration

Authors 1, 2

Affiliations

  1. Departamento de Matemáticas, Universidad Autónoma Mettropolitana-I, Apartado Postal 55-534, 09340 México D.F., Mexico
  2. Departamento de Matemáticas, Cinvestav-IPN, Apartado Postal 14-740, 07000 México D.F., Mexico

Abstract

This paper shows the convergence of the value iteration (or successive approximations) algorithm for average cost (AC) Markov control processes on Borel spaces, with possibly unbounded cost, under appropriate hypotheses on weighted norms for the cost function and the transition law. It is also shown that the aforementioned convergence implies strong forms of AC-optimality and the existence of forecast horizons.

Keywords

ENG
average cost optimality equation, strong average optimality, (discrete-time) Markov control processes, long-run average cost, weighted norms

Bibliography

  1. D. P. Bertsekas, Dynamic Programming : Deterministic and Stochastic Models, Prentice-Hall, Englewood Cliffs, N.J., 1987.
  2. E. B. Dynkin and A. A. Yushkevich, Controlled Markov Processes, Springer, New York, 1979.
  3. J. Flynn, On optimality criteria for dynamic programs with long finite horizons, J. Math. Anal. Appl. 76 (1980), 202-208.
  4. E. Gordienko and O. Hernández-Lerma, Average cost Markov control processes with weighted norms: existence of canonical policies, this volume, 199-218.
  5. O. Hernández-Lerma, Adaptive Markov Control Processes, Springer, New York, 1989.
  6. O. Hernández-Lerma and J. B. Lasserre, A forecast horizon and a stopping rule for general Markov decision processes, J. Math. Anal. Appl. 132 (1988), 388-400.
  7. O. Hernández-Lerma and J. B. Lasserre, Average cost optimal policies for Markov control processes with Borel state space and unbounded costs, Systems Control Lett. 15 (1990), 349-356.
  8. O. Hernández-Lerma and J. B. Lasserre, Linear programming and average optimality of Markov control processes on Borel spaces-unbounded costs, SIAM J. Control Optim. 32 (1994), 480-500.
  9. O. Hernández-Lerma and J. B. Lasserre, Discrete-Time Markov Control Processes, book in preparation.
  10. G. P. Klimov, Existence of a final distribution for an irreducible Feller process with invariant measure, Math. Notes 37 (1985), 161-163.
  11. R. Montes-de-Oca and O. Hernández-Lerma, Value iteration in average cost Markov control processes on Borel spaces, Acta Appl. Math., to appear.
  12. E. Nummelin, General Irreducible Markov Chains and Non-Negative Operators, Cambridge University Press, Cambridge, 1984.
  13. R. Rempała, Forecast horizon in a dynamic family of one-dimensional control problems, Dissertationes Math. 315 (1991).
  14. H. L. Royden, Real Analysis, 2nd ed., Macmillan, New York, 1971.
  15. M. Schäl, Conditions for optimality and for the limit of n-stage optimal policies to be optimal, Z. Wahrsch. Verw. Gebiete 32 (1975), 179-196.
  16. L. I. Sennott, Value iteration in countable state average cost Markov decision processes with unbounded costs, Ann. Oper. Res. 28 (1991), 261-272.
  17. D. J. White, Dynamic programming, Markov chains, and the method of successive approximations, J. Math. Anal. Appl. 6 (1963), 373-376.
Applicationes Mathematicae
1995-1996/23/2
Export to PBN, Opens in new tab
Pages:
219-237
Main language of publication
English
Received
1994-10-02
Published
1995
Exact and natural sciences