Download PDF - Average cost Markov control processes with weighted norms: existence of canonical policiesAll rights reserved. Copyright exceptions and limitations apply

ArticleOriginal scientific text

Title

Average cost Markov control processes with weighted norms: existence of canonical policies

Authors 1, 2

Affiliations

  1. Departamento de Matemáticas, Universidad Autónoma Metropolitana-I, Apartado Postal 55-534, 09340 México D.F., Mexico
  2. Departamento de Matemáticas, Cinvestav-IPN, Apartado Postal 14-740, 07000 México D.F., Mexico

Abstract

This paper considers discrete-time Markov control processes on Borel spaces, with possibly unbounded costs, and the long run average cost (AC) criterion. Under appropriate hypotheses on weighted norms for the cost function and the transition law, the existence of solutions to the average cost optimality inequality and the average cost optimality equation are shown, which in turn yield the existence of AC-optimal and AC-canonical policies respectively.

Keywords

ENG
discounted cost, average cost optimality equation, long run average cost, (discrete-time) Markov control processes, average cost optimality inequality, weighted norms

Bibliography

  1. A. Arapostathis, V. S. Borkar, E. Fernández-Gaucherand, M. K. Ghosh and S. I. Marcus, Discrete-time controlled Markov processes with average cost criterion: a survey, SIAM J. Control Optim. 31 (1993), 282-344.
  2. D. P. Bersekas and S. E. Shreve, Stochastic Optimal Control: The Discrete Time Case, Academic Press, New York, 1978.
  3. E. B. Dynkin and A. A. Yushkevich, Controlled Markov Processes, Springer, New York, 1979.
  4. E. I. Gordienko, Controlled Markov processes with slowly varying characteristics. The problem of adaptive control. I, Soviet J. Comput. Syst. Sci. 23 (1985), 87-95.
  5. E. I. Gordienko and O. Hernández-Lerma, Average cost Markov control processes with weighted norms: value iteration, this volume, 219-237.
  6. O. Hernández-Lerma, Adaptive Markov Control Processes, Springer, New York, 1989.
  7. O. Hernández-Lerma, Average optimality in dynamic programming on Borel spaces-unbounded costs and controls, Systems Control Lett. 17 (1991), 237-242.
  8. O. Hernández-Lerma and J. B. Lasserre, Average cost optimal policies for Markov control processes with Borel state space and unbounded costs, ibid. 15 (1990), 349-356.
  9. O. Hernández-Lerma and J. B. Lasserre, Discrete-Time Markov Control Processes, book in preparation.
  10. O. Hernández-Lerma, R. Montes-de-Oca and R. Cavazos-Cadena, Recurrence conditions for Markov decision processes with Borel state space: a survey, Ann. Oper. Res. 28 (1991), 29-46.
  11. K. Hinderer, Foundations of Non-Stationary Dynamic Programming with Discrete Time Parameter, Lecture Notes Oper. Res. 33, Springer, New York, 1970.
  12. N. V. Kartashov, Inequalities in theorems of ergodicity and stability of Markov chains with common phase space. I, Theory Probab. Appl. 30 (1985), 247-259.
  13. N. V. Kartashov, Inequalities in theorems of ergodicity and stability of Markov chains with common phase space. II, ibid. 30 (1985), 507-515.
  14. N. V. Kartashov, Strongly stable Markov chains, J. Soviet Math. 34 (1986), 1493-1498.
  15. V. K. Malinovskiĭ, Limit theorems for Harris Markov chains, I, Theory Probab. Appl. 31 (1986), 269-285.
  16. R. Montes-de-Oca and O. Hernández-Lerma, Conditions for average optimality in Markov control processes with unbounded costs and controls, J. Math. Systems Estim. Control 4 (1994), 1-19.
  17. R. Montes-de-Oca and O. Hernández-Lerma, Value iteration in average cost Markov control processes on Borel spaces, Acta Appl. Math., to appear.
  18. E. Nummelin, General Irreducible Markov Chains and Non-Negative Operators, Cambridge University Press, Cambridge, 1984.
  19. E. Nummelin and P. Tuominen, Geometric ergodicity of Harris recurrent Markov chains with applications to renewal theory, Stochastic Process. Appl. 12 (1982), 187-202.
  20. S. Orey, Limit Theorems for Markov Chain Transition Probabilities, Van Nostrand Reinhold, London, 1971.
  21. U. Rieder, Measurable selection theorems for optimization problems, Manuscripta Math. 24 (1978), 115-131.
  22. H. L. Royden, Real Analysis, 2nd ed., Macmillan, New York, 1971.
  23. M. Schäl, Conditions for optimality and for the limit of n-stage optimal policies to be optimal, Z. Wahrsch. Verw. Gebiete 32 (1975), 179-196.
  24. M. Schäl, Average optimality in dynamic programming with general state space, Math. Oper. Res. 18 (1993), 163-172.
  25. R. Sznajder and J. A. Filar, Some comments on a theorem of Hardy and Littlewood, J. Optim. Theory Appl. 75 (1992), 201-209.
Applicationes Mathematicae
1995-1996/23/2
Export to PBN, Opens in new tab
Pages:
199-218
Main language of publication
English
Received
1994-10-02
Published
1995
Exact and natural sciences