Download PDF - Infinite-horizon Markov control processes with undiscounted cost criteria: from average to overtaking optimalityAll rights reserved. Copyright exceptions and limitations apply

ArticleOriginal scientific text

Title

Infinite-horizon Markov control processes with undiscounted cost criteria: from average to overtaking optimality

Authors 1, 2

Affiliations

  1. Departamento de Matemáticas, CINVESTAV-IPN, A. Postal 14-740, México D.F. 07000, Mexico
  2. Departamento de Matemáticas, Universidad de Sonora, Blvd. Transversal y Rosales s/n, Hermosillo, Sonora, Mexico

Abstract

We consider discrete-time Markov control processes on Borel spaces and infinite-horizon undiscounted cost criteria which are sensitive to the growth rate of finite-horizon costs. These criteria include, at one extreme, the grossly underselective average cost

Keywords

ENG
uniform ergodicity, Lyapunov stability conditions, (discrete-time) Markov control processes, Poisson's equation, undiscounted cost criteria

Bibliography

  1. A. Arapostathis, V. S. Borkar, E. Fernández-Gaucherand, M. K. Ghosh and S. I. Marcus (1993), Discrete-time controlled Markov processes with average cost criterion: a survey, SIAM J. Control Optim. 31, 282-344.
  2. R. Bellman (1957), A markovian decision process,z J. Math. Mech. 6, 679-684.
  3. D. P. Bertsekas and S. E. Shreve (1978), Stochastic Optimal Control: The Discrete Time Case, Academic Press, New York.
  4. B. W. Brown (1965), On the iterative method of dynamic programming on a finite space discrete time Markov process, Ann. Math. Statist. 33, 719-726.
  5. D. A. Carlson, A. Haurie and A. Leizarowitz (1991), Infinite Horizon Optimal Control: Deterministic and Stochastic Systems, Springer, New York.
  6. E. V. Denardo and U. G. Rothblum (1979), Overtaking optimality for Markov decision chains, Math. Oper. Res. 4, 144-152.
  7. P. K. Dutta (1991), What do discounted optima converge to? A theory of discount rate asymptotics in economic models, J. Econom. Theory 55, 64-94.
  8. E. B. Dynkin and A. A. Yushkevich (1979), Controlled Markov Processes, Springer, New York.
  9. A. Ephremides and S. Verdú (1989), Control and optimization methods in communication network problems, IEEE Trans. Automat. Control 34, 930-942.
  10. E. Fernández-Gaucherand, M. K. Ghosh and S. I. Marcus (1994), Controlled Markov processes on the infinite planning horizon: weighted and overtaking criteria, Z. Oper. Res. 39, 131-155.
  11. J. Flynn (1980), On optimality criteria for dynamic programs with long finite horizons, J. Math. Anal. Appl. 76, 202-208.
  12. D. Gale (1967), On optimal development in a multi-sector economy, Rev. Econom. Stud. 34, 1-19. P. W. Glynn and S. P. Meyn (1996), A Lyapunov bound for solutions of Poisson's equation, Ann. Probab. 24, 916-931.
  13. E. Gordienko and O. Hernández-Lerma (1995a), Average cost Markov control processes with weighted norms: existence of canonical policies, Appl. Math. (Warsaw) 23, 199-218.
  14. E. Gordienko and O. Hernández-Lerma (1995b), Average cost Markov control processes with weighted norms: value iteration, ibid., 219-237.
  15. O. Hernández-Lerma (1989), Adaptive Markov Control Processes, Springer, New York.
  16. O. Hernández-Lerma, J. C. Hennet and J. B. Lasserre (1991), Average cost Markov decision processes: optimality conditions, J. Math. Anal. Appl. 158, 396-406.
  17. O. Hernández-Lerma and J. B. Lasserre (1996), Discrete-Time Markov Control Processes: Basic Optimality Criteria, Springer, New York.
  18. O. Hernández-Lerma and J. B. Lasserre (1997), Policy iteration for average cost Markov control processes on Borel spaces, Acta Appl. Math. 47, 125-154.
  19. O. Hernández-Lerma, R. Montes-de-Oca and R. Cavazos-Cadena (1991), Recurrence conditions for Markov decision processes with Borel state space: a survey, Ann. Oper. Res. 28, 29-46.
  20. O. Hernández-Lerma and M. Muñoz de Ozak (1992), Discrete-time Markov control processes with discounted unbounded cost: optimality criteria, Kybernetika (Prague) 28, 191-212.
  21. C. J. Himmelberg, T. Parthasarathy and F. S. Van Vleck (1976), Optimal plans for dynamic programming problems, Math. Oper. Res. 1, 390-394.
  22. A. Leizarowitz (1988), Controlled diffusion processes on infinite horizon with the overtaking criterion, Appl. Math. Optim. 17, 61-78.
  23. P. Mandl and M. Lausmanová (1991), Two extensions of asymptotic methods in controlled Markov chains, Ann. Oper. Res. 28, 67-79.
  24. S. P. Meyn (1995), The policy improvement algorithm for Markov decision processes with general state space, preprint, Coordinated Science Laboratory, Univ. of Illinois, Urbana, Ill.
  25. S. P. Meyn and R. L. Tweedie (1993), Markov Chains and Stochastic Stability, Springer, London.
  26. R. Montes-de-Oca and O. Hernández-Lerma (1996), Value iteration in average cost Markov control processes on Borel spaces, Acta Appl. Math. 42, 203-222.
  27. A. S. Nowak (1992), Stationary overtaking optimal strategies in Markov decision processes with general state space, preprint, Institute of Mathematics, Technical Univ. of Wrocław.
  28. E. Nummelin (1984), General Irreducible Markov Chains and Non-Negative Operators, Cambridge Univ. Press, Cambridge.
  29. S. Orey (1971), Lecture Notes on Limit Theorems for Markov Chain Transition Probabilities, Van Nostrand Reinhold, London.
  30. M. L. Puterman (1994), Markov Decision Processes, Wiley, New York.
  31. F. P. Ramsey (1928), A mathematical theory of savings, Econom. J. 38, 543-559.
  32. U. Rieder (1978), Measurable selection theorems for optimization problems, Manuscripta Math. 24, 115-131.
  33. P. J. Schweitzer (1985), On undiscounted Markovian decision processes with compact action spaces, RAIRO Rech. Opér. 19, 71-86.
  34. S. Stidham and R. Weber (1993), A survey of Markov decision models for control of networks of queues, Queueing Systems Theory Appl. 13, 291-314.
  35. O. Vega-Amaya (1996), Overtaking optimality for a class of production-inventory systems, preprint, Departamento de Matemáticas, Universidad de Sonora.
  36. A. F. Veinott, Jr. (1966), On finding optimal policies in discrete dynamic programming with no discounting, Ann. Math. Statist. 37, 1284-1294.
  37. C. C. von Weizsäcker (1965), Existence of optimal programs of accumulation for an infinite horizon, Rev. Econom. Stud. 32, 85-104.
  38. A. A. Yushkevich (1973), On a class of strategies in general Markov decision models, Theory Probab. Appl. 18, 777-779.
Applicationes Mathematicae
1998-1999/25/2
Export to PBN, Opens in new tab
Pages:
153-178
Main language of publication
English
Received
1997-03-06
Accepted
1997-07-15
Published
1998
Exact and natural sciences