Funded by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Education and Culture Executive Agency (EACEA). Neither the European Union nor EACEA can be held responsible for them.


Markov decision processes: discrete stochastic dynamic programming

The book gives a very accessible introduction to the concepts of Markov Decision Processes. It is of great value to advanced-level students, researchers, and professional practitioners of this field to have now a complete volume with more than 600 pages. As I had a good feeling for finite state dynamic programming, this book helped me to get a better feeling for stationary processes an mainly value iteration to obtain optimal rules and policies. Discrete-time Markov decision processes require some studying (said in another way, it is not directly easy) and feeling for transient and absorbtion states and how decisions influence the shape of the Markov process. What I like about the book, is that it is rather complete.



Target Group:


Cite as:
M.L. Puterman (1994), Markov decision processes: discrete stochastic dynamic programming. Wiley, New York

Author of the review:
Eligius Hendrix
University of Malaga


You have to login to leave a comment. If you are not registered click here