Neuro-Dynamic Programming (Optimization and Neural Computation Series, 3) [Hardcover]

Dimitri P. Bertsekas (Author), John N. Tsitsiklis (Author), John Tsitsiklis (Author), Dimitri P. Bertsekas (Author), John Tsitsiklis (Author), John N. Tsitsiklis (Author)

Book Description

ISBN-10: 1886529108 | ISBN-13: 978-1886529106 | Publication Date: May 1, 1996 | Edition: 1

This is the first textbook that fully explains the neuro-dynamic programming/reinforcement learning methodology, which is a recent breakthrough in the practical application of neural networks and dynamic programming to complex problems of planning, optimal decision making, and intelligent control.

Neuro-dynamic programming uses neural network approximations to overcome the "curse of dimensionality" and the "curse of modeling" that have been the bottlenecks to the practical application of dynamic programming and stochastic control to complex problems. The methodology allows systems to learn about their behavior through simulation, and to improve their performance through iterative reinforcement.

This book provides the first systematic presentation of the science and the art behind this exciting and far-reaching methodology.

The book develops a comprehensive analysis of neuro-dynamic programming algorithms, and guides the reader to their successful application through case studies from complex problem areas.

Editorial Reviews

Review

Neurodynamic Programming is a remarkable monograph that integrates a sweeping mathematical and computational landscape into a coherent body of rigorous knowledge. The topics are current, the writing is clear and to the point, the examples are comprehensive and the historical notes and comments are scholarly."

"In this monograph, Bertsekas and Tsitsiklis have performed a Herculean task that will be studied and appreciated by generations to come. I strongly recommend it to scientists and engineers eager to seriously understand the mathematics and computations behind modern behavioral machine learning. --George Cybenko in IEEE Computational Science and Engineering, May 1998:

About the Author

The authors are Professors of Electrical Engineering and Computer Science at the Massachusetts Institute of Technology, and members of the National Academy of Engineering.

Product Details

• Hardcover: 512 pages
• Publisher: Athena Scientific; 1 edition (May 1, 1996)
• Language: English
• ISBN-10: 1886529108
• ISBN-13: 978-1886529106
• Product Dimensions: 9.2 x 6.1 x 1.1 inches
• Shipping Weight: 1.8 pounds (View shipping rates and policies)
• Average Customer Review: 5.0 out of 5 stars  See all reviews (2 customer reviews)
• Amazon Best Sellers Rank: #627,791 in Books (See Top 100 in Books)
  • #88 in Books > Computers & Technology > Computer Science > Artificial Intelligence > Neural Networks

More About the Author

Discover books, learn about writers, read author blogs, and more.

Customer Reviews

Most Helpful Customer Reviews

2 of 2 people found the following review helpful:

5.0 out of 5 stars One of my favorites, August 12, 2009

By 

Abhijit Gosavi - See all my reviews

This review is from: Neuro-Dynamic Programming (Optimization and Neural Computation Series, 3) (Hardcover)

This was one of my favorite books as a student and it still is. The book presents much of the theory underlying reinforcement learning (the authors like to call it neuro-dynamic programming) in a clear and compact manner. While there are detailed mathematical proofs in many chapters, the book is actually pretty easy to read! It for sure helped me build an intuitive understanding of the mechanism of reinforcement learning when I was a student. The book is certaintly useful for beginners to this area; Ph.D. students are likely to get even more out of it.

Some special features of the book are:

* A clear discussion on the connection between classical dynamic programming and reinforcement learning (RL) along with the links to the Robbins-Monro algorithm
* Discussion on numerous types of approximate policy iteration
* A clear discussion on TD(lambda)
* Extensions to average reward (cost) problems
* A rigorous discussion of how function approximation works within this framework
* Treatment of the stochastic shortest path problem
* Detailed proofs of convergence of numerous NDP/RL algorithms
* Material/ideas on numerous topics on NDP/RL (for future research)

5 of 7 people found the following review helpful:

5.0 out of 5 stars From the author of Approximate Dynamic Programming, December 15, 2007

By 

Warren B. Powell (Princeton, NJ USA) - See all my reviews
(REAL NAME)   

This review is from: Neuro-Dynamic Programming (Optimization and Neural Computation Series, 3) (Hardcover)

Neuro-Dynamic Programming was, and is, a foundational reference for anyone wishing to work in the field that goes under names such as approximate dynamic programming, adaptive dynamic programming, reinforcement learning or, as a result of this book, neuro-dynamic programming. This is a clearly written treatment of the theory behind methods to solve dynamic programs by approximating the value function (in this book, the cost-to-go function).

The book is primarily for doctoral students and researchers. It provides descriptions of many solution strategies, but not at the level of detailed recipes. The presentation focuses on theory, but at a very readable level. Building on the prior work of the authors, this is the first book that brings together approximation methods in dynamic programming, with the theory of stochastic approximation methods (with its origins in Robbins and Monro) that provide the foundation for convergence proofs.

This book, along with Reinforcement Learning: An Introduction (Adaptive Computation and Machine Learning) by Sutton and Barto, were major references when I started my own work in this field, leading up to my book: Approximate Dynamic Programming: Solving the Curses of Dimensionality (Wiley Series in Probability and Statistics) published by John Wiley and Sons. My students still use Neuro-Dynamic Programming as a reference for their research.

Warren Powell
Professor
Operations Research and Financial Engineering
Princeton University