Reinforcement Learning: An Introduction (Adaptive Computation and Machine Learning) [Hardcover]

Richard S. Sutton (Author), Andrew G. Barto (Author)

Book Description

ISBN-10: 0262193981 | ISBN-13: 978-0262193986 | Publication Date: March 1, 1998

Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability.The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

Editorial Reviews

Review

"This is a groundbreaking work, dealing with a subject that you would have expected to have been sorted out right at the start of AI... This isn't a simple theory but many of the ideas and methods are practically useful and if you have an interest in neural networks or learning systems then you need to study this book for the six months it deserves!" -- Mike James, Computer Shopper, November 1998

About the Author

Richard S. Sutton is Senior Research Scientist, Department of Computer Science, University of Massachusetts. Andrew G. Barto is Professor of Computer Science at the University of Massachusetts.

Product Details

• Reading level: Ages 18 and up
  
• Hardcover: 322 pages
• Publisher: A Bradford Book (March 1, 1998)
• Language: English
• ISBN-10: 0262193981
• ISBN-13: 978-0262193986
• Product Dimensions: 9.4 x 7.3 x 1 inches
• Shipping Weight: 1.8 pounds (View shipping rates and policies)
• Average Customer Review: 4.3 out of 5 stars  See all reviews (15 customer reviews)
• Amazon Best Sellers Rank: #32,775 in Books (See Top 100 in Books)
  • #1 in Books > Computers & Technology > Computer Science > Artificial Intelligence > Theory of Computing
  • #3 in Books > Computers & Technology > Computer Science > Artificial Intelligence > Neural Networks
  • #4 in Books > Computers & Technology > Computer Science > Artificial Intelligence > Computer Mathematics

More About the Author

Discover books, learn about writers, read author blogs, and more.

Customer Reviews

Most Helpful Customer Reviews

24 of 26 people found the following review helpful:

5.0 out of 5 stars An excellent introduction to Reinforcement Learning, February 29, 2000

By 

David Tan (USA) - See all my reviews

This review is from: Reinforcement Learning: An Introduction (Adaptive Computation and Machine Learning) (Hardcover)

This is probably one of the best book I have read in the past 1 year.

The authors present the subject with an excellent balance of mathematical, computational and intuitive material. The book also includes plenty of real-life examples to explain the concepts and motivations for the algorithms.

The book starts with examples and intuitive introduction and definition of reinforcement learning. It follows with 3 chapters on the 3 fundamental approaches to reinforcement learning: Dynamic programming, Monte Carlo and Temporal Difference methods. Subsequent chapters build on these methods to generalize to a whole spectrum of solutions and algorithms.

The book is very readable by average computer students. Possibly the only difficult one is chapter 8, which deals with some neural network concepts.

I highly recommend this book to anyone who wants to learn about this subject.

14 of 15 people found the following review helpful:

5.0 out of 5 stars Excellent introduction to reinforcement learning, August 3, 2003

By 

Mihailo Despotovic (Silicon Valley, CA USA) - See all my reviews
(REAL NAME)   

This review is from: Reinforcement Learning: An Introduction (Adaptive Computation and Machine Learning) (Hardcover)

I have this book more than a year now and I am going through it for the second time, so I think I have a pretty good picture about it.

The book consists of three parts. In the first part, "The Problem", the authors define the scope of issues reinfocement learning is dealing with and they give some interesting introductory examples. Then, they move on to the concept of evaluative feedback and, eventually, define the reinforcement learning problem formally.

The second part, "Elementary Solution Methods" consists of three more-less independent subparts: Dynamic Programming, Monte Carlo Methods and Temporal Difference Learning. All three fundamental reinforcement learning methods are presented in an interesting way and using good examples. Personally, I liked the TD-Learning part best and I agree that this method is indeed the central method and an original contribution of reinforecement learning to the field of machine learning.

The third part, "A Unified View" present more advanced techniques. The last chapter gives the most important case studies in reinforcement learning including Samuel's Checkers Player and Thesauro's TD-Gammon.

The book is very readable and every chapter ends with illustrative exercises (many of them actually are real programming projects!), always useful summary and very valuable bibliographical and historical remarks. Some subchapters are more advanced and therefore marked with '*'. I really recommend first two parts to any student ofd computer science or anyone interested in machine learning and fuzzy computing. The third part is much more advanced but it would be definitely interesting for advanced computer scientists and graduate students.

This is still the first edition of the book which means that the material is almost six years old, but it's the third printing, so there is lot of interest and I would suggest (for second edition) that authors include solutions to (at least selected) exercises, something like Knuth did in "The Art of Computer Programming".

9 of 9 people found the following review helpful:

3.0 out of 5 stars || Good Reference- Much of a draft version||, February 8, 2009

By 

Kausik Ghatak - See all my reviews

This review is from: Reinforcement Learning: An Introduction (Adaptive Computation and Machine Learning) (Hardcover)

I am a software developer and worked on applying Reinforcement Learning (RL) in cognitive fields for my patent work (pending).

This book is highly regarded in RL literature and is probably one of the few hand counted books that explicitly address RL as a subject. The book has good balance between subject matter and theory which makes it unique.

However this book has many serious drawbacks. Had there been excellent books on this subject I would have discouraged you to have this one. Rather my advice would be referring to the book "Approximate Dynamic Programming" by "Warren B. Powell" as well. This DP book has formalized the terms used in Sutton's book. This might save you from the ambiguous terminologies used in Sutton's book.

You might like to refer to these two excellent & precise works by Abhijjit Gosavi: //web.mst.edu/ ~gosavia/ tutorial.pdf and //web.mst.edu/ ~gosavia/ joc.pdf.