The Nature of Statistical Learning Theory [Hardcover]

Vladimir N. Vapnik (Author)

Book Description

ISBN-10: 0387945598 | ISBN-13: 978-0387945590 | Publication Date: December 14, 1998 | Edition: 1st ed. 1995. Corr. 2nd printing

The aim of this book is to discuss the fundamental ideas which lie behind the statistical theory of learning and generalization. It considers learning from the general point of view of function estimation based on empirical data. Omitting proofs and technical details, the author concentrates on discussing the main results of learning theory and their connections to fundamental problems in statistics. These include: - the general setting of learning problems and the general model of minimizing the risk functional from empirical data - a comprehensive analysis of the empirical risk minimization principle and shows how this allows for the construction of necessary and sufficient conditions for consistency - non-asymptotic bounds for the risk achieved using the empirical risk minimization principle - principles for controlling the generalization ability of learning machines using small sample sizes - introducing a new type of universal learning machine that controls the generalization ability.

Editorial Reviews

Review

"This interesting book helps a reader to understand the interconnections between various streams in the empirical modeling realm and may be recommended to any reader who feels lost in modern terminology." V.V. Fedorov, Oak Ridge National Laboratory, USA

Product Details

• Hardcover: 188 pages
• Publisher: Springer; 1st ed. 1995. Corr. 2nd printing edition (December 14, 1998)
• Language: English
• ISBN-10: 0387945598
• ISBN-13: 978-0387945590
• Product Dimensions: 9.3 x 5.6 x 0.8 inches
• Shipping Weight: 1 pounds
• Average Customer Review: 4.2 out of 5 stars  See all reviews (5 customer reviews)
• Amazon Best Sellers Rank: #4,294,083 in Books (See Top 100 in Books)

More About the Author

Discover books, learn about writers, read author blogs, and more.

Customer Reviews

Most Helpful Customer Reviews

14 of 14 people found the following review helpful:

5.0 out of 5 stars A very nice book to get ideas on support vector machines, August 27, 2000

By 

Random Thoughts (Maryland) - See all my reviews

This review is from: The Nature of Statistical Learning Theory (Information Science and Statistics) (Hardcover)

This is a very readable book by an authority on this subject. The book starts with the statistical learning theory, pioneered by the author and co-worker's work, and gradually leads to the path of discovery of support vector machines. An excellent and distinctive property of support vector machines is that they are robust to small data perturbation and have good generalization ability with function complexity being controlled by VC dimension. The treatment of nonlinear kernel classification and regression is given for the first time in the first edition. The 2nd edition includes significant updates including a separate chapter on support vector regression as well as a section on logistic regression using the support vector approach. Most computations involved in this book can be implemented using a quadratic programming package. The connections of support vector machines to traditional statistical modeling such as kernel density and regression and model selection are also discussed. Thus, this book will be an excellent starting point for learning support vector machines.

23 of 26 people found the following review helpful:

3.0 out of 5 stars worth reading, September 21, 2001

By 

a reather presumptous reader (NY) - See all my reviews

This review is from: The Nature of Statistical Learning Theory (Information Science and Statistics) (Hardcover)

A good, albeit highly idiosyncratic, guide to Statistical Learning. The highly personal account of the theory is both the strong point and the drawback of the treatise. On one side, Vapnick never loses sight of the big picture, and gives illuminating insights and formulations of the "basic problems" (as he calls them), that are not found in any other book. The lack of proofs and the slightly erratic organization of the topic make for a brisk, enjoyable reading. On the minus side, the choice of the topics is very biased. In this respect, the book is a self-congratulatory tribute by the author to himself: it appears that the foundations of statistical learning were single-handedly laid by him and his collaborators. This is not really the case. Consistency of the Empircal Risk Measure is rather trivial from the viewpoint of a personal trained in asymptotic statistics, and interval estimators for finite data sets are the subject of much advanced statistical literature. Finally, SVMs and neural nets are just a part of the story, and probably not the most interesting.
In a nutshell, what Vapnick shows, he shows very well, and is able to provide the "why" of things as no one else. What he doesn't show... you'll have to find somewhere else (the recent Book of Friedman Hastie & Tibs is an excellent starting point).
A last remark. The book is rich in grammatical errors and typos. They could have been corrected in the second edition, but do not detract from the book's readability.

3 of 3 people found the following review helpful:

5.0 out of 5 stars Remarkably readable tour of one path into machine learning, May 12, 2008

By 

A. Khalak - See all my reviews
(REAL NAME)   

This review is from: The Nature of Statistical Learning Theory (Information Science and Statistics) (Hardcover)

This book is meant to be a popularization, of sorts, of the material covered in the considerably more formal and detailed treatment, "Statistical Learning Theory." Some of the other reviewers have commented on how Vapnik's subjective perspective is not as evenhanded as they would like. However, I would not have it any other way. I really enjoyed the fact that he has an organic understanding of the field and he expresses his opinions about it in a relatively unvarnished way; it is undeniable that he played a central role in it. Most readers of this kind of thing should be mature enough to deal with the subjectivity that an author must have in talking about the relevance of their own life's work. He is a bit dismissive of work that he believes is either competitive or is derivative/overlapping with his own (as other reviewers pointed out, this includes nearly all of the American work in the 1980's and 90's).

The benefits of such subjectivity is a framing of the problems of machine learning in the context of the grand scheme of mathematics/statistics. The book has many insights that would usually be reserved only for lectures. Since it is subjective, it is not PC and he gives his (rather valuable) opinions and insights. I really appreciated that. The connections to philosophical work in induction (Kant, Popper) and the formalization of this into a study of statistical induction was a brilliant section, though it was clear that the argument was more a interpretation for the risk formulation than an encoding of the philosophical texts. You either find that sort of thing interesting or you don't.

In summary, a unique portal into understanding Vapnik's extremely insightful point of view on the subject. He has obviously thought very deeply about topics that he's writing about, and it came through.