Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond (Adaptive Computation and Machine Learning) [Hardcover]

Bernhard Schlkopf (Author), Alexander J. Smola (Author)

Book Description

Publication Date: December 15, 2001 | ISBN-10: 0262194759 | ISBN-13: 978-0262194754 | Edition: 1st

In the 1990s, a new type of learning algorithm was developed, based on results from statistical learning theory: the Support Vector Machine (SVM). This gave rise to a new class of theoretically elegant learning machines that use a central concept of SVMs -- -kernels--for a number of learning tasks. Kernel machines provide a modular framework that can be adapted to different tasks and domains by the choice of the kernel function and the base algorithm. They are replacing neural networks in a variety of fields, including engineering, information retrieval, and bioinformatics.Learning with Kernels provides an introduction to SVMs and related kernel methods. Although the book begins with the basics, it also includes the latest research. It provides all of the concepts necessary to enable a reader equipped with some basic mathematical knowledge to enter the world of machine learning using theoretically well-founded yet easy-to-use kernel algorithms and to understand and apply the powerful algorithms that have been developed over the last few years.

Editorial Reviews

Review

"Interesting and original. Learning with Kernels will make a fine textbook on this subject."--Grace Wahba, Bascom Professor of Statistics, University of Wisconsin Madison

"This splendid book fills the need for a comprehensive treatment of kernel methods and support vector machines. It collects results, theorems, and discussions from disparate sources into one very accessible exposition. I am particularly impressed that the authors have included problem sets at the end of each chapter; such problems are not easy to construct, but add significantly to the value of the book for the student audience."--Chris J. C. Burges, Microsoft Research

About the Author

Bernhard Scholkopf is Professor and Director at the Max Planck Institute for Biological Cybernetics in Tubingen, Germany. He is coauthor of Learning with Kernels (2002) and is a coeditor of Advances in Kernel Methods: Support Vector Learning (1998), Advances in Large-Margin Classifiers (2000), and Kernel Methods in Computational Biology (2004), all published by the MIT Press. Alexander J. Smola is Senior Principal Researcher and Machine Learning Program Leader at National ICT Australia/Australian National University, Canberra.

Product Details

• Hardcover: 644 pages
• Publisher: The MIT Press; 1st edition (December 15, 2001)
• Language: English
• ISBN-10: 0262194759
• ISBN-13: 978-0262194754
• Product Dimensions: 10.2 x 8.2 x 1.4 inches
• Shipping Weight: 3.3 pounds (View shipping rates and policies)
• Average Customer Review: 4.8 out of 5 stars  See all reviews (10 customer reviews)
• Amazon Best Sellers Rank: #93,564 in Books (See Top 100 in Books)
  • #9 in Books > Computers & Technology > Computer Science > Artificial Intelligence > Theory of Computing
  • #16 in Books > Computers & Technology > Computer Science > Artificial Intelligence > Machine Learning

More About the Author

Discover books, learn about writers, read author blogs, and more.

Customer Reviews

Most Helpful Customer Reviews

29 of 32 people found the following review helpful:

5.0 out of 5 stars machine learning via support vector machines and kernels, January 23, 2008

By 

Michael R. Chernick "statman31147" (Holland PA) - See all my reviews
(REAL NAME)   

This review is from: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond (Adaptive Computation and Machine Learning) (Hardcover)

The authors are young researchers who did their Ph.D. research in this rapidly developing branch of pattern recognition. Because they are young and are at the state of the art in the filed the book has sevral advantages and disadvantages and what I see as a disadvantage someone else might view as an advantage. Anyway here is my view.

Advantage 1: Pattern recognition is a field of many disciplines. It has been studied by statisticians, mathematician, probabilists and engineering and people that call themselves computer scientists specializing in artificial intelligence. The field is old and has a long history but each discipline has developed their own jargon and many times the wheel has been reinvented. The advantage of this book is that these young scientists don't see that awful history. They have learned and mastered their subject in a basically engineering jargon but they include many concepts from statistics and statistical learning theory that are not common to engineering texts. This includes such topics as robust regression, ridge regression and spline estimation. Much of the classical statistical literature is cited. The book contains over 600 references including much of the authors own work.

Disadvantage 1: Because they are young they miss some of the important historical literature and key texts. I found it a little disappointing that the bootstrap which is a statistical tool that has played a major role in discriminant analysis (particularly in the estimation of classification error rates) was completely overlooked. Also although many important texts on pattern recognition, machine learning and discriminant analysis are cited the fine text by McLachlan is overlooked as is the recent relevant text by Hastie, Tibshirani and Friedman.

Advantage 2: This book highlights the work of Vapnik and Chervonenkis and provides nice concise descriptions that one can easily refer to when needed. The mathematics is deep and includes reproducing kernel Hilbert space and many important properties from functional analysis and statistical theory.

Disadvantage 2: The authors are more experienced at writing professional papers than at writing text books. Consequently the book does not flow well and the authors freely admit in their preface that it is best not to read the book in sequential order but rather to take the suggestions in the preface that differ based on the readers background and interest.

Having said all this, for someone like me, who is very knowledgeable about statistical pattern recognition this is a great text for getting me up to speed on an exciting new area that I know very little about. I became curious about it when I started reading Vapnik recently.

I am hoping that a careful reading of this book will give me an intuition about why this approach that incorporates kernel methods can be a powerful tool in pattern recognition and classification.

This book should be a useful reference for anyone interested in this research area. It could be used in an engineering or statistics course in pattern recognition at either the undergraduate or graduate levels depending on what material is covered.

In a recent communication with Bernhard Scholkopf I learned that his book was sent for publication before the Hastie et al. book went to press. So that is the only reason it wasn't referenced. I think that point is worth my mentioning in an editing of this review. Also on reflection I do not think the disadvantages are so great as to remove a star. So it is 5 stars for them.

I can only hope that they will reference the work of McLachlan and Hastie et al. in their future books and research on this subject.

9 of 9 people found the following review helpful:

4.0 out of 5 stars In depth review of kernel methods in machine learning, October 24, 2005

By 

Doron Shamia - See all my reviews
(REAL NAME)   

This review is from: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond (Adaptive Computation and Machine Learning) (Hardcover)

Great book, but a word of caution, it is not for the novice.

Book assumes a lot of background in functional analysis and

probability. True, it has extensive appendixes but they are

short-handing the relevant materials only. However, having said

that, this is a book worth struggling with even if you have not

yet got the intuitions in the above mentioned disciplines.

It is worthwhile (at least as I can tell) to read the book

skipping the tool chapters (2-6) going back to them when one has

a point where those are needed. I found that to be much easier

as it provides a concrete use of the methods putting them

in context.

5 of 5 people found the following review helpful:

5.0 out of 5 stars Excellent overview of the theory of kernel-based methods, June 21, 2007

By 

Gabor Melli - See all my reviews
(REAL NAME)   

This review is from: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond (Adaptive Computation and Machine Learning) (Hardcover)

This book is at the right level if you are already strong in Machine Learning theory. (e.g. Tom Mitchell's "Machine Learning").

Note that it is already getting somewhat dated. It for example includes little information on kernels for discreate structured input, such as trees and graphs.