Customer Reviews

Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond (Adaptive Computation and Machine Learning)

5 star:		(8)
4 star:		(2)
3 star:		(0)
2 star:		(0)
1 star:		(0)

 

 

The authors are young researchers who did their Ph.D. research in this rapidly developing branch of pattern recognition. Because they are young and are at the state of the art in the filed the book has sevral advantages and disadvantages and what I see as a disadvantage someone else might view as an advantage. Anyway here is my view.
Advantage 1: Pattern recognition is a field of many disciplines. It has been studied by statisticians, mathematician, probabilists and engineering and people that call themselves computer scientists specializing in artificial intelligence. The field is old and has a long history but each discipline has developed their own jargon and many times the wheel has been reinvented. The advantage of this book is that these young scientists don't see that awful history. They have learned and mastered their subject in a basically engineering jargon but they include many concepts from statistics and statistical learning theory that are not common to engineering texts. This includes such topics as robust regression, ridge regression and spline estimation. Much of the classical statistical literature is cited. The book contains over 600 references including much of the authors own work.
Disadvantage 1: Because they are young they miss some of the important historical literature and key texts. I found it a little disappointing that the bootstrap which is a statistical tool that has played a major role in discriminant analysis (particularly in the estimation of classification error rates) was completely overlooked. Also although many important texts on pattern recognition, machine learning and discriminant analysis are cited the fine text by McLachlan is overlooked as is the recent relevant text by Hastie, Tibshirani and Friedman.

Advantage 2: This book highlights the work of Vapnik and Chervonenkis and provides nice concise descriptions that one can easily refer to when needed. The mathematics is deep and includes reproducing kernel Hilbert space and many important properties from functional analysis and statistical theory.

Disadvantage 2: The authors are more experienced at writing professional papers than at writing text books. Consequently the book does not flow well and the authors freely admit in their preface that it is best not to read the book in sequential order but rather to take the suggestions in the preface that differ based on the readers background and interest.

Having said all this, for someone like me, who is very knowledgeable about statistical pattern recognition this is a great text for getting me up to speed on an exciting new area that I know very little about. I became curious about it when I started reading Vapnik recently.

I am hoping that a careful reading of this book will give me an intuition about why this approach that incorporates kernel methods can be a powerful tool in pattern recognition and classification.

This book should be a useful reference for anyone interested in this research area. It could be used in an engineering or statistics course in pattern recognition at either the undergraduate or graduate levels depending on what material is covered.

In a recent communication with Bernhard Scholkopf I learned that his book was sent for publication before the Hastie et al. book went to press. So that is the only reason it wasn't referenced. I think that point is worth my mentioning in an editing of this review. Also on reflection I do not think the disadvantages are so great as to remove a star. So it is 5 stars for them.

I can only hope that they will reference the work of McLachlan and Hastie et al. in their future books and research on this subject.

Great book, but a word of caution, it is not for the novice.
Book assumes a lot of background in functional analysis and
probability. True, it has extensive appendixes but they are
short-handing the relevant materials only. However, having said
that, this is a book worth struggling with even if you have not
yet got the intuitions in the above mentioned disciplines.

It is worthwhile (at least as I can tell) to read the book
skipping the tool chapters (2-6) going back to them when one has
a point where those are needed. I found that to be much easier
as it provides a concrete use of the methods putting them
in context.

This book is at the right level if you are already strong in Machine Learning theory. (e.g. Tom Mitchell's "Machine Learning").

Note that it is already getting somewhat dated. It for example includes little information on kernels for discreate structured input, such as trees and graphs.

It is the best book on kernel methods. It covers a wide range of subjects.

The best thing is that after finishing one or two basic chapters, you can read the rest of the book in any order; most chapters are almost independent to each other. At the beginning of a chapter, the authors list the prerequistites, so a reader knows whether he will be able to understand the chapter.

For now the book still reflects the state of art. But it is a fast changing field. I hope the authors will update the book in the future.

To start with, this book is rather technical with many theorems, assumptions, lengthy discussions. At the same time, it allows a beginner to start with some chapters and start exploring this field - from understanding point of view and even practical aspects. That was the case for me, started with suggested sections, i understood why this or that theorem is essential and what is the reasoning behind.
I find this book rather deep in learning with kernels and i suggest it as a reading for starting phD (and master?) students to get in grasp with concepts. And then delve into more details.

The book itself is nicely organized by topics and is not at all to be read chapter by chapter. Authors give some guidelines what to start according to your prior knowledge or experience.

I take one star off to show that "may be" there was a way to write about the same complex and fast developing domain better and easier to grasp for a reader :)

Excellent theory on SVMs and VC dimensionality. However, I found the chapters on optimization a bit terse. Otherwise, an essential reference for those interested in using SVMs in classification and regression.

To really get the most out of this book you will need to have good knowledge in functional analysis, constrained convex optimization and prob/stats. That being said, it is a great book to learn about basic SVM theory. I say "basic" theory because a big issue today is to learn hyperparameters of SVMs like sigma. Although chapter 16 of this book does talk about the Gaussian Process framework and how it can be used to estimate this hyperparameter, it is a little short and insufficient. I would recommend "Gaussian Processes for Machine Learning" by Rasmussen and Williams in order to get a good understanding of this more recent topic.

When I purchased this book I was overwhelmed by the math at first and had to put it down until this year. Now that my math skill have improved, I have gained a new appreciation for this book because of it through treatment of the subject. The math is advanced requiring graduate level skills. If you have never heard of a sigma-algebra, Borell set, or Hilbert space then you may want to read the easier book Learning With Kernels by John Shawe-Taylor and Nello Cristianini (though that book requires solid Linear Algebra skills too).

Learning With Kernels is the best book I have found on the subject. If you want to really understand Kernels you should learn the math just to read this book.

You will learn a lot from this book if you have a strong background in the following parts of mathematics:

1. Linear & Matrix Algebra
2. Functional analysis, (derivatives, gradients,...)
3. Optimization Theory (min ,max problems, convex optimization, unconstrained ,constrained optimizations)
4. Probability theory ( Bayes rule,and others like PDF ...)

Probably, the book will be a challenge for those who hasn't majored in applied mathematics. If you think, you don't have enough background, at a time you will get stuck in understanding a theorem.
Overall, it is the unique book that provides in depth analysis and explanation of kernel methods and support vector machines.

This book should be on the bookshell of anyone interested in kernel methods. The authors managed to make a clear and comprehensive enough textbook such that it is well suited for graduate students. But it also contains all the state of the art results of the domain, and its scope is wider than other similar books. For this reason, this book should be very useful to any researcher in the machine learning field.