Customer Reviews

Pattern Recognition and Machine Learning (Information Science and Statistics)

5 star:		(29)
4 star:		(13)
3 star:		(6)
2 star:		(6)
1 star:		(6)

 

 

This new book by Chris Bishop covers most areas of pattern recognition quite exhaustively. The author is an expert, this is evidenced by the excellent insights he gives into the complex math behind the machine learning algorithms. I have worked for quite some time with neural networks and have had coursework in linear algebra, probability and regression analysis, and found some of the stuff in the book quite illuminating.

But that said, I must point out that the book is very math heavy. Inspite of my considerable background in the area of neural networks and statistics, I still was struggling with the equations. This is certainly not the book that can teach one things from the ground up, and thats why I would give it only 3 stars. I am new to kernels, and I am finding the relevant chapters difficult and confusing. This book wont be very useful if all you want to do is write machine learning code. The intended audience for this book I guess are PhD students/researchers who are working with the math related aspects of machine learning. Undergraduates or people with little exposure to machine learning will have a hard time with this book. But that said, time spent in struggling with the contents of this book will certainly pay-off, not instantly though.

I can appreciate others who might think that this is a great book.... but I am a student using it and I have some very different opinions of it.

First, although Mr. Bishop is clearly an expert in Machine Learning, he is also obviously a HUGE fan of Bayesian Statistics. The title of the book is misleading as it makes no mention of Bayes at all but EVERY CHAPTER ends with how all of the chapter's contents are combined in a Bayes method. That's not bad it's just not clear from the title. The title should be appended with "... using Bayesian Methods"

Second, while it is certainly a textbook, the author clearly has an understanding of the material that seems to undermine his ability to explain it. Though there are mentions of examples there are, in fact, none. There are many graphics and tiny, trivial indicators, but I can't help to think that every single one of the concepts in the book would have benefited from even a single application. There aren't any. I am lead to believe that if you are already aware of many of the methods and techniques that this would be an excellent reference or refresher. As a student starting out I almost always have no idea what his intentions are.

To make matter worse, he occasionally uses symbols that are flat-out confusing. Why would you use PI for anything other than Pi or Product? He does. Why use little k, Capital K, and Greek Letter Kappa (a K!) in a series of explanations. He does. He even references articles that he has written... in 2008!!

Every chapter seems to be an exercise to see how many equations he can stuff in it. There are 300 in Chapter 2 alone. Over and over and over again I have the feeling that he is trying to TELL me how to ride a bicycle when it would have been so much easier to at least let me see the view from behind the handle bars with my feet on the pedals. Chapter five on Neural Nets, for example, is abysmally over-complicated. Would you hand someone a dictionary and ask them to write a poem? ("Hey, all the words you need are in here!") Of course not.

Third, the book mentions that there is a lot of information available on the web site. The only info available on his website is a brief overview of the text, a detailed overview of the text (that's not a typo.... he has both), an example chapter, links to where the book can be purchased, and (actually, quite useful for creating slides) an archive of all of the figures available in the book. There are no answers to problems or explorations of any part of the material. The upcoming book might be amazing and exactly what I am looking for but it could be months away and another $50 or so to purchase it. Hardly ideal. How about putting some of that MatLab code on your site? *Something* to crystalize the concepts!

Finally, while the intro indicates this might be a good book for Computer Scientists it would actually make more sense to call it a Math book. More specifically a Statistics book. There are no methods, no algorithms, no bits of pseudo-code, and (again) no applications are in the text. Even examples that actually used hard numbers and/or elements from a real problem and explained would be much appreciated.

Maybe I am being a little critical and perhaps I want for too much but in my mind if you are writing a book with the goal of TEACHING a subject, it would be in your interest to make things clear and illustrative. Instead, the book feels more like a combination of "I am smart. Just read this!" and a reference text.

The book is worth a look, but after some of 5 star reviews i read here, it was quite a disappointment. Yes, the book covers a lot of ground. Yes, the book has lots of nice pictures and easy examples, but that is exactly the problem. There are lots and lots of simple examples to explain the most basic concepts, but when it gets complicated the book often sounds as if the text was taken out of a mathematics book. For example: the basics of probability theory are introduced for over 5 pages with the example of "two coloured boxes each containing fruit". Nothing wrong with that. Then the chapter continues with probability densities which are covered within 2 pages and contain sentences like "Under a nonlinear change of variable, a probability density transforms differently from a simple function, due to the Jacobian factor". There is no mentioning how a simple function exactly transforms, what a Jacobian factor actually is and why we would be interested in a nonlinear change. Surely, some of the introductory pages could have been thrown out to explain in depth the more difficult issues. Unfortunately, this is not the only time, where easy concepts get a lot of attention and the truly important complex concepts are skimmed over. All in all, still worth a read, though do not expect too much.

I have been working in the field of signal processing and speech for more
than 40 years at AT&T Bell labs and, more recently, as a professor at
Rutgers University and at the Univ. of California at Santa Barbara where I
teach courses in digital speech processing and speech recognition. I am
extremely impressed with Chris Bishop's "Pattern Recognition and Machine
Learning." The writing style is such that understanding is maximized by the
clarity of thought and examples provided. He did a very nice job with the
Hidden Markov Model material. He is to be congratulated on this excellent
addition to the literature.

This book (PRML) should be re-titled as "PRML: a bayesian approach". Yes, bayesian approach is very useful for machine learning, and sometimes the final goal of learning is to maximize some sort of posterior probability. However, if the author is such a huge fun of bayes statistics, please tell perspective readers in a clear way. Emphasize bayes aspects too much really hurt the quality of this book as a general-purpose textbook of machine learning.

For a better textbook of machine learning, I recommend:
1) The elements of statistical learning (perhaps this book a little hard for beginner in this field -- but as least better than PRML -- you can compare their chapters about linear regression to see which one is better).
2) Pattern classification (focus on classification, not regression. Also not very easy -- anyway, machine learning is not an easy field ^_^).
3) Machine Learning (a little old, but great for beginner.)

These three book also mention bayesian statistics, but in a proper way. If you have some experience in machine learning and have engineering-level math background, just choose the 1) or 2). If you are completely a beginner, first take a glance on 3), and then go to 1) or 2).

Finally, if you want a book that discusses machine learning purely from bayesian perspective, PRML is good.

I started to read this book after I gave up the book "element of statisitcal learning" which I read about 80 pages. I won't say that the latter book EoSL is bad, but it definitely assumes a much higher math background. Also it doesn't give all the derivations and reasonings, so it may take a long time to understand a single paragraph. The reading is slow and frustrating. I read each chapter twice, but still do not think I did get it in my heart.

By contrast, the book "Pattern Recognition and machine learning" assumes much less math background, and usually gives complete derivation and reasoning, which makes it a pleasure to read. Therefore, if you are not in statistics major (but a CS major with reasonable statistics background), I recommend you to start this book.
Answers to some problems are posted in the author's website (just google the author's name). It is a big plus to me.

First of all, as some other reviewers have pointed out, the subtitle of the book should include the word 'Bayesian' in some form or the other. The reason this is important is because the Bayesian approach, although an important one, is not adapted across the board in machine learning, and consequently, an astonishing number of methods presented in the book (Bayesian versions of just about anything) are not mainstream. The recent Duda book gives a better idea of the mainstream in this sense, but because the field has evolved in such rapidity, it excludes massive recent developments in kernel methods and graphical models, which Bishop includes.

Pedagogically, however, this book is almost uniformly excellent. I didn't like the presentation on some of the material (the first few sections on linear classification are relatively poor), but in general, Bishop does an amazing job. If you want to learn the mathematical base of most machine learning methods in a practical and reasonably rigorous way, this book is for you. Pay attention in particular to the exercises, which are the best I've seen so far in such a text; involved, but not frustrating, and always aiming to further elucidate the concepts. If you want to really learn the material presented, you should, at the very least, solve all the exercises that appear in the sections of the text (about half of the total). I've gone through almost the entire text, and done just that, so I can say that it's not as daunting as it looks. To judge your level regarding this, solve the exercises for the first two chapters (the second, a sort of crash course on probability, is quite formidable). If you can do these, you should be fine. The author has solutions for a lot of them on his website, so you can go there and check if you get stuck on some.

As far as the Bayesian methods are concerned, they are usually a lot more mathematically involved than their counterparts, so solving the equations representing them can only give you more practice. Seeing the same material in a different light can never hurt you, and I learned some important statistical/mathematical concepts from the book that I'd never heard of, such as the Laplace and Evidence Approximations. Of course, if you're not interested, you can simply skip the method altogether.

From the preceding, it should be clear that the book is written for a certain kind of reader in mind. It is not for people who want a quick introduction to some method without the gory details behind its mathematical machinery. There is no pseudocode. The book assumes that once you get the math, the algorithm to implement the method should either become completely clear, or in the case of some more complicated methods (SVMs for example), you know where to head for details on an implementation. Therefore, the people who will benefit most from the book are those who will either be doing research in this area, or will be implementing the methods in detail on lower level languages (such as C). I know that sounds offputting, but the good thing is that the level of the math required to understand the methods is quite low; basic probability, linear algebra and multivariable calculus. (Read the appendices in detail as well.) No knowledge is needed, for example, of measure-theoretic probability or function spaces (for kernel methods) etc. Therefore the book is accessible to most with a decent engineering background, who are willing to work through it. If you're one of the people who the book is aimed at, you should seriously consider getting it.

Edited to Add:
I've changed my rating from 4 stars to 5. Even now, 4-5 years later, there is simply no good substitute for this book.

It's hard to figure out who would actually benefit from this book - it amounts to seven hundred pages of equations interrupted by blocks of text that fail to provide any intuition whatever for the techniques they are describing, and the occasional graph which is remarkable in the universe of graphs as being scarcely more informative than the equations it is meant to illustrate.

Seriously, you have to wonder wtf Bishop thought he was doing here. As a catalog of equations for people who already thoroughly understand the learning algorithms I suppose the book can be considered adequate. For any didactic purpose you're wasting your time - you can find dense, technically correct but incomprehensible descriptions for any of these methods online, for free. A textbook ought to aspire to more - should bring some order to the chaos, re-tell a technical story in a new light to make it more sensible and intuitive. This book is so bad in these regards that it makes me angry.

On a related note, I can't believe that Duda and Hart is still the best machine learning / pattern rec. book on the market after thirty years or whatever. This field is dying for a book by someone with even an INKLING of how to teach, or at least willing to make an effort to try.

I was a big fan of Bishop's earlier "Neural Networks for Pattern Recognition" despite my not being particularly interested in neural networks (as opposed to other aspects of machine learning), and so I was pretty excited when I heard about this book. Reading it has not left me disappointed. Like his earlier book, this text is quite mathematically oriented, and not well-suited for people who aren't comfortable with calculus. However, also like in "NNPR", the writing style here is very clear, and everything past basic calculus and linear algebra is well-explained before it's needed. The appendices alone are a goldmine. (Appendix B is a great "cheat sheet" for commonly used probability distributions; Appendix C has lots of useful matrix properties you may have forgotten or never known; Appendix D quickly explains what you need to know about the calculus of variations; and Appendix E does the same for Lagrange multipliers.) The author also does an excellent job throughout the text of marrying math and intuition without giving either short shrift.

However, note that the material covered is inherently pretty complex, so the book can still be intimidating in parts despite the excellent writing. It's more appropriate for, say, Ph.D. students and professional researchers in statistics or machine learning than people who just want to crank out code for a simple classifier. There is very little pseudocode (although copious MATLAB code will supposedly be made available in a companion book due out in 2008), and the book's overall approach to machine learning is basically hard-core Bayesian statistics. If you are not willing to scratch your head for a while over lots and lots of equations, this may not be the book for you.

On the flip side, people who are already experts in machine learning may be mildly disappointed with the lack of coverage some of their pet topics get. For example, while the chapter on graphical models is excellent as far as it goes, it only mentions the problem of learning graphical model structures (one of my areas of interest) in passing. Reinforcement learning (another personal area of interest) is discussed briefly in the introduction and then written off as beyond the scope of the book.

However, the book is already a fabulous resource as it stands; complaining there's not even more of it would be gauche. The cover may look like goat barf, and there are some innocuous missing words here and there (hey, it's a first edition), but if you're serious about machine learning and not afraid of a little math, you should definitely own this book. I can only imagine how much cooler my own thesis research might have been if this book had been around a few years earlier.

This books reviews the personal illuminations of the author about the fields of Pattern Recognition and Machine Learning. As such it is quite interesting, but only if you have a deep understanding of the field already and want to see a new view on the field. In an interview on Microsoft's Channel 9 (the author works for Microsoft Research, Cambridge, England), the author mentions that this book provides a unifying view of the field with recent break-throughs. The unification is through Bayesian approach. For those that don't know, Bayesian means lots of math and integrals, lots of computation and "you should believe me, this is the right model, because it's Bayesian". From what I've seen Bayesian methods and graphical models seem to work best for images. In practice one is faced with much more acute problems than finding the right values for the hyper-parameters. Topics like feature selection, data imbalance are not discussed.
My feeling is that this book is not appropriate for practitioners in Machine learning and Pattern Recognition. It does not offer any statistical intuition why methods work and how to reason about problems. It's very math heavy, but in my opinion the more math-heavy a machine learning algorithm, the worst it is in practice. Statistical intuition and under what conditions a method works are absent.
I like the book of Hastie, Tibshirani, Friedman: "The elements of statistical learning" that offers much more intuition(discussion), less useless math (from practical point of view) and more experiments. Friedman has produced one of the best-working machine learning algorithms to-date: gradient-boosted decision trees, by putting together four components proven to work in practice (boosting, decision-trees, bagging, and linear combination of week classifiers). Hastie, Tibshirani, Friedman have took lifetimes to think what works and how models related to each other. Graphical models, kernel methods, neural networks: that stuff is only good if you want to write papers, but not solve problems.
Again, a very nice theoretical book, but not useful for the practitioner. This books should be called "My personal unifying theory of Machine Learning and Pattern Recognition using the Bayesian Approach".