Machine Learning: A Probabilistic Perspective (Adaptive Computation and Machine Learning series) 1st Edition

(Disclaimer: I have worked with a draft of the book and been allowed to use the instructor's review copy for this review. I have bought the book from Amazon.co.uk, but apparently this Amazon.com review can't be tagged "verified purchase". I don't receive any compensation whatsoever for writing this review. I hope it will help you chose a machine learning textbook.)

Similar textbooks on statistical/probabilistic machine learning (links to book websites, not Amazon pages):
- Barber's Bayesian Reasoning and Machine Learning ("BRML", Cambridge University Press 2012)
- Koller and Friedman's Probabilistic Graphical Models ("PGM", MIT Press 2009)
- Bishop's Pattern Recognition and Machine Learning ("PRML", Springer 2006)
- MacKay's Information Theory, Inference and Learning Algorithms ("ITILA", CUP 2003)
- Hastie, Tibshirani and Friedman's Elements of Statistical Learning ("ESL", Springer 2009)

* Perspective: My perspective is that of a machine learning researcher and student, who has used these books for reference and study, but not as classroom textbooks.

* Audience/prerequisites: they are comparable among all the textbooks mentioned. BRML has lower expected commitment and specialization, PGM requires more scrupulous reading. The books differ in their topics and disciplinary approach, some more statistical (ESL), some more Bayesian (PRML, ITILA), some focused on graphical models (PGM, BRML). K Murphy compares MLAPP to others here. For detailed coverage comparison, read the table of contents on the book websites.

* Main strength: MLAPP stands out for covering more advanced and current research topics: there is a full chapter on Latent Dirichlet Allocation, learning to rank, L1 regularization, deep networks; in the basics, the decision theory part is quite thorough (e.g. will mention Jeffrey's/uninformative priors). The book is "open" and vivid, doesn't shy away from current research and advanced concepts. This seems to be purposeful, as it shows in many aspects:

- quotes liberally from web sources, something usually not done in academic publications

- borrows "the best" from other authors (always with permission and acknowledgment, of course): most importantly the best pictures and diagrams, but also tables, recaps, insightful diagrams. Whereas other books will produce their own pictures and diagrams themselves (eg, PRML has a distinctive clarity and style in its illustrations), MLAPP takes many of its colour illustrations from other people's publications; therefore it can select the most pithy and relevant pictures to make a point. You could think that reproductions may be illegible and require extra effort to interpret because they come from a variety of sources; I have found that the bonus coming from having precisely the right image prevails.

- frequent references to the literature, mentions of extensions and open questions, as well as computational complexity considerations: for instance, the section on HMMs will mention duration modeling and variable-duration Markov models, and a comparison of the expressive power of hierarchical HMMs versus stochastic context-free grammars, complete with relevant citations, and a brief mention of the computational complexity results from the publications. All this connects the material with research and new ideas in a fine way -- which other textbooks don't achieve, I find. For instance, PGM defers references to a literature section at the end of each chapter, resulting in a more self-contained, but more poorly "linked" text.

* Didactic aids: Another distinctive feature is that the author clearly has tried to include didactic aids gathered over the years, such as recaps, comparative tables, diagrams, much in the spirit of the "generative model of generative models" (Roweis and Ghahramani): e.g. table comparing all models discussed, pros and cons of generative vs. discriminative models, recap of operations on HMMs (smoothing, filtering etc), list of parameter estimation methods for CRFs.

* Editorial features: Other editorial features worth mentioning are

- compared to others, helpful mentions of terminology, e.g. jargon, nomenclature, concept names, in bold throughout the text ("you could also devise a variant thus; this is called so-and-so")

- mathematical notation relatively clear and consistent, occasional obscurities. PGM stands out as excruciatingly precise on this aspect.

- boxes/layout: no "skill boxes" or "case study boxes" (PGM), not many roadmap/difficulty indications like ITILA or PGM, examples are present but woven into the text (not separated like PGM or BRML). Layout rather plain and homogeneous, much like PRML.

- sadly lacks list of figures and tables, but has index of code

* Complete accompanying material:

- interesting exercises (yet fewer than PRML, BRML, PGM); solutions, however, are only accessible to instructors (same with BRML, PGM), which in my experience makes them only half as useful for the self-learner. PRML and ITILA have some solutions online resp. in the book.

- accompanying Matlab/Octave source code, which I found more readily usable than BRML's. PGM and PRML have no accompanying source code, even though the toolkit distributed with Koller's online PGM class might qualify as one. I find accompanying code a truly useful tool for learning; there's nothing like trying to implement an algorithm, checking your implementation against a reference, having boilerplate/utility code for the parts of the algorithm you're not interested in re-implementing. Also, code may clarify an algorithm, even when presented in pseudo-code. By the way, MLAPP has rather few pseudo-code boxes (like BRML or PRML, while PGM is very good here).

- MLAPP is not freely available as a PDF (unlike BRML, closest topic-wise, ESL, or ITILA). This will no doubt reduce its diffusion. My own take on the underlying controversy is in favor of distributing the PDF: makes successful books widely popular and cited (think ITILA or Rasmussen and Williams' Gaussian Processes), increases the book's overall value, equips readers with a weightless copy to annotate with e-ink, or consult on the go. I believe PDF versions positively impact sales, too: impact neutral-to-positive to course textbook/university library sales, indifferent to sales in countries with widely different purchase power, positive to all other segments due to enormous diffusion/popularity.

* Conclusion:
The closest contender to this book I believe is BRML. Both are excellent textbooks and have accompanying source code.

BRML is more accessible, has a free PDF version, and a stronger focus on graphical models.
MLAPP has all the qualities of an excellent graduate textbook (unified presentation, valuable learning aids), and yet is unafraid of discussing detail points (e.g. omnipresent results on complexity), as well as advanced and research topics (LDA, L1 regularization).

I echo other comments regarding typos and inconsistent writing. As an applied statistician, I also have the sense of a book written by an academic mathematician and computer scientist who has had limited experience analyzing real-world data. It is impressive in its coverage, however, and I did learn quite a bit from it.

This book is fantastic ... very well written, but be careful it requires patience and good math backgroung

"Machine Learning: A Probabilistic Perspective" is truly ambitious in its scope. A major selling point of this work is that it covers material, such as deep learning, that other large surveys leave out. There are also some genuine insights in the book. A new perspective is always welcome in this growing technical field, and Dr. Murphy appears to have a lot to offer here.

However, the book suffers in its current form (my copy is the second printing) from the author's and publisher's hurry to get it in print. (The author admits, on the site devoted to the book, that such a rush existed.) The result is a book that, as other reviewers have noted, is less than perfectly organized and riddled with typographical and technical errors.

The errata notes on the author's site suggest that the third printing will be much improved over the first and second (including, among other things, an essentially rewritten chapter). My guess, though, is that it will take a good few years before the book's errors are altogether extirpated. (Aside: Why is it nearly impossible to determine the current printing of a book before you're actually holding it in your hands? This is pretty important if you're trying to get the cleanest copy possible.)

This book has all the ingredients of a five-star ML text. At the moment, though, it is hard to justify more than a three-star rating (especially given its hefty price). I suspect that my ultimate rating will be a function of the book's printing number, its exact shape depending largely on the diligence of the publisher, the author, and his readership.

Review of the kindle version. The content is ok so far, but many variables are not defined in the text or the notation section. In other cases, variables are used as they are vernacularly in the field; in many of these cases, this is not the precise use of the variable. Texts should be precise and not take up idiosyncratic notations of practitioners. This sloppy production makes it hard to objectively review the content.

Aside from this book, I am very dissatisfied with mathematical notation in kindle books, FWIW.

One of the best books on probabilistic approaches to machine learning. I like that the author not only describes various techniques but also takes the time to try to pull out the intuitions behind concepts, compares techniques to help the reader understand what makes a technique special and provides a detailed bibliography so that details can be followed up in the original literature. The book covers a huge swath of material from basic theory of Bayesian reasoning all the way to recent work in deep learning models. I find I learn something new every time I pick it up. The first edition had a lot of errors, but there are published errata and I understand that later editions have been fixed.

I've read the first 6 chapters and I don't think this book is for beginner of ML. It's not a good purchase as a introductory book. That's fine. The think I care the most is that there are multiple errors in the book (missing label for axis in the figure, saying that something will be defined below and it isn't, etc...) Cons is that it covers broad important topics in ML and it's still good to keep a copy.

How couldn't be less for a masterpiece?